Information processing apparatus and information processing method

Abstract

To provide a technique to improve the compressibility of information on reflection characteristics, while preventing a deterioration in texture expression.SOLUTION: An information processing apparatus sets a compression parameter based on a luminous environment, and compresses information on mirror reflection on a surface of an object based on the set compression parameter.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a technique for compressing information related to reflection characteristics.

Background Art

[0002] Measurement data of reflection characteristics that change depending on the type of illumination, illumination direction, and observation direction are used to express the texture of the material and coating of an object. Also, two-dimensional measurement data of reflection characteristics are used to express spatial differences in reflection characteristics. Two-dimensional reflection characteristic data is generally approximated and expressed as three types of information: diffuse reflection information, specular reflection information, and fine unevenness information. However, compared to still images, the amount of data is three times larger, which has become an issue for widespread use among general users.

[0003] In response to this problem, a method of individually compressing each of the diffuse reflection information, specular reflection information, and fine unevenness information has been proposed. For example, in Patent Document 1, depth information, which is one of the diffuse reflection information and the fine unevenness information, is compressed using conventional two-dimensional image compression methods.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when the spatial frequency of illumination information includes high-frequency components, such as in the arrangement of spotlights or fluorescent tubes, even a slight change in the fine unevenness information among the reflection characteristic data significantly affects the "appearance", and the texture expression after compression may deteriorate. The present invention provides a technique for improving the compression rate of information related to reflection characteristics while suppressing deterioration of texture expression.

Means for Solving the Problems

[0006] One aspect of the present invention is characterized by comprising setting means for setting compression parameters based on an illumination environment, and compression means for compressing information regarding specular reflection on the surface of an object based on the compression parameters set by the setting means.

Effects of the Invention

[0007] According to the configuration of the present invention, it is possible to improve the compression rate of information regarding reflection characteristics while suppressing deterioration of texture expression.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant explanations are omitted.

[0010] [First Embodiment] In the present embodiment, compression parameters are set based on the ratio of high-frequency components in the lighting environment, and information on the specular reflection component on the surface of an object is compressed based on the set compression parameters. In the present embodiment, a case will be described in which environment map information is used as information representing the lighting environment in real space (three-dimensional space), and a normal map is used as information on the specular reflection component of an object (specular reflection light information).

[0011] <Explanation of reflection characteristics> Before entering into the detailed description, the reflection characteristics of an object will be described. FIG. 1 is a diagram for explaining the reflection characteristics on the surface of an object. In FIG. 1, the intensity distribution 1104 of the reflected light when light is irradiated from the light source 1101 toward the point 1102 on the object surface with the direction 1103 as the normal direction is shown. The light reflected from the object surface can be separated into a diffuse reflection component 1105 and a specular reflection component 1106. The diffuse reflection component is a component generated by the irregular reflection of the incident light inside the measurement surface and is observed with a substantially uniform intensity in multiple directions. On the other hand, the specular reflection component is a component related to the glossiness generated by the specular reflection of the incident light on the surface of the measurement surface and is observed with an intensity biased in the direction opposite to the illumination direction with respect to the direction 1103.

[0012] Hereinafter, information that affects the intensity, width, or direction of the specular reflection component 1106 in an arbitrary object region will be referred to as "specular reflection light information". The specular reflection light information includes information defining the normal line on the object surface, information regarding fine irregularities on the object surface, information regarding the intensity, width, and direction of the specular reflection component determined by physical property values, and the like.

[0013] <Example of hardware configuration of information processing apparatus> An example of the hardware configuration of the information processing apparatus according to the present embodiment will be described with reference to the block diagram of FIG. 2. As the information processing apparatus according to the present embodiment, computer apparatuses such as a PC (personal computer), a smartphone, and a tablet terminal device are applicable.

[0014] The CPU 101 executes various processes using computer programs and data stored in the main memory 102. Thereby, the CPU 101 controls the operation of the entire information processing apparatus and executes or controls each process described as being performed by the information processing apparatus.

[0015] The main memory 102 has an area for storing computer programs and data loaded from the storage device 103, a work area used when the CPU 101 executes various processes, and the like. In this way, the main memory 102 can appropriately provide various areas.

[0016] The storage device 103 is a large-capacity information storage device such as a hard disk drive. The storage device 103 stores an OS (operating system), computer programs and data for causing the CPU 101 to execute or control each process described as being performed by the information processing device, and the like. The computer programs and data stored in the storage device 103 are appropriately loaded into the main memory 102 according to the control by the CPU 101 and become processing targets by the CPU 101.

[0017] The input device 104 is a user interface such as a keyboard, a mouse, or a touch panel screen, and various instructions can be input to the CPU 101 by the user's operation.

[0018] The display device 105 has a liquid crystal screen or a touch panel screen and displays the processing result by the CPU 101 as an image, characters, or the like. Note that the display device 105 may be a projection device such as a projector that projects an image or characters.

[0019] The CPU 101, the main memory 102, the storage device 103, the input device 104, and the display device 105 are all connected to the system bus 106. Note that the configuration shown in FIG. 2 is an example of a configuration capable of executing various processes described below, and can be appropriately changed / modified.

[0020] <Functional configuration example of information processing device> A functional configuration example of the information processing apparatus according to the present embodiment will be described with reference to the block diagram of FIG. 3. In the following, the functional units shown in FIG. 3 will be described as the main bodies of processing. However, in reality, the functions of these functional units are realized by the CPU 101 executing a computer program for causing the CPU 101 to realize the functions of the functional units. Note that the functional units shown in FIG. 3 may be implemented in hardware.

[0021] The acquisition unit 201 acquires specular reflection light information. The acquisition unit 202 acquires environment map information. The setting unit 203 sets a compression parameter that determines the compression rate of the specular reflection light information acquired by the acquisition unit 201 using the environment map information acquired by the acquisition unit 202. The acquisition unit 204 creates compressed specular reflection light information by compressing the specular reflection light information acquired by the acquisition unit 201 using the compression parameter set by the setting unit 203.

[0022] <Compression Process of Specular Reflection Light Information> The processing performed by the information processing apparatus 2 to compress specular reflection light information will be described with reference to the flowchart of FIG. 4. In step S11, the acquisition unit 201 reads out specular reflection light information from among various pieces of information related to the reflection characteristics stored in the storage device 103 into the main memory 102. In the present embodiment, a case where the specular reflection light information is "normal information that defines the normal direction at each position on the surface of an object" will be described. In the present embodiment, a case where the normal information is a "map in 8-bit RGB image format" having a size of 128 pixels in the vertical direction × 128 pixels in the horizontal direction, each pixel having a pixel value of 8 bits for each of R, G, and B, and having a resolution of 150 dpi will be described. The pixel value of the R component at the pixel position (x, y) is a value obtained by normalizing the X component value (a real number value between -1 and 1) of the normal vector at the position on the object surface corresponding to the pixel position (x, y) to a value between 0 and 255. The pixel value of the G component at the pixel position (x, y) is a value obtained by normalizing the Y component value (a real number value between -1 and 1) of the normal vector at the position on the object surface corresponding to the pixel position (x, y) to a value between 0 and 255. The pixel value of the B component at the pixel position (x, y) is a value obtained by normalizing the Z component value (a real number value between -1 and 1) of the normal vector at the position on the object surface corresponding to the pixel position (x, y) to a value between 0 and 255.

[0023] In step S12, the acquisition unit 202 reads out the environment map information stored in the storage device 103 into the main memory 102. In the present embodiment, a case where the environment map information is a "map in 8-bit RGB image format" having a size of 720 pixels in the vertical direction × 1440 pixels in the horizontal direction, each pixel having a pixel value representing "the intensity of light of the color component from the direction corresponding to the pixel" for each of the color components of R, G, and B will be described.

[0024] Here, the environmental map information will be described with reference to FIG. 5. As shown in FIG. 5(a), using an omnidirectional camera 301 arranged at an arbitrary point X in the real space, the intensity of light is measured for the line-of-sight directions (θi, φi) (i = 0, 1, …, (the number of line-of-sight directions to be measured - 1)) from the point X. Then, the intensity of light measured for the line-of-sight direction (θi, φi) from the point X is set as the pixel value of the pixel corresponding to the pixel position (θi, φi) in the environmental map information (image) shown in FIG. 5(b). Here, (θ, φ) has the point X on the object 301 as the origin, θ is the polar angle with respect to the normal n on the surface of the object 301, and φ is the azimuth angle with respect to the reference direction t. Also, i indicates the measurement number at the time of measurement for each Δθ and Δφ, and on the environmental map information, i corresponds in ascending order with the upper left pixel being 0. For pixel positions where the pixel value is not set, a pixel value obtained by known interpolation processing or the like from the pixel values at the surrounding pixel positions is set.

[0025] In this way, environmental map information can be created in which the pixel value of each pixel is set to the intensity of light measured for the line-of-sight direction corresponding to the pixel. Such environmental map information is created in advance and stored in the storage device 103.

[0026] Next, in step S13, the setting unit 203 sets a compression parameter that determines the compression rate of the specular reflection light information acquired in step S11 based on the environmental map information acquired in step S12. The details of the processing in step S13 will be described according to the flowchart of FIG. 6.

[0027] In step S131, the setting unit 203 converts the environmental map information acquired in step S12 into grayscale environmental map information. In this embodiment, the "environmental map information in 8-bit RGB image format" is converted into an "8-bit Gray image format map". The pixel value at the pixel position (θ, φ) in the "8-bit Gray image format map" represents the luminance value of the pixel at the pixel position (θ, φ) in the "environmental map information in 8-bit RGB image format". Since such conversion is a well-known technique, the description thereof is omitted.

[0028] Then, the setting unit 203 divides the grayscale environment map information into a plurality of divided regions having a region size specified in advance. In the present embodiment, the environment map information having a size of 720 pixels in the vertical direction × 1440 pixels in the horizontal direction is divided into a plurality of divided regions having a size of 72 pixels in the vertical direction × 72 pixels in the horizontal direction (in this case, it is divided into 10 divisions with respect to the θ axis and 20 divisions with respect to the φ axis, for a total of 200 divisions). Further, the setting unit 203 initializes the value of the counter used in the subsequent processing to 0.

[0029] In step S132, the setting unit 203 selects one unselected region among the plurality of divided regions divided in step S131 as the selected divided region. The selection order from the plurality of divided regions is not limited to a specific selection order. FIG. 7(a) shows the environment map information 401 and the selected divided region 402 selected for the first time.

[0030] In step S133, the setting unit 203 converts the pixel value group in the selected divided region into two-dimensional spatial frequency information. For example, the setting unit 203 applies a two-dimensional FFT to the selected divided region to convert the pixel value group in the selected divided region into two-dimensional spatial frequency information, which is information in the frequency space. Then, the setting unit 203 obtains the intensity for each frequency by integrating the angular direction for each frequency in the two-dimensional spatial frequency information (converts the two-dimensional spatial frequency information into one-dimensional spatial frequency information with the frequency as the axis). FIG. 7(b) shows the result of applying the processing in step S133 to the selected divided region 402 shown in FIG. 7(a).

[0031] In step S134, the setting unit 203 determines whether the one-dimensional spatial frequency information of the selected divided region converted in step S133 contains high-frequency components. In this embodiment, 18 [cycles / 72 pixels] or more is defined as the high-frequency component. And in this embodiment, the setting unit 203 sets the integrated value of the frequency components of the entire selected divided region to 100%. If the high-frequency components account for 20% (threshold) or more of this value, it is determined that "the one-dimensional spatial frequency information of the selected divided region contains high-frequency components". On the other hand, when the setting unit 203 sets the integrated value of the frequency components of the entire selected divided region to 100% and the proportion of the high-frequency components is less than 20%, it is determined that "the one-dimensional spatial frequency information of the selected divided region does not contain high-frequency components". Examples where the high-frequency components are considered to be few and examples where the high-frequency components are considered to be many according to the environment map information will be described separately later.

[0032] As a result of such determination, if it is determined that the one-dimensional spatial frequency information of the selected divided region converted in step S133 contains high-frequency components, the process proceeds to step S135. On the other hand, if it is determined that the one-dimensional spatial frequency information of the selected divided region converted in step S133 does not contain high-frequency components, the process proceeds to step S136.

[0033] In step S135, the setting unit 203 increments (counts) the value of the counter by one. In step S136, the setting unit 203 determines whether all of the plurality of divided regions divided in step S131 have been selected as the selected divided region. And as a result of such determination, if all of the plurality of divided regions divided in step S131 have been selected as the selected divided region, the process proceeds to step S137. On the other hand, if there is one or more remaining divided regions among the plurality of divided regions divided in step S131 that have not yet been selected as the selected divided region, the process proceeds to step S132.

[0034] In step S137, the setting unit 203 sets compression parameters according to the value (count number) of the counter. In the present embodiment, a known JPEG compression method is adopted for compressing the specular reflection light information performed in step S14 described later. The JPEG compression method is a compression method capable of specifying compression parameters that determine the degree of compression. When parameter value 0 is specified as the compression parameter, compression with the highest compression rate can be specified, and when parameter value 12 is specified as the compression parameter, compression with the lowest compression rate can be specified. In the present embodiment, information indicating the correspondence relationship between the count number and the compression parameter is registered in the storage device 103 in advance, and the setting unit 203 refers to such information and sets the compression parameter corresponding to the count number.

[0035] A graph which is an example of information indicating the correspondence relationship between the count number and the compression parameter is shown in FIG. 7(c). FIG. 7(c) shows a graph indicating the relationship between the count number and the compression parameter. In the graph of FIG. 7(c), the horizontal axis represents the count number and the vertical axis represents the compression parameter. As shown in FIG. 7(c), the smaller the count number, that is, the smaller the high-frequency component, the more a compression parameter for specifying compression with a higher compression rate is set. On the other hand, the larger the count number, that is, the more the high-frequency component, the more a compression parameter for specifying compression with a lower compression rate is set.

[0036] Therefore, when using the graph of FIG. 7(c) as information indicating the correspondence relationship between the count number and the compression parameter, the setting unit 203 specifies from the graph the compression parameter corresponding to the count number. Then, the setting unit 203 sets the specified compression parameter as the compression parameter to be used for compressing the specular reflection light information.

[0037] In this way, in the present embodiment, the compression rate of the specular reflection light information is changed according to the ratio of the high-frequency component contained. Note that the theory in the features of such a present embodiment will be described in "Influence on the Appearance of Illumination Spatial Frequency Information" described later. When the process of step S137 ends, the process proceeds to step S14 in FIG. 4.

[0038] In step S14, the compression unit 204 compresses the specular reflection light information obtained in step S11 according to the JPEG compression method using the compression parameters set in step S13, thereby generating the compressed specular reflection light information as the compressed specular reflection light information. Then, the compression unit 204 outputs the compressed specular reflection light information. The output destination of the compressed specular reflection light information is not limited to a specific output destination. For example, the compression unit 204 may store the compressed specular reflection light information in the storage device 103, may display it on the display device 105, or may transmit it to an external device via a network interface (not shown).

[0039] In this embodiment, the case where the specular reflection light information is the normal information as described above has been described. However, as described above, the specular reflection light information may be information that affects the intensity, width, or direction of the specular reflection component in an arbitrary object region. That is, the specular reflection light information may be information that defines the normal represented by the normal information or height information, or information on the intensity, width, and direction of the specular reflection component determined by the unevenness information finer than the normal information or the physical property value. Specifically, the specular reflection light information may be a data storing a normal map, a height map, a bump map, a depth map, a displacement map, a specular map, a substance map, a gloss map, a roughness map, or physical-based parameters of the specular reflection component represented by the Cook-Torrance model. Furthermore, regarding the data size, resolution, bit depth, and format, it is not limited to the above examples.

[0040] Also, in this embodiment, a case where a Latitude-Longitude type (also called Cylindrical type) environmental map with the polar angle and azimuth angle at the time of measurement as axes is used as environmental map information has been described, but it is not limited to this. For example, as environmental map information, an environmental map obtained by converting omnidirectional information typified by Latitude-Longitude type, Cube type, and Sphere type into a two-dimensional image format may be adopted. Also, regarding information in which the arrangement and direction of illumination are specified in real space, since it can be converted into the above environmental map by performing baking processing generally used in rendering technology, such an environmental map may be used as environmental map information.

[0041] Also, in this embodiment, as a method for creating environmental map information, a method of generating environmental map information based on the intensity of light from various line-of-sight directions measured using an omnidirectional camera has been described, but the method for creating environmental map information is not limited to this. For example, the intensity of light from various line-of-sight directions may be measured using a camera with a fish-eye lens or a mirror ball. Also, environmental map information may be estimated from information at the time of measurement such as non-omnidirectional camera information, GPS information, and gyro sensor information, or environmental map information corresponding to the information at the time of estimation may be specified by referring to a database of pre-prepared environmental map information.

[0042] Also, in this embodiment, the case where the environmental map information is divided into a plurality of divided regions in step S13 and it is determined whether high-frequency components are included in each divided region has been described. However, it is only necessary to be able to set compression parameters according to the ratio of high-frequency components included in the environmental map information, and this case described in this embodiment is just an example. For example, the value obtained by integrating the frequency components of the entire environmental map information is set to 100%. Then, if the ratio occupied by the high-frequency components among them is equal to or higher than the threshold value, it is determined that the environmental map information includes high-frequency components, and if the ratio is less than the threshold value, it is determined that the environmental map information does not include high-frequency components. And when it is determined that the environmental map information includes high-frequency components, the first compression parameter is set, and when it is determined that the environmental map information does not include high-frequency components, the second compression parameter with a higher compression rate than the first compression parameter is set. In this case, the compression unit 204 compresses the specular reflection light information using the first compression parameter or the second compression parameter.

[0043] Also, in this embodiment, in step S131, the environmental map information acquired in step S12 is converted into grayscale environmental map information. The process of this step S131 is a preprocess for performing frequency conversion on the selected divided region in the environmental map information in step S133. Therefore, such grayscale conversion does not have to be performed as long as it is a method of performing frequency conversion on each color component in the map in RGB image format. Also, in this embodiment, preprocessing other than the grayscale conversion in step S131 (preprocessing before frequency conversion) is not shown, but preprocessing such as noise removal and distortion correction may be added if necessary.

[0044] Also, in this embodiment, the case where the size of the divided region is 72 pixels in height × 72 pixels in width has been described, but the size of the divided region is not limited to 72 pixels in height × 72 pixels in width, and any size can be used as long as the low-frequency components and high-frequency components necessary for the determination in step S134 can be separated.

[0045] Also, in the present embodiment, in step S133, conversion was performed into two-dimensional spatial frequency information for each divided region, and the two-dimensional spatial frequency information was converted into one-dimensional spatial frequency information. Such a conversion method is not limited to a specific method, and for example, Wavelet transform may be applied. Also, in accordance with the determination method in step S134, two-dimensional spatial frequency information or information created based on the two-dimensional spatial frequency information may be used.

[0046] Also, the threshold values described in the present embodiment are merely examples and are not limited thereto. For example, the threshold value used to determine whether the one-dimensional spatial frequency information of the selected divided region converted in step S133 contains high-frequency components may be any threshold value that separates high-frequency components from low-frequency components. For example, the user may manually set the threshold value by subjectively evaluating the texture of the rendering result of the specular reflection light information after compression, or may manually set the threshold value based on the evaluation value related to the texture.

[0047] The influence on the appearance by the illumination spatial frequency information (spatial frequency information of the illumination environment) will be described with reference to FIG. 8. FIG. 8(a) shows an example of observing the reflection characteristics under diffused illumination 501 where the outline of the illumination is not clear, such as is typical outdoors. The incident light 501a from the diffused illumination 501 is reflected at the point 502 on the object having the normal n, and the specular reflection component of the reflected light is most strongly observed at the observation position 503 in the direct reflection direction. At the observation position 504 slightly deviated from the observation position 503, the reflected light of the incident light 501b from the diffused illumination 501 is observed. Since the incident light 501a and the incident light 501b have a small intensity difference because the illumination itself is the diffused illumination 501, the reflected light observed at the observation position 503 and the observation position 504 also has a small intensity difference. That is, when "the high-frequency components included in the illumination environment are few", such as when the outline of the illumination is not clear or the illumination intensity is uniform, it can be said that the influence on the appearance due to changes in the observation position and the normal direction is small.

[0048] On the one hand, Fig. 8(b) shows an example of observing the reflection characteristics under the array illumination 505 in which the illuminations are arranged at regular intervals, such as the fluorescent tube illumination array. The incident light 505a from the array illumination 505 is reflected at the point 506 on the object having the normal line n, and the specular reflection component of the reflected light is most strongly observed at the observation position 507 in the specular reflection direction. On the other hand, at the observation position 508 slightly deviated from the observation position 507, the reflected light of the incident light 505b from the interval part (the part without illumination) of the illumination array will be observed. Therefore, the intensity difference of the reflected light observed at the observation position 507 and the observation position 508 is large. That is, when there are a plurality of illuminations arranged at regular intervals or the contour of the illumination is clear, that is, "when there are many high-frequency components in the illumination environment", it can be said that the influence on the appearance due to the change in the observation position or the normal direction is large.

[0049] In the above example, for the sake of clarity of explanation, only the direction and intensity of the specular reflection component are explained, but the same applies even if the specular reflection component has a spread. Specifically, considering the case where the object reflects with a spread of specular reflection in Fig. 8(a), at the observation position 503, the reflected light obtained by integrating the reflected light of the incident light 501a and the reflected light spread by the reflection of the incident light 501b is observed. The same is true at the observation position 504. Thus, since the integrated value of the adjacent incident light is observed, it is rare for the magnitude of the spread width of the specular reflection to be significantly observed. It can be said that when "there are few high-frequency components in the illumination environment", the influence of the spread of the specular reflection is small.

[0050] On the other hand, when the object reflects with a spread of specular reflection in Fig. 8(b), at the observation position 508, the reflected light spread by the reflection of the incident light 505a will be observed. Although the reflected light observed at the observation position 508 increases, there is a difference in intensity from the reflected light observed at the observation position 507. Also, the intensity of the reflected light observed at the observation position 507 is significantly proportional to the magnitude of the spread width of the specular reflection. It can be said that when "there are many high-frequency components in the illumination environment", the influence of the spread amount of the specular reflection is large.

[0051] <Example of illumination spatial frequency information> An example of lighting spatial frequency information corresponding to environmental map information in various lighting environments (an example of the spatial frequency information of environmental map information) is shown in FIG. 9. The environmental map information 601a is a map representing a lighting environment of a blurred point light source, such as represented by stars or a night view. The lighting spatial frequency information 601b is the spatial frequency information of the environmental map information 601a.

[0052] The environmental map information 602a is a map representing a lighting environment of a point light source with a clear contour, such as represented by a spotlight. The lighting spatial frequency information 602b is the spatial frequency information of the environmental map information 602a.

[0053] The environmental map information 603a is a map representing a lighting environment of an area light source that has a substantially uniform light amount in the plane and the light amount gradually decreases towards the edge, such as represented by the outdoors or a light board. The lighting spatial frequency information 603b is the spatial frequency information of the environmental map information 603a.

[0054] The environmental map information 604a is a map representing a lighting environment of an area light source that has a substantially uniform light amount in the plane and a clear contour, such as represented by light from a window. The lighting spatial frequency information 604b is the spatial frequency information of the environmental map information 604a.

[0055] The environmental map information 605a is a map representing a lighting environment of a blurred line light source array, such as represented by a fluorescent tube lighting arrangement with a diffuser plate often used in offices. The lighting spatial frequency information 605b is the spatial frequency information of the environmental map information 605a.

[0056] The environmental map information 606a is a map representing a lighting environment of a line light source array with a clear contour, such as represented by a fluorescent tube lighting arrangement without a diffuser plate often used in facilities for observing the reflection characteristics of objects. The lighting spatial frequency information 606b is the spatial frequency information of the environmental map information 606a.

[0057] For example, it can be seen that in the environmental map information 602a, the intensity of the high-frequency component region (the fronto-parallel region surrounded by the dotted line in FIG. 9) in the illumination spatial frequency information 602b (the intensity of the high-frequency component) is high. The same applies to the environmental map information 604a and the environmental map information 606a. On the other hand, in the environmental map information 601a, 603a, and 605a, it can be seen that the intensity of the high-frequency component is low. In the present embodiment, in an illumination environment with many high-frequency components as shown in the environmental map information 602a, 604a, 606a and the illumination spatial frequency information 602b, 604b, 606b, the compression rate of the specular reflection light information is set low. On the other hand, in the present embodiment, in an illumination environment with few high-frequency components as shown in the environmental map information 601a, 603a, 605a and the illumination spatial frequency information 601b, 603b, 605b, the compression rate of the specular reflection light information is set high.

[0058] <Effect of the First Embodiment> The effect of the present embodiment will be described using FIG. 10 showing examples of a rendering image rendered in an illumination environment with many high-frequency components and a rendering image rendered in an illumination environment with few high-frequency components.

[0059] The upper rendering images 701a, 701b, and 701c in FIG. 10 are rendering results corresponding to an illumination environment with few high-frequency components (illumination condition A). The rendering image 701b is an image compressed at a higher compression rate than the rendering image 701a and has a smaller data amount than the rendering image 701a. Here, "compression of the rendering image" includes compression of the specular reflection light information of the rendering image, and "data amount of the rendering image" includes the data amount of the specular reflection light information of the rendering image. The rendering image 701c is an image compressed at a higher compression rate than the rendering image 701b and has a smaller data amount than the rendering image 701b. That is, the rendering image 701a is the image compressed at the lowest compression rate and having the largest data amount, and the rendering image 701c is the image compressed at the highest compression rate and having the smallest data amount.

[0060] In addition, the lower rendering images 702a, 702b, and 702c in FIG. 10 are rendering results corresponding to an illumination environment with many high-frequency components (illumination condition B). The rendering image 702b is an image compressed at a higher compression rate than the rendering image 702a and has a smaller data amount than the rendering image 702a. The rendering image 702c is an image compressed at a higher compression rate than the rendering image 702b and has a smaller data amount than the rendering image 702b. That is, the rendering image 702a is the image compressed at the lowest compression rate and having the largest data amount, and the rendering image 702c is the image compressed at the highest compression rate and having the smallest data amount.

[0061] When the present embodiment is applied, for example, in illumination condition A, since there are few high-frequency components in the illumination environment, a compression parameter that realizes the compression rate corresponding to the rendering image 701c with the highest compression rate and the smallest data amount is selected. Also, for example, in illumination condition B, since there are many high-frequency components in the illumination environment, a compression parameter that realizes the compression rate corresponding to the rendering image 702b with a low to medium compression rate and a large to medium data amount is selected.

[0062] That is, since the appearances of the rendering images 701a, 701b, and 701c do not differ much from each other, in the present embodiment, the compression parameter of the rendering image 701c is selected as the compression parameter that realizes a higher compression rate within a range where the influence on the appearance is small. Also, the rendering image 702c does not show a detailed change in gloss compared to the rendering image 702a and has a large influence on the appearance. In contrast, the rendering image 702b has a small difference in the detailed change in gloss compared to the rendering image 702a and has a small influence on the appearance. Therefore, in the present embodiment, the compression parameter of the rendering image 702b is selected as the compression parameter that realizes a higher compression rate within a range where the influence on the appearance is small. Thus, according to the present embodiment, it is possible to improve the compression rate of the specular reflection light information of the object while suppressing the deterioration of the texture expression of the object.

[0063] [Second Embodiment] In each of the subsequent embodiments including this embodiment, the differences from the first embodiment will be described, and it is assumed that they are the same as the first embodiment unless otherwise specified hereinafter. In this embodiment, compression parameters are set according to information indicating the type of illumination in the illumination environment, and specular reflection light information is compressed based on the set compression parameters.

[0064] <Functional Configuration Example of Information Processing Apparatus> A functional configuration example of the information processing apparatus 2 according to this embodiment will be described with reference to the block diagram of FIG. 11. Hereinafter, the functional units shown in FIG. 11 will be described as the main bodies of processing. In reality, however, the functions of these functional units are realized by the CPU 101 executing a computer program for causing the CPU 101 to realize the functions of these functional units. Note that the functional units shown in FIG. 11 may be implemented in hardware.

[0065] The configuration shown in FIG. 11 has a configuration in which the acquisition unit 202 in the configuration shown in FIG. 3 is replaced with an acquisition unit 205. The acquisition unit 205 acquires information (illumination type information) indicating the type of illumination in the illumination environment, such as the illumination space frequency information described above.

[0066] <Compression Processing of Specular Reflection Light Information> The processing performed by the information processing apparatus 2 to compress the specular reflection light information will be described according to the flowchart of FIG. 12. In FIG. 12, the same processing steps as those shown in FIG. 4 are assigned the same step numbers, and the description of the processing steps will be omitted.

[0067] In step S21, the acquisition unit 205 reads out the illumination type information stored in the storage device 103 to the main memory 102. The illumination type information is text information indicating the type of illumination in the illumination environment, such as "spotlight" and "fluorescent tube lighting array without diffuser plate", as shown in the table of FIG. 13.

[0068] The acquisition unit 205 may acquire the preset lighting type information, or may acquire the lighting type information selected by the user operating the input device 104. Further, the acquisition unit 205 may acquire the lighting type information associated with the specular reflection light information or the lighting type information associated with other information indicating the reflection characteristics of the object. That is, the acquisition of the lighting type information is not limited to a specific acquisition method.

[0069] Further, the lighting type information is not limited to text information, and may be, for example, an identification ID corresponding to the lighting type, and is not limited to a specific format of information as long as it is information in a format capable of specifying the lighting type.

[0070] In step S22, the setting unit 203 reads the table of FIG. 13 stored in the storage device 103 into the main memory 102. The table of FIG. 13 is a table in which various lighting type information and compression parameters corresponding to the lighting type information are registered in association with each other, and is created in advance and stored in the storage device 103. In the table, a compression parameter specifying a lower compression rate is associated with the lighting type information of the lighting type that contains more high-frequency components. Also, in the table, a compression parameter specifying a higher compression rate is associated with the lighting type information of the lighting type that contains fewer high-frequency components.

[0071] Then, the setting unit 203 sets, in the table read into the main memory 102, the compression parameter registered in association with the lighting type information read into the main memory 102 in step S21 as the compression parameter to be used for compressing the specular reflection light information. Note that the format of the table for holding the lighting type information and the compression parameter in association with each other is not limited to a specific format as shown in FIG. 13.

[0072] <Effect of the Second Embodiment> In the first embodiment, even when the type of illumination is unknown, it was possible to obtain compression parameters by analyzing the ratio of high-frequency components in the environmental map information. On the other hand, if the type of illumination is known, as in this embodiment, compression parameters can be obtained from the text information indicating the type of illumination. In this case, compared to obtaining compression parameters by analyzing the ratio of high-frequency components in the environmental map information as in the first embodiment, compression parameters can be obtained with less processing cost.

[0073] [Third Embodiment] In this embodiment, specular reflection light information (map) is divided into a plurality of divided regions, and compression parameters are set for each divided region. The compression parameter of a divided region is set according to the ratio of high-frequency components in the region on the environmental map information (map) based on the normal direction on the surface of the object corresponding to the divided region and the observation direction with respect to the object. Then, in this embodiment, each divided region is compressed based on the compression parameter set for the divided region.

[0074] [Functional Configuration Example of Information Processing Apparatus] A functional configuration example of the information processing apparatus 2 according to this embodiment will be described with reference to the block diagram of FIG. 14. In FIG. 14, the same reference numerals are given to the functional units similar to those shown in FIG. 3, and the description related to the functional units is omitted. In the following, the functional units shown in FIG. 14 will be described as the main body of processing. Actually, a computer program for causing the CPU 101 to realize the functions of the functional units is executed by the CPU 101, whereby the functions of the functional units are realized. Note that the functional units shown in FIG. 14 may be implemented in hardware.

[0075] The acquisition unit 206 acquires line-of-sight information indicating the observation direction with respect to the object. The setting unit 207 sets compression parameters for each divided region in the specular reflection light information based on the specular reflection light information, the environmental map information, and the line-of-sight information. The compression unit 208 compresses each divided region in the specular reflection light information based on the compression parameter set for the divided region.

[0076] <Compression Processing of Specular Reflection Light Information> The processing performed by the information processing apparatus 2 to compress the specular reflection light information will be described according to the flowchart of FIG. 15. In FIG. 15, the same processing steps as those shown in FIG. 4 are assigned the same step numbers, and the descriptions related to these processing steps are omitted.

[0077] In step S31, the acquisition unit 206 reads the line-of-sight information stored in the storage device 103 into the main memory 102. The line-of-sight information is information indicating the observation direction (θc, φc) with respect to an object.

[0078] Note that the line-of-sight information is not limited to being fixed information stored in the storage device 103 in advance. For example, it may be line-of-sight information indicating an observation direction changed by the user operating the input device 104, or it may be information indicating an observation direction measured by a sensor or the like.

[0079] In step S32, the setting unit 207 sets compression parameters for each divided region in the specular reflection light information based on the specular reflection light information acquired in step S11, the environmental map information acquired in step S12, and the line-of-sight information acquired in step S31. The details of the processing in step S32 will be described according to the flowchart of FIG. 16.

[0080] In step S321, the setting unit 207 converts the environmental map information acquired in step S12 into grayscale environmental map information in the same manner as in step S131 above. Also, the setting unit 207 divides the specular reflection light information into a plurality of divided regions having a predetermined region size in the same manner as the division of the above environmental map information. In this embodiment, specular reflection light information having a size of 128 pixels in the vertical direction × 128 pixels in the horizontal direction is divided into a plurality of divided regions having a size of 16 pixels in the vertical direction × 16 pixels in the horizontal direction (in this case, it will be divided 8 times in the vertical direction, 8 times in the horizontal direction, for a total of 64 divisions).

[0081] In step S322, the setting unit 207 selects one unselected region among the plurality of divided regions divided in step S321 as the selected divided region. The selection order from the plurality of divided regions is not limited to a specific selection order. FIG. 17(a) shows the specular reflection light information 801 and the selected divided region 802 selected for the first time.

[0082] In step S323, the setting unit 207 specifies the reference range in the environment map information from the pixel value (normal direction) of each pixel in the selected divided region and the observation direction indicated by the line-of-sight information. The process of step S323 will be described taking FIG. 17(b) as an example. First, the setting unit 207 obtains the observation direction (θc, φc) 803 and the reference direction (θr, φr) in the specular reflection direction with respect to the representative normal direction n on the surface of the object corresponding to the selected divided region. The representative normal direction n may be, for example, the normal direction indicated by the pixel value of the pixel at the center position of the selected divided region, or the average value (average normal direction) of the pixel values of the pixels in the selected divided region. Further, the setting unit 207 obtains the observation direction (θc, φc) 803 and the reference direction in the specular reflection direction with respect to the normal direction indicated by the pixel value of each of a plurality of pixels (which may be all pixels in the selected divided region or a part of the pixel group) in the selected divided region.

[0083] Then, for each reference direction obtained in this way, the setting unit 207 specifies the pixel position closest to the reference direction in the environment map information, and specifies a reference range 804 that includes the pixel positions specified for each of the reference directions.

[0084] In step S324, the setting unit 207 cuts out from the environment map information a rectangular region including the reference range specified in step S323 as the cutout region. FIG. 17(c) shows a cutout region 805 (a rectangular region indicated by a solid line) including the reference range 804 (a circular region indicated by a dotted line) in the environment map information. Note that the shape of the cutout region is not limited to a rectangle and may be an arbitrary shape such as a circle.

[0085] In step S325, the setting unit 207 converts the group of pixel values in the cutout area into two-dimensional spatial frequency information in the same manner as in step S133 above, and converts the two-dimensional spatial frequency information into one-dimensional spatial frequency information with frequency as the axis.

[0086] In step S326, the setting unit 207 specifies the ratio of the high-frequency components in the one-dimensional spatial frequency information of the cutout area converted in step S325 in the same manner as in the first embodiment, and sets the compression parameter of the selected division area according to the specified ratio.

[0087] In this embodiment, information indicating the correspondence between the ratio of high-frequency components and the compression parameter is registered in the storage device 103 in advance, and the setting unit 207 refers to such information and sets the compression parameter corresponding to the ratio of high-frequency components.

[0088] A graph which is an example of the information indicating the correspondence between the ratio of high-frequency components and the compression parameter is shown in FIG. 17(d). FIG. 17(d) shows a graph indicating the relationship between the ratio of high-frequency components and the compression parameter. In the graph of FIG. 17(d), the horizontal axis represents the ratio of high-frequency components, and the vertical axis represents the compression parameter. As shown in FIG. 17(d), the smaller the ratio of high-frequency components, the more the compression parameter for specifying compression with a higher compression rate is set. On the other hand, the larger the ratio of high-frequency components, the more the compression parameter for specifying compression with a lower compression rate is set. Note that the theory in such a feature of this embodiment was explained in the above "Influence on the Appearance of Illumination Spatial Frequency Information".

[0089] In step S327, the setting unit 207 determines whether all of the plurality of divided areas divided in step S321 are selected as the selected division areas. And as a result of such determination, when all of the plurality of divided areas divided in step S321 are selected as the selected division areas, the process proceeds to step S34 in FIG. 15. On the other hand, when one or more divided areas that have not yet been selected as the selected division areas remain among the plurality of divided areas divided in step S321, the process proceeds to step S322.

[0090] And in step S34, the compression unit 208 compresses each divided region in the specular reflection light information based on the compression parameter set in step S32 for the divided region.

[0091] <Effect of the Third Embodiment> In the first embodiment, one compression parameter was set for the entire specular reflection light information, and compression was performed according to the same one compression parameter for the entire specular reflection light information. In contrast, in this embodiment, considering the observation direction with respect to the object, a compression parameter is set for each divided region in the specular reflection light information, and compression is performed according to an individual compression parameter for each divided region. Thereby, for example, for a region that contains many high-frequency components and is illuminated, the compression rate is lowered to minimize the influence on the appearance, and for a region that contains few high-frequency components and is illuminated, the compression rate is increased while reducing the influence on the appearance.

[0092] Also, the numerical values, processing timings, processing orders, processing entities, structures / sending destinations / sending sources / storage locations of data (information), etc. used in the above-described embodiments and modification examples are given as examples for the purpose of specific explanation, and are not intended to be limited to such examples.

[0093] Also, some or all of the above-described embodiments and modification examples may be used in appropriate combinations. Also, some or all of the above-described embodiments and modification examples may be selectively used.

[0094] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0095] The invention is not limited to the above embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, the claims are appended to disclose the scope of the invention.

Explanation of Signs

[0096] 201: Acquisition Unit 202: Acquisition Unit 203: Setting Unit 204: Compression Unit

Claims

1. Setting means for setting a compression parameter based on an illumination environment; Compression means for compressing information regarding specular reflection on the surface of an object based on the compression parameter set by the setting means An information processing apparatus comprising the above.

2. The information processing apparatus according to claim 1, wherein the setting means acquires a map representing the intensity of light measured in each direction in the illumination environment, and sets a compression parameter based on the ratio of high-frequency components in the map.

3. The information processing apparatus according to claim 2, wherein the setting means sets a first compression parameter when the ratio of high-frequency components in the map is equal to or greater than a threshold value, and sets a second compression parameter with a higher compression rate than the first compression parameter when the ratio of high-frequency components in the map is less than the threshold value.

4. The information processing apparatus according to claim 2, wherein the setting means divides the map into a plurality of divided regions, and sets a compression parameter that specifies compression with a higher compression rate as the number of divided regions in which the ratio of high-frequency components is equal to or greater than the threshold value is smaller, and sets a compression parameter that specifies compression with a lower compression rate as the number of such regions is larger.

5. The information processing apparatus according to claim 1, wherein the setting means sets a compression parameter associated with illumination type information indicating the type of illumination in the illumination environment.

6. The setting means acquires a first map representing the intensity of light measured in each direction in the illumination environment, divides a second map indicating the component of specular reflection into a plurality of divided regions, and sets a compression parameter for the divided region based on the ratio of high-frequency components in the region of the first map based on the normal direction of the surface of the object corresponding to the divided region and the observation direction with respect to the object. The compression means compresses each of the plurality of divided regions based on the compression parameter of the divided region. The information processing apparatus according to claim 1, characterized by the above.

7. The information processing apparatus according to claim 6, wherein the setting means obtains a reference direction that is in the observation direction and the specular reflection direction with respect to the normal direction of each surface of the object corresponding to the divided region, and sets a compression parameter for the divided region based on the ratio of high-frequency components in a region including the pixel position closest to the reference direction in the first map.

8. The setting means sets a compression parameter that specifies compression with a higher compression ratio as the ratio of the high-frequency components in the region including the pixel position closest to the reference direction in the first map is smaller, and sets a compression parameter that specifies compression with a lower compression ratio as the ratio is larger. The information processing apparatus according to claim 7, characterized in that.

9. The information regarding the specular reflection is a map of the normal line on the surface of the object. The information processing apparatus according to any one of claims 1 to 8, characterized in that.

10. The information regarding the specular reflection is information regarding the intensity, width, or direction of the specular reflection light on the surface of the object. The information processing apparatus according to any one of claims 1 to 8, characterized in that.

11. A setting step of setting a compression parameter based on the illumination environment, A compression step of compressing information regarding specular reflection on the surface of the object based on the compression parameter set in the setting step An information processing method characterized by comprising:

12. A computer program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 10.

Images