Information processing device, information processing method, and program

Abstract

To provide processing for reducing the amount of texture data while suppressing the deterioration of the texture of an object expressed using the texture data.SOLUTION: An information processing device acquires texture data representing the texture of the surface of an object, and outputs texture data encoded by an encoding method determined according to the texture data. The encoding method is either an encoding method in which encoding is performed by dividing into layers in the bit plane direction, or an encoding method in which encoding is performed by dividing into layers in the resolution direction.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a technique for compressing data relating to the texture of an object. [Background technology]

[0002] In order to reproduce the texture of an object's material or paint, measurement data of reflection characteristics according to the lighting direction and observation direction is used. Reflection characteristic data generally includes information on the diffuse reflection and specular reflection of the object, as well as information on minute surface irregularities, and is characterized by a larger data volume than still image data. Patent Document 1 discloses a technology for compressing texture information data by separating an image of a subject into components with approximately the same feature values ​​based on the feature values ​​of the texture information data and associating each component with a representative value of the feature value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-149390 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, when targeting images that include components that look different even if the feature amounts are similar, image quality may deteriorate depending on how the representative value is selected.

[0005] Therefore, an object of the present invention is to provide a process for reducing the amount of texture data while suppressing deterioration of the texture of an object expressed using texture data. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to the present invention includes: an acquisition means for acquiring texture data representing the texture of an object surface; a selection means for selecting, based on the texture data, either an encoding method in which a hierarchy is established in bit planes and encoding is performed, or an encoding method in which a hierarchy is established in resolution and encoding is performed; The aforementioned Selected an output means for outputting the texture data encoded by the encoding method; have It is characterized by: [Effects of the Invention]

[0007] According to the present invention, it is possible to reduce the amount of texture data while suppressing deterioration of the texture of an object expressed using texture data. [Brief explanation of the drawings]

[0008] [Figure 1] Diagram for explaining the reflection characteristics of an object [Figure 2] Block diagram showing the functional configuration of an information processing device [Figure 3] 1 is a flowchart showing a process executed by an information processing device; [Figure 4] Block diagram showing the functional configuration of an information processing device [Figure 5] A diagram showing an example of an object with regions made of different materials [Figure 6] A diagram showing an example of the file format [Figure 7] 10 is a flowchart showing a process for analyzing texture data. [Figure 8] Block diagram showing the hardware configuration of an information processing device DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all of the combinations of features described in each embodiment are necessarily essential to the solution of the present invention.

[0010] [First embodiment] In the first embodiment, when encoding texture data that represents the texture of an object, the hierarchical encoding method is switched depending on the type of texture data.

[0011] First, the reflection characteristics of an object will be described. FIG. 1 is a diagram illustrating the reflection characteristics of an object. FIG. 1(a) shows a distribution 104 of reflected light when light is irradiated from a light source 101 toward a point 102 on the object surface having a normal 103. The light reflected from the object surface is separated into diffusely reflected light 105 shown in FIG. 1(b) and specularly reflected light 106 shown in FIG. 1(c). The diffusely reflected light 105 is incident light that is scattered inside the object and emitted to the outside of the object, and is observed with approximately uniform intensity in all directions. The intensity of the diffusely reflected light 105 is called the diffuse reflection intensity. On the other hand, the specularly reflected light 106 is incident light that is reflected from the object surface so that the angle of incidence and the angle of reflection are equal. The reflection intensity in the direction in which the intensity of the specularly reflected light is greatest is called the specular reflection intensity. Furthermore, the specularly reflected light has a spread 107 around the specular reflection direction due to minute irregularities on the object surface. The width of the spread 107 of specularly reflected light is called the specular reflection width.

[0012] <Hardware configuration of information processing device> FIG. 8 is a block diagram showing the hardware configuration of the information processing device 1. The information processing device 1 includes a CPU 801, a ROM 802, and a RAM 803. The information processing device 1 also includes a VC (video card) 804, a general-purpose I / F (interface) 805, a SATA (serial ATA) I / F 806, and a NIC (network interface card) 807. The CPU 801 uses the RAM 803 as a work memory to execute an OS (operating system) and various programs stored in the ROM 802, a HDD (hard disk drive) 813, etc. The CPU 801 also controls each component via a system bus 808. Note that, in the processing of the flowcharts described below, program codes stored in the ROM 802, the HDD 813, etc. are loaded into the RAM 803 and executed by the CPU 801. A display device 815 is connected to the VC 804. An input device 810 such as a mouse and a keyboard, and an imaging device 811 are connected to the general-purpose I / F 805 via a serial bus 809. A general-purpose drive 814 that reads from and writes to an HDD 813 and various recording media is connected to the SATA I / F 806 via a serial bus 812. The NIC 807 inputs and outputs information to and from external devices. The CPU 801 uses the HDD 813 and various recording media mounted on the general-purpose drive 814 as storage locations for various data. The CPU 801 displays a UI (user interface) provided by a program on a display device 815, and receives inputs such as user instructions received via the input device 810. The display device 815 may be a touch panel display having a touch panel function that detects the position of a touch made by a pointer such as a finger.

[0013] <Functional configuration of information processing device> Fig. 2 is a block diagram showing the functional configuration of the information processing device 1. The CPU 801 uses the RAM 803 as a work memory and reads and executes a program stored in the ROM 802 or the HDD 813, thereby functioning as the functional configuration shown in Fig. 2. Note that it is not necessary for all of the processes described below to be executed by the CPU 801, and the information processing device 1 may be configured so that part or all of the processes are executed by one or more processing circuits other than the CPU 801.

[0014] The information processing device 1 includes a texture data acquisition unit 201, an encoding unit 202, and an output unit 203. The texture data acquisition unit 201 acquires texture data representing the texture of an object from a storage device such as an HDD 813. The texture data includes diffuse reflection data representing the diffuse reflection intensity for each position on the object, and specular reflection data representing the specular reflection intensity and specular reflection width for each position on the object. Specifically, the texture data is image data that stores texture information for each position on the object in association with pixels. For example, the diffuse reflection data is image data having a diffuse reflection intensity for each pixel. Note that the texture data may be acquired from another device, such as the imaging device 811, rather than from a storage device. Furthermore, the texture data does not necessarily need to include specular reflection data and diffuse reflection data, and may include, for example, only diffuse reflection data. The encoding unit 202 encodes the texture data acquired by the texture data acquisition unit 201. The output unit 203 outputs the encoded texture data to a storage device such as the HDD 813. The output destination of the encoded texture data is not limited to a storage device, and the encoded texture data may be output to, for example, a device that performs rendering based on the texture data. The output unit 203 may also function as a transmission unit that transmits the encoded texture data to another device.

[0015] <Processing performed by the information processing device> The flow of processing executed by the information processing device 1 in this embodiment will be described with reference to the flowchart in Fig. 3. The processing shown in the flowchart in Fig. 3 starts when a user inputs an instruction via the input device 810 and the CPU 801 accepts the input instruction. Hereinafter, each step (process) will be represented by adding an S before the reference number.

[0016] In S301, the texture data acquisition unit 201 acquires texture data representing the texture of an object from the HDD 813. In S302, the encoding unit 202 receives the texture data acquired by the texture data acquisition unit 201 and selects one of the data included in the received texture data. The texture data in this embodiment includes diffuse reflection intensity as diffuse reflection data, and specular reflection intensity and specular reflection width as specular reflection data.

[0017] In S303, the encoding unit 202 determines whether the selected data is diffuse reflection data. If the selected data is diffuse reflection data, the process proceeds to S304. If the selected data is not diffuse reflection data, the process proceeds to S305. In S304, if the selected data is diffuse reflection data, the encoding unit 202 encodes the selected data by dividing it into layers in the bit plane direction because changes in resolution have a greater impact on image quality. In S305, if the selected data is not diffuse reflection data, the encoding unit 202 encodes the selected data by dividing it into layers in the resolution direction because changes in gradation have a greater impact on image quality than resolution.

[0018] Here, hierarchical coding is a method for efficiently encoding data from multiple layers together. For example, JPEG2000 (ISO / IEC15444) can encode multiple pieces of data with different spatial resolutions as a single data set by performing a discrete wavelet transform. The JPEG coding standard (ISO / IEC10918-1, ITU-T.81) also includes a function for progressive display as an extended process. This function is hereinafter referred to as progressive JPEG. Two progressive JPEG methods are specified. One method divides DCT coefficients into multiple bands and encodes each band separately. First, only the lowest frequency component band is encoded, followed by the remaining bands, one by one. The other method first encodes only a certain number of the most significant bits of all DCT coefficients, followed by the subsequent lower-order bits, one by one. In this embodiment, a method of encoding each frequency component band of progressive JPEG is used as hierarchical encoding in the resolution direction, and a method of encoding DCT coefficients bit by bit is used as hierarchical encoding in the bit plane direction.

[0019] In S306, the encoding unit 202 determines whether all of the texture data acquired by the texture data acquisition unit 201 has been encoded. If all of the texture data has been encoded, the process proceeds to S307. If all of the texture data has not been encoded, the process returns to S302, and unselected texture data is selected. In S307, the output unit 203 outputs each of the encoded texture data to the HDD 813 as individual encoded data.

[0020] As described above, when encoding diffuse reflection data, the information processing device of this embodiment performs hierarchical encoding in the bit plane direction to maintain the resolution of the diffuse reflection data. Furthermore, when encoding data other than diffuse reflection data, the information processing device performs hierarchical encoding in the resolution direction because it is not necessary to maintain the resolution as much as that of the diffuse reflection data.

[0021] When the output unit 203 transmits the encoded data via a communication channel, it can transmit each piece of encoded texture data in order, starting with the most important layer. Therefore, even if the bandwidth of the communication channel is narrow, it is possible to perform decoding processing and texture reproduction processing as soon as data of an important layer is received. By selecting an encoding method according to the texture data, it is possible to reduce the amount of texture data while suppressing deterioration of the texture of an object represented using the texture data.

[0022] <Modification> In this embodiment, it is determined whether the texture data to be coded is diffuse reflection data, and the hierarchical coding method is switched based on the determination result, but the process for switching the coding method is not limited to this. For example, consider a case where the texture data includes diffuse reflection intensity, specular reflection intensity, specular reflection width, and object normal as data. In this case, the diffuse reflection intensity and specular reflection intensity may be hierarchically coded in the bit plane direction, and the specular reflection width and normal may be hierarchically coded in the resolution direction.

[0023] Furthermore, in this embodiment, each piece of encoded texture data is output as individual encoded data, but each piece of encoded data may be stored and output in a single file. FIG. 6 shows an example of an encoded data string (file format) stored in a single file. When diffuse reflection intensity, specular reflection intensity, specular reflection width, and normal are acquired and encoded as texture data, information capable of identifying each encoding method is stored in the metadata section of the encoded data string as metadata. Each piece of encoded data is stored in the corresponding storage section. In the example of FIG. 6, the encoding methods are described in the metadata in the order in which the texture data is stored. Note that it is not necessary to store one encoding method for each piece of texture data as in FIG. 6, and metadata may be generated in which texture data is associated with an encoding method.

[0024] [Second embodiment] In the first embodiment, encoding was performed by switching the hierarchical encoding method depending on the type of texture data. In this embodiment, the hierarchical encoding method is switched by analyzing diffuse reflection data from the texture data. Note that the hardware configuration of the information processing device in this embodiment is the same as that in the first embodiment, so a description thereof will be omitted. The following mainly describes the differences between this embodiment and the first embodiment. Note that the same components as in the first embodiment will be described using the same reference numerals.

[0025] <Functional configuration of information processing device> Fig. 4 is a block diagram showing the functional configuration of the information processing device 1. The CPU 801 uses the RAM 803 as a work memory and reads and executes a program stored in the ROM 802 or the HDD 813, thereby functioning as the functional configuration shown in Fig. 4. Note that it is not necessary for all of the processes shown below to be executed by the CPU 801, and the information processing device 1 may be configured so that part or all of the processes are executed by one or more processing circuits other than the CPU 801.

[0026] The information processing device 1 has a texture data acquisition unit 201, an analysis unit 401, an encoding unit 202, and an output unit 203. The analysis unit 401 analyzes the texture data acquired by the texture data acquisition unit 201. Specifically, the analysis unit 401 converts the diffuse reflection intensity represented by the diffuse reflection data into spatial frequency components, and determines whether the diffuse reflection intensity contains a large number of high-frequency components or a large number of low-frequency components. Note that the texture data acquisition unit 201 in this embodiment acquires diffuse reflection data representing the diffuse reflection intensity, specular reflection data representing the specular reflection intensity and specular reflection width, and normal data representing the normal distribution on the object.

[0027] <Processing performed by the information processing device> The flow of processing executed by the information processing device 1 in this embodiment will be described with reference to the flowchart in Fig. 7. The processing shown in the flowchart in Fig. 7 starts when a user inputs an instruction via the input device 810 and the CPU 801 accepts the input instruction.

[0028] In S301, the texture data acquisition unit 201 acquires texture data representing the texture of an object from the HDD 813. In S701, the analysis unit 401 acquires diffuse reflection data from the texture data acquired by the texture data acquisition unit 201. In S702, the analysis unit 401 converts the diffuse reflection intensity represented by the acquired diffuse reflection data into spatial frequency components. For the conversion into spatial frequency components, known methods such as Fourier transform, wavelet transform, and DCT transform can be used.

[0029] In S703, the analysis unit 401 determines whether the diffuse reflection data is image data containing a large number of high-frequency components or image data containing a large number of low-frequency components based on the spatial frequency components of the diffuse reflection intensity. If it is determined that the diffuse reflection data is image data containing a large number of high-frequency components, the process proceeds to S304, and if it is determined that the diffuse reflection data is image data containing a large number of low-frequency components, the process proceeds to S305. The processes of S304 to S307 are the same as those in the first embodiment, and therefore will not be described here.

[0030] As described above, for images containing many high-frequency components, resolution can be maintained by performing hierarchical coding in the bit plane direction. Furthermore, for images containing many low-frequency components, resolution is less important than for images containing many high-frequency components, so hierarchical coding in the resolution direction is performed so that gradation can be maintained. By selecting an encoding method according to the results of analyzing the texture data, it is possible to reduce the amount of texture data while suppressing degradation of the texture of objects represented using the texture data.

[0031] <Modification> In this embodiment, the analysis of diffuse reflection data is performed for the entire image, but it may also be performed for each region of the image. FIG. 5 is a diagram showing an example of an object containing regions of different materials. Region 501 corresponds to a fluffy material like the fur of a stuffed toy. Region 502 corresponds to a material with a smooth plastic surface. When compressing texture data of an object containing regions of different materials, as in the example of FIG. 5, performing the same hierarchical coding on the entire image may result in degradation of image quality. Therefore, the image may be divided into multiple regions, and an analysis may be performed on each region before determining the coding method. Alternatively, after conversion to spatial frequency components, the regions may be divided after analyzing the spatial frequency components of each region, and a coding method may be determined for each divided region.

[0032] In this embodiment, the encoding method for all of the texture data is determined based on the analysis results of the diffuse reflection data. However, the encoding method for other texture data may be determined based on the analysis results of any one of the data included in the texture data. For example, the analysis unit 401 analyzes the specular reflection intensity, and the encoding unit 202 switches the encoding method for the specular reflection width based on the analysis results of the specular reflection intensity. Specifically, since the specular reflection width tends to be small when the specular reflection intensity value is large, the encoding unit 202 performs hierarchical encoding in the bit plane direction, taking into account the significant impact that a decrease in resolution has on image quality. Since the specular reflection width tends to be large when the specular reflection intensity value is small, the encoding unit 202 performs hierarchical encoding in the resolution direction, taking into account the significant impact that a decrease in gradation has on image quality.

[0033] Furthermore, the encoding unit 202 may determine a method for encoding normal information based on the analysis result of the specular reflection width. Specifically, when the value of the specular reflection width is small, detailed information is required as normal information, and the encoding unit 202 performs hierarchical encoding in the bit plane direction to maintain resolution. When the value of the specular reflection width is large, detailed information is not required as normal information, and the encoding unit 202 performs hierarchical encoding in the resolution direction.

[0034] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0035] 1. Information processing equipment 201 Texture data acquisition unit 203 Output section

Claims

1. an acquisition means for acquiring texture data representing the texture of an object surface; a selection means for selecting, based on the texture data, either an encoding method in which a hierarchy is established in bit planes and encoding is performed, or an encoding method in which a hierarchy is established in resolution and encoding is performed; an output means for outputting the texture data encoded by the selected encoding method; An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein the texture data is at least one of data representing a diffuse reflection intensity on the object, data representing a specular reflection intensity on the object, data representing a spread of specular reflected light on the object, and data representing a normal to the surface of the object.

3. further comprising an encoding means for encoding the texture data based on the selected encoding method, 3. The information processing apparatus according to claim 1, wherein the output means outputs the texture data encoded by the encoding means.

4. 2. The information processing apparatus according to claim 1, wherein the selection means analyzes the texture data and selects an encoding method for encoding the texture data based on the analysis result.

5. 5. The information processing apparatus according to claim 1, wherein the selection means, when the material appearance data is data relating to diffuse reflection intensity, selects an encoding method in which a hierarchy is established in the bit planes and encoding is performed.

6. 5. The information processing apparatus according to claim 1, wherein the selection means, when the material appearance data is data relating to specular reflection intensity, selects an encoding method in which a hierarchy is established in the bit planes and encoding is performed.

7. 5. The information processing device according to claim 1, wherein the selection means, when the texture data is data relating to the spread of specularly reflected light, selects an encoding method in which a hierarchy is set for the resolution and encoding is performed.

8. 5. The information processing apparatus according to claim 1, wherein the selection means, when the material appearance data is data relating to normals, selects an encoding method in which encoding is performed by providing a hierarchy for the resolution.

9. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 8.

10. an acquisition step of acquiring texture data representing the texture of an object surface; a selection step of selecting, based on the texture data, either an encoding method in which a hierarchy is established in bit planes and encoding is performed, or an encoding method in which a hierarchy is established in resolution and encoding is performed; an output step of outputting the texture data encoded by the selected encoding method; An information processing method comprising:

Images