Information processing device, information processing system, information processing method, and program

Abstract

To reduce the amount of illumination information when performing texture representation using the illumination information, while preventing deterioration of texture representation.SOLUTION: An information processing apparatus has: acquisition means that acquires reflection characteristic information to be rendered; and determination means that determines the level of resolution of illumination information used in performing rendering based on mirror reflection information in the reflection characteristic information. As a mirror reflection width representing spreading of a mirror reflection component becomes larger, the determination means decreases the level of resolution.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a technology for performing processing using information on the reflection characteristics of an object. [Background technology]

[0002] In recent years, improvements in CG technology have made it possible to express the texture of objects. Texture is the impression given by the unevenness and gloss of a surface, which depends on the material, and is perceived from the reflection of the light irradiated on the subject. Therefore, when trying to express realistic textures using CG, not only material information but also lighting information is important. A commonly used technique is a rendering method that uses an environment map as omnidirectional lighting information. The environment map uses images or videos captured with an omnidirectional camera or fisheye lens. Patent Document 1 discloses a method that determines the environment map area to be used based on the field of view specified during rendering, and then increases the resolution of only that area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-98921 Summary of the Invention [Problem to be solved by the invention]

[0004] Omnidirectional lighting information such as an environment map requires a large amount of information, which poses problems in terms of memory loading, storage, and communication, so there is a need to reduce the amount of information.However, reducing the amount of lighting information poses the problem of degrading the texture representation.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to reduce the amount of lighting information when texture expression is performed using lighting information, while suppressing deterioration of texture expression. [Means for solving the problem]

[0006] The information processing device according to the present invention includes an acquisition means for acquiring reflection characteristic information of a rendering target, and a determination means for determining a resolution level of illumination information to be used when performing rendering based on specular reflection information of the reflection characteristic information. The specular reflection information is a specular reflection width map that stores, in a map format, a specular reflection width that represents the spread of a specular reflection component, and the determining means determines the resolution level based on the proportion of an area in which the specular reflection width is equal to or less than a threshold value to the entire area of ​​the specular reflection width map. It is characterized by: [Effects of the Invention]

[0007] According to the present invention, when texture representation is performed using lighting information, it is possible to reduce the amount of lighting information while suppressing deterioration of the texture representation. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is an explanatory diagram of the reflection characteristics of an object. [Figure 2] FIG. 10 is a diagram illustrating the influence of specular reflection components on appearance. [Figure 3] 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to a first embodiment. [Figure 4] 1 is a diagram illustrating an example of a functional configuration of an information processing device according to a first embodiment. [Figure 5] 4 is a flowchart showing the processing of the information processing device according to the first embodiment. [Figure 6] FIG. 10 is an explanatory diagram of specular reflection width information. [Figure 7] FIG. 10 is an explanatory diagram of a method for determining the resolution level of an environment map. [Figure 8] FIG. 1 is an explanatory diagram of the effect according to the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of the hardware configuration of an information processing system according to a second embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the functional configuration of an information processing system according to a second embodiment. [Figure 11] FIG. 10 is a sequence diagram showing processing of an information processing system according to a second embodiment. [Figure 12A] 10 is a flowchart showing the processing of the server device of the second embodiment. [Figure 12B]10 is a flowchart showing the processing of a client device according to the second embodiment. [Figure 13] FIG. 10 is an explanatory diagram of line-of-sight information. [Figure 14] FIG. 2 is an explanatory diagram of an environment map. [Figure 15] FIG. 10 is an explanatory diagram of a method for converting JPEG compression parameters. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention. Note that the same components will be described with the same reference numerals.

[0010] <Embodiment 1> (Reflection characteristics of objects) Before going into more detail, let us use Figure 1 to explain the reflection characteristics of an object. Figure 1 shows the intensity 1104 of reflected light when light is irradiated from a light source 1101 toward a point 1102 on the object surface with a normal direction 1103. Light reflected from the object surface can be separated into a diffuse reflection component 1105 and a specular reflection component 1106. The diffuse reflection component 1105 is a component that occurs when incident light is diffusely reflected within the measurement surface and is usually observed with uniform intensity in all directions. On the other hand, the specular reflection component 1106 is a component related to the glossiness that occurs when incident light is specularly reflected on the surface of the measurement surface and is observed with an intensity biased in the direction opposite to the illumination direction (specular reflection direction) based on the normal direction 1103. The component with the highest intensity (maximum intensity) of the specular reflection component 1106 is called the specular reflection intensity 1107. The component representing the spread (dispersion) of the specular reflection component 1106 is called the specular reflection width 1108. On the surface of an object made of a material with low gloss, the specular reflection intensity 1107 is small and the specular reflection width 1108 is large.

[0011] (Effect on specular reflection component appearance) Next, we will use Figure 2 to explain the effect of specular reflection components on the appearance. Figure 2(a) shows an example of an environment map with high resolution for an object with reflection characteristics that have a small specular reflection width. Figure 2(b) shows an example of an environment map with low resolution for an object with reflection characteristics that have a small specular reflection width. Figure 2(c) shows an example of an environment map with high resolution for an object with reflection characteristics that have a large specular reflection width. Figure 2(d) shows an example of an environment map with low resolution for an object with reflection characteristics that have a large specular reflection width.

[0012] For a subject with reflective characteristics that have a small specular reflection width, the observed appearance of the object reflected by the environment map is that of a blurred environment map, as shown in Figure 2(a). Therefore, as shown in Figure 2(b), if the resolution of the environment map is reduced, the observed appearance of the object reflected by the environment map is that of a blurred environment map, resulting in a difference in appearance. Therefore, if the resolution of the environment map is reduced, the texture of the object will not be accurately expressed.

[0013] On the other hand, for a subject with reflection characteristics that have a large specular reflection width, the environment map appears blurred when observed reflected from the subject, as shown in FIG. 2(c). Therefore, as shown in FIG. 2(d), even if the resolution of the environment map is reduced, there is little difference between the appearance observed when reflected from the subject. In other words, a low-resolution environment map is sufficient for texture representation. Therefore, the information processing device according to this embodiment reduces the resolution level of the environment map used during rendering when the specular reflection width of the subject to be rendered is large. This makes it possible to reduce the amount of data in the environment map while suppressing degradation of texture representation. Here, the resolution level is information indicating how much the resolution of the environment map should be increased or decreased. In other words, it is information for specifying resolution conversion for the environment map.

[0014] (Configuration of information processing device according to embodiment 1) FIG. 3 shows an example of the hardware configuration of an information processing device according to this embodiment. The information processing device 100 includes a CPU 101, a main memory 102, a storage device 103, an input device 104, and a display device 105. These components are interconnected via a bus 106. The CPU 101 controls the entire information processing device 100. The CPU 101 executes programs stored in the main memory 102, etc., to realize processing shown in the flowcharts described below. The main memory 102 stores various programs, data, etc. required for processing. The storage device 103 is a hard disk, and stores various programs, data, etc. The input device 104 is a keyboard or mouse, and is used to input user operations. The display device 105 is composed of a CRT, LCD screen, etc., and displays images and numerical data generated by the CPU 101 under the control of the CPU 101.

[0015] FIG. 4 shows an example of the functional configuration of the information processing device according to this embodiment. The information processing device 100 functions as a reflection characteristic information acquisition unit 201 and a resolution level determination unit 202 by the CPU 101 executing a program stored in the main memory 102 or the like. The storage device 103 also stores a conversion table storage unit 203 that stores a conversion table for converting reflection characteristic information into the resolution level of an environment map. The reflection characteristic information acquisition unit 201 acquires reflection characteristic information of a subject to be rendered. In this embodiment, specular reflection width information is acquired from the reflection characteristic information. The specular reflection width information is an example of specular reflection information. The resolution level determination unit 202 determines the resolution level of the environment map from the reflection characteristic information acquired by the reflection characteristic information acquisition unit 201. In this embodiment, a conversion table is read from the conversion table storage unit 203, and the resolution level of the environment map is determined from the specular reflection width information using the conversion table. The determined resolution level of the environment map is provided to another system that performs rendering. The environment map is an example of lighting information used for rendering.

[0016] (Description of Processing According to First Embodiment) Next, the main processing of the information processing device according to this embodiment will be described using the flowchart in Fig. 5. The processing of this flowchart is realized by the CPU 101 executing a program stored in the main memory 102 or the like. In the following description of the flowchart, each process (step) is denoted by adding an S to the beginning, and the process (step) will not be described in detail.

[0017] First, in S11, the reflection property information acquisition unit 201 reads the reflection property information stored in the storage device 103 into the main memory 102. In this embodiment, the reflection property information is acquired from the storage device 103, but it may also be acquired from an external device or the like. In this embodiment, specular reflection width information is read in map format (hereinafter referred to as specular reflection width map) as reflection characteristic information and used in the following steps. In this embodiment, the specular reflection width map is assumed to have a size of 128 x 128 pixels and a resolution of 150 dpi. The specular reflection width map is in an 8-bit gray image format, and stores the specular reflection width at each pixel position in association with each pixel. Fig. 6(a) shows an example of a specular reflection width map. In the example of Fig. 6(a), area 301 indicates an area with a large specular reflection width, and area 302 indicates an area with a small specular reflection width. The stored specular reflection width is calculated by averaging the variance σ when the specular reflection value is Gaussian-approximated. 2 FIG. 6(b) shows an example of the distribution of specular reflection values. As shown in FIG. 6(b), the specular reflection width in the standard case (width 304 in the figure) is smaller than the variance σ 2 The value of becomes smaller, and conversely, in the case of broad specular reflection width (width 305 in the figure), the dispersion σ 2 The value of becomes larger.

[0018] In S12, the resolution level determination unit 202 determines the resolution level of the environment map. The detailed processing of this step will be described with reference to FIG. In S121, the resolution level determination unit 202 calculates the frequency of the specular reflection width. Specifically, for each range of the specular reflection width, the number of pixels within that range is counted. FIG. 7(a) shows the frequency of the specular reflection width in the specular reflection width map shown in FIG. 6(a). A pre-specified number of bins and bin width are used to calculate the frequency of the specular reflection width. In the example of FIG. 7(a), for the specular reflection width expressed as a value from 0 to 255, the frequency is calculated using a bin count of 8 and a bin width of 32, and the frequency is normalized so that the total value of the frequency is 100%.

[0019] Next, in S122, the resolution level determination unit 202 calculates the proportion of small specular reflection widths from the frequency of the specular reflection widths calculated in S121. The following describes this using FIG. 7A as an example. The proportion of small specular reflection widths is calculated by the total frequency of the specular reflection widths from the bin containing the minimum value of the specular reflection width to the bin containing a pre-specified threshold value 401 for determining the proportion of small specular reflection widths. In other words, the proportion of pixels whose specular reflection widths are equal to or smaller than the threshold value to all pixels constituting the specular reflection width map is calculated. The proportion of small specular reflection widths corresponds to the proportion of the area whose specular reflection width is equal to or smaller than the threshold value to the entire area of ​​the specular reflection width map. In the example of FIG. 7A, the proportion of small specular reflection widths is the total frequency within the range of width 402. In the example of FIG. 7A, 25% of the specular reflection width is specified as the threshold value, and the proportion of small specular reflection widths is 20%.

[0020] Next, in S123, the resolution level determination unit 202 reads a conversion table stored in advance in the conversion table storage unit 203. FIG. 7(b) shows an example of a graph of the conversion table. In this embodiment, the conversion table is a table for converting the proportion of small specular reflection width into the resolution level of the environment map. The conversion table tends to increase the resolution of the environment map as the proportion of small specular reflection width increases, and conversely, decrease the resolution of the environment map as the proportion of small specular reflection width decreases. Note that the resolution level of the environment map is expressed as 1-bit data from 0 to 7, but the bit depth and format are not limited to this as long as they are parameters for specifying resolution conversion. Finally, in S124, the resolution level determination unit 202 uses the conversion table acquired in S123 to convert the proportion of small specular reflection width calculated in S122 into the resolution level of the environment map. By using the conversion table, the resolution level determination unit 202 increases the resolution level when the proportion of small specular reflection width is large, and decreases the resolution level when the proportion of small specular reflection width is small. Thereafter, the processing of S12 ends. When the process of S12 is completed, the CPU 101 outputs the determined resolution level of the environment map to another system that performs rendering, and then the series of steps in the flowchart shown in FIG.

[0021] In this embodiment, specular reflection width information is used as specular reflection information, but as described in the description of reflection characteristics, specular reflection intensity information may also be used. Furthermore, data storing a scalar value of specular reflection width, a scalar value of specular reflection intensity, a specular map, a gloss map, a roughness map, or physically based parameters of specular reflection components such as the Cook-Torrance model may also be used. Furthermore, the data size, resolution, bit depth, and format of the specular reflection width map are not limited to those described above.

[0022] In addition, in S12, the resolution level determination unit 202 determined the resolution level of the environment map based on the proportion of small specular reflection widths, but this is not limited to the process as long as the resolution is increased when the specular reflection width is small and decreased when the specular reflection width is large. For example, the process may determine the minimum value of the specular reflection width from the specular reflection width map and determine the resolution level based on this minimum value. Specular reflection intensity information may also be used. In this case, the resolution level determination unit 202 reads the specular reflection intensity information in map format (hereinafter referred to as the specular reflection intensity map). The specular reflection intensity map stores the specular reflection intensity at each pixel position in association with each pixel. In the example of FIG. 6(b), the stored specular reflection intensity is the peak value of the specular reflection value. The resolution level determination unit 202 may obtain the maximum value of the specular reflection intensity from the specular reflection intensity map and determine the resolution level based on this maximum value. Furthermore, the resolution may be increased when the specular reflection intensity is high and decreased when the specular reflection intensity is low. Alternatively, the resolution level determination unit 202 may determine whether or not to perform resolution conversion based on the proportion of the small specular reflection width. Specifically, the resolution level determination unit 202 allows resolution reduction when the proportion of the small specular reflection width is small, and does not allow resolution reduction when the proportion of the small specular reflection width is large. In this case, the CPU 101 outputs information indicating whether or not to perform resolution conversion of the environment map to another system that performs rendering.

[0023] (Explanation of Effects According to Embodiment 1) The effects of this embodiment will be described with reference to FIG. The upper part of Fig. 8 shows a rendering result 502 when the environment map is high resolution and a rendering result 503 when the environment map is low resolution for a specular reflection width map 501 that has areas with small specular reflection width. The lower part of Fig. 8 shows a rendering result 505 when the environment map is high resolution and a rendering result 506 when the environment map is low resolution for a specular reflection width map 504 that has an overall large specular reflection width. Rendering result 502 and rendering result 506 were obtained by applying this embodiment.

[0024] In the rendering result 502, the reflection of the environment map is not blurred in the region 502a where the specular reflection width is small, and the texture is accurately reproduced. On the other hand, in the rendering result 503, the reflection of the environment map is blurred in the region 503a where the specular reflection width is small, resulting in degradation of the texture reproduction. The information processing device 100 of this embodiment determines that a high-resolution environment map is required for the specular reflection width map 501, which has a region where the specular reflection width is small. Therefore, as shown in the rendering result 502, there is no degradation in the texture reproduction. In the rendering result 505, the reflection of the environment map is reproduced blurred in the region 505a where the specular reflection width is large, and it can be said that the texture reproduction is correct. On the other hand, in the rendering result 506, the reflection of the environment map is also reproduced blurred in the region 506a where the specular reflection width is large, and there is little difference between the rendering result 505 and the rendering result 506. In this embodiment, the information processing device 100 determines that a low-resolution environment map is sufficient for the specular reflection width map 504, which has an overall large specular reflection width. Therefore, as shown in the rendering result 506, it is possible to reduce the data volume of the environment map while suppressing deterioration in the texture reproduction.

[0025] As described above, according to this embodiment, when rendering is performed, it is possible to reduce the amount of data in the environment map while suppressing deterioration in texture expression.

[0026] <Embodiment 2> In the first embodiment, an example in which the present invention is applied to a standalone information processing device has been described. In this embodiment, an example in which the present invention is applied to an information processing system including a server device that performs rendering and a client device that acquires an environment map and displays the rendering results will be described. The following will mainly describe the parts that are different from the first embodiment.

[0027] (Configuration of information processing system according to embodiment 2) 9 shows an example of the hardware configuration of an information processing system according to this embodiment. The information processing system includes a server device 620 and a client device 630, which are connected to each other via a network. The server device 620 includes a CPU 601, a main memory 602, a storage device 603, a GPU 604, and a communication device 605. These components are interconnected via a bus 606. The CPU 601 controls the entire server device 620. The CPU 601 executes programs stored in the main memory 602, etc., thereby realizing the processes on the server device 620 side among the processes shown in sequence diagrams and flowcharts described below. The main memory 602 stores various programs, data, etc. required for the processes. The storage device 603 is a hard disk, and stores various programs, data, etc. The GPU 604 performs arithmetic processing specialized for image processing and rendering processing under the control of the CPU 601. The communication device 605 is a device connected to various networks. The CPU 601 communicates data with external devices, including a client device 630, via the communication device 605.

[0028] The client device 630 includes a CPU 607, a main memory 608, a storage device 609, an input device 610, a display device 611, a photographing device 612, and a communication device 613. These components are interconnected via a bus 614. The CPU 607, the main memory 608, and the storage device 609 can be configured similarly to the CPU 601, the main memory 602, and the storage device 603 of the server device 620 described above, and therefore a description thereof will be omitted. The input device 610 is a keyboard, a mouse, or a touch panel, and is used to input user operations. The display device 611 is configured with a CRT or LCD screen, and displays images and GUIs received from the server device 620 under the control of the CPU 607. The photographing device 612 is a device for acquiring ambient lighting information as an environment map, and is typically a camera with a wide viewing angle, such as an omnidirectional camera or a fisheye lens camera. The communication device 613 is a device for connecting to various networks. The CPU 607 performs data communication with external devices including a server device 620 via a communication device 613 .

[0029] FIG. 10 shows an example of the functional configuration of an information processing system according to this embodiment. The server device 620 has the functions of a line-of-sight information acquisition unit 701, a shape information acquisition unit 702, a reflection characteristic information acquisition unit 704, a resolution level determination unit 706, an illumination information acquisition unit 707, and a rendering unit 708. The server device 620 functions as each of these functional units by the CPU 601 executing a program stored in the main memory 602 or the like. The storage device 603 also stores a shape information storage unit 703 that stores shape information as a database, and a reflection characteristic information storage unit 705 that stores reflection characteristic information as a database. The client device 630 has the functions of a rendering condition instruction unit 709, an environment map acquisition unit 710, a resolution conversion unit 711, and a rendering result display unit 712. The client device 630 functions as each of these functional units by the CPU 607 executing a program stored in the main memory 608 or the like.

[0030] The line-of-sight information acquisition unit 701 acquires line-of-sight information from the rendering condition instruction information received from the client device 630 . The shape information acquisition unit 702 acquires shape information from the shape information storage unit 703 based on the shape information instruction information included in the rendering condition instruction information received from the client device 630 . The shape information storage unit 703 reads the shape information from the database in response to an instruction from the shape information acquisition unit 702 . The reflection property information acquisition unit 704 acquires reflection property information from the reflection property information storage unit 705 based on the reflection property information specification information included in the rendering condition instruction information received from the client device 630 . The reflection characteristic information storage unit 705 reads the reflection characteristic information from the database in response to an instruction from the reflection characteristic information acquisition unit 704 . The resolution level determination unit 706 determines the resolution level of the environment map based on the acquired reflection characteristic information. Note that the method for determining the resolution level of the environment map is the same as in the first embodiment, and therefore a description thereof will be omitted. The resolution level determination unit 706 transmits the determined resolution level to the client device 630. The client device 630 inputs the resolution level received from the server device 620 to the resolution conversion unit 711. The lighting information acquisition unit 707 acquires the environment map after resolution conversion received from the client device 630 as lighting information. The rendering unit 708 performs rendering processing based on the acquired line-of-sight information, shape information, reflection characteristic information, and lighting information, and transmits the rendering results to the client device 630. The client device 630 inputs the rendering results received from the server device 620 to the rendering result display unit 712.

[0031] The rendering condition instruction unit 709 transmits rendering condition instruction information, which is generated using application software that allows a user to specify rendering conditions and display rendering results, to the server device 620. The rendering condition instruction information includes rendering conditions related to line-of-sight information, shape information, and reflection characteristic information. The server device 620 inputs the rendering condition instruction information received from the client device 630 to the line-of-sight information acquisition unit 701, shape information acquisition unit 702, and reflection characteristic information acquisition unit 704, respectively. The environment map acquisition unit 710 acquires the environment map to be used for rendering from the image capture device 612 . The resolution conversion unit 711 converts the resolution of the environment map acquired by the environment map acquisition unit 710 based on the resolution level received from the server device 620, and transmits the environment map after the resolution conversion to the server device 620. The server device 620 inputs the environment map received from the client device 630 to the illumination information acquisition unit 707. The rendering result display unit 712 displays the rendering result received from the server device 620 on the display device 611 .

[0032] (Description of Processing According to Second Embodiment) Fig. 11 is a sequence diagram showing the processing of the information processing system according to this embodiment. Note that Fig. 11 explains an overview of the processing, and the details of the processing of each step executed by the server device 620 and the client device 630 will be described later using flowcharts. First, in S201, the client device 630 transmits a rendering request to the server device 620. Next, in S202, the server device 620 transmits rendering order acceptance information to the client device 630. Next, in S203, the client device 630 transmits rendering condition instruction information to the server device 620. Next, in S204, the server device 620 reads the shape information stored in the shape information holding unit 703 based on the shape information instruction information included in the rendering condition instruction information received in S203. Next, in S205, the server device 620 reads the reflection characteristic information stored in the reflection characteristic information storage unit 705 based on the reflection characteristic information instruction information included in the rendering condition instruction information received in S203. Next, in S206, the server device 620 determines the resolution level of the environment map based on the specular reflection width map of the reflection characteristic information read in S205. During the execution of steps S204 to S206, the client device 630 acquires an environment map in step S207. The environment map is acquired by the image capturing device 612 that is part of the client device 630. Note that the client device 630 may also be configured to acquire the environment map from an external camera. Next, in S208, the server device 620 notifies the client device 630 of the resolution level determined in S206. Next, in S209, the client device 630 performs resolution conversion of the environment map acquired in S207 based on the resolution level received from the server device 620. Next, in S210, the client device 630 transmits the environment map whose resolution has been converted in S209 to the server device 620. Next, in S211, the server device 620 performs rendering based on the line of sight information included in the rendering condition instruction information received in S203, the shape information read in S204, the reflection characteristic information read in S205, and the environment map received in S210. Next, in S212, the server device 620 transmits the rendering result of S211 to the client device 630. Finally, in S213, the client device 630 displays the rendering results received from the server device 620 on the display device 611, which is a part of the client device 630. In this manner, the series of processes shown in the sequence diagram of FIG. 11 is completed.

[0033] 12A is a flowchart showing details of processing by server device 620 according to this embodiment. The processing of this flowchart is realized by CPU 601 executing a program stored in main memory 602 or the like. CPU 601 also transmits and receives data to and from client device 630 using communication device 605. In S301, the CPU 601 receives a rendering request signal from the client device 630. In S302, the CPU 601 determines whether it is possible to accept a rendering order. In this embodiment, the criterion used for this determination is whether a predetermined number of simultaneous renderings has been exceeded. If the CPU 601 determines that it is possible to accept a rendering order, the processing from S304 onwards is executed, and if it determines that it is not possible to accept a rendering order, the processing from S303 onwards is executed. Note that other criteria may also be used as the determination criterion. In S303, the CPU 601 transmits a signal indicating that the rendering order result is unsuccessful to the client device 630. After that, the series of processes in the flowchart ends. In S304, the CPU 601 transmits a signal indicating that the rendering order has been accepted to the client device 630.

[0034] In S305, the CPU 601 receives rendering condition instruction information from the client device 630. The line-of-sight information included in the rendering condition instruction information is information indicating the position and distance when observing the subject to be rendered in rendering. FIG. 13 is a diagram illustrating line-of-sight information. In this embodiment, a line-of-sight vector 801 is used as the line-of-sight information, as shown in FIG. 13. In the example of FIG. 13, the line-of-sight vector 801 is expressed as V(θv, φv). The shape information specified in the rendering condition instruction information is information for specifying the three-dimensional shape of the subject. The shape information specified in this embodiment is assumed to be a plate-like shape with a uniform normal vector N. The reflection characteristic information specified in the rendering condition instruction information is information for specifying the diffuse reflection characteristic and the specular reflection characteristic. The reflection characteristic information is assumed to include a set of three pieces of information: a diffuse reflection light map, a normal map, and a specular reflection width map. Furthermore, the reflection characteristic information specified in this embodiment includes a diffuse reflection light map with uniform diffuse reflection light, a normal map with uniform normal vector N, and a specular reflection width map with a portion having an area with small specular reflection width. This specular reflection width map is the same as the specular reflection width map shown in Figure 6(a) of the first embodiment.

[0035] In S306, the CPU 601 reads the shape information stored in the shape information holding unit 703 based on the shape information instruction information included in the rendering condition instruction information received in S305. In S307, the CPU 601 reads the reflection characteristic information stored in the reflection characteristic information storage unit 705 based on the reflection characteristic information instruction information included in the rendering condition instruction information received in S305. In S308, the CPU 601 determines the resolution level of the environment map based on the specular reflection width map included in the reflection characteristic information read in S307. In this embodiment, the resolution level is expressed by 1-bit data of 0-7, as in the first embodiment. In S309, the CPU 601 notifies the client device 630 of the resolution level determined in S308. In S310, the CPU 601 receives an environment map from the client device 630. In this embodiment, the environment map is assumed to be of a latitude-longitude type (also called a cylindrical type) in which the polar angle and azimuth angle at the time of measurement are taken as axes, and what is received is assumed to be a single RGB image format.

[0036] Next, details of the environment map will be explained using FIG. 14. FIG. 14(a) shows an example of the configuration of the measurement device when acquiring the environment map. FIG. 14(b) shows an example of the created environment map. In the measurement device shown in FIG. 14(a), the environment map can be acquired by an omnidirectional camera 901 (corresponding to the image capture device 612) placed at an arbitrary point X. Specifically, the environment map is created by measuring the light intensity in the line of sight (θi, φi) from the arbitrary point X and storing the measured intensity for each line of sight (θi, φi). In FIG. 14(b), (θ, φ) are set with point X on the object at the position of the omnidirectional camera 901 as the origin, θ indicates the polar angle relative to the surface normal N, and φ indicates the azimuth angle relative to the reference direction. Furthermore, i indicates the measurement number when measuring for each Δθ and Δφ, and i in the environment map corresponds to ascending order, with the upper left pixel being 0.

[0037] In S311, the CPU 601 performs rendering processing based on the line-of-sight information, shape information, reflection property information, and environment map acquired in each step described above. Details of the rendering processing will not be explained here. The rendering result represents how an object having specified shape information and reflection property information appears when illuminated by the environment map and observed from the direction indicated by the specified line-of-sight information. The rendering result is assumed to be in RGB image format. In S312, the CPU 601 transmits the rendering result obtained in S311 to the client device 630. In this way, the series of processes in the flowchart shown in FIG. 12A is completed.

[0038] In S305, the line-of-sight information is expressed as a line-of-sight vector V (θv, φv), but is not limited to this. Any information that can define the line-of-sight information used in the rendering process may be used, and the format is not limited. Line-of-sight distance information and field of view information may also be included. In addition, in S305, the reflection characteristic information includes a set of three pieces of information: a diffuse reflection map, a normal map, and a specular reflection width map. However, the information is not limited to this. There are no limitations on the type or format as long as it includes information that can define the reflection characteristic information used in the rendering process. In addition, in S310, the environment map is described as being of the latitude-longitude type, with the polar angle and azimuth angle at the time of measurement taken as axes, but it is not limited to this as long as it stores three-dimensional lighting information. Specifically, it is sufficient for the environment map to be an environment map in which lighting information in all directions is converted into a two-dimensional image format, as typified by latitude-longitude, cube, and sphere types. Information specifying the arrangement and direction of lighting in three-dimensional space can also be applied, as it can be converted into the above-mentioned environment map by baking processing, which is commonly used in rendering technology. Furthermore, although the environment map received in S310 and the rendering results created in S311 are described as information in RGB image format, this is not limiting. For example, if the environment map is in gray image format, the rendering results contain almost no RGB information, so they may be in either RGB or gray image format. Furthermore, if the environment map is omnidirectional video information, in S311 the CPU 601 may perform rendering processing for each video frame of the environment map and create the rendering results in video format.

[0039] 12B is a flowchart showing details of processing by client device 630 according to this embodiment. The processing of this flowchart is realized by CPU 607 executing a program stored in main memory 608 or the like. CPU 607 also transmits and receives data to and from server device 620 using communication device 613. In S401, the CPU 607 transmits a rendering request signal to the server device 620. In S402 , the CPU 607 receives the rendering order acceptance result from the server device 620 . In S403, the CPU 607 determines the rendering order result received in S402. If the CPU 607 determines that the rendering order result is acceptable, the processing from S404 onward is executed, and if the rendering order result is unacceptable, the processing of the series of flowcharts ends. In S404, the CPU 607 transmits rendering condition instruction information specified by the user to the server device 620. The user specifies the rendering conditions using a GUI provided by application software that displays the rendering results. Specifically, for shape information and reflection characteristic information, a list of information stored in advance in the shape information storage unit 703 and reflection characteristic information storage unit 705 in the server device 620 is presented to the user, and the user selects and specifies from the list. For line-of-sight information, the user specifies the observation direction of the subject by specifying on the touch panel.

[0040] In S405, the CPU 607 acquires an environment map using the image capturing device 612. The method for acquiring the environment map is as described above in S310. In S406 , the CPU 607 receives the resolution level of the environment map from the server device 620 . In S407, the CPU 607 performs a process of converting the resolution of the environment map acquired in S405 based on the resolution level received in S406. In this embodiment, the resolution conversion process uses the JPEG compression method. Specifically, a pre-stored conversion table is used to convert the resolution level, which is represented by one bit from 0 to 7, into JPEG compression parameters. FIG. 15 shows an example of a graph of the conversion table. In this embodiment, the conversion table converts the resolution level into JPEG compression parameters. The conversion table tends to lower the compression rate for higher resolution levels and, conversely, to higher compression rate for lower resolution levels. The CPU 607 performs JPEG compression using the converted JPEG compression parameters. Details of the JPEG compression method are not described here, as they are well-known techniques. In S408, the CPU 607 transmits the environment map whose resolution has been converted in S407 to the server device 620. In S409 , the CPU 607 receives the rendering result from the server device 620 . In S410, the CPU 607 displays the rendering results received in S409 on the display device 611 to present them to the user. In this way, the series of processes in the flowchart shown in FIG. 12B is completed.

[0041] In S404, the method of specifying the rendering conditions by the user has been described, but the present invention is not limited to this. As long as the rendering condition instruction information includes information specifying the reflection characteristic information, other rendering conditions may be specified on the server side. The rendering conditions may also be automatically specified by application software. They may also be automatically specified based on output values ​​from the image capture device 612 or gyro sensor of the client device 630. The GUI provided by the application software is also not limited to the above example.

[0042] In addition, in S405, the CPU 607 photographs the environment map using the photographing device 612, and in S407, converts the resolution of the environment map, but this is not limiting. For example, a database of environment maps may be stored in the main memory 608 of the client device 630, and the CPU 607 may retrieve the specified environment map from the database according to its type and resolution. In addition, in S407, the CPU 607 starts resolution conversion of the environment map regardless of the resolution level of the environment map, but it may determine whether or not to perform resolution conversion and start resolution conversion depending on the result of the determination. For example, if the resolution level of the environment map is the highest resolution level, the CPU 607 may not perform resolution conversion and may send the environment map acquired in S405 to the server device 620 as is. In addition, in S407, the resolution conversion of the environment map uses the JPEG compression method, but this is not limited to this. The purpose of the resolution conversion of the environment map is to remove high-frequency components from the environment map and reduce the data volume of the environment map, and other methods may be used as long as they meet the purpose. For example, a process of reducing the image size of the environment map, a process of reducing high-frequency information in the environment map, or a process of performing image compression using resolution conversion may be used. In addition, in this embodiment, the JPEG compression parameters used for resolution conversion are determined on the client side, but they may also be determined on the server side. In this case, in S308, the server device 620 determines JPEG compression parameters based on the resolution level and notifies the client device 630 of the JPEG compression parameters.

[0043] Furthermore, although the present embodiment has been described as being applied to a server-client system, the applicable device configuration is not limited to this. For example, the present invention can be similarly applied to a system in which communication is performed between an image processing device and a camera device. If the camera device that acquires the environment map does not have a calculation unit, the image processing device can simply specify the resolution and image size when acquiring the environment map. The present invention can also be applied to a system in which communication is performed between an image processing device and a database device. When an environment map for each resolution level is stored in the database device, the image processing device can specify the type and resolution level of the environment map and read it from the database device.

[0044] (Explanation of Effects According to Example 2) The effects of this embodiment are similar to those of embodiment 1. Furthermore, by using the JPEG compression method, the data volume of a high-resolution environment map can be reduced to about 40% of that before compression, and the data volume of a low-resolution environment map can be reduced to about 10% of that before compression.

[0045] As described above, according to this embodiment, when rendering is performed, it is possible to reduce the amount of data in the environment map while suppressing deterioration in texture expression.

[0046] Although the present invention has been described above with reference to the embodiments, the above embodiments are merely illustrative of specific examples of how the present invention can be implemented, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0047] As a modification of each of the above-described embodiments, a learning model that learns the relationship between specular reflection width information and the resolution level of the environment map may be generated, and the resolution level of the environment map may be determined using the learning model. In this case, the resolution level determination unit 202 determines the resolution level of the environment map using a machine-learned learning model. For example, the information processing device 100 generates the learning model by performing machine learning using multiple pairs of input data and training data on a learning model based on a machine learning algorithm such as an SVM (support vector machine) algorithm. The input data is a specular reflection width map. The training data is a resolution level that is determined to result in no degradation in the reproducibility of the reflection of the environment map. By performing learning using these data, a learning model that determines the resolution level of the environment map from the specular reflection width map is generated. When using a learning model, the resolution level determination unit 202 determines the resolution level of the environment map using output data obtained by inputting the specular reflection width map into the learning model.

[0048] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0049] 100: Information processing device, 620: Server device, 630: Client device

Claims

1. an acquisition means for acquiring reflection characteristic information of a rendering target; a determination means for determining a resolution level of illumination information to be used when performing rendering based on specular reflection information among the reflection characteristic information; and The specular reflection information is a specular reflection width map that stores a specular reflection width representing the spread of a specular reflection component in a map format, The information processing apparatus is characterized in that the determination means determines the resolution level based on the proportion of an area where the specular reflection width is equal to or less than a threshold to the entire area of ​​the specular reflection width map.

2. 2. The information processing apparatus according to claim 1, wherein said determining means increases the resolution level when the ratio is greater than a predetermined threshold value, and decreases the resolution level when the ratio is smaller than the predetermined threshold value.

3. An acquisition means for acquiring reflection characteristic information of a rendering target; a determination means for determining a resolution level of illumination information to be used when performing rendering based on specular reflection information among the reflection characteristic information; and The specular reflection information is a specular reflection width map that stores a specular reflection width representing the spread of a specular reflection component in a map format, The information processing apparatus is characterized in that the determination means determines the resolution level based on a minimum value of the specular reflection width in the specular reflection width map.

4. An information processing device described in any one of claims 1 to 3, characterized in that the determination means lowers the resolution level when the specular reflection width, which represents the spread of the specular reflection component, is large, and raises the resolution level when the specular reflection width is small.

5. The information processing device according to claim 1 or 3, characterized in that the determination means determines to allow conversion of the resolution of the lighting information when the specular reflection width, which represents the spread of the specular reflection component, is large, and to not allow conversion of the resolution of the lighting information when the specular reflection width is small.

6. 6. The information processing apparatus according to claim 1, wherein the illumination information is an environment map that stores illumination information in all directions in a two-dimensional image format.

7. 7. The information processing apparatus according to claim 6, wherein said determining means determines, as said resolution level, a parameter used in a conversion process for converting the resolution of said environment map.

8. 8. The information processing apparatus according to claim 7, wherein the conversion process is a process of converting the image size of the environment map, a process of reducing high frequency information of the environment map, or a compression process involving resolution conversion of the environment map.

9. 9. The information processing apparatus according to claim 1, wherein the resolution level is information for specifying resolution conversion of the illumination information.

10. an acquisition means for acquiring reflection characteristic information of a rendering target; a determination means for determining a resolution level of illumination information to be used when performing rendering based on output data obtained by inputting specular reflection information from the reflection characteristic information into a learning model; An information processing device comprising:

11. An information processing system including a first device that performs rendering and a second device connected to the first device, The first device is a first acquisition means for acquiring reflection characteristic information of a rendering target; a determination means for determining a resolution level of illumination information to be used when performing rendering based on specular reflection information among the reflection characteristic information; a notification means for notifying the second device of the resolution level; receiving means for receiving the illumination information from the second device; a rendering means for performing rendering using the reflection characteristic information and the illumination information; and The second device is a second acquisition means for acquiring the illumination information; a conversion means for converting the acquired illumination information at the resolution level notified by the first device; a transmitting means for transmitting the converted illumination information to the first device; and The specular reflection information is a specular reflection width map that stores a specular reflection width representing the spread of a specular reflection component in a map format, The information processing system is characterized in that the determination means determines the resolution level based on the proportion of the area where the specular reflection width is equal to or less than a threshold to the entire area of ​​the specular reflection width map.

12. An information processing system including a first device that performs rendering and a second device connected to the first device, The first device is a first acquisition means for acquiring reflection characteristic information of a rendering target; a determination means for determining a resolution level of illumination information to be used when performing rendering based on specular reflection information among the reflection characteristic information; a notification means for notifying the second device of the resolution level; receiving means for receiving the illumination information from the second device; a rendering means for performing rendering using the reflection characteristic information and the illumination information; and The second device is a second acquisition means for acquiring the illumination information; a conversion means for converting the acquired illumination information at the resolution level notified by the first device; a transmitting means for transmitting the converted illumination information to the first device; and The specular reflection information is a specular reflection width map that stores a specular reflection width representing the spread of a specular reflection component in a map format, The information processing system is characterized in that the determination means determines the resolution level based on a minimum value of the specular reflection width in the specular reflection width map.

13. an acquisition step of acquiring reflection characteristic information of a rendering target; a determination step of determining a resolution level of illumination information to be used when performing rendering based on specular reflection information among the reflection characteristic information; Including, The specular reflection information is a specular reflection width map that stores a specular reflection width representing the spread of a specular reflection component in a map format, An information processing method characterized in that in the determination step, the resolution level is determined based on the proportion of the area where the specular reflection width is equal to or less than a threshold value to the entire area of ​​the specular reflection width map.

14. An acquisition step of acquiring reflection characteristic information of a rendering target; a determination step of determining a resolution level of illumination information to be used when performing rendering based on specular reflection information among the reflection characteristic information; Including, The specular reflection information is a specular reflection width map that stores a specular reflection width representing the spread of a specular reflection component in a map format, The information processing method, wherein the determination step determines the resolution level based on a minimum value of the specular reflection width in the specular reflection width map.

15. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 10.

Images