Information processing device, information processing method, and program

Abstract

To provide a processing for easily confirming the texture of an object.SOLUTION: Disclosed is an information processing device which includes: switching means for switching a mode from a first mode for displaying a first image, which is a still image of an object, to a second mode for displaying a second image of the object generated in response to the visual line information; and display controlling means for displaying the second image, in which the virtual lighting is reflected on the object, on the display means in accordance with the switching by the switching means.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a technique for setting lighting conditions for confirming the texture of an object. [Background technology]

[0002] In recent years, with the advancement of printing and CG (computer graphics) technology, it has become possible to reproduce the texture of objects with high precision. Texture refers to the impression given by the surface irregularities and gloss that depend on the material of the object. Since the appearance of an object is perceived based on the reflection of light irradiated on the object, it is important to determine how to set the lighting conditions when expressing texture. Patent Document 1 discloses a technology that achieves optical consistency between real space and virtual space by controlling the virtual light source in the virtual space and the reflection of the surrounding environment on the virtual object according to the position and orientation of the real light source. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-18173 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, when no illumination is reflected on an object, the shadows caused by gloss or unevenness are not expressed, which makes it difficult to confirm the texture of the object.

[0005] The present invention has been made in view of the above-mentioned problems, and has an object to provide a process for easily checking the texture of an object. [Means for solving the problem]

[0006] In order to solve the above problems, an information processing device according to the present invention is provided, and virtual lighting is not reflected on the object. From a first mode displaying a first image, To the object According to gaze information Virtual lighting is reflected a switching means for switching to a second mode for displaying a second image, in response to the switching by the switching means; Switch the displayed image from the first image to the second image. Display means Control and a display control means. [Effects of the Invention]

[0007] According to the present invention, the texture of an object can be easily confirmed. [Brief explanation of the drawings]

[0008] [Figure 1] Block diagram showing the hardware configuration of an information processing device [Figure 2] Block diagram showing the logical configuration of an information processing device [Figure 3] 1 is a flowchart showing a process executed by an information processing device; [Figure 4] A diagram showing the UI for acquiring texture information [Figure 5] A diagram showing the UI for displaying images. [Figure 6] Block diagram showing the logical configuration of an information processing device [Figure 7] 1 is a flowchart showing a process executed by an information processing device; DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all of the combinations of features described in each embodiment are necessarily essential to the solution of the present invention.

[0010] [First embodiment] In this embodiment, when switching from a first viewing mode that displays a still image of an object to a second viewing mode that displays an image of the object generated according to line-of-sight information, virtual lighting is reflected on the object.

[0011] <Hardware configuration of information processing device> FIG. 1 is a block diagram showing the hardware configuration of an information processing device 1. The information processing device 1 includes a CPU 101, a ROM 102, and a RAM 103. The information processing device 1 also includes a VC (video card) 104, a general-purpose I / F (interface) 105, a SATA (serial ATA) I / F 106, and a NIC (network interface card) 107. The CPU 101 uses the RAM 103 as a work memory to execute an OS (operating system) and various programs stored in the ROM 102, a HDD (hard disk drive) 113, etc. The CPU 101 also controls each component via a system bus 108. Note that, in the processing of the flowcharts described below, program codes stored in the ROM 102, the HDD 113, etc. are loaded into the RAM 103 and executed by the CPU 101. A display device 115 is connected to the VC 104. An input device 110, such as a mouse or a keyboard, and an imaging device 111 are connected to the general-purpose I / F 105 via a serial bus 109. A general-purpose drive 114 that reads from and writes to an HDD 113 and various recording media is connected to the SATA I / F 106 via a serial bus 112. The NIC 107 inputs and outputs information to and from external devices. The CPU 101 uses the HDD 113 and various recording media mounted on the general-purpose drive 114 as storage locations for various data. The CPU 101 displays a UI (user interface) provided by a program on a display device 115, and receives inputs such as user instructions received via the input device 110. The display device 115 may be a touch panel display having a touch panel function that detects the position of a touch by a pointer such as a finger.

[0012] <Logical configuration of information processing device> Fig. 2 is a block diagram showing the logical configuration of the information processing device 1. The CPU 101 uses the RAM 103 as a work memory and reads and executes a program stored in the ROM 102 or the HDD 113, thereby functioning as the logical configuration shown in Fig. 2. It is not necessary for all of the processes shown below to be executed by the CPU 101, and the information processing device 1 may be configured so that part or all of the processes are executed by one or more processing circuits other than the CPU 101.

[0013] The information processing device 1 includes a texture acquisition unit 201, a first image generation unit 202, an initial line of sight acquisition unit 203, a setting unit 204, a dynamic line of sight acquisition unit 205, a second image generation unit 206, and a display control unit 207. The texture acquisition unit 201 acquires, as texture information, a diffuse reflection intensity ρ d (x,y), specular reflection intensity ρ s The first image generation unit 202 acquires the diffuse reflection intensity ρ(x, y), the glossiness σ(x, y), and the normal N(x, y). d Based on (x, y), first image data representing a first image I1(x, y) to be displayed in the first viewing mode is generated. The initial line-of-sight acquisition unit 203 acquires line-of-sight information when switching from the first viewing mode to the second viewing mode. The setting unit 204 sets the lighting direction L and lighting intensity E as lighting information based on the line-of-sight information and texture information. The dynamic line-of-sight acquisition unit 205 acquires line-of-sight information that is dynamically changed in the second viewing mode. The second image generation unit 206 uses the line-of-sight information, lighting information, first image data, and texture information to generate second image data representing a second image I2(x, y) to be displayed in the second viewing mode. The display control unit 207 controls the display device 115 to display the first image I1(x, y), the second image I2(x, y), etc. on the display device 115.

[0014] <Processing Executed by Information Processing Device 1> The flow of processing executed by the information processing device 1 in this embodiment will be described with reference to the flowchart in Fig. 3. The processing shown in the flowchart in Fig. 3 starts when a user inputs an instruction via the input device 110 and the CPU 101 accepts the input instruction. Hereinafter, each step (process) will be represented by adding an S before the reference number.

[0015] In S301, the texture acquisition unit 201 acquires the diffuse reflection intensity ρ d (x,y), specular reflection intensity ρ s (x, y), glossiness σ(x, y), and normal N(x, y) are acquired. Figure 4 shows an example of a UI for acquiring texture information. The texture acquisition unit 201 acquires the diffuse reflection intensity ρ d (x,y), specular reflection intensity ρ s The texture acquisition unit 201 sets storage locations for the texture information designated by the user for each of the diffuse reflection intensity ρ(x,y), glossiness σ(x,y), and normal N(x,y). In response to pressing the read button, the texture acquisition unit 201 reads the diffuse reflection intensity ρ d (x,y), specular reflection intensity ρ s (x, y), glossiness σ(x, y), and normal N(x, y) are acquired. In this embodiment, the texture information is stored in the HDD 113, but it may be stored in another device connected to the information processing device 1, or the information processing device 1 may have a storage unit. In this embodiment, the diffuse reflection intensity ρ d (x,y) and specular reflection intensity ρ s(x, y) is read in a color image format in which pixel values ​​(R, G, B) are expressed in 8 bits. Each intensity, expressed in a value ranging from 0 to 1, is associated with a value ranging from 0 to 255. In this embodiment, the glossiness σ(x, y) is read in a grayscale image format in which pixel values ​​are expressed in 8 bits. As the glossiness value increases, the spread of specular reflection narrows, resulting in an appearance of strong gloss. In this embodiment, the normal N(x, y) is read in a color image format in which pixel values ​​(R, G, B) are expressed in 8 bits. The x component of the normal (a value ranging from -1 to 1) is associated with the R component of the pixel value (a value ranging from 0 to 255), and the y component of the normal (a value ranging from -1 to 1) is associated with the G component of the pixel value (a value ranging from 0 to 255). Furthermore, the z component of the normal (a value ranging from -1 to 1) is associated with the B component of the pixel value (a value ranging from 0 to 255).

[0016] In S302, the first image generation unit 202 generates first image data representing a first image I1(x,y) to be displayed in the first viewing mode based on the texture information acquired in S301. The first image I1(x,y) is a still image of an object to be displayed in the first viewing mode, and the diffuse reflection intensity ρ d Generated using (x, y) according to equation (1). I1(x,y)=E d ρ d (x,y) Equation (1) Here, E d is the illumination intensity, and in order to make the first image visible, the maximum value of I1(x, y) is set to be 50% of the maximum pixel value (255), for example.

[0017] In S303, the display control unit 207 displays the first image I1(x,y) represented by the first image data generated in S302 on the display device 115. FIG. 5 shows an example of a UI for displaying an image. The display control unit 207 displays the first image I1(x,y) on the image display unit 501. In S304, the display control unit 207 accepts an instruction to switch from the first viewing mode to the second viewing mode via the UI. The display of the first image I1(x,y) in S303 continues until a switching instruction is received from the user, and when a switching instruction is received from the user, the process proceeds to S305. The switching instruction from the user is input, for example, by pressing the texture viewing mode button 502 on the UI shown in FIG.

[0018] In S305, the initial line-of-sight acquisition unit 203 acquires line-of-sight information when the mode switching instruction is received in S304. The line-of-sight information is a line-of-sight vector V=(Vx, Vy, Vz) that indicates the direction of the line of sight. The line-of-sight vector V is input by, for example, the line-of-sight information input unit 503 in the UI of FIG. 5. Specifically, Vx and Vy of the line-of-sight vector V (in the range of -1 to 1 and Vx 2 +Vy 2 <1) is designated by the pointer 504. The initial line-of-sight acquisition unit 203 derives Vz according to equation (2).

[0019]

number

[0020] In S306, the setting unit 204 sets lighting information based on the line-of-sight vector V, which is the line-of-sight information acquired in S305, and the texture information acquired in S301. The lighting information is a lighting vector L = (Lx, Ly, Lz) that represents the direction of lighting, and lighting intensity E, and is set so that the desired image features can be confirmed by the user. In this embodiment, the number of pixels at which the pixel value saturates is used to determine whether the desired image features can be confirmed. First, the setting unit 204 sets the lighting vector L so that it has a specular relationship with the line-of-sight vector V, based on a normal that represents the average direction of the object. Specifically, the setting unit 204 derives the lighting vector L according to equation (3). L=2dot(N',V)N'-V...Equation (3) Here, N' is the average vector of the normal N(x, y), which is obtained by averaging the x, y, and z components of the normal and then normalizing the length to 1. Also, dot(·,·) represents the dot product operation between vectors.

[0021] Next, the setting unit 204 sets the illumination intensity E. The setting unit 204 sets the illumination vector L, the line-of-sight vector V, the diffuse reflection intensity ρ d (x,y), specular reflection intensity ρ s The illumination intensity E is set according to the number of saturated pixel values ​​in an image generated using (x, y), the normal N(x, y), and the glossiness σ(x, y). First, the setting unit 204 generates an image I(x, y) according to equation (4), using an equation based on Lambert's cosine law corresponding to diffuse reflection and the Blinn-Phong model corresponding to specular reflection. I(x,y)=E(ρ d (x,y)dot'(N(x,y),L)+ρ s (x,y)(dot'(N(x,y),H)) σ(x,y) )...Equation (4) Here, dot'(·,·) represents the dot product operation between vectors, and if the dot product is negative, it is replaced with 0. Furthermore, H is an intermediate vector (half vector) between the illumination vector L and the line of sight vector V, and is derived by H=(L+V) / |L+V|. The setting unit 204 calculates the image I(x, y) while increasing the illumination intensity E in equation (4), and sets the illumination intensity E when the number of pixels whose pixel values ​​are saturated exceeds a threshold. The threshold for the number of pixels whose pixel values ​​are saturated is set in advance to, for example, 1% of the total number of pixels.

[0022] In S307, second image generation unit 206 starts repeating the processes of S308 to S310, and ends the repeating processes in S311 when the user instructs to end the repeating processes. In S308, dynamic line-of-sight acquisition unit 205 acquires line-of-sight vector V=(Vx, Vy, Vz) as line-of-sight information dynamically input in the second viewing mode. Line-of-sight vector V is acquired based on Vx and Vy designated by pointer 504 in line-of-sight information input unit 503 in the UI of Fig. 5 and Vz derived according to equation (2).

[0023] In S309, the second image generation unit 206 generates a second image based on the illumination vector L, the illumination intensity E, the line-of-sight vector V, the first image I1, and the diffuse reflection intensity ρ d , specular reflection intensity ρ s , normal N, and glossiness σ to generate second image data representing a second image I2(x,y). Specifically, the second image generation unit 206 generates the second image I2(x,y) by superimposing an image I'(x,y) generated using texture information, lighting information, and line-of-sight information onto the first image I1(x,y). The second image generation unit 206 superimposes the first image I1(x,y) and the image I'(x,y) in accordance with equations (5) and (6). I2(x,y)=βI1(x,y)+(1-β)I'(x,y)...Equation (5) I'(x,y)=E(ρ d (x,y)dot'(N(x,y),L)+ρ s (x,y)(dot'(N(x,y),H)) σ(x,y) )...Equation (6) Here, β is the synthesis rate of the first image I1(x, y) and the image I'(x, y), and is set to, for example, β=0.5.

[0024] In S310, the display control unit 207 displays the second image I2(x, y) generated in S309 on the display device 115. Specifically, the display control unit 207 displays the second image I2(x, y) on the image display unit 501 in the UI of FIG.

[0025] <Effects of the first embodiment> As described above, the information processing device of this embodiment reflects virtual lighting onto the displayed image when switching from a first viewing mode that displays a still image of an object to a second viewing mode that displays an image of the object generated according to line-of-sight information. This allows the user to easily check the texture of the object, such as its unevenness and gloss, by using the lighting reflected onto the object when switching to the second viewing mode for checking the texture of the object.

[0026] [Second embodiment] In the first embodiment, the first image displayed in the first viewing mode is generated based on texture information, but a still image to be displayed as the first image may be prepared separately. In this embodiment, a pre-generated image is acquired as the first image displayed in the first viewing mode, and the second image displayed in the second viewing mode is generated based on the texture information. Note that the hardware configuration of the information processing device in this embodiment is the same as that in the first embodiment, and therefore a description thereof will be omitted. The following mainly describes the differences between this embodiment and the first embodiment. Note that the same components as those in the first embodiment are described using the same reference numerals.

[0027] <Logical configuration of information processing device 1> Fig. 6 is a block diagram showing the logical configuration of the information processing device 1. The CPU 101 functions as the logical configuration shown in Fig. 6 by using the RAM 103 as a work memory and reading and executing a program stored in the ROM 102 or the HDD 113. Note that it is not necessary for all of the processes shown below to be executed by the CPU 101, and the information processing device 1 may be configured so that part or all of the processes are executed by one or more processing circuits other than the CPU 101.

[0028] The information processing device 1 has a first image acquisition unit 601, a texture acquisition unit 201, an initial gaze acquisition unit 203, a setting unit 204, a dynamic gaze acquisition unit 205, a second image generation unit 602, and a display control unit 207. The first image acquisition unit 601 acquires first image data representing a first image I1(x,y) to be displayed in a first viewing mode. The second image generation unit 602 uses gaze information, lighting information, and texture information to generate second image data representing a second image I2(x,y) to be displayed in a second viewing mode.

[0029] <Processing Executed by Information Processing Device 1> The flow of processing executed by the information processing device 1 in this embodiment will be described with reference to the flowchart in Fig. 7. The processing shown in the flowchart in Fig. 7 starts when a user inputs an instruction via the input device 110 and the CPU 101 accepts the input instruction.

[0030] S301 is the same process as in the first embodiment, and therefore description thereof will be omitted. In S701, the first image acquisition unit 601 acquires first image data representing a first image I1(x,y) from a data storage destination based on an instruction from a user. The first image I1(x,y) is a color image in which pixel values ​​(R,G,B) are expressed in 8 bits. The first image acquisition unit 601 displays a UI similar to the UI in FIG. 4, sets a storage destination for the first image data, and acquires the first image data from the data storage destination in response to pressing a read button.

[0031] In S702, the display control unit 207 displays the first image I1(x,y) represented by the first image data acquired in S701 on the display device 115. The displayed UI is the same as the UI in FIG. 5, and the display control unit 207 displays the first image I1(x,y) on the image display unit 501. Since S304, S305, and S306 are the same processes as in the first embodiment, their explanations will be omitted. In S703, the second image generation unit 602 starts repeated processing of S308, S704, and S705, and ends the repeated processing in S706 when an instruction to end the repeated processing is given by the user. Since S308 is the same process as in the first embodiment, its explanation will be omitted.

[0032] In S704, the second image generation unit 602 calculates the line of sight vector V, the illumination vector L, the illumination intensity E, and the diffuse reflection intensity ρ d , specular reflection intensity ρ s , the normal N, and the glossiness σ to generate second image data representing the second image I2(x,y). Specifically, the second image generation unit 602 generates the second image I2(x,y) according to equation (7). I2(x,y)=E(ρ d (x,y)dot'(N(x,y),L)+ρ s (x,y)(dot'(N(x,y),H)) σ(x,y) )...Equation (7) In S705, the display control unit 207 displays the second image I2(x, y) generated in S704 on the display device 115. Specifically, the display control unit 207 displays the second image I2(x, y) on the image display unit 501 in the UI of FIG.

[0033] <Effects of the second embodiment> As described above, the information processing device in this embodiment acquires an image generated in advance as a first image to be displayed in the first browsing mode, and displays the first image until switching to the second browsing mode. By making it possible to specify the image to be displayed in the first browsing mode, it is possible to set an image that is more suitable for the user to be presented to the user in the first browsing mode.

[0034] [Variations] In the above-described embodiment, the UI and images are displayed using a display device 115 externally connected to the information processing device, but a hardware configuration such as a tablet in which the information processing device and a display unit are integrated may also be used.

[0035] In the above-described embodiment, the initial gaze information when switching to the second viewing mode and the gaze information used when generating the second image are input from the UI. When a hardware configuration such as a tablet is used for the information processing device 1, the gaze information may be acquired using a gyro sensor, an in-camera, or the like.

[0036] In addition, in the first embodiment described above, the illumination intensity when generating the first image was set so that the maximum value of I1(x, y) was 50% of the maximum pixel value (255), but it may also be set to other ratios.

[0037] Furthermore, in the first embodiment described above, the first image is generated by multiplying the diffuse reflection intensity by a constant, but an image obtained by calculation according to the Lambert model using normals as well may also be used.

[0038] In the above-described embodiment, normals representing the orientation of the surfaces are acquired as shape information of the object included in the texture information, but a height map representing the height distribution of the object may also be acquired. In this case, the height map is converted into a normal map by differentiating it, and then the above-described processing is performed.

[0039] Furthermore, in the above-described embodiment, diffuse reflection intensity, specular reflection intensity, glossiness, and normals were acquired as texture information, but other information may be acquired in accordance with the texture to be expressed, such as an anisotropy map that indicates the distribution direction of specular reflection.

[0040] Furthermore, in the above-described embodiment, texture information is used in an image format in which pixel values ​​are represented by 8 bits, but other image formats (such as a 16-bit image format) or other information formats (such as a binary file listing data, text data, etc.) may also be used.

[0041] In addition, in the above-described embodiment, the Blinn-Phong model was used as a model formula for specular reflected light when generating the second image, but other model formulas such as the Torrance-Sparrow model or an approximation using a Gaussian function may also be used.

[0042] In addition, in the above-described embodiment, the lighting information when generating the second image was set based on the number of pixels whose pixel values ​​are saturated, but it may also be set based on a predetermined ratio of specular reflection to diffuse reflection or local contrast.

[0043] In the above-described embodiment, the synthesis ratio with the first image when generating the second image is set to 0.5, but other ratios may also be used.

[0044] Furthermore, in the above-described embodiment, the number of saturated pixels when setting the illumination intensity is set to 1% of the total number of pixels in the image, but it may be set to another threshold value.

[0045] In the above-described embodiment, the image displayed on the image display unit 501 is updated every time new line-of-sight information is acquired, but multiple images with different line-of-sight information may be displayed together. By comparing the multiple displayed images, the user can easily confirm the difference in appearance when observing an object with different line-of-sight information.

[0046] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0047] 1. Information processing equipment 206 Second image generation unit 207 Display control unit 602 Second image generation unit

Claims

1. a switching means for switching from a first mode in which a first image, which is a still image of an object and in which virtual illumination is not reflected on the object, to a second mode in which a second image in which virtual illumination is reflected on the object in accordance with line-of-sight information is displayed; a display control means for controlling the display means so as to switch the displayed image from the first image to the second image in response to the switching by the switching means; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the second image is generated based on texture information representing the texture of the object.

3. a first acquisition means for acquiring the texture information; a generating means for generating the second image based on the texture information so that virtual lighting is reflected on the object; 3. The information processing apparatus according to claim 2, further comprising:

4. 4. The information processing apparatus according to claim 3, wherein the generating means generates the second image by superimposing an image in which the virtual illumination is reflected on the first image.

5. further comprising a setting means for setting information on the virtual lighting to be reflected on the object based on the line of sight information and the texture information; 5. The information processing apparatus according to claim 2, wherein the second image is generated based on information about the set virtual lighting.

6. 6. The information processing apparatus according to claim 5, wherein the information about the virtual lighting is the direction and intensity of the virtual lighting.

7. 7. The information processing apparatus according to claim 5, wherein the setting means sets the virtual lighting information based on the number of pixels at which pixel values ​​are saturated.

8. 7. The information processing apparatus according to claim 5, wherein the setting means sets the information about the virtual illumination based on a preset ratio between specular reflection and diffuse reflection.

9. 7. The information processing apparatus according to claim 5, wherein the setting means sets the information about the virtual illumination based on a local contrast.

10. 10. The information processing apparatus according to claim 2, wherein the texture information includes at least a diffuse reflection intensity, a specular reflection intensity, and a normal.

11. a second acquisition means for acquiring the line-of-sight information; 11. The information processing device according to claim 1, wherein the display control means causes the display means to display an image generated based on the new line-of-sight information each time the line-of-sight information is newly acquired.

12. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 11.

13. a switching step of switching from a first mode in which a first image, which is a still image of an object and in which virtual illumination is not reflected on the object, to a second mode in which a second image in which virtual illumination is reflected on the object in accordance with line-of-sight information is displayed; a display control step of controlling a display means to switch the image to be displayed from the first image to the second image in response to the switching in the switching step; An information processing method comprising:

Images