Information processing device, information processing method, and program

Abstract

To provide an information processor, an information processing method, and a program for performing processing for setting a desired lighting condition with a simple operation by a user.SOLUTION: An information processor receives at least one of an instruction to enlarge and display a target object on a screen and an instruction to contract and display the target object on the screen, and displays on the screen an image of the target object irradiated with light under different lighting conditions according to the received instruction.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to setting lighting conditions in a virtual space. [Background technology]

[0002] In recent years, advances in computer graphics (CG) technology and imaging technology have made it possible to represent the shape and texture of objects on information terminals. Patent Document 1 discloses a technology for linking the position and orientation of a light source device held by a user to the position and orientation of a virtual light source. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-18173 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, the user needs to move the light source device until the desired lighting conditions are achieved, and operating the light source device can be a burden for the user.

[0005] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a process for setting desired lighting conditions with a simple operation by the user. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to the present invention includes: a receiving means for receiving at least one of an instruction to enlarge and display a target object on a screen and an instruction to reduce and display the target object on the screen; and a display control means for displaying, on the screen, an image of the target object illuminated with light under different lighting conditions in accordance with the received instruction; and a setting means for setting the illumination conditions based on the occupancy of the target object in the image displayed on the screen, wherein the display control means displays an image of the target object illuminated with light under the set illumination conditions on the screen. It is characterized by: [Effects of the Invention]

[0007] According to the present invention, the user can set desired lighting conditions with a simple operation. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a hardware configuration of an information processing device. [Figure 2] FIG. 1 is a diagram showing the functional configuration of an information processing device; [Figure 3] 1 is a flowchart showing a process executed by an information processing device; [Figure 4] A diagram to explain the ambient light map [Figure 5] FIG. 10 is a diagram showing an example of an image displayed on a display device. [Figure 6] Flowchart showing the process of setting the position and orientation of a virtual light source [Figure 7] A diagram to explain the direction of the virtual light source [Figure 8] FIG. 10 is a diagram for explaining a process for deriving the occupancy of a virtual object in a display image. [Figure 9] A diagram showing the relationship between occupancy and lighting ratio [Figure 10] 1 is a flowchart showing a process executed by an information processing device; DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all of the combinations of features described in each embodiment are necessarily essential to the solution of the present invention.

[0010] [First embodiment] In recent years, there has been an increasing demand for the ability to confirm the shape and texture of products when shopping online. Therefore, when displaying product images, it is necessary to reproduce the appearance of the product in real space. The elements of the appearance of the object to be reproduced include the appearance of the product in various indoor and outdoor environments, and the detailed shape and texture of the product. The appearance of a product affected by the surrounding environment can be expressed by illuminating the product with a light source in that environment. The detailed shape and texture of the product can be expressed by illuminating a region of interest on the product. However, a light source in a certain environment does not necessarily illuminate the region of interest on the product, and a virtual light source illuminating the region of interest alone may not be able to express the appearance of the product in the real environment. Therefore, in order to confirm both the appearance of the product in a certain environment and the detailed shape and texture of the product, it is necessary to view the product under multiple lighting conditions.

[0011] Therefore, in this embodiment, the ratio of illumination by a light source in a certain environment to illumination by a virtual light source is changed according to the occupancy of a virtual object in a displayed image. The occupancy of a virtual object can be changed by a simple operation such as a zoom-in instruction or a zoom-out instruction by the user, so desired lighting conditions can be set with little effort.

[0012] <Hardware configuration of information processing device> FIG. 1 is a block diagram showing the hardware configuration of an information processing device 1. The information processing device 1 includes a CPU 101, a ROM 102, and a RAM 103. The information processing device 1 also includes a VC (video card) 104, a general-purpose I / F (interface) 105, a SATA (serial ATA) I / F 106, and a NIC (network interface card) 107. The CPU 101 uses the RAM 103 as a work memory to execute an OS (operating system) and various programs stored in the ROM 102, a HDD (hard disk drive) 113, etc. The CPU 101 also controls each component via a system bus 108. Note that, in the processing of the flowcharts described below, program codes stored in the ROM 102, the HDD 113, etc. are loaded into the RAM 103 and executed by the CPU 101. A display device 115 is connected to the VC 104. An input device 110, such as a mouse or a keyboard, and an imaging device 111 are connected to the general-purpose I / F 105 via a serial bus 109. A general-purpose drive 114 that reads from and writes to an HDD 113 and various recording media is connected to the SATA I / F 106 via a serial bus 112. The NIC 107 inputs and outputs information to and from external devices. The CPU 101 uses the HDD 113 and various recording media mounted on the general-purpose drive 114 as storage locations for various data. The CPU 101 displays a UI (user interface) provided by a program on a display device 115, and receives inputs such as user instructions received via the input device 110. The display device 115 may be a touch panel display having a touch panel function that detects the position of a touch by a pointer such as a finger.

[0013] <Functional configuration of information processing device> Fig. 2 is a block diagram showing the functional configuration of the information processing device 1. The CPU 101 uses the RAM 103 as a work memory and reads and executes programs stored in the ROM 102 or the HDD 113, thereby functioning as the functional configuration shown in Fig. 2. It is not necessary for all of the processes shown below to be executed by the CPU 101, and the information processing device 1 may be configured so that part or all of the processes are executed by one or more processing circuits other than the CPU 101.

[0014] The information processing device 1 includes an object information acquisition unit 201, a viewpoint information acquisition unit 202, a condition setting unit 203, a ratio setting unit 204, an image generation unit 205, and a display control unit 206. The object information acquisition unit 201 acquires texture information representing the texture of a target object and three-dimensional shape information representing the three-dimensional shape of the target object. The object information acquisition unit 201 acquires, as texture information, diffuse reflection intensity, specular reflection intensity, glossiness, and a normal map related to the target object. The viewpoint information acquisition unit 202 acquires, as viewpoint information, a relative position and orientation of the virtual viewpoint with respect to an initial position and orientation of the virtual viewpoint. The position and orientation of the virtual viewpoint are changed based on an instruction from a user via the input device 110. The condition setting unit 203 acquires a two-dimensional ambient light map as information on lighting conditions (hereinafter referred to as ambient lighting conditions) for expressing the appearance of a target object illuminated with ambient light. Furthermore, based on the three-dimensional shape information and viewpoint information of the target object, condition setting unit 203 sets the position and orientation of the virtual light source as lighting conditions for expressing the detailed shape and texture of the target object (hereinafter referred to as texture lighting conditions). Note that condition setting unit 203 sets the position and orientation of the virtual light source as the texture lighting conditions so that specular reflected light is reflected in a region of interest on the surface of the target object.

[0015] The ratio setting unit 204 derives the occupancy of the target object in the display image when observed from the virtual viewpoint, based on the three-dimensional shape information and viewpoint information of the target object. Furthermore, when the occupancy of the target object is large, the ratio setting unit 204 sets the illumination ratio so that the ratio of the texture illumination condition to the ambient illumination condition is high. In other words, when the target object is displayed large on the screen of the display device 115, it is considered that the user wants to confirm the texture of the target object, so the ratio of light illuminating the attention area is set higher than the ambient light. The image generation unit 205 generates an image I1 representing the appearance of the target object from the virtual viewpoint when the target object is illuminated under an illumination condition obtained by mixing two illumination conditions according to the illumination ratio. Furthermore, the image generation unit 205 generates an image I2 in which a UI is superimposed on the image I1. The display control unit 206 displays the image I2 on the display device 115.

[0016] <Processing performed by the information processing device> The flow of processing executed by the information processing device 1 in this embodiment will be described with reference to the flowchart in Fig. 3. The processing shown in the flowchart in Fig. 3 starts when the user inputs an instruction via the input device 110 and the CPU 101 accepts the input instruction. Hereinafter, each step (process) will be represented by adding an S before the reference numeral. In S301, the object information acquisition unit 201 acquires the diffuse reflection intensity ρ d (x,y,i), specular reflection intensity ρ sThe object information acquisition unit 201 acquires (x, y, i), glossiness σ(x, y, i), and normal N(x, y, k). Specifically, the object information acquisition unit 201 acquires texture information from a storage device such as the HDD 113 based on a user instruction. Here, (x, y) indicates the position coordinates on the surface of the target object. i indicates which of the color signals the value corresponds to: R, G, or B. k indicates which of the x, y, or z components of the normal direction the value corresponds to. In this embodiment, the diffuse reflection intensity, specular reflection intensity, and glossiness are arrays having values ​​expressed in 8 bits for each position coordinate and for each color signal. The normal direction is an array having values ​​expressed in 8 bits for each position coordinate and for each component, with values ​​ranging from -1 to 1 for each component corresponding to values ​​ranging from 0 to 255. Note that the format of the texture information in this embodiment is merely an example; for example, the specular reflection intensity may be acquired as 32-bit floating-point information. Furthermore, the texture information does not have to be the four types of information described above as long as it can identify the texture of the target object; for example, it may be two types, diffuse reflection intensity and specular reflection intensity, or it may be five or more types.

[0017] Furthermore, the object information acquisition unit 201 acquires three-dimensional shape information that represents the three-dimensional shape of the target object. The three-dimensional shape information in this embodiment is information that holds, as attribute information, multiple three-dimensional shapes of the target object and a truth value B indicating whether each three-dimensional shape affects each process of the condition setting unit 203 and the ratio setting unit 204. The three-dimensional shape information is information in OBJ format that stores the three-dimensional coordinates and normal directions of each vertex.

[0018] In S302, the condition setting unit 203 generates an environmental light map E(θ) as lighting condition information corresponding to the environmental lighting conditions based on an instruction from the user. a ,φ a , i) is acquired from a storage device such as the HDD 113. a ,φ a , i), and Fig. 4(b) shows the created ambient light map E(θ a ,φ a, i). The ambient light map E can be acquired by an omnidirectional camera 401 installed at an arbitrary point O. Specifically, when the ambient light map E is acquired from the arbitrary point O in the line of sight direction (θ a ,φ a ) by measuring the light intensity in the line of sight (θ a ,φ a ) and the measured light intensity for each color signal i. a ,φ a , i) is created. Here, the position O of the image capturing plane of the omnidirectional camera 401 is set as the origin, and θ a , φ a These are the polar angle and azimuth angle with respect to the normal direction N of the imaging surface. a ,φ a ,i) stores intensity information expressed in 8 bits for each angle and each color signal. The number of pixels is 1920 x 1080. Note that the format of the lighting condition information in this embodiment is an example; for example, the signal intensity of the ambient light map may be stored as 32-bit floating-point information. Using the above method, an ambient light map 402 such as that shown in FIG. 4(b) can be created.

[0019] In S303, the image generation unit 205 generates an ambient light map E(θ a ,φ a , i), an image I1(X, Y, i) is drawn, which represents the appearance of the target object when illuminated using the 3D coordinates C c =(x c ,y c ,z c ) and the posture of the virtual viewpoint is determined by the roll angle R c , pitch angle P c , yaw angle Y c The virtual viewpoint posture C is determined by r is Cr=(R c ,P c ,Y c) The initial position of the virtual viewpoint is expressed as C c =(4h,4h,4h). The initial orientation of the virtual viewpoint is set, for example, so that it faces the direction of the origin and the positive direction of the z axis coincides with the upward direction of the viewpoint.

[0020] The image generation unit 205 generates an ambient light map E(θ a ,φ a , i), an image I1(X,Y,i) is drawn as the appearance of the target object illuminated using the virtual viewpoint. The size of image I1(X,Y,i) is, for example, 720 × 720. Furthermore, the image generation unit 205 draws image I2(X,Y,i) by overlaying a UI on image I1(X,Y,i). An example of image I2 is shown in FIG. 5(a). In this embodiment, three-dimensional shapes 502, 503, and 504 are arranged in the virtual space as target objects. Image I1(X,Y,i) 501 is composed of areas representing the appearance of these three three-dimensional shapes and an area 505 where no three-dimensional shapes exist. The image generation unit 205 draws a zoom button 506, a zoom button 507, a translation button 508, and a rotation button 509 as UIs for receiving instructions from the user to change the position and orientation of the virtual viewpoint. The size of the image I2(X, Y, i) is, for example, 900 × 1080. The display control unit 206 causes the display device 115 to display the image I2(X, Y, i).

[0021] The processing from S304 to S308 is repeatedly executed until the viewpoint information acquisition unit 202 receives an end instruction from the user. In S304, the viewpoint information acquisition unit 202 determines whether or not an instruction to change the position and orientation of the virtual viewpoint has been received from the user. The viewpoint information acquisition unit 202 determines that an instruction to change the position and orientation has been received when the user presses any one of the enlarge button 506, the reduce button 507, the translation button 508, and the rotation button 509 in FIG. 5A via the input device 110. When it is determined that an instruction to change the position and orientation of the virtual viewpoint has been received from the user, the viewpoint information acquisition unit 202 changes the position and orientation of the virtual viewpoint in accordance with the instruction from the user. When the enlarge button 506 is pressed, the viewpoint information acquisition unit 202 moves the viewpoint position in the line of sight without changing the viewpoint orientation, so as to reduce the distance between the virtual viewpoint and a three-dimensional shape of the target object present in the line of sight direction by 10%. When the zoom out button 507 is pressed, the viewpoint information acquisition unit 202 moves the viewpoint position in the line of sight direction without changing the viewpoint orientation so as to increase the distance between the virtual viewpoint and the three-dimensional shape of the target object present in the line of sight direction by 10%. When the parallel movement button 508 is pressed, the viewpoint information acquisition unit 202 moves the viewpoint position in the direction of the pressed arrow button on a plane perpendicular to the line of sight without changing the viewpoint orientation. The distance of the parallel movement is 10% of the distance between the two farthest vertices among the group of vertices that make up the target object. When the rotation button 509 is pressed, the viewpoint information acquisition unit 202 rotates the virtual viewpoint 15 degrees around the origin of the virtual space in the axial direction of the pressed rotation button. Note that the above-described method of changing the position and orientation of the virtual viewpoint is an example, and other values ​​may be used for the movement distance, rotation angle, etc.

[0022] If the viewpoint information acquisition unit 202 determines that it has not received an instruction from the user to change the position and orientation of the virtual viewpoint, it proceeds to S309 and determines whether it has received an end instruction from the user. If it determines that it has not received an end instruction from the user, it returns the process to S304. If it determines that it has received an end instruction from the user, it ends the process.

[0023] In S305, the viewpoint information acquisition unit 202 determines whether or not an enlargement instruction has been received from the user. The viewpoint information acquisition unit 202 determines that an enlargement instruction has been received when the enlarge button 506 in FIG. 5(a) is pressed by the user. If the viewpoint information acquisition unit 202 determines that an enlargement instruction has been received from the user, it proceeds to S306. If the viewpoint information acquisition unit 202 determines that an enlargement instruction has not been received from the user, it proceeds to S307. Note that the viewpoint information acquisition unit 202 may also determine that an enlargement instruction has been received when the distance between the virtual viewpoint and the target object becomes smaller.

[0024] In S306, the condition setting unit 203 sets the position and orientation of the virtual light source as the texture lighting condition. FIG. 6 is a flowchart showing the details of the processing in S306. In S601, the condition setting unit 203 derives a region of interest on the surface of the target object based on the position and orientation of the virtual viewpoint and the shape of the target object. In this embodiment, the condition setting unit 203 determines the intersection O=(x o ,y o ,z o ) is set as the region of interest. Note that the above-mentioned method of setting the region of interest is just one example, and for example, a circular region on the target object centered on the above-mentioned intersection point O may be set as the region of interest. Alternatively, the region of interest may be set as the intersection point between a half-ray obtained by translating a half-ray extending in the line of sight from the position of the virtual viewpoint by a certain distance and the surface of the target object.

[0025] In S602, condition setting unit 203 derives the direction (posture) of a virtual light source to be installed. Fig. 7 is a diagram for explaining the direction of the virtual light source. An attention area 703 is set on the surface of a target object 701 as an intersection with a half line extending from a virtual viewpoint 702 in the line of sight direction, and a light source direction 704 is derived based on target object 701 and virtual viewpoint 702. A vector S representing the direction of the virtual viewpoint is derived as shown in Equation (1).

[0026]

number

[0027] When the normalized normal vector in the region of interest 703 is N, the light source direction L is expressed as in equation (2).

[0028]

number

[0029] Here, the parameter d is a real number equal to or greater than 0. If the region of interest has a large area, the normal vector N may be a vector representing the average normal direction in the region of interest. By determining the light source direction L in this way, the user can visually recognize the specular reflected light in the region of interest and can confirm the texture, such as the shape and specular reflection intensity.

[0030] In S603, the condition setting unit 203 determines the parameter d and derives the position of the virtual light source to be installed. The condition setting unit 203 uses the light source direction L determined in S602 to calculate 20 different light source positions L shown in Equation (3). j Set.

[0031]

number

[0032] where d j is determined as in equation (4).

[0033]

number

[0034] Here, j=1, . . . , 20, and the value a is set to, for example, 10 times the height h. The condition setting unit 203 calculates the normal vector N at each point in the partial region of the target object visible from the viewpoint position and the normalized light source position L jThe controller 202 derives the inner product of the parameter d and derives the ratio between the inner product equal to or greater than threshold A and the inner product less than threshold A. The controller 202 sets the light source position from the 20 light source positions whose derived ratio is closest to ratio B. The threshold A is set to, for example, 0.2, and the ratio B is set to, for example, 4:1. By determining the light source position in this manner, even if the viewpoint position changes, it is possible to cast shadows on the surface of the target object visible from the viewpoint position at approximately the same ratio as before the change, allowing the user to effectively confirm the shape of the target object. Note that the search method for determining the parameter d described above is merely an example, and other optimization methods may also be used. In S604, the condition setting unit 203 sets a point light source with the orientation derived in S602 at the position derived in S603.

[0035] In S307, the ratio setting unit 204 derives the proportion of the target object occupying the image I1 as the occupancy, and sets an illumination ratio for blending two illumination conditions based on the occupancy of the target object. FIGS. 8(a), 8(b), and 8(c) are diagrams for explaining the process of deriving the occupancy. First, the ratio setting unit 204 generates an image 801 representing the occlusion relationship when the target object is viewed from a virtual viewpoint, as shown in FIG. 8(a). In generating the image 801, it is not necessary to add texture information; for example, it is sufficient to simply add a monochromatic diffuse reflection intensity to the surface of the target object. The ratio setting unit 204 generates the image 801 by setting the alpha value representing opacity to 0 for a region 802 where no three-dimensional shape exists and a three-dimensional shape whose truth value B is false, and then derives the occupancy according to Equation (5).

[0036]

number

[0037] The size of image 801 is assumed to be the same as, for example, image I1(X,Y,i). Fig. 8(b) shows image 801 when the truth value B is true for all three three-dimensional shapes, and region 803 is a region drawn with an alpha value of 0. Fig. 8(c) shows image 801 when the truth value B for only one of the three three-dimensional shapes is true and the other two are false, and region 804 is a region drawn with an alpha value of 0. Note that, although the alpha value is used to separate the target object from the rest (background) in order to derive the occupancy rate of the target object in the image, any method may be used as long as it is possible to obtain the size of the target object as seen from the virtual viewpoint.

[0038] Next, the ratio setting unit 204 sets the illumination ratio m of the two illumination conditions based on the occupancy of the target object. FIG. 9 shows an example of the relationship between occupancy and illumination ratio. Here, the occupancy Th1 at which the illumination ratio m begins to decrease is set to 30%, and the occupancy Th2 at which the illumination ratio m reaches 0 is set to 80%. By increasing the ratio of the texture illumination condition as the occupancy increases, the appearance of the entire target object can be confirmed when the target object is displayed small (reduced display) on the screen. Furthermore, when the target object is displayed large (enlarged display) on the screen, the texture and shape of the target area can be confirmed. Note that the relationship shown in FIG. 9 is merely an example, and the illumination ratio m may be changed discretely depending on the occupancy. For example, when the occupancies Th1 and Th2 are both set to 50%, the illumination switches discretely between the ambient illumination condition and the texture illumination condition at an occupancy of around 50%. In this case, the user can clearly recognize the change in illumination condition.

[0039] In S308, the image generation unit 205 generates an image I1(X,Y,i) that represents how the target object appears from the virtual viewpoint when the target object is illuminated under lighting conditions obtained by mixing two lighting conditions based on the lighting ratio m. The image generation unit 205 also generates an image I2(X,Y,i) by overlaying a UI on the image I1(X,Y,i). Specifically, first, the image generation unit 205 generates an image i1(X,Y,i) using an image-based lighting technique that uses the ambient lighting conditions, as in S303. Then, the image generation unit 205 generates an image i2(X,Y,i) using texture lighting conditions based on the Blinn-Phong model. The image generation unit 205 derives pixel values ​​of the image I1(X,Y,i) using the lighting ratio m, as shown in Equation (6).

[0040]

number

[0041] When generating images i1 and i2, the image generation unit 205 changes the display color of areas where no three-dimensional shapes exist and the transparency of three-dimensional shapes other than the three-dimensional shape where the region of interest exists, based on the illumination ratio m. Figures 5(a) and 5(b) show changes in the background display color and the transparency of three-dimensional shapes when the illumination ratio m is different.

[0042] As the illumination ratio m decreases, the color of the area 505 where no three-dimensional shape exists is changed, and the transparency of the three-dimensional shapes 503 and 504 where no region of interest exists is increased. By changing the background display color, the user can recognize which lighting conditions are used, and by changing the transparency of the three-dimensional shapes, it is possible to prevent the three-dimensional shape that the user is focusing on from being obscured by other three-dimensional shapes. Note that the transparency may be changed only for three-dimensional shapes whose truth value B is false. The UI to be overlaid on image I1(X,Y,i) is the same as the UI described in S303, so its description will be omitted. The display control unit 206 displays image I2(X,Y,i) on the display device 115.

[0043] As described above, the information processing device according to this embodiment illuminates the target object using an ambient light map when the target object is displayed small on the screen, and illuminates the target object's focus area with a virtual light source when the target object is displayed large on the screen. This allows the user to confirm the appearance of the target object in a certain environment and the shape and texture of the focus area on the target object without the need to set the light source position or change the light source type. Furthermore, by updating the position and orientation of the virtual light source in S306 when an enlargement instruction is received, the virtual light source can be appropriately positioned even if the user's focus area on the target object changes. Furthermore, by not updating the position and orientation of the virtual light source for reasons other than an enlargement instruction, flickering that occurs when the virtual light source is repositioned can be suppressed. Furthermore, by acquiring, as attribute information in S301, a boolean value B indicating whether each three-dimensional shape is considered when setting the illumination, it is possible to prevent three-dimensional shapes that the user does not intend to focus on from being illuminated.

[0044] [Second embodiment] In the first embodiment, a virtual light source was installed to illuminate the target object as a texture lighting condition. In this embodiment, an ambient light map is used as the texture lighting condition. Note that the hardware configuration and functional configuration of the information processing device in this embodiment are the same as those in the first embodiment, and therefore a description thereof will be omitted. Below, differences between this embodiment and the first embodiment will be mainly described. Note that the same components as those in the first embodiment will be described using the same reference numerals.

[0045] <Processing performed by the information processing device> The flow of processing executed by the information processing device 1 in this embodiment will be described with reference to the flowchart of Fig. 10. The processing shown in the flowchart of Fig. 10 starts when a user inputs an instruction via the input device 110 and the CPU 101 accepts the input instruction. Note that the processing other than S1001, S1002, and S1003 is the same as in the first embodiment, and therefore description thereof will be omitted.

[0046] In S1001, the condition setting unit 203 generates an environmental light map E2(θ) as lighting condition information corresponding to the environmental lighting conditions based on an instruction from the user. a ,φ a , i) is acquired from a storage device such as the HDD 113. Details of the ambient light map are the same as those explained in S302 in the first embodiment. In S1002, the condition setting unit 203 acquires the ambient light map E2(θ a ,φ a , i) to set texture lighting conditions that illuminate the target area. The method of deriving the light source direction L is the same as in S602 in the first embodiment, so a description thereof will be omitted. Furthermore, the condition setting unit 203 sets the ambient light map E2(θ a ,φ a , i), grayscaling is performed to identify the area with the highest brightness. Specifically, the ambient light map, which is an array having 8-bit intensity information for each coordinate and each color signal, is converted into an array having brightness values ​​represented by 8 bits for each coordinate. The conversion method is a known technique, so a description thereof will be omitted. Note that the format of the ambient light map after grayscaling described above is an example, and brightness values ​​may be held as 32-bit floating-point information, for example. The condition setting unit 203 derives the coordinates (Θ, Φ) with the highest brightness value in the ambient light map after grayscaling. The angle component (θ) of the derived light source direction L when expressed in polar coordinates is used as the l ,φ l The ambient light map is transformed so that the coordinates (Θ,Φ) match.

[0047] In S1003, the image generation unit 205 generates an image I1(X,Y,i) that represents how the target object appears from the virtual viewpoint when the target object is illuminated under lighting conditions obtained by mixing two lighting conditions based on the lighting ratio m. The image generation unit 205 also generates an image I2(X,Y,i) by overlaying a UI on the image I1(X,Y,i). The display control unit 206 causes the display device 115 to display the image I2(X,Y,i). Note that the specific processing is the same as in the first embodiment except that the image i2(X,Y,i) is generated by the image-based lighting technique using the texture lighting conditions, and therefore a description thereof will be omitted.

[0048] As described above, the information processing device in this embodiment illuminates the target object using an ambient light map when the target object is displayed small on the screen, and illuminates the target object's region of interest using another ambient light map when the target object is displayed large on the screen. In particular, by acquiring an ambient light map including a large number of light sources and various light source shapes as lighting condition information corresponding to texture lighting conditions, when the user displays the target object large on the screen, the user can confirm the shape and texture of the region of interest through a rich lighting effect using virtual light sources.

[0049] [Variations] In the above-described embodiment, the UI and images are displayed using a display device externally connected to the information processing device, but a hardware configuration such as a tablet in which the information processing device and the display device are integrated may also be used.

[0050] In the above-described embodiment, instructions regarding the position and orientation of the virtual viewpoint are received from the user via the UI, but when using a hardware configuration such as a tablet, the position and orientation may be acquired using an in-camera, a gyro sensor, etc. Also, the position and orientation of the virtual viewpoint may be acquired using an eye-tracking device.

[0051] Furthermore, in the above-described embodiment, the light source position under the texture lighting condition is determined only by the condition setting unit 203, but a change in the light source position may also be received from the user via a UI or the like.

[0052] In the above-described embodiment, an image i1 when the target object is illuminated under the ambient lighting condition and an image i2 when the target object is illuminated under the texture lighting condition are generated, and then mixed according to the lighting ratio m, but other methods of mixing lighting may be used. For example, in the first embodiment, the luminance of the ambient light map corresponding to the ambient lighting condition may be multiplied by m, and the light source intensity corresponding to the texture lighting condition may be multiplied by (1-m), and an image I1 representing the appearance of the target object illuminated under both lighting conditions may be generated.

[0053] In addition, in the above-described embodiment, two lighting conditions are set and the ratio between the two lighting conditions is determined according to the occupancy of the target object in the displayed image, but there may be three or more lighting conditions, and the ratio between the three or more lighting conditions may be determined according to the occupancy.

[0054] Furthermore, in the above-described embodiment, there are multiple three-dimensional shapes of the target object, but there may be one three-dimensional shape.

[0055] In addition, in the above-described embodiment, processing was performed according to the flow shown in Figures 3, 6, and 10, but the processing flow is only an example, and for example, the order of processing S301 and S302 may be reversed, or processing may be performed in parallel.

[0056] In addition, in the above-described embodiment, the UI is superimposed on the image when lighting is performed using an ambient light map in S303, but the UI may also be superimposed on the image when lighting is performed using a virtual light source.

[0057] In the first embodiment described above, the Blinn-Phong model is used as a method for generating the image i2 when the target object is illuminated under textured lighting conditions, but other model formulas may be used, such as the Torrance-Sparrow model or an approximation using a Gaussian function.

[0058] Furthermore, in the first embodiment described above, a point light source is set as the texture lighting condition, but a virtual light source having a shape such as a spot light source or a surface light source may also be set. In this case, the orientation of the virtual light source is determined so as to illuminate the region of interest. Also, an ambient light map with similar lighting conditions may be generated to illuminate the region of interest. In this case, the luminance near the position coordinates of the ambient light map corresponding to the light source direction derived by the condition setting unit 203 is set to a high value, and the luminance of other regions is set to a low value.

[0059] Furthermore, in the second embodiment described above, if the ambient light map acquired as the lighting condition information corresponding to the texture lighting condition has high luminance in all directions, it is not necessary to perform coordinate conversion processing.

[0060] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0061] 1. Information processing equipment 202 Viewpoint information acquisition unit 206 Display control unit

Claims

1. a receiving means for receiving at least one of an instruction to enlarge and display a target object on a screen and an instruction to reduce and display the target object on the screen; a display control means for displaying on the screen an image of the target object illuminated with light under different lighting conditions in accordance with the received instruction; a setting means for setting the illumination conditions based on the occupancy of the target object in the image displayed on the screen; and The information processing apparatus is characterized in that the display control means displays on the screen an image of the target object illuminated with light under the set lighting conditions.

2. 2. The information processing apparatus according to claim 1, wherein said setting means sets the lighting conditions by combining a plurality of lighting conditions based on the occupancy rate.

3. 3. The information processing apparatus according to claim 2, wherein the plurality of lighting conditions include a lighting condition using an environmental light map and a lighting condition using a virtual light source.

4. The information processing device described in claim 3, characterized in that the lighting conditions using the environmental light map are lighting conditions when a target object placed in a certain environment is illuminated by a light source included in the environment, and the lighting conditions using the virtual light source are lighting conditions when a focus area on the target object is illuminated by the virtual light source.

5. 5. The information processing apparatus according to claim 3, wherein the lighting conditions using the virtual light source are lighting conditions under which light specularly reflected in a region of interest on the target object is visually recognized by a user.

6. 6. The information processing apparatus according to claim 3, wherein the setting means increases a ratio of the illumination condition using the virtual light source when the occupancy is large.

7. 2. The information processing apparatus according to claim 1, wherein said setting means sets the lighting conditions by switching the lighting conditions based on the occupancy rate.

8. 8. The information processing apparatus according to claim 1, wherein the occupancy is determined based on a size of the target object when viewed from a viewpoint in a virtual space.

9. a first acquisition means for acquiring information about the position and orientation of a viewpoint from which the target object is viewed in a virtual space; 9. The information processing apparatus according to claim 1, further comprising: a generating unit that generates an image of the target object based on information about the position and orientation of the viewpoint.

10. The method further includes a second acquisition means for acquiring information about the texture of the target object, The information processing apparatus according to claim 9 , wherein the generating means generates an image of the target object based on the information relating to the texture.

11. 11. The information processing device according to claim 9, wherein the generating means, when generating an image of the target object, changes the transparency of a target object that does not have a region of interest among the plurality of target objects based on the lighting conditions.

12. 12. The information processing device according to claim 9, wherein the generating means, when generating the image of the target object, changes a color of a background where the target object does not exist based on the lighting conditions.

13. a receiving means for receiving at least one of an instruction to enlarge and display a target object on a screen and an instruction to reduce and display the target object on the screen; a display control means for controlling the display so as to change the color of the background where the target object does not exist in response to the received instruction; An information processing device comprising:

14. A receiving means for receiving at least one of an instruction to enlarge and display a target object on a screen and an instruction to reduce and display the target object on the screen; a display control means for displaying on the screen an image of the target object illuminated with light under different lighting conditions in accordance with the received instruction; a first acquisition means for acquiring information about the position and orientation of a viewpoint from which the target object is viewed in a virtual space; a generating means for generating an image of the target object based on information about the position and orientation of the viewpoint; and The generating means generates an image of the target object based on the lighting conditions.

10. An information processing apparatus comprising: an information processing device for changing a transparency of a target object that does not have a region of interest among the target objects;

15. A receiving means for receiving at least one of an instruction to enlarge and display a target object on a screen and an instruction to reduce and display the target object on the screen; a display control means for displaying on the screen an image of the target object illuminated with light under different lighting conditions in accordance with the received instruction; a first acquisition means for acquiring information about the position and orientation of a viewpoint from which the target object is viewed in a virtual space; a generating means for generating an image of the target object based on information about the position and orientation of the viewpoint; and The generating means generates an image of the target object based on the lighting conditions. An information processing device that changes the color of a background where no target object is present.

16. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 15.

17. receiving at least one of an instruction to enlarge and display a target object on a screen and an instruction to reduce and display the target object on the screen; displaying on the screen an image of the target object illuminated with light under different lighting conditions in accordance with the received instruction; setting the illumination conditions based on the occupancy of the target object in the image displayed on the screen; and an image of the target object illuminated under the set lighting conditions is displayed on the screen;

Images