Information processing device, information processing method, and program

Abstract

To provide a technique for grasping a sight line direction suitable for observation of a texture of an object.SOLUTION: An information processing device includes: texture information acquisition means which acquires texture information indicating a texture of an object; illumination information acquisition means which acquires illumination information for illuminating the object; sight line direction calculation means which calculates an observation sight line direction suitable for observation of the texture of the object on the basis of the texture information and the illumination information; and drawing means which draws information indicating the observation sight line direction.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, advances in CG technology, imaging technology, and computer performance have made it possible to express the texture of objects on information terminals. In particular, there is a growing demand for the ability to check the texture of products in various indoor and outdoor environments in order to increase the appeal of products in online shopping.

[0003] In the method described in Patent Document 1, the content creator manually sets the gaze direction in advance and presents it to the user. By presenting the user with a gaze direction that is likely to reveal the glossiness, color, and other textures of the product, the user can effectively check the texture of the product without having to go to the trouble of changing the gaze direction in various ways. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 11-265462 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the technology described in Patent Document 1, the content creator needs to manually specify a gaze direction suitable for observing the texture of an object. Generally, when the surrounding environment changes, the gaze direction suitable for observing the texture also changes. Therefore, when the surrounding environment changes, it is difficult to present the user with a gaze direction suitable for observing the texture.

[0006] The present invention has been made in view of the above-mentioned problems, and aims to provide a technique for determining a gaze direction suitable for observing the texture of an object. [Means for solving the problem]

[0007] To achieve the above object, an information processing device according to one aspect of the present invention comprises: a texture information acquisition means for acquiring texture information indicating the texture of an object; an illumination information acquisition means for acquiring illumination information for illuminating the object; The texture of the object is determined based on the texture information and the lighting information. of observation In order to a gaze direction calculation means for calculating an observation gaze direction; a drawing means for drawing information indicating the observation line of sight direction; The present invention is characterized by comprising: [Effects of the Invention]

[0008] According to the present invention, it is possible to easily grasp the gaze direction suitable for observing the texture of an object, thereby enabling the user to effectively check the texture of the object without having to take the effort of changing the gaze direction in various ways. [Brief explanation of the drawings]

[0009] [Figure 1] Conceptual diagram explaining color and gloss [Figure 2] Conceptual diagram explaining the unevenness [Figure 3] FIG. 1 is a diagram showing an example of the hardware appearance of an information processing device according to first, second, and third embodiments. [Figure 4] FIG. 1 is a diagram showing an example of the hardware configuration of an information processing device according to first, second, and third embodiments. [Figure 5] FIG. 1 is a diagram showing an example of the functional configuration of an information processing device according to first and second embodiments. [Figure 6] Main processing flow in embodiments 1 and 2 [Figure 7] Environment map illustration [Figure 8] Detailed processing flow of S503 in the first embodiment [Figure 9] Illustrative diagram of calculation of reflection area [Figure 10]Detailed processing flow of S507 in the first embodiment [Figure 11] Example of user interface in embodiments 1 and 2 [Figure 12] Detailed processing flow of S503 in the second embodiment [Figure 13] Supplementary figure for S1109 in embodiment 2 [Figure 14] Supplementary diagram of the use example of embodiment 3 [Figure 15] FIG. 10 is a diagram showing an example of the system configuration of an information processing device according to a third embodiment. [Figure 16] Main processing flow in embodiment 3 [Figure 17] FIG. 10 is a diagram showing an example of a user interface in the third embodiment. [Figure 18] Detailed processing flow of S1612 in the third embodiment DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] (Embodiment 1) In this embodiment, an example is described in which a gaze direction suitable for observing the texture of an object (target) displayed on a display screen is calculated and presented based on the surrounding environment (lighting environment). This allows the user to check the texture of the target object in the current lighting environment as if it were actually located there. For example, when browsing fabric products on an online shopping site, an image of the fabric product may be displayed on a tablet information terminal or the like. In this case, the system displays how the product will look in the lighting environment of the room where the user is located, and allows the user to understand from which direction on the display screen the texture is best viewed.

[0012] Before describing this embodiment, examples of elements that constitute texture and the line of sight directions suitable for observing them will be described.

[0013] <Explanation of texture elements> The elements of the texture of an object (material, target object) include color, gloss, and roughness. Color can be perceived from the diffuse reflection component of the material, and gloss can be perceived from the specular reflection component of the material.

[0014] FIG. 1 illustrates diffuse reflection and specular reflection. Light 102 incident on a material surface 101 is reflected according to an intensity distribution 103 that depends on the observation angle. The reflected light intensity distribution 103 can be expressed as the sum of a diffuse reflection component 104 caused by irregular reflection within the material and a specular reflection component 105 caused by reflection on the material surface. The specular reflection component 105 has a peak in the angular region where the angle of incidence and the angle of reflection are equal. Therefore, at gaze direction position 106, which is the direction of specular reflection of incident light 102, a larger proportion of the specular reflection component 105 is observed compared to other gaze directions, making it suitable for observing the glossiness of a material. Similarly, at gaze direction position 107, which is away from the specular reflection direction of incident light 102, a diffuse reflection component is observed more predominantly compared to the specular reflection component, making it suitable for observing the color of a material.

[0015] The sense of unevenness can be perceived from the change in appearance of diffuse reflection and specular reflection according to the minute unevenness of the material surface. Figure 2(a) is a diagram showing a material surface with minute unevenness. There are minute unevennesses on the material surface 101, and the normal direction 108 changes depending on the location. In the case of a material in which the specular reflection intensity is stronger than the diffuse reflection intensity and the reflected light intensity changes significantly near the specular reflection direction, the minute normal changes on the material surface can be perceived as changes in the specular reflection component. In this case, the sense of unevenness can be effectively observed by observing from the specular reflection direction of a small light source, as will be explained below.

[0016] Figures 2(b) and 2(c) show specular reflection from two surfaces with different normal directions N on a material surface. When specularly reflected light from two surfaces with different normal directions on material surface 205 is observed at gaze direction 204, two light rays 202 and 203 are observed. When light source 200 illuminates, light ray 202 originates in an area where no light source is present and is therefore not observed in gaze direction 204. Therefore, the difference in the normals of the two surfaces is perceived as a difference in brightness due to the presence or absence of specularly reflected light. On the other hand, when light source 201, which is larger than light source 200, illuminates, both light rays 202 and 203 are observed, making it difficult to distinguish the difference in the normals of the two surfaces from the specularly reflected light. Therefore, for materials with strong specular reflection, the texture can be effectively observed by observing them from the specular reflection direction of a small light source.

[0017] Additionally, the texture can be effectively observed by observing from the direction of specular reflection of a light source with a clear edge. Examples of light sources with a clear edge include fluorescent lamps and spotlights, while examples of light sources with an unclear edge include external light from a curtained window. Figure 2(d) shows images of a material observed from the direction of specular reflection of a light source with a clear edge and from the direction of specular reflection of a light source with an unclear edge. In image 206 observed from the direction of specular reflection of a light source with a clear edge, fluctuations at the edges of the specular reflection image of the light source reflect subtle changes in the normal of the material surface, making the texture of the material visible. On the other hand, in image 207 observed from the direction of specular reflection of a light source with an unclear edge, the brightness difference at the edges of the specular reflection image of the light source is small, making it difficult to visually recognize the texture of the surface from the reflected image.

[0018] In addition, for materials where the specular reflection intensity is weaker than the diffuse reflection intensity and the change in the reflected light intensity near the specular reflection is small, minute changes in the normal to the surface of the material can be perceived as changes in the diffuse reflection intensity. In such cases, the unevenness can be effectively observed by observing from a direction perpendicular to the light beam, as will be explained below.

[0019] Figure 2(e) is a diagram explaining how light from a light source is incident on the uneven surface of a material and diffusely reflected. Figure 2(e) shows ray 208 from light source 201 incident on a surface of the material with a normal horizontal to the light source direction, and ray 209 incident on a surface with a normal that makes an angle θ with the light source direction. If the respective diffuse reflection intensities are In and Iθ, the relationship between these two intensities can be approximately expressed by Lambert's cosine law, as follows:

[0020]

number

[0021] Differentiating equation 1 with respect to θ gives the following equation:

[0022]

number

[0023] The region of θ where the diffuse reflection intensity changes most is when θ is near 90 degrees. Figure 2(f) shows the distribution of the normal direction of the material. The normal to the material surface has a distribution 210 centered on the average normal direction Nave. Within distribution 210, the surface in region 211 where the angle between the incident light and the normal vector is near 90 degrees has the greatest change in brightness.

[0024] To observe the surface where the angle between the incident light and the normal vector is near 90 degrees, the surface is observed from a direction perpendicular to the light source. Figure 2(g) shows the appearance of surfaces where the angle between the normal and the incident light is 0, 5, 85, and 90 degrees when viewed from two directions: a viewing direction parallel to the incident light and a viewing direction perpendicular to the incident light. In the view 212 from a viewing direction parallel to the incident light, the surface where the angle between the normal and the incident light is small appears larger. On the other hand, in the view 213 from a viewing direction perpendicular to the incident light, the surface where the angle between the normal and the incident light is close to 90 degrees appears larger. Furthermore, when the angle between the normal and the incident light is near 90 degrees, the diffuse reflection intensity changes significantly with changes in the normal, so subtle changes in the normal of the material surface can be observed as changes in brightness. Therefore, when the specular reflection intensity is weak, the texture can be effectively observed by observing from a direction perpendicular to the light.

[0025] In this embodiment, a tablet-type information terminal is used as the information processing device, and gaze directions suitable for observing color and glossiness, two of the texture elements mentioned above, are calculated and presented. An application is envisioned in which a planar model onto which texture information is mapped is displayed as an object on the display of a tablet-type information terminal with a fixed position and orientation, and the appearance of the object is dynamically changed in response to changes in the user's gaze to observe the texture.

[0026] <Application scene> An example of an application scenario using the tablet information terminal according to this embodiment will be described. For example, when browsing and checking products displayed on a web site, such as during internet shopping, a situation may be envisioned in which the user wants to check the texture of the product. When browsing fabric as an example of a product, the tablet information terminal acquires texture information for the fabric that has been stored in advance on the web site in response to the user selecting the fabric product.

[0027] Next, to understand how the fabric will be viewed under the lighting environment of the room the user is currently in, lighting information about the user's room is acquired. The lighting information can be acquired by the user holding a tablet information terminal in the center of the room, for example, and rotating it to capture images of the entire room. Alternatively, the lighting information can be acquired using an omnidirectional camera placed in the center of the room.

[0028] When a user displays an image of a fabric on a tablet information terminal in a room, the appearance of the fabric changes depending on the position and orientation of the tablet information terminal and the user's line of sight. When the image of a fabric is displayed on the tablet information terminal, the user is presented with a situation in which the fabric is being held at the position of the tablet information terminal, and the appearance of the fabric at that time is displayed. At this time, by displaying an observation direction suitable for observing the texture, the user can easily recognize that the current line of sight is deviated from the observation direction suitable for observing the texture. Therefore, while keeping the tablet information terminal held in a fixed position (i.e., without moving the spatial position and orientation of the tablet information terminal), the user can easily change the line of sight to approach the observation direction suitable for observing the texture. For example, if the user initially observes the fabric from directly in front of the tablet information terminal screen, the user can recognize that observing from a slightly diagonal angle to the right is an observation direction suitable for observing the texture. By shifting the user's head to the right and changing the line of sight so that the user views the tablet information terminal screen diagonally, the appearance of the fabric image changes, making it easier to observe the texture.

[0029] <Hardware configuration> 3 is an external view of an information processing device described in this embodiment. Reference numeral 1 denotes the main body of the information processing device. The information processing device 1 includes a display 304 with a touch panel function, which is made up of a panel such as a liquid crystal or organic EL panel, and a camera 308 that is located on the same surface as the display 304. The camera 308 acquires a two-dimensional 8-bit RGB image of 1280 × 720 pixels using, for example, a CMOS sensor.

[0030] 4 is a block diagram showing the hardware configuration of the information processing device 1. In addition to the above-mentioned display 304 and camera 308, the information processing device 1 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, and an HDD (Hard Disk Drive) 305. It also includes an acceleration sensor 306, a direction sensor 307, and a NIC (Network Interface Card) 309.

[0031] The CPU 301 uses the RAM 303 as work memory to execute the OS (operating system) and various programs stored in the ROM 302, HDD 305, etc. The CPU 301 controls each component via the system bus 310. The NIC 309 connects to the Internet and inputs and outputs information to and from external devices. The CPU 301 uses the HDD 305, ROM 302, and recording media on the Internet as data storage areas. The CPU 301 also displays a user interface (UI) provided by a program on the display 304 and accepts input from the user via the touch panel of the display 304. The acceleration sensor 306 outputs acceleration in the three axes x, y, and z of the coordinate axes shown in Figure 3, for example, using a capacitance method. When the display 304 is facing directly up (toward the sky), this acceleration is output as (0, 0, g), where g is the acceleration of gravity. The orientation sensor 307 is a sensor that measures geomagnetism and outputs a three-dimensional vector indicating the direction of north in the x, y, and z coordinate system shown in Figure 2.

[0032] <Functional configuration> 5 is a configuration diagram showing the functional configuration of the information processing device of this embodiment. OS3 is an operating system, and is a set of instructions that controls input / output and launches and switches applications. Device drivers 408 are a set of instructions included in OS3, and control the display 304, camera 308, and various sensors built into the information processing device 1. Various applications can control these devices by sending predetermined commands to OS3.

[0033] The texture display application 2 is a group of commands that reads information from the HDD 305 or recording media on the Internet and displays it on the display 304. The texture display application 2 includes a texture information acquisition unit 401, an illumination information acquisition unit 402, a user gaze direction acquisition unit 403, a texture observation gaze direction calculation unit 404, an object image drawing unit 405, a texture observation gaze direction drawing unit 406, and an output image synthesis unit 407.

[0034] The texture information acquisition unit 401 acquires diffuse reflection intensity, specular reflection intensity, glossiness, and a normal map as texture information of the object. The lighting information acquisition unit 402 acquires an environment map as lighting information. Details of the environment map will be described later. The user gaze direction acquisition unit 403 estimates the position of the user's eyes relative to the information processing device 1 from the RGB image acquired by the camera 308 and acquires the gaze direction when viewing the object displayed on the display 304. The texture observation gaze direction calculation unit 404 calculates the gaze direction in which the texture of the object is easily visible from the environment map. The object image drawing unit 405 draws an image I1 representing how the object looks from the user's gaze direction when illuminated by the environment map, based on the texture information, the environment map, and the user's gaze direction. The texture observation gaze direction drawing unit 406 draws an image I2 indicating the user's gaze direction and the texture observation gaze direction. The output image synthesis unit 407 synthesizes the image I1 and the image I2 to generate an image to be displayed on the display 304.

[0035] <Processing> Fig. 6 is a flow diagram showing the processing executed by the texture display application 2. Details of the processing executed by the texture display application 2 will be explained below with reference to Fig. 6. Below, each step will be represented by adding an S before the reference numeral.

[0036] In S501, the texture information acquisition unit 401 acquires texture information, such as diffuse reflection intensity ρd(x, y, i), specular reflection intensity ρs(x, y, i), glossiness σ(x, y, i), and normal N(x, y, k), from a data storage area based on a user's instruction. Here, the data storage area refers to, for example, information pre-stored on an Internet shopping website accessed by a user about products viewed on the website. Here, x and y indicate position coordinates on the object. i indicates whether the color signal is R, G, or B. k indicates whether the normal direction is the x, y, or z component. In this embodiment, the diffuse reflection intensity, specular reflection intensity, and glossiness are arrays with 8 bits of information for each position coordinate and each color signal. The normal direction is an array with 8 bits of information for each position coordinate and each component, with each component ranging from -1 to 1 corresponding to a value from 0 to 255. The format of the texture information in this embodiment is an example, and for example, specular reflection intensity may be stored as 32-bit floating-point information.

[0037] In S502, the illumination information acquisition unit 402 acquires an environment map E(θa, φa, i) as illumination information from a data storage area based on an instruction from a user. After that, the environment map E(θa, φa, i) is rotated based on the orientation of the information processing device 1 so that the coordinate system is consistent with the real world.

[0038] Here, details of the environment map will be described using FIG. 7(a) showing a method for creating the environment map E (θa, φa, i) and FIG. 7(b) showing an example of the created environment map E. The environment map E can be acquired by an omnidirectional camera 601 installed at an arbitrary point O (for example, the center of the floor of a room or a position where the information processing device 1 is expected to be operated). Specifically, the environment map E is created by measuring the light intensity in the line of sight (θa, φa) from the arbitrary point O and storing the light intensity measured for each line of sight (θa, φa) and for each color signal i. The environment map E is an image that represents the light intensity, and is like an omnidirectional image.

[0039] Here, the position O of the imaging surface of the omnidirectional camera 601 is set as the origin, and θa and φa represent the polar angle and azimuth angle, respectively, with respect to the normal direction N of the imaging surface. In this embodiment, an environment map E (θa, φa, i) is acquired where θa = 0 for directly above (the sky) and φa = 0 for the north direction, and the environment map E stores 8-bit intensity information for each angle and each color signal. The number of pixels is set to 1920 x 1080.

[0040] In addition, instead of using the omnidirectional camera 601, the information processing device 1 may be configured so that the user holds it at an arbitrary point O and rotates it 360° at that position while photographing the surroundings, thereby obtaining an environmental map E at that position.

[0041] Note that the format of the lighting information in this embodiment is just an example; for example, the signal intensity of the environment map may be stored as 32-bit floating-point information, and the origins of the polar angle and azimuth angle may not need to be determined as described above.

[0042] Next, a three-dimensional vector d representing the north direction is acquired via the orientation sensor 307 of the information processing device 1, and a three-dimensional vector a representing the acceleration of the information processing device 1 is acquired via the acceleration sensor 306. Based on the two vectors d and a, three rotation angles (r, p, y) in the roll, pitch, and yaw directions are calculated, which represent the transformation from a reference coordinate system in which the x-axis points east, the y-axis points north, and the z-axis points directly above (sky) to a coordinate system fixed to the information processing device 1 shown in FIG. 3 . This calculation method is well known, so a description thereof will be omitted. Based on the rotation angles (r, p, y), the environment map E(θa, φa, i) is rotated so that the directly above (sky) direction and north direction of the environment map E(θa, φa, i) coincide with the directly above (sky) direction and north direction in the real space, thereby aligning the coordinate system with the real space. As a result, optical consistency with the real space can be maintained, particularly when using an environment map that reflects the ambient lighting environment during observation.

[0043] In S503, the texture observation gaze direction calculation unit 404 calculates a gaze direction suitable for observing color and glossiness from the lighting information. Fig. 8 is a flow diagram showing the details of S503. The following description will be given with reference to Fig. 8.

[0044] In S701, the user gaze direction acquisition unit 403 first acquires an RGB image including the user's face via the camera 308. Then, based on the RGB image, the gaze direction (R, Θ, Φ) from the position of the user's eyes to an object displayed at the center of the display 304 is estimated. Here, the gaze direction (R, Θ, Φ) is expressed in a three-dimensional polar coordinate system with Θ as the polar angle of the normal direction of the display 304, Φ as the azimuth angle, and the origin as the center point of the display 304. From the distance of the user's eyes from the camera 308 estimated using a well-known depth estimation technique using a monocular RGB image and the two-dimensional position of the user's eyes in the image, a vector (xr, yr, zr) pointing from the camera 308 to the eye position is obtained. Using the vector (X, Y, 0) pointing from the camera 308 to the center point of the display 304, the gaze direction (R, Θ, Φ) when viewing an object displayed on the display 304 is calculated using the following equation:

[0045]

number

[0046] From S702 to S707, the process is performed while scanning the observation direction, and when there are no more observation directions to scan, the repetition is terminated in S707.

[0047] The scanning observation direction is determined as follows. First, consider a three-dimensional polar coordinate system in which the polar angle of the average normal direction of the object is θ, the azimuth angle is φ, and the center position of the object is the origin. A hemispherical region of the environment map, which is within the range of θ from 0 to 90 degrees and φ from 0 to 360 degrees of the object, illuminates the object. Observation directions within this hemispherical region are scanned sequentially, and a process described below determines whether they are suitable for observing the color and gloss of the object. At this time, the distance from the origin of the scanning viewpoint is fixed and set to be equal to the distance R from the object to the eye obtained in S701. In this embodiment, θ and φ in the hemispherical region are each divided into 20 parts, and the scanning observation direction (θl, φl) is determined.

[0048] 9(a) is a diagram for explaining the scanning order. The scanning observation directions are ordered by the distance 801 between the scanning observation direction 803 and the acquired user gaze direction 802, and scanning is performed in order from the closest distance. This allows gaze directions closest to the user gaze direction to be presented preferentially, thereby shortening the distance the user must travel when moving from their original gaze direction to the texture observation gaze direction, which is expected to improve convenience. Note that in this embodiment, distance is used to order the scanning observation directions, but the scanning observation directions may also be ordered by, for example, the magnitude of the angle relative to the user gaze direction.

[0049] In S702, the texture observation gaze direction calculation unit 404 ranks the observation directions using the method described above and selects the observation direction that is closest to the user's gaze direction. From the second repetition onwards, the observation directions that are closest to the user are selected in order from the ranked observation directions. In this way, the direction that forms the smallest angle with the user's gaze direction is determined to be the observation gaze direction suitable for observing the texture of the object.

[0050] In S703, the texture observation line of sight direction calculation unit 404 calculates a partial area (reflected area) of the environment map that is reflected in the object 805.

[0051] Here, Fig. 9(b) is a diagram for explaining the calculation of the reflection area, and Fig. 9(c) is a diagram showing a partial area of ​​the environment map that becomes the reflection area. First, directions 806 (θjc, φjc) from the scanning gaze direction position 804 toward the four corners of a rectangular object 805 are calculated. Here, j is a subscript that represents each corner of the object 805. For example, when the vertical length of the display 304 of the information processing device 1 is d and the horizontal length is w, the object 805 is a rectangular area where -w / 4≦x≦w / 4, -d / 4≦y≦d / 4, z=0. Specifically, (θjc, φjc) are calculated using the following equations.

[0052]

number

[0053] Here, (xj, yj) represent the x and y components (-w / 4, -d / 4), (-w / 4, d / 4), (w / 4, -d / 4), and (w / 4, d / 4) of the coordinates of four points representing the four corners of the object 805. Based on the average normal direction, the direction 807 (θjr, φjr) in which each direction 806 is specularly reflected is calculated. Specifically, this is obtained using the following equation.

[0054]

number

[0055] A partial region 808 of the environment map including the four directions 807 is designated as the reflection region. As a method of designation, for example, the region of the environment map corresponding to the range of the polar angle θ of the four directions 807 from the minimum value to the maximum value and the range of the azimuth angle φ from the minimum value to the maximum value is designated as the reflection region.

[0056] In S704, the texture observation gaze direction calculation unit 404 determines a gaze direction suitable for observing the glossiness of an object. As described above, the gaze direction suitable for observing glossiness is one in which a light source is present in the specular reflection direction of the gaze and specular reflection intensity and glossiness can be visually recognized. In other words, the observation direction in which the portion of the illumination information where the illumination (light source) is present is reflected on the object is determined as the observation gaze direction suitable for observing the glossiness of the object. In this embodiment, the reflected area is grayscaled to determine the gaze direction. The environment map, which is an array containing 8-bit intensity information for each coordinate and each color signal, is converted into an array holding 8-bit luminance values ​​for each coordinate. Since the conversion method is a well-known technique, a description thereof will be omitted. Note that the format used for grayscaling is merely an example; for example, luminance values ​​may be held as 32-bit floating-point information.

[0057] When the number of pixels in the reflection area whose brightness is equal to or greater than the threshold E1 is equal to or greater than the threshold S1, the observation direction at that time is determined as the gloss observation gaze direction. Here, the threshold E1 may be, for example, the maximum brightness value of the entire environment map area minus the standard deviation of brightness values, and the threshold S1 may be, for example, 10% of the total pixels in the reflection area. This step S704 is repeatedly executed when scanning the observation direction, but once the gloss observation gaze direction has been determined, a flag is set so that determination of the gloss observation gaze direction is skipped in subsequent scans.

[0058] In S705, the texture observation gaze direction calculation unit 404 determines a gaze direction suitable for observing the object's color. As described above, a gaze direction suitable for color observation is one in which a light source is not present in the specular reflection direction of the gaze and in which diffuse reflection components are dominant compared to specular reflection components. In this embodiment, the reflection area is grayscaled, and when the maximum luminance of the reflection area is equal to or less than a threshold value E2, the observation direction at that time is determined as the color observation gaze direction. Here, the threshold value E2 can be, for example, the first quartile of the luminance of the entire environment map area. In other words, the observation direction in which a portion of the illumination information that does not have an illumination (light source) is reflected on the object is determined as the observation gaze direction suitable for observing the object's color. Step S705 is repeatedly executed when scanning the observation direction. However, once the color observation gaze direction has been determined, a flag is set to skip determining the color observation gaze direction in subsequent scans.

[0059] In S706, if both the gloss observation viewing direction and the color observation viewing direction have been determined, the scanning ends. If neither has been determined, the process proceeds to S707.

[0060] In S707, the texture observation line of sight direction calculation unit 404 determines whether or not there are other observation directions to scan. If there are other observation directions to scan, the process returns to S702. On the other hand, if there are no other observation directions to scan, the process ends.

[0061] This concludes the description of the flow chart of FIG. 8, and we will return to the description of the flow chart of FIG.

[0062] The subsequent processes from S504 to S509 are repeated until an end instruction is received from the user.

[0063] In S505, the user's line of sight direction acquisition unit 403 acquires the user's line of sight direction (R, Θ, Φ). This process is the same as that in S701, and therefore a description thereof will be omitted.

[0064] In S506, the object image rendering unit 405 uses a known image-based lighting technique to render an image I1 that represents how the object 805 looks in the user's line of sight (R, Θ, Φ) when illuminated by the environment map E(θa, φa, i). The size of the image I1 is the same as the size of the object 805 used in the calculation in S704, for example, and is 50% of the size of the display 304 of the information processing device 1. The image I1 is also rendered so that the normal to the display 304 and the average normal to the object 805 coincide with each other. At this time, the coordinate system of the user's line of sight (R, Θ, Φ) coincides with the coordinate system used in the calculation in S704.

[0065] In S507, the texture observation line of sight direction drawing unit 406 draws an image I2 that presents the texture observation line of sight direction and the user line of sight direction. Details of this process will be described later.

[0066] In S508, the output image synthesis unit 407 synthesizes the two images I1 and I2 to generate an image to be displayed on the display 304.

[0067] 10 is a detailed flow diagram of S507, and FIG. 11(a) is an example of an image output by the output image synthesis unit 407. An object image 1001 shown in FIG. 11(a) is image I1 drawn by the object image drawing unit 405. Image I2 is composed of a three-dimensional hemispherical model 1002, a user gaze direction marker 1003, a texture observation gaze direction marker 1004, and a region of interest marker 1005. The processing of S507 will be described in detail below with reference to FIG. 10.

[0068] First, in S901, the texture observation line of sight drawing unit 406 determines whether the texture observation line of sight direction has been determined in the above-mentioned processes S704 and S705. If it has been determined, the process proceeds to S903. On the other hand, if it has not been determined, the process proceeds to S902.

[0069] In S902, the texture observation gaze direction drawing unit 406 notifies the user that there is no gaze direction for texture observation, thereby encouraging the user who wants to perform texture observation to change the tablet orientation or the environment map. Here, FIG. 11(b) is a diagram showing an example of a warning display. The user is notified by a warning display 1006. In the illustrated example, it is shown that a gaze direction suitable for observing the color of the object has not been determined.

[0070] In S903, the texture observation line of sight drawing unit 406 calculates the relative angles (Δθc, Δφc) (Δθl, Δφl) between the acquired user line of sight direction (Θ, Φ) and the two types of texture confirmation line of sight directions (θc, φc) (θl, φl). The relative angles are calculated, for example, using the following formula:

[0071]

number

[0072] In S904, the texture observation gaze direction drawing unit 406 draws an image indicating the relative angle with respect to the user gaze direction calculated in S903. This image corresponds to the three-dimensional hemispherical model 1002, user gaze direction marker 1003, and texture observation gaze direction marker 1004 in Fig. 11. If the polar angle and azimuth angle of the three-dimensional hemispherical model 1002 relative to the direction from the center of the bottom surface to the zenith are T and P, respectively, the user gaze direction marker 1003 is displayed on the spherical surface at T = 0, P = 0. The texture observation gaze direction marker 1004 is displayed at two locations on the spherical surface at (T, P) = (Δθc, Δφc) and (Δθl, Δφl). The spherical surface of the three-dimensional hemispherical model is displayed, for example, as a grid in which the polar angle T and azimuth angle P are divided at regular intervals so that the general shape can be seen.

[0073] In S905, the texture observation line of sight drawing unit 406 determines the drawing transparency of the attention area marker 1005 to be drawn in S906 from the magnitude of the relative angle calculated in S903. The magnitude A of the relative angle is defined, for example, as follows:

[0074]

number

[0075] Here, Δθ and Δφ respectively represent the relative polar angle and azimuth angle between the two types of gaze directions for confirming texture and the user's gaze direction. When the magnitude of the relative angle A is large, the rendering transparency of the attention area marker 1005 is set to 0%, and as the magnitude of the relative angle A decreases, the rendering transparency of the attention area marker 1005 approaches 100%. For example, when A is 90 degrees or greater, the transparency is set to 0%, and when A is 10 degrees or less, the transparency is set to 100%, with the transparency varying linearly with the value of A between these values. As a result, the user's observation of the texture of the attention area is not obstructed by the attention area marker 1005, and when searching for the gaze direction for observing the texture, the attention area marker 1005 can help identify the area to focus on.

[0076] In S906, the texture observation line of sight drawing unit 406 draws the attention area marker 1005 based on the transparency determined in S905. The attention area marker 1005 is centered at the center position of the object image I1, and its size is set to 10% of the size of the display 304, for example.

[0077] In this way, the zenith of the spherical surface of the three-dimensional hemispherical model (e.g., 1003) is shown as the user's gaze direction, and the three-dimensional hemispherical model is drawn showing the observation gaze direction suitable for observing the texture of the object as a point on the spherical surface (e.g., 1004) as a direction relative to the user's gaze direction. By looking at this display, the user can intuitively grasp how to change their gaze direction to achieve a gaze direction suitable for observing the texture.

[0078] This concludes the description of the processing flow in FIG.

[0079] <Effects of the embodiment> As described above, in this embodiment, texture information indicating the texture of an object and lighting information (environment map information) for illuminating the object are acquired. Then, based on this information, an observation gaze direction suitable for observing the texture of the object (a gaze direction that makes it easy to visually recognize the color and glossiness of the object) is calculated, and information indicating this observation gaze direction is drawn and presented to the user.

[0080] As a result, the user can easily determine from which direction to observe, and can effectively check the color and texture, such as glossiness, of the object.

[0081] <Modification> 6 is executed after the environment map and texture information are acquired in response to a user instruction, but the processing order may be changed as follows: The texture observation gaze direction is calculated in advance from the environment map and texture information using a process similar to S503 and stored in a data storage location. Thereafter, when the environment map and subject information are acquired in response to a user instruction, the corresponding texture observation gaze direction is simultaneously acquired from the data storage location and presented.

[0082] In S502, an environment map was obtained as lighting information, but it is also possible to obtain the three-dimensional position, direction, and intensity information of the light source as lighting information and convert it into an environment map by performing baking processing, which is a common CG rendering technique.

[0083] In S505 and S701, the user's gaze direction is obtained using an RGB camera built into the information processing device. However, the gaze direction may be estimated using a distance sensor in combination, or parallax information obtained from multiple cameras may be used. A camera other than a visible light camera, such as an infrared camera, may also be used. Furthermore, the camera does not have to be built-in as long as the relative position with respect to the display can be obtained. Alternatively, the user's gaze direction may be obtained using an eye-tracking device.

[0084] The calculation method for the reflected area described in S704 is an example, and the effect of changes in the reflected area due to variations in the normal of the object may be incorporated, for example, by expanding the partial area of ​​the environment map in accordance with variations in the normal of the object. Also, the reflected area taking into account the normal direction for each position of the object may be found by actually tracing rays in the reverse direction for each point on the object.

[0085] (Embodiment 2) In the first embodiment, an example was described in which a gaze direction suitable for observing the color and glossiness of a subject is calculated and presented. In the present embodiment, an example will be described in which a gaze direction suitable for observing the texture of a subject is calculated and presented.

[0086] The appearance, hardware configuration, functional configuration, and processing flow other than S503 of the information processing device in the second embodiment are the same as those in the first embodiment, and therefore will not be described further. Details of the processing of S503 in this embodiment are shown in Fig. 12. Details of the processing will be described below with reference to Fig. 12.

[0087] <Processing> In S1101, the user's line of sight direction acquisition unit 403 acquires the user's line of sight direction. This process is the same as S701, and therefore a description thereof will be omitted.

[0088] In S1102, the texture observation gaze direction calculation unit 404 determines the strength of the glossiness of the object. In this embodiment, the diffuse reflection intensity ρs and the specular reflection intensity ρd are converted to grayscale to obtain an array that holds, for example, 8-bit brightness value information for each coordinate. It is then determined whether the ratio of the average specular reflection brightness to the average diffuse reflection brightness over the entire area of ​​the object is equal to or less than a threshold value R1. Here, the threshold value R1 is set to 1, for example.

[0089] The processes from S1103 to S1108 are steps for calculating the gaze direction suitable for observing the texture when the object has a strong glossiness. As described above, the gaze direction suitable for observing the texture in this case is the direction of specular reflection from a small light source or a light source with a clear edge. When such light sources are present in the reflected area, the reflected area contains many components with high spatial frequencies. Therefore, the presence or absence of such light sources can be determined by calculating the spatial frequency information of the reflected area.

[0090] The processing of S1103, S1104, and S1108 is the same as that of S702, S703, and S707, respectively, and therefore the description thereof will be omitted.

[0091] In S1105, the texture observation line of sight direction calculation unit 404 performs two-dimensional FFT processing on the reflection area to convert it into spatial frequency information.

[0092] In S1106, the texture observation gaze direction calculation unit 404 determines whether the observation direction is suitable for observing the texture feel. When the spatial frequency components of the reflection area include a threshold value F1 or more, in absolute value, of which the number of components is equal to or greater than the threshold value D1, the observation direction (θb, φb) at that time is determined as the texture feel observation gaze direction. The threshold value F1 is set to, for example, 10 times the reciprocal 2 / w of the width of the object 805, and the threshold value D1 is set to, for example, 10% of all frequency components included in the reflection area 808.

[0093] In this way, when the specular reflection intensity is equal to or greater than the diffuse reflection intensity, the observation direction in which the portion of the illumination information containing spatial frequency components above a threshold is reflected on the object is determined to be the observation line of sight direction suitable for observing the unevenness of the object.

[0094] In S1107, if the roughness observation line of sight direction is determined in S1106, the texture observation line of sight direction calculation unit 404 ends scanning of the observation direction. On the other hand, if the roughness observation line of sight direction is not determined, the process proceeds to S1108.

[0095] Meanwhile, the processes from S1109 to S1111 are steps for calculating a gaze direction suitable for observing the textured appearance when the object has a weak glossiness. As described above, the gaze direction suitable for observing the textured appearance when the object has a weak glossiness is a direction perpendicular to the light source. However, an environment map may contain multiple light sources. Therefore, in this embodiment, the gaze direction suitable for observing the textured appearance when the object has a weak glossiness is an observation direction that is orthogonal to the direction of the strongest light source contained in the environment map and the direction obtained by integrating the other light sources.

[0096] Here, FIG. 13 is a diagram for explaining a method for calculating a line of sight direction suitable for observing the texture of an object having a weak glossiness.

[0097] In S1109, the texture observation viewing direction calculation unit 404 converts a partial region of the environment map corresponding to 0°≦θ≦90°, where θ is the polar angle of the average normal direction of the object and φ is the azimuth angle, into a grayscale image. Then, θ and φ are each divided into ten, for example, to divide into small regions 1201. When the small regions 1201 are expressed as θ≦θ≦θ2 and φ1≦φ≦φ2 using parameters θ1, θ2, φ1, and φ2, a vector 1202 whose magnitude is the integral value of the brightness of the small region and whose direction is (θ12, φ12) expressed by the following equation is calculated for each small region.

[0098]

number

[0099] In S1110, the texture observation viewing direction calculation unit 404 defines the vector with the largest magnitude among the vectors calculated for each small region as the main light source direction L1, and defines the sum of the vectors of each small region excluding the main light source direction L1 as the secondary light source direction L2.

[0100] In S1111, the texture observation gaze direction calculation unit 404 determines the gaze direction (θb, φb) closest to the user gaze direction (Θ, Φ) among the gaze directions perpendicular (orthogonal) to both the main light source direction L1 and the secondary light source direction L2 as the roughness texture observation gaze direction.

[0101] In this way, when the specular reflection intensity is smaller than the diffuse reflection intensity, the observation direction perpendicular to the light source included in the illumination information is determined as the observation line direction suitable for observing the texture of the object.

[0102] This concludes the description of the processing flow in FIG.

[0103] <Effects of the embodiment> As described above, the information processing device according to this embodiment calculates the gaze direction in which the unevenness of an object can be easily recognized from the environment map information and presents the direction to the user, thereby enabling the user to effectively recognize the unevenness of the object.

[0104] <Modification> In this embodiment, in S1103, the determination of whether the object is highly glossy is made based on the ratio of specular reflection intensity to diffuse reflection intensity, but the determination may also be made based on other function values ​​of specular reflection intensity and diffuse reflection intensity based on, for example, visual characteristics.

[0105] Furthermore, in S1105, the frequency transformation method is not limited to two-dimensional FFT, and may be, for example, Wavelet transformation.

[0106] (Embodiment 3) In this embodiment, as in the first and second embodiments, a tablet information terminal is used as the information processing device. Fig. 14 shows an example of use of this embodiment. An object 1401, which is a three-dimensional shape model to which texture information has been added, is displayed so as to be fixed at coordinates in real space, and an image 1402 that reproduces how it will look when illuminated by an environment map is displayed. In addition, an application is envisioned in which a direction perpendicular to the surface of the display 304 of the information processing device 1 is calculated as the user's gaze direction 1403 from the position and orientation of the information processing device 1, and the appearance is dynamically changed in response to dynamic changes in the gaze direction to observe the texture of the object 1401. In this embodiment, a gaze direction suitable for observing the texture of the object 1401 is presented in the application.

[0107] The hardware configuration of the information processing device according to this embodiment is the same as that of the first embodiment, and therefore a description thereof will be omitted.

[0108] <Functional configuration> 15 is a diagram showing the functional configuration of an information processing device according to this embodiment. The texture display application 2 of the information processing device 1 according to this embodiment further includes a shape information acquisition unit 409, a partial region designation unit 410, and a texture observation mode switching unit 411 in addition to the components shown in embodiment 1.

[0109] The shape information acquisition unit 409 acquires shape information that represents the three-dimensional shape of the object. Based on an instruction from the user, the texture observation mode switching unit 411 can transition from a first display mode that displays only the appearance of the object to a second display mode that presents not only the appearance of the object but also a gaze direction suitable for texture observation. The partial area designation unit 410 designates a partial area on the object shape to be used by the texture observation gaze direction calculation unit 404. The other components are the same as those assigned the same numbers in embodiment 1, and therefore description thereof will be omitted.

[0110] <Processing> 16 is a flowchart showing the processing executed by the texture display application 2 of the information processing device 1 according to this embodiment. Details of the processing will be described below with reference to FIG.

[0111] In S1601, the texture information acquisition unit 401 acquires texture information of the object. This process is the same as S501, and therefore a description thereof will be omitted.

[0112] In S1602, the shape information acquisition unit 409 acquires shape information that represents the three-dimensional shape of the object. The shape information is, for example, three-dimensional shape data written in Wavefront OBJ format.

[0113] In S1603, the illumination information acquisition unit 402 acquires an environment map E(θa, φa, i) as illumination information based on an instruction from the user. The data format is the same as that described in S502, but in this embodiment, the environment map must also be rotated dynamically to dynamically change the attitude of the information processing device 1. Therefore, rotation is not performed during this process.

[0114] The processing of S1604 and S1605 is processing for realizing the first display mode in which only the appearance of the object is displayed. In S1604, the user's gaze direction acquisition unit 403 acquires the attitude of the information processing device 1 via the acceleration sensor 306 and the orientation sensor 307. The negative direction of the z axis of the coordinate system fixed to the information processing device 1 in FIG. 3 is set as the gaze direction. The gaze direction of the information processing device 1 acquired when processing S1604 for the first time is set as the initial gaze direction, and the position of the information processing device 1 is set as the initial gaze position. The initial gaze position cannot be acquired directly, but the relative position from the initial gaze position can be acquired by integrating the acceleration vector a output by the acceleration sensor 306.

[0115] In S1605, the object image rendering unit 405 renders an image I1 that represents the appearance of an object fixed in real space from the user's line of sight and line of sight, illuminated by an environment map fixed in real space. The virtual three-dimensional coordinate system in which the object exists and the environment map are rotated according to the attitude of the information processing device 1, and the virtual three-dimensional coordinate system in which the object exists is translated according to the relative position from the initial line of sight, thereby displaying the environment map and the object as if they were fixed in real space. The environment map is displayed so that the direction directly above (sky) and the direction north of the environment map coincide with the direction directly above (sky) and the direction north in real space. The details of this processing are not the focus of the present invention, so a detailed description will be omitted.

[0116] In S1606, the texture observation mode switching unit 411 determines whether to switch to the second display mode, which presents a line of sight direction suitable for observing the texture, based on an instruction from the user. If an instruction from the user is received, the switching is performed. If an instruction from the user is not received, the process returns to S1604.

[0117] 17 shows an example of an image displayed in the display area of ​​the display 304 of the information processing device 1. In the first display mode, an image 1702 showing how an object illuminated by an environment map looks from the current user's line of sight is displayed in the display area 1701 of the display 304. When the user presses the texture confirmation button 1703, the display mode transitions to the second display mode.

[0118] In S1607, the partial area designation unit 410 designates an area (region of interest) on the object surface where the user will check the texture. The region of interest must be designated in order to determine the gaze direction for texture confirmation in the next step, S1608. In this embodiment, the region of interest is set to the area around a point (xc, yc, zc) on the object surface that is displayed at the center of the screen when the button is pressed. A square region is set with the point on the object surface as its center and a normal direction that is the same as the normal direction at the point on the object surface. The side length 1 of this square is set to 1 / 100 of the largest of the height, width, and depth of the entire object. However, this method of setting the region of interest is merely an example, and the center of the region of interest may be determined, for example, by the user selecting a partial area on the image of the object displayed on the display 304. The region of interest may also be set using, for example, a rectangle or a circle, and the size of the region of interest may be determined, for example, according to the curvature at the point on the object surface, or by the sum of the distances between each point on the object surface and the region of interest.

[0119] In S1608, the texture observation gaze direction calculation unit 404 determines a gaze direction suitable for observing the texture. This process is the same as the process of S503 in the second embodiment, but the size of the object in the second embodiment is calculated by replacing it with the length 1 of the side of the region of interest.

[0120] Then, the processes from S1609 to S1614 are repeated until a stop instruction is received from the user.

[0121] The processing in S1610 and S1611 is the same as that in S1604 and S1605, respectively, and therefore a description thereof will be omitted.

[0122] In S1612, the texture observation line of sight drawing unit 406 draws an image I2 that presents the texture observation line of sight. Image I2 includes at least one of a three-dimensional arrow model 1705 indicating the texture observation line of sight in FIG. 17, a two-dimensional plane model 1706 indicating the texture observation line of sight, and a region of interest marker 1707. FIG. 18 is a diagram showing a detailed processing flow of S1612. The following will be described with reference to FIG. 18. Since S1801, S1802, S1803, and S1804 are the same as S901, S902, S903, and S905 described in the first embodiment, respectively, description thereof will be omitted.

[0123] In S1805, the image I2 is drawn with the rendering transparency determined in S1804. Specifically, the closer the user's line of sight direction, which is the line of sight direction of the user observing the information processing device 1, is to an observation line of sight direction suitable for observing the texture of the object, the lower the rendering transparency of at least one of the three-dimensional arrow model 1705 or the two-dimensional plane model 1706 is performed.

[0124] For the three-dimensional arrow model 1705, the start point vi and end point vf of the arrow are expressed using the center point (xc, yc, zc) of the region of interest, the length l of the side of the region of interest, and the texture observation viewing direction (θb, φb). Here, the texture observation viewing direction (θb, φb) is obtained by converting the texture observation viewing direction calculated in S1608 into a polar angle and azimuth angle in a three-dimensional polar coordinate system, which has the center point of the region of interest as the origin and the polar angle and azimuth angle relative to the direction directly above (sky) as two angular coordinates. Specifically, the start point and end point of the arrow are determined using the following equations.

[0125]

number

[0126] Each coordinate represents the x-coordinate, y-coordinate, and z-coordinate in the virtual three-dimensional coordinate system where the object exists.

[0127] Furthermore, with regard to the two-dimensional plane model 1706 indicating the gaze direction for texture observation, a plane expressed by the following formula is drawn as a plane that passes through the center point (xc, yc, zc) of the region of interest and is perpendicular to the gaze direction for texture observation (θb, φb). The size is, for example, 20 times the length l of the side of the region of interest.

[0128]

number

[0129] A square model representing the region of interest marker 1707 is displayed. The three-dimensional arrow model 1705, two-dimensional plane model 1706, and region of interest marker 1707, which indicate the direction of the gaze for texture observation, are superimposed on the virtual three-dimensional coordinate system where the object exists, and are drawn taking into consideration the occlusion relationship with the object. In this embodiment, in which the user's eyes are always directly facing the display 304, this method of presenting the texture observation direction allows the user to grasp the texture observation direction more intuitively than the method of presenting the texture observation direction in embodiment 1.

[0130] The method for determining the model position and orientation that represent the gaze direction for texture observation is not limited to the method described above, and other positions and orientations may be used as long as they are sufficient to present the gaze direction for texture observation. This concludes the explanation of the processing flow in Figure 18, and we will now return to the explanation of the processing flow in Figure 16.

[0131] In S1613, the output image synthesis unit 407 synthesizes the image I1 drawn by the object image drawing unit 405 and the image I2 drawn by the texture observation gaze direction drawing unit 406 to generate an image to be displayed on the display 304 of the information processing device 1.

[0132] This completes the description of the processing flow in FIG.

[0133] <Effects of the embodiment> As described above, the information processing device according to this embodiment calculates the gaze direction that makes it easy to visually recognize the texture of an object having a three-dimensional shape from the environment map information and presents the direction to the user, thereby enabling the user to effectively recognize the texture of the object.

[0134] <Modification> In this embodiment, a viewing direction suitable for observing the textured feel is presented, but a viewing direction suitable for observing other texture elements may also be displayed, and multiple viewing directions for observing texture may be displayed simultaneously or switched at the user's instruction.

[0135] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0136] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0137] 1: Information processing device, 401: Texture information acquisition unit, 402: Lighting information acquisition unit, 403: User gaze direction acquisition unit, 404: Texture observation gaze direction calculation unit, 405: Object image drawing unit, 406: Texture observation gaze direction drawing unit, 407: Output image synthesis unit

Claims

1. a texture information acquisition means for acquiring texture information indicating the texture of an object; an illumination information acquisition means for acquiring illumination information for illuminating the object; a gaze direction calculation means for calculating an observation gaze direction for observing the texture of the object based on the texture information and the illumination information; a drawing means for drawing information indicating the observation line of sight direction; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the texture information includes information on diffuse reflection intensity and specular reflection intensity.

3. 3. The information processing device according to claim 2, wherein, when the specular reflection intensity is equal to or greater than the diffuse reflection intensity, the gaze direction calculation means determines, as the observation gaze direction for observing the unevenness of the object, an observation direction in which a portion of the illumination information having spatial frequency components equal to or greater than a threshold is reflected on the object.

4. 4. The information processing device according to claim 2, wherein when the specular reflection intensity is smaller than the diffuse reflection intensity, the gaze direction calculation means determines an observation direction perpendicular to a light source included in the illumination information as the observation gaze direction for observing the unevenness of the object.

5. 5. The information processing device according to claim 2, wherein the gaze direction calculation means determines an observation direction in which a portion of the illumination information in which no illumination is present is reflected on the object as the observation gaze direction for observing the color of the object.

6. 6. The information processing device according to claim 2, wherein the gaze direction calculation means determines an observation direction in which a portion of the illumination information where illumination is present is reflected on the object as the observation gaze direction for observing the glossiness of the object.

7. The information processing device according to any one of claims 1 to 6, characterized in that the drawing means draws at least one of information indicating a viewing direction for observing the glossiness of the object, information indicating a viewing direction for observing the texture of the object, and information indicating a viewing direction for observing the color of the object.

8. 8. The information processing apparatus according to claim 1, wherein the drawing means displays a warning when there is no observation line of sight direction for observing the texture of the object.

9. 9. The information processing apparatus according to claim 1, further comprising: a user line of sight direction acquisition unit for acquiring a user line of sight direction of a user observing the information processing apparatus.

10. 10. The information processing apparatus according to claim 9, wherein the gaze direction calculation means determines a direction that forms a small angle with the user's gaze direction as the observation gaze direction for observing the texture of the object.

11. The information processing device according to claim 9 or 10, characterized in that the drawing means draws the three-dimensional hemispherical model in such a way that the zenith of the spherical surface of the three-dimensional hemispherical model is indicated as the user's gaze direction, and an observation gaze direction for observing the texture of the object is indicated as a point on the spherical surface as a direction relative to the user's gaze direction.

12. a shape information acquisition means for acquiring shape information representing the shape of the object; a partial area designation means for designating a partial area of ​​the shape information; 12. The information processing apparatus according to claim 1, further comprising:

13. 13. The information processing apparatus according to claim 12, wherein the partial area designation means designates the partial area in response to a designation received from a user of the information processing apparatus.

14. 14. The information processing apparatus according to claim 12, wherein the gaze direction calculation means calculates an observation gaze direction for observing the texture of the object by referring to the partial region.

15. 15. The information processing apparatus according to claim 12, wherein the drawing means further draws the partial region.

16. 16. The information processing device according to claim 1, wherein the drawing means draws at least one of an arrow model pointing in the same direction as the observation line of sight for observing the texture of the object, and a plane model perpendicular to the observation line of sight for observing the texture of the object.

17. The information processing device according to claim 16, characterized in that the drawing means reduces the drawing transparency of at least one of the arrow model or the plane model as the user's line of sight, which is the line of sight of the user observing the information processing device, approaches the observation line of sight for observing the texture of the object.

18. A control method for an information processing device, comprising: a texture information acquisition step of acquiring texture information indicating the texture of an object; an illumination information acquisition step of acquiring illumination information for illuminating the object; a gaze direction calculation step of calculating an observation gaze direction for observing the texture of the object based on the texture information and the illumination information; a drawing step of drawing information indicating the observation line of sight direction; 1. A method for controlling an information processing device, comprising:

19. A program for causing a computer to function as the information processing device according to any one of claims 1 to 17.

Images