Display device and display method

Abstract

A display device and a display method, comprising: a generating unit, which generates, based on an image of a certain object, generated images when the object is observed from multiple angles in such a manner that the angle of the object relative to a virtual light source is changed; a selecting unit, which selects, as a selected image, a first image in which the object is observed from a first angle in the generated images; a converting unit, which converts the first image into a converted image in which the object is observed from a second angle different from the first angle, while maintaining a positional relationship between the virtual light source and the object in the first image; and a display unit, which displays the converted image.

Description

Technical Field

[0001] The invention relates to a display device and a display method. Background Art

[0002] Conventionally, as a technology related to a display device, for example, a technology disclosed in Patent Document 1 has been proposed.

[0003] Patent document 1 is a device constructed in the following manner, that is, comprising: a diffuse reflection image acquisition unit, which acquires a diffuse reflection image of the object surface; a mirror reflection image acquisition unit, which acquires a mirror reflection image of the object surface; a differential image acquisition unit, which acquires a differential image between the diffuse reflection image and the mirror reflection image; a reflectance distribution function calculation unit, which uses the diffuse reflection image and the differential image to calculate the reflectance distribution function of the object surface; and a display unit, which uses the reflectance distribution function to display the reflection color of the object surface corresponding to the change in the orientation of the object surface.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2019-082838 Summary of the invention

[0005] An object of the present invention is to be able to display an image in which an object is observed from another angle while maintaining the positional relationship between a virtual light source and the object.

[0006] The invention described in claim 1 is a display device comprising: a generating unit that generates, based on an image of a certain object, generated images when the object is observed from a plurality of angles in such a manner that the angle of the object relative to a virtual light source is changed;

[0007] a selection unit that selects, from among the generated images, a first image in which the object is observed from a first angle as a selected image;

[0008] a conversion unit that converts the first image into a converted image in which the object is observed from a second angle different from the first angle, while maintaining a positional relationship between the virtual light source and the object in the first image;

[0009] The display unit displays the conversion image.

[0010] The invention described in claim 2 is the display device described in claim 1, further comprising a designating unit that designates the second angle.

[0011] The invention described in claim 3 is the display device described in claim 2, wherein the image of the object that falls within a preset angle range relative to the second angle designated by the designating unit is set as the image of the object observed from the second angle.

[0012] The invention described in claim 4 is the display device described in claim 2, wherein the designation unit automatically designates an image having the largest area of ​​the image of the object as the image of the object observed from the second angle.

[0013] The invention described in claim 5 is the display device described in claim 3, wherein the image of the object observed from the second angle is a planar image of the object.

[0014] The invention described in claim 6 is characterized in that, in the display device described in claim 1, an adjustment unit is provided for adjusting the position and illumination intensity of the virtual light source when the display unit that also displays the generated image displays the generated image and the converted image.

[0015] The invention described in claim 7 is the display device described in claim 6, wherein the virtual light source includes a point light source and a parallel light source.

[0016] The invention described in claim 8 is the display device described in claim 1, wherein the display unit also displays the selected image.

[0017] The invention described in claim 9 is the display device described in claim 1, wherein the display unit displays the generated images observed from the plurality of angles.

[0018] The invention described in claim 10 is the display device described in claim 9, wherein the display unit displays the generated images observed from the plurality of angles in a designated order.

[0019] The invention described in claim 11 is the display device described in claim 9, wherein the display unit rearranges and displays the generated images observed from the plurality of angles in such a manner that the amount of change becomes gentle.

[0020] The invention described in Scheme 12 is such that in the display device described in Scheme 1, the generating unit generates an image to which a gloss component is added, multiplied by a coefficient to which a slope is added according to an area on the surface of the object, when generating an image of the object when the object is observed from multiple angles in a manner that changes the angle of the object relative to the imaginary light source.

[0021] The invention described in claim 13 is the display device described in claim 12, wherein the slope corresponding to the area on the surface of the object is composed of a parallel linear slope corresponding to the surface light source and a concentric circular slope corresponding to the point light source.

[0022] The invention described in Scheme 14 is a display method, which includes the following steps: a generation step, based on an image of a certain object, generating an image when the object is observed from multiple angles in a manner that changes the angle of the object relative to an imaginary light source; a selection step, in which, in the generated image, a first image in which the object is observed from a first angle is selected as a selection image; a conversion step, in which a state in which the positional relationship between the imaginary light source and the object in the first image is maintained is converted into a conversion image in which the object is observed from a second angle different from the first angle; and a display step, in which the conversion image is displayed.

[0023] Effects of the Invention

[0024] According to the first aspect of the present invention, it is possible to display an image obtained by observing the object from another angle while maintaining the positional relationship between the virtual light source and the object.

[0025] According to the second aspect of the present invention, compared with a case where no designating unit for designating the second angle is provided, an image observed from a designated angle can be displayed.

[0026] According to the third aspect of the present invention, even if a detailed designation is not made when designating, an intended angle can be designated.

[0027] According to the fourth aspect of the present invention, it is possible to save the time and effort required for user selection.

[0028] According to the fifth aspect of the present invention, a two-dimensional image of an object can be automatically displayed.

[0029] According to the sixth aspect of the present invention, the evaluation of the object image is improved compared to a case where no adjustment unit for adjusting the position and illumination intensity of the virtual light source is provided.

[0030] According to the seventh aspect of the present invention, the width of the image representing the object becomes wider compared to the case where the virtual light source does not include a point light source and a parallel light source.

[0031] According to the eighth aspect of the present invention, the image of the object can be selected more easily than in a case where the display unit does not display the image of the object generated by the generating unit.

[0032] According to the ninth aspect of the present invention, compared with a case where the display unit does not display the images of the object observed from multiple angles generated by the generating unit, it is possible to select an arbitrary image from the images of the object observed from multiple angles.

[0033] According to the tenth aspect of the present invention, the image can be selected more easily than in a case where the display unit displays only the images of the object viewed from a plurality of angles generated by the generating unit.

[0034] According to the eleventh aspect of the present invention, compared with a case where the display unit randomly displays images of the object viewed from a plurality of angles generated by the generating unit, changes in the image of the object can be easily recognized.

[0035] According to the twelfth scheme of the present invention, compared with a case where the generating unit does not generate an image to which a gloss component is added by multiplying a coefficient to which a slope is added according to an area on the surface of the object when generating an image of the object when the object is observed from multiple angles in a manner that changes the angle of the object relative to a virtual light source, a gloss etc. that is closer to the actual one can be displayed.

[0036] According to the thirteenth aspect of the present invention, the slope can be easily set compared to a case where the slope corresponding to the area on the surface of the object is not composed of parallel linear slopes corresponding to the surface light source and concentric circular slopes corresponding to the point light source.

[0037] According to the fourteenth aspect of the present invention, it is possible to display an image of the object observed from another angle while maintaining the positional relationship between the virtual light source and the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Embodiments of the present invention will be described in detail with reference to the following drawings.

[0039] Figure 1 is a schematic structural diagram showing an image display system to which the display device according to Embodiment 1 of the present invention is applied;

[0040] Figure 2 is a block diagram showing a server device;

[0041] Figure 3 is a block diagram showing a user terminal device;

[0042] Figure 4 is a block diagram showing a main part of a display device according to Embodiment 1 of the present invention;

[0043] Figure 5 is a three-dimensional structural diagram showing a tablet terminal device;

[0044] Figure 6 It is a structural diagram representing a texture scanner;

[0045] Figure 7 It is an explanatory diagram showing the geometric conditions for applying Lambert's hypothetical model;

[0046] Figure 8 It is a graph showing the measured BRDF;

[0047] Fig. 9 is an explanatory diagram showing types of imaginary light sources;

[0048] Fig.10 is an explanatory diagram showing the geometric conditions of the hypothetical model to which Phong is applied;

[0049] Fig.11 is a schematic diagram representing specular reflection;

[0050] Fig.12 is an example of estimating parameters in a reflection model, which is expressed in the same way as Figure 8 A plot of the predicted BRDF fitted to the measured BR DF at low coverage as shown;

[0051] Fig.13 is a block diagram showing a main part of a display device according to Embodiment 1 of the present invention;

[0052] Fig.14 It is a three-dimensional structural diagram that shows the three-dimensional configuration of an object;

[0053] Fig.15 It is a three-dimensional structural diagram that shows the three-dimensional configuration of an object;

[0054] Fig.16 It is a structural diagram representing an image of an object;

[0055] Fig.17 It is a three-dimensional structural diagram that shows the three-dimensional configuration of an object;

[0056] Fig.18 It is a structural diagram representing an image of an object;

[0057] Fig.19 It is an illustration showing the rotation of an object;

[0058] Fig. 20 It is an illustration of an image representing an object;

[0059] Fig.21 It is an illustration of an image representing an object;

[0060] Fig. 22 It is a three-dimensional structural diagram that shows the three-dimensional configuration of an object;

[0061] Fig.23 is a block diagram showing a main part of a display device according to Embodiment 2 of the present invention;

[0062] Fig.24 It is an explanatory diagram showing an image of an object displayed by the display device according to the second embodiment of the present invention.

[0063] Explanation of symbols

[0064] 1-image display system, 20-display device, 205-display unit, 206-acquisition unit, 207-generation unit, 208-selection unit, 209-conversion unit. DETAILED DESCRIPTION

[0065] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0066] [Implementation Method 1]

[0067] Figure 1 This is a diagram showing an image display system to which the display device according to Embodiment 1 is applied.

[0068] <Overall structure of image display system using display device>

[0069] In order to take the possibilities of on-demand printing to a new level, it is becoming possible to print on-demand images such as shells, butterfly and dragonfly wings, belts woven with gold or silver threads, and Japanese paintings drawn with gold or silver foil, and images based on various colors that change due to changes in light, viewing angles and viewpoints.

[0070] Along with this, display devices that display images are also required to be able to display the texture of the surface of an object, such as glossiness and unevenness.

[0071] As an image display system 1 to which the display device according to the first embodiment is applied, Figure 1 As shown, the following image display system can be cited, which includes: a user terminal device 2, which is an example of a display device that can display an object with gloss or convexity and concavity using 3DCG (Three-Dimensional Computer Graphics) or the like; a tablet terminal device 7, which can communicate via a wireless LAN (not shown) and can display an object with gloss or convexity and concavity using 3DCG or the like; a server device 4, to which the user terminal device 2 or the tablet terminal device 7 is connected via a network 3; a texture scanner 6, which is an example of an image reading device that reads an image of an object with gloss or convexity and the like; and an image forming device 5, which prints image information as needed.

[0072] The user terminal device 2 is constituted by, for example, a personal computer. However, the user terminal device 2 is not limited to a personal computer, and may be a computer including one or more processors, a memory, an input / output interface, a communication interface, and a display unit.

[0073] Figure 2 This is a block diagram showing the server device 4 to which the image display system 1 according to the first embodiment is applied.

[0074] like Figure 2As shown, the server device 4 includes a control unit 401 as an example of a control unit, a storage unit 402 , an operation unit 403 , and a communication unit 404 .

[0075] The control unit 401 has a CPU (Central Processing Unit) or a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 401 performs reading or reading of a database composed of a plurality of 3DCG data, etc., which is created by a 3DCG program stored in a storage unit 402 composed of a hard disk or the like. The control unit 401 controls the storage unit 402, the operation unit 403 or the communication unit 404, and is connected to the user terminal device 2, the tablet terminal device 7, the texture scanner 6 or the image forming device 5 via the communication unit 404 and the network 3.

[0076] Figure 3 1 is a block diagram showing a user terminal device as an example of a display device in the image display system 1 according to the first embodiment. Figure 3 The user terminal device 2 shown is the same.

[0077] like Figure 3 As shown, the user terminal device 2 includes a control unit 201 as an example of a control unit, a storage unit 202 , an operation unit 203 , a communication unit 204 , and a display unit 205 .

[0078] The control unit 201 has a CPU (Central Processing Unit) or a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 201 executes a 3DCG program read from the server device 4 or pre-stored in the storage unit 202 composed of a hard disk or the like, and performs various functions as a display device. The control unit 201 controls the storage unit 202, the operation unit 203, and the communication unit 204 or the display unit 205, and transmits and receives data with the server device 4 and the image forming device 5 via the communication unit 204.

[0079] And, if Figure 4As shown in FIG. 1 , the control unit 201 functions as the following units: an acquisition unit 206 that acquires image information of an object; a generation unit 207 that generates an image equivalent to the generated image in this embodiment, that is, an image of the object when the object is observed from multiple angles in a manner that changes the angle of the object relative to the virtual light source from the image of the object; a selection unit 208 that selects the selected image in this embodiment, that is, the image of the object observed from the first angle, from the image of the object generated by the generation unit 207; and a conversion unit 209 that converts the image of the object selected by the selection unit 208 into the converted image in this embodiment, that is, the image of the object observed from the second angle different from the first angle while maintaining the positional relationship between the virtual light source and the object. The image of the object converted by the conversion unit 209 is displayed by the display unit 205. In addition, the display unit 205 of this embodiment displays not only the converted image but also the generated image on the display unit 205, and when the selected image is displayed, selection can be made while observing the display unit 205.

[0080] The selection unit 208 is composed of, for example, a keyboard or a mouse as an example of the operation unit 203 of the user terminal device 2. In addition, the display unit 205 is composed of a liquid crystal display or an organic electroluminescent (EL) display of the user terminal device 2. Figure 5 As shown, the tablet terminal device 7 has a display unit 71 composed of a liquid crystal display or an organic electroluminescent (EL) display, which is the same as the display unit 205 of the user terminal device 2, but a touch panel 72 is provided on the surface of the display unit 71 as an example of an operating unit, which is operated by the user by touching it with a finger.

[0081] The configuration of the user terminal device 2 as an example of a display device will be described later.

[0082] Figure 6 This is a diagram showing the structure of a texture scanner as an example of an image reading device for acquiring an image of a planar object.

[0083] like Figure 6 As shown, the texture scanner 6 optically reads the surface characteristics of a planar object 10 as an example of an object, and outputs an image signal (image data) as a result of the reading. The image signal output by the texture scanner 6 includes, in addition to the color image signal of the planar object 10, a diffuse reflection image signal based on diffuse reflection light and a mirror reflection image signal based on mirror reflection light.

[0084] The texture scanner 6 includes a platen glass 11 composed of a transparent glass plate or the like, on the upper end of which a planar object 10 is placed. The texture scanner 6 may also include a platen cover (not shown) that blocks external light by covering the planar object 10 placed on the platen glass 11. A reciprocating table 12 that can move along the surface of the platen glass 11 in the direction indicated by the arrow A is arranged below the platen glass 11. The reciprocating table 12 is equipped with first light sources 13 and 14 for reading color images and diffuse reflection images; a second light source 15 for reading mirror reflection images; an imaging optical system 16 composed of an imaging lens or a reflective mirror (not shown); a sensor 17, etc.

[0085] The texture scanner 6 is provided with components such as the first light source 13, 14, the second light source 15, the sensor 17 and the reciprocating stage 12 in a predetermined width in a direction perpendicular to the drawing. The direction perpendicular to the drawing is the main scanning direction of the texture scanner 6. In addition, the direction indicated by the arrow A in the figure is the moving direction of the reciprocating stage 12, that is, the sub-scanning direction of the texture scanner 6.

[0086] The reciprocating table 12 is configured to move in the sub-scanning direction A at a preset reading speed when reading an image of the planar object 10. The first light source 13 is a front light source, and irradiates light for reading diffusely reflected light from the planar object 10 by the sensor 17 by irradiating light at a first incident angle of, for example, 45° relative to the normal direction in the reading position of the planar object 10. The first light source 14 is a rear light source, and irradiates light for reading diffusely reflected light from the planar object 10 by irradiating light at a first incident angle of, for example, 45° relative to the normal direction of the planar object 10. Furthermore, the second light source 15 is a rear light source, and irradiates light for reading specularly reflected light from the planar object 10 by irradiating light at a second incident angle of, for example, 10° relative to the normal direction in the reading position of the planar object 10.

[0087] The second light source 15 is disposed at a position where the main light beam of the reflected light is not blocked. In the first embodiment, the incident angle of the light irradiated by the second light source 15 is set to 10°. This is not limited to the above, and the incident angle of the light irradiated by the second light source 15 can also be set to a small angle of about 5° to 10°. The light traveling in the normal direction of the reading position of the planar object 10 among the reflected light irradiated by the second light source 15 is read by the sensor 17 via the imaging optical system 16 such as an imaging lens.

[0088] Furthermore, the second light source 15 preferably emits light at a narrow angle, for example. When the angle of the light emitted by the second light source 15 is large, a cover or the like that limits the angle of the light emitted by the second light source 15 may be provided. Furthermore, the second light source 15 is used to read gloss information based on the mirror reflection of the planar object 10, and therefore, compared with the first light sources 13 and 14, for example, the brightness in the main scanning direction is preferably as uniform and continuous as possible.

[0089] As a light source that meets the requirements of the second light source 15, for example, a fluorescent lamp or a rare gas fluorescent lamp (xenon fluorescent lamp, etc.) can be used. In addition, the second light source 15 can also be a light source that arranges a plurality of white LEDs along the main scanning direction and uses a diffusion plate or the like to make the brightness distribution in the main scanning direction uniform.

[0090] The reciprocating table 12 also has an imaging optical system 16 and a sensor 17 inside. The imaging optical system 16 is composed of a reflector or an imaging lens, and images the diffuse reflection light and the mirror reflection light components from the plane object 10. The sensor 17 receives the diffuse reflection light and the mirror reflection light components imaged by the imaging optical system 16 and generates an image signal corresponding to the received light. The sensor 17 is composed of a light receiving element such as a CCD linear image sensor or a CMOS image sensor, and converts the received light into a signal indicating its intensity. In addition, the sensor 17 has a color filter composed of RGB that can be selectively loaded and unloaded, and generates a color image signal indicating the color of the plane object 10. In addition to the color image signal, the sensor 17 also outputs the diffuse reflection image signal obtained by receiving the diffuse reflection light and the mirror reflection image signal obtained by receiving the mirror reflection light to an external device such as a user terminal device 2, a tablet terminal device 7 or a server device 4 via a network 3.

[0091] The texture scanner 6 outputs a color image signal, a diffuse reflection image signal, and a mirror reflection image signal, respectively, by scanning the plane object 10 placed on the platen glass 11 three times, for example. The color image signal is output by installing a color filter in the sensor 17 and illuminating the plane object 10 placed on the platen glass 11 with the first light sources 13 and 14. The diffuse reflection image signal is output by removing the color filter from the sensor 17 and illuminating the plane object 10 placed on the platen glass 11 with the first light sources 13 and 14. The mirror reflection image signal is output by removing the color filter from the sensor 17 and illuminating the plane object 10 placed on the platen glass 11 with the second light source 15.

[0092] In addition, the diffuse reflection condition (light source incident angle 45°) and the specular reflection condition (light source incident angle 10°) are calibrated using the same white calibration plate. Therefore, when only diffuse reflection light is generated from the flat object 10 placed on the platen glass 11 and no specular reflection light is generated, the output signals from the sensor 17 become equal, and gloss information can be extracted by a simple differential operation.

[0093] However, the image of the planar object 10 read by the texture scanner 6 is not limited to ordinary images of text, photos, illustrations, etc., but also includes images of objects having gloss or uneven surfaces.

[0094] There are printers that can form glossy images using silver or gold metallic colors and YMCK (yellow, magenta, cyan and black) color toners, or printers that can print images that show gloss by applying a transparent toner on the color toner or by using a cooling device to increase the gloss after fixing the toner.

[0095] like Figure 6 As shown, by using the texture scanner 6 to acquire image data of a diffuse reflection image and a specular reflection image in addition to a color image of the planar object 10, image information of an object having gloss or unevenness on the surface is obtained.

[0096] The image information of an object with gloss or uneven surfaces is expressed using a bidirectional reflectance distribution function called BRDF (Bidirectional Reflectance Distribution Function). When light hits the surface of an object, it is specularly reflected on the surface, or is absorbed by the inside of the object, or is scattered and emitted from the surface. Figure 7 It is a diagram showing the geometric conditions of BRDF (bidirectional reflectance distribution function). BRDF is a function that shows the ratio of the radiance dL0 of the reflected light at a small corner in the line of sight direction V to the irradiance dE of the light incident from a small corner in the light source direction L at a certain point x on the reflective surface. It is usually measured by a measuring instrument that can change the incident angle or the angle of light reception, such as a variable angle photometer. This measured value is called the measured BRDF.

[0097] exist Figure 7 In, θ i is the angle between the incident light and the normal vector N at a point x on the reflecting surface, θ o is the angle between the reflected light and the normal vector N at a point x on the reflecting surface, φ i is the angle between the incident light and the tangent vector t at a point x on the reflecting surface, φ o It is the angle between the reflected light and the tangent vector t at a point x on the reflecting surface.

[0098] BRDF is expressed as follows:

[0099] [Formula 1]

[0100]

[0101] Figure 8 1 is a diagram showing an example of the measured BRDF of a printed product output by an electrophotographic system using a silver toner with a different coverage. Figure 8 In the figure, the light source direction vector L and the sight direction vector V are located on the same plane (without φ component), and the incident angle θ i The horizontal axis represents the light receiving angle θ0. The mirror reflection angle when the incident angle is 45° is 45°.

[0102] By Figure 8 It is clear that the higher the coverage (metallic luster) of the silver toner is, the higher the brightness of the specularly reflected light becomes.

[0103] As a model for simulating the reflection of light on the surface of an object, the Phong reflection model can be used. In the Phong reflection model, the reflection of the surface of an object is simulated based on three elements: ambient light, diffuse reflection of direct light, and specular reflection of direct light.

[0104] In addition, if Fig. 9 As shown, light sources can be divided into point light sources and parallel light sources. Point light sources are light sources that spread light from a certain position in space to all directions. Parallel light sources are light sources that have the same direction at all positions in space.

[0105] The intensity of the reflected light from the object is the sum of the intensities of ambient light, diffuse reflected light of direct light, and specular reflected light of direct light.

[0106] I r =I a +I d +I s

[0107] Here, I a ,I d ,I s Represents the brightness of ambient light, diffuse light, and specular light respectively. a Can be treated as a constant.

[0108] It is known that the intensity of diffusely reflected light from an object follows Lambert's cosine theorem. Lambert's cosine theorem states that the intensity of radiation reflected from a tiny surface is related to the surface normal and the incident angle θ of the light, as shown in the following equation. i The cosine proportionality theorem. In addition, Fig.10This is a diagram showing the geometric conditions in Lambert's cosine theorem and Phong's reflection model described below.

[0109] I=I i ·(ρ d ·cosθ i )

[0110] Here, I i is the intensity of the incident light, ρ d is the diffuse reflectance, θ i is the angle of incidence.

[0111] The intensity of specular light from an object is the reflection produced on a smooth surface such as metal or plastic. In a perfect mirror, light is reflected only in the direction of the incident angle, but in real objects, such as Fig.11 As shown, it has diffusion properties in a range centered on the regular reflection direction.

[0112] In Phong's reflection model, the radiation intensity I of specular reflected light is simulated as follows.

[0113] I=I i ·(ρ s ·cos n γ)

[0114] Here, ρ s is the specular reflectivity, n is the specular reflection index (n≥1), and γ is the angle between the specular reflection direction and the line of sight direction.

[0115] Therefore, the intensity of the reflected light from the object is obtained by adding the intensities of the diffuse reflected light of the direct light and the specular reflected light of the direct light, and is expressed by the following equation.

[0116] I=I i ·(ρ d ·cosθ i )+I i ·(ρ s ·cos n γ)

[0117] In this formula, the smaller the angle γ is, the larger the value becomes. Also, as the value of n increases, the value of the angle γ decreases rapidly.

[0118] In this reflection model, parameters of the diffuse reflectance ρd, the specular reflectance ρs, and the specular reflection index n are obtained based on the image data of the diffuse reflection image and the specular reflection image output from the texture scanner 6 .

[0119] Fig.12 is an example of estimating parameters in a reflection model, which means Figure 8The predicted BRDF fits the measured BRDF at low coverage as shown. Fig.12 It is clear that in the reflection model, by appropriately estimating and determining parameters based on the output of the texture scanner 6, a BRDF suitable for the actually measured low coverage range is determined.

[0120] However, the display device 20 according to the first embodiment is configured as follows.

[0121] The display device 20 uses 3DCG or the like to three-dimensionally display an image of an object having a glossy or uneven surface.

[0122] Before displaying the image of the object having a glossy or uneven surface, the display device 20 calculates the reflectivity distribution function of the planar object 10 based on the image data of the diffuse reflection image and the specular reflection image of the planar object 10 read by the texture scanner 6 .

[0123] Fig.13 1 is a block diagram showing a process of calculating a reflectance distribution function of the planar object 10 based on image data of a diffuse reflection image and a specular reflection image of the planar object 10 read by the texture scanner 6 .

[0124] like Fig.13 As shown, the calculation process of the reflectance distribution function includes a diffuse reflection image acquisition unit 21, a mirror reflection image acquisition unit 22, a differential image acquisition unit 23, a diffuse reflectance distribution function calculation unit 24, a mirror reflectance distribution function calculation unit 25, a parameter adjustment unit 26, a reflectance distribution function calculation unit 27, a light source information acquisition unit 28 and a camera information acquisition unit 29.

[0125] The diffuse reflection image acquisition unit 21 and the specular reflection image acquisition unit 22 respectively acquire the diffuse reflection image and the specular reflection image obtained by the texture scanner 6. The diffuse reflection image acquisition unit 21 and the specular reflection image acquisition unit 22 may be respectively connected to the texture scanner 6 and acquire these images from the texture scanner 6, or may acquire these images from the server device 4 connected to the texture scanner 6 via the network 3.

[0126] The difference image acquisition unit 23 calculates the difference between the diffuse reflection image and the specular reflection image to acquire the difference image. The difference image acquisition unit 23 calculates the difference image which is (specular reflection image - diffuse reflection image).

[0127] The diffuse reflectance distribution function calculation unit 24 uses the diffuse reflection image to calculate the diffuse reflectance distribution function of the planar object 10. For example, the diffuse reflectance distribution function calculation unit 24 sets ρd as the diffuse reflectance relative to the incident light, θi as the incident angle, and the diffuse reflectance distribution function as ρd·cosθi, and calculates the diffuse reflectance ρd as a parameter from the diffuse reflection image according to Lambert's cosine theorem.

[0128] The specular reflectivity distribution function calculation unit 25 uses the differential image to calculate the specular reflectivity distribution function of the planar object 10. For example, the specular reflectivity distribution function calculation unit 25 uses the Phong reflection model to set ρs as the specular reflectivity, γ as the angle between the specular reflection direction and the line of sight direction, n as the specular reflection index, and the specular reflectivity distribution function as ρs·cos n γ, the specular reflectance ρs is calculated as a parameter from the difference image.

[0129] The reflectivity distribution function calculation unit 27 calculates the reflectivity distribution function of each pixel of the planar object 10 using the diffuse reflectivity distribution function calculated by the diffuse reflectivity distribution function calculation unit 24 and the specular reflectivity distribution function calculated by the specular reflectivity distribution function calculation unit 25. For example, the reflectivity distribution function calculation unit 27 calculates the reflectivity distribution function according to Lambert's cosine theorem and Phong's reflection model, and according to reflectivity distribution function = diffuse reflectivity distribution function + specular reflectivity distribution function.

[0130] The parameter adjustment unit 26 adjusts various parameters when the reflectance distribution function calculation unit 27 calculates the reflectance distribution function.

[0131] The light source information acquisition unit 28 acquires the light source information (light source direction) that illuminates the planar object 10. Also, the camera information acquisition unit 29 acquires the camera information (line of sight direction).

[0132] These various parameters and light source information (light source direction) or camera information (line of sight direction) are appropriately set by the user using the operation unit 203 of the terminal device 2 or the like.

[0133] like Figure 4 As shown, the display device 20 involved in this embodiment 1 is configured to include: an acquisition unit 206, which acquires an image including a surface reflection component of an object; a generation unit 207, which generates, based on the image of the object acquired by the acquisition unit 206, an image of the object when the object is observed from multiple angles in a manner that changes the angle of the object relative to a virtual light source; a selection unit 208, which selects the image of the object observed from a first angle from the image of the object generated by the generation unit 207; a conversion unit 209, which converts the image of the object selected by the selection unit 208 into an image of the object observed from a second angle different from the first angle while maintaining the positional relationship between the virtual light source and the object; a display unit 205, which displays the image of the object converted by the conversion unit 209; and a storage unit 202, which stores various image data.

[0134] The acquisition unit 206 acquires the diffuse reflection image and the specular reflection image acquired by the texture scanner 6, and then acquires the color image. In addition, as described above, the acquisition unit 206 performs calculation processing of the reflectance distribution function based on the diffuse reflection image and the specular reflection image acquired by the texture scanner 6. The diffuse reflection image and the specular reflection image acquired by the acquisition unit 206, as well as the color image information, are temporarily stored in the storage unit 202.

[0135] The generating unit 207 generates, based on the image of the object acquired by the acquiring unit 206 , an image of the object when the object is observed from a plurality of angles such that the angle of the object relative to the virtual light source is changed.

[0136] If further explanation is given, Fig.14 As shown, the generating unit 207 generates an image in which the image of the object acquired by the acquiring unit is pasted on the surface (upper end surface) of the rectangular parallelepiped object 100 arranged in the world coordinate system using a 3DCG program.

[0137] The image of the object acquired by the acquisition unit 206 may be directly used as a two-dimensional image, but here, in order to easily identify the tilted state of the two-dimensional image, the two-dimensional image is generated as an image pasted on the surface (upper end face) of the rectangular parallelepiped.

[0138] In the world coordinate system, the point light source is arranged at the coordinates (XYZ) as the light source information (light source direction) for illuminating the planar object 10 acquired by the light source information acquisition unit 28. Furthermore, the camera information acquisition unit 29 acquires an image of the rectangular parallelepiped observed from the Z axis direction as camera information (viewing direction).

[0139] like Fig.15 As shown, the generating unit 207 generates image information by using a 3DCG program to tilt or rotate the image of the rectangular parallelepiped object 100 arranged in the world coordinate system at an arbitrary angle in a direction designated by the mouse as the operating unit 203 of the user terminal device 2 .

[0140] like Fig.16 As shown, the image information generated by the generating unit 207 is rendered using a 3DCG program and displayed on the display unit 205 of the user terminal device 2 as a three-dimensional image.

[0141] The selection unit 208 selects an image of the object observed from the first angle from among the images of the object 100 generated by the generation unit 207. Here, the first angle is an angle that can be arbitrarily selected by the user. The selection unit 208 is composed of a mouse or the like as the operation unit 203 of the user terminal device 2. The operation unit 203 of the user terminal device 2, i.e., the mouse, which is an example of the selection unit 208, performs a process of selecting an image of the object observed from the first angle from among the images of the object generated by the generation unit 207 by observing the image of the object 100 displayed on the display unit of the user terminal device 2 and clicking the left button.

[0142] like Fig.17 As shown, the conversion unit 209 converts the image of the object 100 selected by the selection unit 208 into an image of the object observed from a second angle different from the first angle while maintaining the positional relationship between the virtual light source and the object.

[0143] The display device 20 includes a designation unit for designating the second angle. The designation unit is constituted by, for example, a mouse as the operation unit 203 of the user terminal device 2. The operation unit 203 of the user terminal device 2, i.e., the mouse, as an example of the designation unit, performs a process of moving the mouse so that the image of the object viewed from the second angle desired by the user is displayed on the display unit while the user observes the image of the object displayed on the display unit of the user terminal device 2 and clicks the left button.

[0144] The image of the object within the preset angle range (for example, ±5 degrees) with respect to the second angle designated by the designation unit is set as the image of the object observed from the second angle. More specifically, for example, in relation to the resolution of the display device, if it is desired to display at an angle of 2.57888° from the horizontal, it cannot be displayed sharply but can be displayed more clearly at 2.5°, it is set to display at 2.5°. Alternatively, for example, when an angle substantially perpendicular to the plane of the object is selected, the angle perpendicular to the plane, which is estimated to be the vertical angle that the user may instruct, may be set as the second angle for display.

[0145] Furthermore, the selection unit 208 may be configured to automatically select the image with the largest area of ​​the image of the object as the image of the object observed from the second angle. The image of the object observed from the second angle is, for example, a planar image of the object.

[0146] The conversion unit 209 performs a process of converting the image of the object 100 selected by the selection unit 208 into an image of the object observed from a second angle different from the first angle while maintaining the positional relationship between the virtual light source and the object.

[0147] like Fig.19As shown, the processing performed by the conversion unit 209 is a rotation processing for rotating the coordinate system, and is executed by multiplying a 4×4 rotation matrix by the (XYZ) coordinates.

[0148] The display unit 205 displays the image of the object on the way including the image of the object converted by the conversion unit 209 .

[0149] At this time, the display unit 205 may be configured to display the image of the object generated by the generating unit 207. Furthermore, the display unit 205 preferably displays, as the selected image, a first image in which the object is viewed from a first angle in the generated image generated by the generating unit 207.

[0150] Furthermore, the display unit 205 may be configured to display images of the object observed from multiple angles generated by the generation unit 207. In this way, the display unit 205 displays images of the object observed from multiple angles generated by the generation unit 207, so that the user is not limited to selecting the image with the highest brightness, and can easily select an image of the object that suits his or her preferences.

[0151] Furthermore, the display unit 205 may be configured to display the images of the object observed from multiple angles generated by the generation unit 207 in a designated order. Here, the designated order may be an order sequentially specified by a user using a mouse or the like, or the brightness of the image of the object may be automatically calculated by calculation and the descending order of brightness may be specified in advance, thereby displaying the image of the object in the designated descending order of brightness.

[0152] Furthermore, the display unit 205 may be configured to redisplay the image of the object observed from multiple angles generated by the generating unit 207 in a manner that makes the amount of change smooth. In this way, by redisplaying the image of the object observed from multiple angles generated by the generating unit 207 in a manner that makes the amount of change smooth, the user can easily recognize the amount of change in the image of the object and easily select an image that suits his or her preferences.

[0153] <Operation of the display device>

[0154] In the display device 20 according to the present embodiment, it is possible to display an image of an object observed from another angle while maintaining the positional relationship between the virtual light source and the object in the following manner.

[0155] Here, consider the following situation, that is, if Fig.14 As shown, in the world coordinate system (XYZ space), the coordinates of the center O1 of the surface (upper end face) 101 of the rectangular object 100 are (X1, Y1, Z1), and the side faces 102 to 105 and the bottom face 106 are arranged parallel to the XY plane, YZ plane, and ZX plane, respectively.

[0156] like Fig. 20 As shown, an image 107 that appears to emit light in an elliptical shape when tilted in a diagonal direction is formed in the center of the surface 101 of the object 100. The image 107 is a portion that cannot be seen because it emits light when not tilted, but has a glossy image or the like formed by coating the surface or using a toner that does not diffusely reflect but has strong directivity, such as silver toner.

[0157] Similarly, there is luster in the surroundings, but in terms of the relationship with the imaginary light source, only image 107 appears to be glowing in an elliptical state, or it is formed as an image different from the surroundings and is particularly conspicuous at the angle of a certain imaginary light source. In this case, the displayed part is displayed in this way, so that this time the image becomes image 107 that glows in an elliptical shape.

[0158] The image of the surface 101 of the object 100 is, for example, an image on which an image read by the texture scanner 6 is pasted.

[0159] The display device 20 according to the first embodiment displays an image of the object 100 on the display unit 205 of the user terminal device 2 based on the color image information of the surface 101 of the object 100 read by the texture scanner 6 , the diffuse reflection image information based on the diffuse reflection light, and the specular reflection image information based on the specular reflection light.

[0160] In the display device 20, as shown in FIG. Fig.13 As shown, the diffuse reflectance distribution function calculation unit 24 calculates the diffuse reflectance distribution function of the planar object 10 based on diffuse reflection image information based on diffuse reflection light and specular reflection image information based on specular reflection light of the surface 101 of the object 100 read by the texture scanner 6 .

[0161] The diffuse reflectance distribution function calculation unit 24 calculates the diffuse reflectance ρd as a parameter from the diffuse reflection image according to Lambert's cosine theorem, with ρd being the diffuse reflectance with respect to incident light, θi being the incident angle, and the diffuse reflectance distribution function being ρd·cosθi.

[0162] The specular reflectivity distribution function calculation unit 25 uses the differential image to calculate the specular reflectivity distribution function of the planar object 10. For example, the specular reflectivity distribution function calculation unit 25 uses the Phong reflection model to set ρs as the specular reflectivity, γ as the angle between the specular reflection direction and the line of sight direction, n as the specular reflection index, and the specular reflectivity distribution function as ρs·cos n γ, the specular reflectivity ρs calculated as a parameter from the difference image.

[0163] Then, the reflectance distribution function calculation unit 27 calculates the reflectance distribution function according to reflectance distribution function=diffuse reflectance distribution function+specular reflectance distribution function.

[0164] I=I i ·(ρ d ·cosθ i )+I i ·(ρ s ·cos n γ)

[0165] like Figure 4 As shown, the display device 20 performs the acquisition process of the reflectance distribution function as described above through the acquisition unit 206.

[0166] like Fig.21 As shown, the reflectance distribution function acquired by the acquisition unit 206 is temporarily stored in the storage unit 202 of the user terminal device 2 for each pixel together with the color image information.

[0167] Next, the generating unit 207 generates an image of the object 100 based on the image information temporarily stored in the storage unit 202 of the user terminal device 2. Fig.15 As shown in FIG. 1 , an image of the object is generated when the object 100 is observed from a plurality of angles so that the angle of the object 100 relative to the virtual light source is changed. Here, the plurality of angles also include the angle at which the object 100 is initially observed.

[0168] like Fig.15 As shown, the generating unit 207 generates image information by using a 3DCG program to tilt or rotate a rectangular parallelepiped image arranged in the world coordinate system at an arbitrary angle in a direction designated by a mouse as the operating unit 203 of the user terminal device 2 .

[0169] like Fig.16 As shown, the image information generated by the generating unit 207 is rendered using a 3DCG program and displayed as a three-dimensional image on the display unit 205 of the user terminal device 2. The displayed image corresponds to the generated image in this embodiment.

[0170] Next, the user operates a mouse as an example of the selection unit 208 of the user terminal device 2 to select the image of the object 100 viewed from the first angle from among the images of the object 100 generated by the generation unit 207 .

[0171] At this time, the user tilts or rotates the image of the object 100 by operating a mouse, which is an example of a selection unit of the user terminal device 2, thereby enabling the user to visually view a reflected image reflected from the surface of the object 100 illuminated by the fixed imaginary light source 400 as an image displayed on the display unit 205 of the user terminal device 2.

[0172] Here, the user tilts the image of the object 100 , and the elliptical portion of the surface of the object specularly reflects the light from the virtual light source, and the specularly reflected light enters the camera.

[0173] To further explain, when the user tilts the image of the object 100, the main angle θ i and the angle γ changes.

[0174] I=I i ·(ρ d ·cosθ i )+I i ·(ρ s ·cos n γ)

[0175] So, in Fig. 22 When the object is tilted so that the incident angle and the reflection angle are equal, the brightness of the specularly reflected light from the elliptical portion becomes the highest. In this embodiment, the position where the brightness becomes the highest is set as the first angle, and the image of the object 100 observed from this angle is selected.

[0176] The user specifies the second angle using the mouse as the operation unit 203 of the user terminal device 2 .

[0177] Next, the conversion unit 209 converts the image of the object 100 selected by the selection unit 208 into an image of the object 100 observed from a second angle different from the first angle while maintaining the positional relationship between the virtual light source 400 and the object 100 .

[0178] Moreover, if Fig.18 As shown, the image of the object 100 converted by the conversion unit 209 is displayed on the display unit 205 of the user terminal device 2.

[0179] As described above, according to the display device 20 according to the first embodiment, it is possible to display an image of the object observed from another angle while maintaining the positional relationship between the virtual light source 400 and the object 100 .

[0180] In the present embodiment, the position where the brightness is highest is specified as the first angle. However, in the case where glare is not desired, a position where the brightness is not too high may be specified as the first angle.

[0181] Furthermore, the second angle may be specified before selecting the first angle or initially displaying the object at multiple angles.

[0182] For example, when a plurality of objects are displayed at a certain angle and one object is selected and the brightness of the displayed object is to be changed, the angle at which the object is displayed together with other objects may be designated in advance as the second angle.

[0183] Furthermore, although it is assumed here that a plurality of angles are changed by a mouse, in the case of a handheld display device, the generated image may be changed by tilting the display device itself.

[0184] When the conversion image and the generated image are displayed together on the display device, the generated image may be changed by changing the display angle using a mouse or by detecting the inclination of the display device, while the conversion image may not be changed.

[0185] [Implementation Method 2]

[0186] Fig.23 and Fig.24 1 is a diagram showing a display device according to Embodiment 2. In the display device according to Embodiment 2, a generating unit is configured to generate an image to which a gloss component is added by multiplying a coefficient α to which a slope is added according to an area on the surface of the object when generating an image of the object when the object is observed from a plurality of angles in a manner that changes the angle of the object relative to a virtual light source.

[0187] Furthermore, in the display device according to the second embodiment, the slope corresponding to the region on the surface of the object is configured to be composed of parallel linear slopes corresponding to the surface light source and concentric circular slopes corresponding to the point light source.

[0188] That is, in the display device 20 according to the second embodiment, Fig.23 As shown, in the image of the object read by the texture scanner 6 , a coefficient (gradient) α multiplied according to the type of the virtual light source is specified for the diffuse reflection image and the specular reflection image representing the gloss component. The parameter adjustment unit 26 specifies the coefficient (gradient) α.

[0189] like Fig.24 As shown in (a), when the point light source as the imaginary light source 400 exists at a certain distance above the planar object 100, as shown in Fig.24 As shown in (b), a plurality of contour lines parallel to each other in a direction orthogonal to the light from the imaginary light source 400 are set in such a manner that the glossiness of the end portion of the planar object 100 close to the imaginary light source 400 is relatively high and the glossiness of the end portion far from the imaginary light source 400 is relatively low, and a coefficient (slope) α is specified according to the plurality of contour lines in such a manner that the glossiness gradually decreases.

[0190] like Fig.24As shown in (c), when a point light source serving as an imaginary light source 400 exists at a certain distance above the central portion of a planar object 100, a plurality of concentric contour lines are set with a vertical line extending from the imaginary light source 400 to the planar object 100 as the center, and a coefficient (slope) α is specified based on the plurality of contour lines in such a manner that the glossiness gradually decreases from the center to the periphery.

[0191] This makes it possible to display a more natural gloss or the like in accordance with the type of the virtual light source 400 .

[0192] The other structures and functions are the same as those in the first embodiment, so their description is omitted.

[0193] The above-mentioned embodiments of the present invention are provided for the purpose of example and explanation. In addition, the embodiments of the present invention do not fully and exhaustively include the present invention, and do not limit the present invention to the disclosed methods. Obviously, various modifications and changes are self-evident to those skilled in the art to which the present invention belongs. The present embodiment is selected and described in order to most easily understand the principles of the present invention and its application. Thus, other technicians in the technical field can understand the present invention by using various modified examples optimized for the determination of the assumed various embodiments. The scope of the present invention is defined by the above claims and their equivalents.

Claims

1. A display device comprising: A generating unit that generates, based on an image of a certain object, generated images when the object is observed from a plurality of angles in such a manner that an angle of the object relative to a virtual light source is changed; a selection unit that selects, from among the generated images, a first image in which the object is observed from a first angle as a selected image; a conversion unit that converts the first image into a converted image in which the object is observed from a second angle different from the first angle, while maintaining a positional relationship between the virtual light source and the object in the first image; A display unit, which displays the conversion image; and a designating unit for designating the second angle, The designating unit automatically designates an image having the largest area among images of the object as the image of the object observed from the second angle.

2. The display device according to claim 1, wherein: The image of the object that falls within a preset angle range with respect to the second angle designated by the designating unit is set as the image of the object observed from the second angle.

3. The display device according to claim 2, wherein: The image of the object observed from the second angle is a planar image of the object.

4. The display device according to claim 1, comprising: The adjustment unit adjusts the position and illumination intensity of the virtual light source when the display unit that also displays the generated image displays the generated image and the converted image.

5. The display device according to claim 4, wherein: The imaginary light source includes a point light source and a parallel light source.

6. The display device according to claim 1, wherein: The display unit also displays the selected image.

7. The display device according to claim 1, wherein: The display unit displays the generated images observed from the plurality of angles.

8. The display device according to claim 7, wherein: The display unit displays the generated images observed from the plurality of angles in the specified order.

9. The display device according to claim 7, wherein: The display unit rearranges and displays the generated images observed from the plurality of angles so that the amount of change becomes gentle.

10. The display device according to claim 1, wherein: When generating an image of the object when the object is observed from a plurality of angles in a manner that changes the angle of the object relative to the virtual light source, the generating unit generates an image to which a gloss component multiplied by a coefficient to which a slope is added according to an area on the surface of the object is added.

11. The display device according to claim 10, wherein: The slope corresponding to the region on the surface of the object is composed of a parallel linear slope corresponding to the surface light source and a concentric circular slope corresponding to the point light source.

12. A display method comprising the following steps: A generating step of generating, based on an image of a certain object, generated images when the object is observed from a plurality of angles in such a manner that the angle of the object relative to a virtual light source is changed; A selection step of selecting, from among the generated images, a first image in which the object is observed from a first angle as a selected image; a conversion step of converting the state in which the positional relationship between the imaginary light source and the object in the first image is maintained into a converted image in which the object is observed from a second angle different from the first angle; A display step of displaying the converted image; and Specifying step, specifying the second angle, The designating step automatically designates an image having the largest area as the image of the object observed from the second angle.

Images