Image display device, image display method, and program

Abstract

To provide an image display device capable of appropriately suppressing occurrence of crosstalk, an image display method, and a program.SOLUTION: An image display device comprises: a visual line detection section for detecting a visual line direction of a user; a rendering section for drawing a stereoscopic image on a display device in accordance with the visual line direction of the user; a light source range identification section for identifying an unwanted light source range indicating a range of candidates of pixels to be a light source of crosstalk light indicating a beam of light passing a position of the stereoscopic image, the beam of light being caused by a light source other than a light source of the drawn stereoscopic image; a pixel identification section for identifying light source pixels indicating pixels to be the light source of crosstalk light out of the pixels in the unwanted light source range; and a beam-of-light processing section for performing predetermined processing on the identified light source pixels or a beam of light caused by the light source pixels in such a manner that the beam of light caused by the light source pixels does not pass the position of the stereoscopic image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an image display device, an image display method, and a program. [Background technology]

[0002] For example, progress is being made in the development of light field displays that reproduce the light rays emitted by an object captured by a light field camera on a display.

[0003] The following Patent Document 1 discloses an eyeball projection type image display device that includes a field lens, a microlens array behind it, and an emissive display device such as an LED array as an image display means for displaying an image behind it. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 10-319342 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in light field displays, crosstalk occurs in which the stereoscopic image is affected by light emitted from pixels other than those that serve as the light source for generating the desired stereoscopic image, which can result in blurring, uneven color, and uneven brightness in the stereoscopic image.

[0006] In view of the above-mentioned problems, an object of the present disclosure is to provide an image display device, an image display method, and a program capable of appropriately suppressing the occurrence of crosstalk. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the image display device of the present disclosure includes a gaze detection unit that detects the direction of the user's gaze, a rendering unit that causes the display device to draw a stereoscopic image according to the direction of the user's gaze, a range identification unit that identifies an unnecessary light source range that indicates a range of candidates for pixels that can become a source of crosstalk light, which indicates light rays from a light source other than the light source of the drawn stereoscopic image and which pass through the position of the stereoscopic image, a pixel identification unit that identifies light source pixels that indicate pixels that can become the source of crosstalk light among the pixels in the unnecessary light source range, and a light ray processing unit that performs a predetermined processing on the identified light source pixels or the light rays from the light source pixels, and does not allow the light rays from the light source pixels to pass through the position of the stereoscopic image.

[0008] In order to solve the above-mentioned problems and achieve the objective, the image display method of the present disclosure includes the steps of detecting a user's gaze direction, causing a display device to render a stereoscopic image in accordance with the user's gaze direction, identifying an unnecessary light source range indicating a range of candidate pixels that can become a source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and which pass through a position of the stereoscopic image, identifying light source pixels among the pixels in the unnecessary light source range that indicate pixels that can become a source of crosstalk light, and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels to prevent the light rays from the light source pixels from passing through the position of the stereoscopic image.

[0009] In order to solve the above-mentioned problems and achieve the objective, the program of the present disclosure causes a computer to execute the steps of detecting the user's gaze direction, causing a display device to draw a stereoscopic image according to the user's gaze direction, identifying an unnecessary light source range indicating a range of candidate pixels that can be the source of crosstalk light, which indicates light rays from a light source other than the light source of the drawn stereoscopic image and which pass through the position of the stereoscopic image, identifying light source pixels among the pixels in the unnecessary light source range that indicate pixels that can be the source of crosstalk light, and performing a predetermined processing on the identified light source pixels or the light rays from the light source pixels, and preventing the light rays from the light source pixels from passing through the position of the stereoscopic image. Effect of the Invention

[0010] According to the present disclosure, it is possible to provide an image display device, an image display method, and a program capable of appropriately suppressing the occurrence of crosstalk. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an image display device according to the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating a configuration example of the control device according to the present disclosure. [Diagram 3] FIG. 3 is a diagram illustrating an example of information stored in the map information storage unit of the control device according to the present disclosure. [Figure 4] FIG. 4 is a diagram illustrating an example of information stored in a drawing image storage unit of a control device according to the present disclosure. [Diagram 5] FIG. 5 is a diagram illustrating an example of the configuration of a gaze detection unit of the control device according to the present disclosure. [Figure 6] FIG. 6 is a diagram illustrating the image rendering process performed by the image display device according to the present disclosure. [Figure 7] FIG. 7 is a diagram illustrating crosstalk that may occur during rendering by the image display device according to the present disclosure. [Figure 8] FIG. 8 is a diagram for explaining the description of a ray by a Plenoptioc Function. [Figure 9] FIG. 9 is a diagram illustrating the processing of the range specifying unit and the light-off instruction unit of the image display device according to the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating a configuration example of a light beam processing unit of the control device according to the present disclosure. [Figure 11] FIG. 11 is a diagram illustrating the processing of the light beam processing unit of the image display device according to the present disclosure. [Figure 12] FIG. 12 is a diagram illustrating a shielding filter of the image display device according to the present disclosure. [Figure 13]FIG. 13 is a diagram showing a configuration example of a display device according to the present disclosure. [Figure 14] FIG. 14 is a flowchart showing the flow of the image display method according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments described below.

[0013] (Configuration of image display device) First, the configuration of an image display device 1 according to the present disclosure will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an image display device according to the present disclosure. As shown in Fig. 1, the image display device 1 according to the present disclosure includes a control device 100, a camera 200, and a display device 300. The configurations of these components will be briefly described below.

[0014] The control device 100 executes various types of arithmetic processing and processing for realizing functions, and controls the display device 300 to render a stereoscopic image. The control device 100 detects the user's line of sight based on the user's face image captured by the camera 200, and causes the display device 300 to display a stereoscopic image according to the detected line of sight direction.

[0015] Camera 200 captures a facial image of a user. Camera 200 includes an optical element and an imaging element. The optical element is an element that constitutes an optical system, such as a lens, a mirror, a prism, and a filter. The imaging element is an element that converts light incident through the optical element into an image signal, which is an electrical signal. The imaging element may be, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0016] The display device 300 is a device that displays a stereoscopic image. The display device 300 may be a light field display. The light field display displays a stereoscopic image by passing light rays emitted from the display through a microlens to refract the light rays and reproduce the light rays emitted by an object. The light field display may be realized by, for example, an integral imaging method.

[0017] Note that image display device 1 does not need to be an integrated device including control device 100, camera 200, and display device 300, and each may be an independent device. For example, image display device 1 may be realized by connecting control device 100, camera 200, and display device 300 to a network and transmitting and receiving information to and from each other via the network. In this case, image display device 1 can be said to be an image display system including control device 100, camera 200, display device 300, and the network.

[0018] (Control device configuration) Next, the configuration of the control device 100 according to the present disclosure will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the control device according to the present disclosure. As shown in Fig. 2, the control device 100 according to the present disclosure includes an I / F unit 110, a storage unit 120, and a control unit 130. Below, these components will be described in order.

[0019] The I / F unit 110 is responsible for transmitting and receiving control signals, video signals, and the like, by wire or wirelessly, between the camera 200 and the display device 300. In the case of a wired connection, it may be realized by, for example, a Universal Serial Bus (USB), a Serial Digital Interface (SDI), or a High-Definition Multimedia Interface (HDMI (registered trademark)). In the case of a wireless connection, it may be realized by a wireless Local Area Network (LAN) defined in IEEE802.11.

[0020] The storage unit 120 is a storage device that stores various types of information. The storage unit 120 includes a main storage device and an auxiliary storage device. The main storage device may be realized by a semiconductor memory element such as a random access memory (RAM), a read only memory (ROM), or a flash memory. The auxiliary storage device may be realized by a hard disk or a solid state drive (SSD), for example.

[0021] 2, the storage unit 120 includes a map information storage unit 121 and a drawing image storage unit 122. An example of information stored in these components will be described below in order.

[0022] The map information storage unit 121 stores information regarding the relationship between the line of sight direction and the unnecessary light source range. Here, an example of information stored in the map information storage unit 121 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of information stored in the map information storage unit of the control device according to the present disclosure.

[0023] As shown in FIG. 3, the map information storage unit 121 stores information relating to the items "line of sight direction", "X coordinate range", and "Y coordinate range".

[0024] The "gaze direction" is information indicating the direction of the user's gaze, and for example, the user's gaze direction expressed by a pleoptioc function may be stored. The "X coordinate range" is information indicating the X coordinate value of the unnecessary light source range associated with the "gaze direction". The "Y coordinate range" is information indicating the Y coordinate value of the unnecessary light source range associated with the "gaze direction". The X coordinate and the Y coordinate are coordinates indicating a position on the surface of the flat panel display 310, and the two are perpendicular to each other.

[0025] That is, FIG. 3 shows an example in which the X coordinate range of the unnecessary light source range associated with the line of sight direction "LOS#1" is stored as "XCRR#1" and the Y coordinate is stored as "YCRR#1."

[0026] Note that the map information storage unit 121 is not limited to information relating to the items of “line of sight direction”, “X coordinate range”, and “Y coordinate range”, and may store information relating to the relationship between any other line of sight direction and unnecessary light source range.

[0027] The drawing image storage unit 122 stores information related to an image to be drawn by the display device 300. An example of information stored in the drawing image storage unit 122 will now be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of information stored in the drawing image storage unit of the control device according to the present disclosure.

[0028] As shown in FIG. 4, the drawing image storage unit 122 stores information relating to the items "image ID" and "image data".

[0029] The "video ID" is an identifier for identifying the image for drawing, and is represented by a character string, a number, etc. The "video data" is the video data of the image for drawing identified by the "video ID", and may be, for example, video data captured by a light field camera or video data generated by CG (Computer Graphics).

[0030] That is, FIG. 4 shows an example in which video data "VDDT#1" of a drawing video identified by a video ID "VDID#1" is stored.

[0031] It should be noted that the drawn image storage section 122 is not limited to storing information related to the items of "image ID" and "image data", and may store any other information related to an image to be drawn by the display device 300.

[0032] The control unit 130 is a controller that executes various types of arithmetic processing and processing for implementing functions. The control unit 130 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs stored in the storage unit 120 using a RAM as a working area. The control unit 130 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0033] As shown in FIG. 2, the control unit 130 includes a gaze detection unit 131, a rendering unit 132, a range specification unit 133, a pixel specification unit 134, a light processing unit 135, and a brightness increase unit 136. The control unit 130 realizes these functions and executes these processes by reading and executing a program (software) from the storage unit 120. Note that these functions of the control unit 130 may be realized by an electronic circuit. Furthermore, the control unit 130 may execute these processes by one CPU, or may include multiple CPUs and execute these processes in parallel by the multiple CPUs. Below, these configurations will be described in order.

[0034] The gaze detection unit 131 detects the user's gaze direction based on image data in which the user's face is captured. That is, the gaze detection unit 131 detects the user's gaze direction from the image data using an image processing technique. Specifically, the gaze detection unit 131 may detect the user's gaze direction by using the configuration described below.

[0035] The configuration of the gaze detection unit 131 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the configuration of the gaze detection unit of the control device according to the present disclosure. As shown in Fig. 5, the gaze detection unit 131 includes an acquisition unit 1311, a face detection unit 1312, an pupil information detection unit 1313, a face direction detection unit 1314, a gaze detection unit 1315, and an output unit 1316. Below, these configurations will be described in order.

[0036] The acquisition unit 1311 acquires image data captured by the camera 200. Here, the image data acquired by the acquisition unit 1311 includes the user's face. After acquiring the image data, the acquisition unit 1311 stores the acquired image data in the storage unit 120 and then outputs the image data to the face detection unit 1312.

[0037] The face detection unit 1312 detects a face area including at least a part of the user's face from the image data acquired by the acquisition unit 1311. For example, the face detection unit 1312 may detect the face area using a classifier such as a convolutive neural network (CNN) that has been trained in advance to detect the face area. Note that the face area is detected as a rectangular area having a size that includes the entire face, for example.

[0038] The pupil information detection unit 1313 detects pupil information indicating information about the pupil of the user based on the image data. The pupil information includes, for example, coordinate data indicating the outer edge of the user's pupil, or information indicating the length of the outer edge of the user's pupil, such as the radius or diameter, and coordinate data of the center of the user's pupil. First, the pupil information detection unit 1313 detects an eye area including the user's eyes from the face area detected by the face detection unit 1312. The eye area may be detected using a classifier such as CNN that has been trained in advance to detect the face area. Next, the pupil information detection unit 1313 detects the user's pupil. A region estimated to include the pupil is detected by narrowing down the eye area based on a predetermined criterion. Based on this, coordinate data indicating the outer edge of the pupil, or information indicating the length of the outer edge of the pupil, such as the radius or diameter, and coordinate data of the center of the pupil are detected.

[0039] The face direction detection unit 1314 detects facial feature points from the face area detected by the face detection unit 1312, and detects face direction information indicating the direction of the user's face from the arrangement pattern of the detected feature points. The process of extracting the facial feature points from the face area may be realized by using a classifier such as a CNN that has learned facial features. Note that the facial feature points are one or more points that are located at characteristic positions in each of multiple parts that make up the face, such as the outer corners and inner corners of the eyes, the facial outline, the bridge of the nose, the corners of the mouth, and the eyebrows.

[0040] The gaze detection unit 1315 detects the gaze direction indicating the user's gaze, based on the pupil information detected by the pupil information detection unit 1313 and the face direction information detected by the face direction detection unit 1314. The gaze detection unit 1315 may detect the gaze information by applying a known gaze detection process that detects the gaze using, for example, a three-dimensional eyeball model to the pupil information and the face direction information. Here, the gaze information may include a vector that three-dimensionally indicates the direction of the user's gaze, or may include coordinate data of a gaze point on a predetermined target surface (for example, the display device 300).

[0041] The output unit 1316 outputs the user's gaze direction detected by the gaze detection unit 1315. Specifically, the output unit 1316 outputs the detected user's gaze direction to the rendering unit 132 and the range identification unit 133.

[0042] The rendering unit 132 causes the display device 300 to render a stereoscopic image according to the user's line of sight. That is, the rendering unit 132 gives a rendering instruction to the display device 300 so that the stereoscopic image is positioned in the user's line of sight. The rendering unit 132 determines pixel positions in the flat panel display 310 that generate light rays capable of reproducing a stereoscopic image in accordance with the user's eye positions and line of sight, and causes the pixels, taking color into consideration, to emit light at a brightness that matches the image signal. Specifically, this may be realized by performing the process described below.

[0043] The processing of the rendering unit 132 will be described with reference to FIG. 6. FIG. 6 is a diagram for explaining the processing of rendering an image by the image display device according to the present disclosure. FIG. 6 shows a flat panel display 310, which will be described later, a microlens array 320, a lens LN that enlarges the pupil of the user's eye, and the user's line of sight AX1. As shown in FIG. 6, the rendering unit 132 gives a drawing instruction to the flat panel display 310 of the display device 300 to cause the pixels of the flat panel display 310 to emit light. For example, the red light emitters R1, R2, and R3 shown in FIG. 6 are turned on. As a result, the red light beams RL1, RL2, and RL3 intersect to form a red stereoscopic image RPLS1. Note that when the green light emitters G1, G2, and G3 shown in FIG. 6 are also turned on in the same manner, the green light beams GL1, GL2, and GL3 intersect to form a green stereoscopic image GPLS1.

[0044] Next, crosstalk will be described with reference to FIG. 7. FIG. 7 is a diagram for explaining crosstalk that may occur in rendering by the image display device according to the present disclosure. As shown in FIG. 7, when the red light emitters R1, R2, and R3 are turned on to form a red stereoscopic image RPLS2, the green light emitters G1, G2, and G3 are also turned on. Then, as shown in FIG. 7, the crosstalk light CSTL of the green light beam emitted from the green light emitter G1 passes through the position where the red stereoscopic image RPLS2 is formed, so crosstalk occurs in which green is mixed into the red stereoscopic image RPLS2. As shown in the example shown in FIG. 7, when the colors of the crosstalk light are different, the color is mixed into the stereoscopic image and becomes a different color from the original, for example, a slightly bright yellow bright spot is formed in a dark red background, which is particularly noticeable. Even if the crosstalk light is also light from the same red pixel, the reproduced stereoscopic image will be slightly blurred or partly brighter than the original.

[0045] The range specifying unit 133 specifies an unnecessary light source range indicating a range of candidates for pixels that are the source of crosstalk light, which indicates a light ray from a light source other than the light source of the drawn stereoscopic image that passes through the position of the stereoscopic image. In other words, it can be said that the range specifying unit 133 specifies a range in which there is a possibility that a pixel that is the source of crosstalk light exists.

[0046] Specifically, the area specifying unit 133 specifies the unnecessary light source area based on the size of the user's pupil and the positional relationship between the user's pupil, the stereoscopic image, and the flat panel display. The process of the area specifying unit 133 will be described with reference to Figs. 8 and 9.

[0047] Figure 8 is a diagram explaining the description of light rays by the Plenoptioc Function. As shown in Figure 8, any light ray L in space can generally be expressed by the Plenoptioc Function proposed by Adelson and Bergen using three dimensions (x, y, z) that represent the light emission position and two dimensions (θ, φ) that represent the direction.

[0048] Using FIG. 9, the case of a light field display will be considered. FIG. 9 is a diagram for explaining the processing of the range specification unit and the turn-off instruction unit of the image display device according to the present disclosure. (x, y, z) are the position coordinates of the spatial stereoscopic image to be displayed, and (θ, φ) are the line of sight direction toward the pupil detected by the line of sight detection unit 131. Then, a line of sight axis AX1 connecting the pupil to the spatial stereoscopic image and the central pixel of the flat panel display 310 for displaying it is determined. Next, a virtual pupil circle C0 is assumed to be slightly larger than the pupil size with the pupil as the center, and the circle C is reduced in proportion to the distance from the spatial stereoscopic image to the pupil and the distance from the spatial stereoscopic image to the flat panel display 310. The circle C is projected onto the flat panel display 310 and specified as the unnecessary light source range RG. In other words, pixels other than the pixels for displaying the desired stereoscopic image that fall within the unnecessary light source range RG are candidates for pixels that may cause crosstalk.

[0049] For example, if the size of the circle on the pupil side is 4 mm in radius, and you are viewing a stereoscopic image displayed 50 mm in front of flat panel display 310 600 mm away, i.e., 550 mm away, then proportionally, the circle on flat panel display 310 will have a radius of 0.37 mm, and any pixels within this circle other than the pixels displaying the desired stereoscopic image can be determined to be pixels that may cause crosstalk.

[0050] Furthermore, the range specifying unit 133 may specify, as the unnecessary light source range, a range corresponding to a predetermined angle centered on the line of sight direction, taking into consideration the discrimination field of view, which is a range in which high visual acuity can be obtained without moving the eyeballs, with respect to the unnecessary light source range. That is, the range specifying unit 133 may set the unnecessary light source range for checking whether crosstalk light is occurring to about ±2.5 degrees centered on the line of sight of the discrimination field of view, which is a range in which high visual acuity can be obtained without moving the eyeballs, taking into consideration the visual field characteristics of the human eye. This makes it possible to efficiently prevent crosstalk in a range where visual acuity is high and crosstalk is easily noticeable, and to reproduce a high-quality stereoscopic image.

[0051] Specifically, the crosstalk check range is a radius of 24mm + 0.37mm, equivalent to ±2.5 degrees, centered on the line of sight around a stereoscopic image displayed 550mm ahead. When viewing a range that is more than ±2.5 degrees away from the center of the visual field, the eye naturally moves in that direction, changing the direction of the line of sight and changing the rendering state. Therefore, there is no need to constantly be aware of and deal with crosstalk light that is more than ±2.5 degrees away. In terms of human visual field characteristics, the range in which humans can obtain high visual acuity without moving their eyes (discrimination visual field) is approximately ±1.5 to 2.5 degrees.

[0052] Furthermore, the range specifying unit 133 may specify the unnecessary light source range according to the gaze direction by reading the unnecessary light source range from the map information storage unit 121 based on the gaze direction. As described above, the unnecessary light source range according to the gaze direction is stored in advance in the map information storage unit 121. Therefore, if the gaze detection unit 131 can detect the gaze direction of the user, the unnecessary light source range can be specified by reading the unnecessary light source range according to the gaze direction from the map information storage unit 121.

[0053] In addition, when there are multiple users, the range identification unit 133 may identify the unnecessary light source range by performing the above-mentioned processing for each user according to the gaze direction detected by the gaze detection unit 131 for each user.

[0054] The pixel specifying unit 134 specifies light source pixels that indicate pixels that are light sources of crosstalk light among the pixels in the unnecessary light source range. Specifically, the pixel specifying unit 134 specifies pixels in the unnecessary light source range, in which a straight line connecting the pixel and the center of the corresponding microlens passes through the stereoscopic image, as pixels related to the occurrence of crosstalk.

[0055] Specifically, the pixel specifying unit 134 specifies, among the pixels in the unnecessary light source range, those pixels in which a straight line connecting the centers of the microlenses in which the pixels are within the range of the viewing angle of the microlenses passes through the target stereoscopic image as pixels that cause crosstalk. Explaining with reference to Fig. 9, the green light ray G4 emitted from the green light emitter G1 located in the unnecessary light source range passes through the microlens and the red stereoscopic image RPLS3, and is therefore specified as the crosstalk light CSLT.

[0056] The light ray processing unit 135 performs a predetermined process on the identified light source pixel or the light ray by the light source pixel, and does not allow the light ray by the light source pixel to pass through the position of the stereoscopic image. The configuration of the light ray processing unit 135 that executes such a process will be described with reference to FIG. 10. FIG. 10 is a diagram showing an example of the configuration of the light ray processing unit of the control device according to the present disclosure. As shown in FIG. 10, the light ray processing unit 135 according to the present disclosure includes at least one of a turn-off instruction unit 1351 and a shield instruction unit 1352.

[0057] The light-off instruction unit 1351 turns off the light emission of the identified light source pixel. That is, when the light ray processing unit 135 has the light-off instruction unit 1351, the light ray processing unit 135 issues a light-off instruction to the pixel identified by the pixel identification unit 134 to turn off the light emission of the identified pixel, thereby preventing the light ray from the light source pixel from passing through the position of the stereoscopic image. Explained with reference to FIG. 9, if the green light emitter G1 is identified as the pixel of the light source of the crosstalk light, turning off the green light emitter G1 turns off the crosstalk light CSLT and the green light ray GL1. This makes it possible to turn off the light emission of the pixel that causes the crosstalk, and therefore it is possible to appropriately suppress the crosstalk. In addition, in a display, a pixel that emits unnecessary light due to a defective lighting is generally more noticeable than a black spot that does not emit light, so it is advantageous not to emit the crosstalk light.

[0058] The light-off instruction unit 1351 may check whether the pixel causing the crosstalk is a different color from the stereoscopic image, and may turn off the pixel causing the crosstalk light only if the crosstalk light is a different color. Since crosstalk is most noticeable when the colors are different, the process can be simplified by ignoring crosstalk of the same color.

[0059] The shielding instruction unit 1352 instructs the shielding filter 350 placed between the flat panel display 310 and the microlens array 320 to shield the pixel of the shielding filter 350 corresponding to the position of the identified light source pixel. That is, when the light ray processing unit 135 has a configuration including the shielding instruction unit 1352, the shielding filter 350 corresponding to the position of the identified light source pixel is shielded by the shielding filter 350, so that the light ray from the light source pixel is not passed through the position of the stereoscopic image. The process of the shielding instruction unit 1352 will be specifically described with reference to FIG. 11.

[0060] Fig. 11 is a diagram for explaining the processing of the shading instruction unit of the image display device according to the present disclosure. As shown in Fig. 11, the display device 300 may include a shading filter 350 between the front of the flat panel display 310 and the microlens array 320. The configuration of the shading filter 350 will be described later.

[0061] 11, the shielding instruction unit 1352 issues an instruction to shield the pixel of the shielding filter 350 corresponding to the position of the identified light source pixel, so that the light ray GL4 from the identified light source pixel (green light emitter G1) does not pass through the position of the stereoscopic image. This makes it possible to shield the light ray from the light emitted from the pixel that causes crosstalk and not allow it to pass through the position of the stereoscopic image, thereby making it possible to appropriately suppress crosstalk.

[0062] The size of one pixel of the shielding filter 350 may be half the size of a pixel of the flat panel display 310. An embodiment of such a shielding filter 350 will be described with reference to FIG. 12. FIG. 12 is a diagram illustrating a shielding filter of an image display device according to the present disclosure. As shown in FIG. 12, the size of one pixel of the shielding filter 350 may be half the size of a pixel of the flat panel display 310.

[0063] Further, the shading instruction unit 1352 may give an instruction to shading two pixels of the shading filter 350 corresponding to the position of the identified light source pixel, in other words, of the two pixels of the shading filter 350 that face the identified light source pixel perpendicularly, the pixel on the side of the stereoscopic image where crosstalk occurs. In this way, in the example shown in Fig. 12, the light ray GL1 of the light source G can pass through the shading filter 350 as usual to form a stereoscopic image, so that a decrease in the brightness of the stereoscopic image due to the light ray from the light source G can be suppressed compared to the case where the light source G is turned off. In other words, while suppressing crosstalk due to the emission of the pixel that causes crosstalk, a decrease in the brightness of other stereoscopic images where no crosstalk occurs can also be suppressed.

[0064] The luminance increasing unit 136 increases the luminance of pixels that are not turned off among the pixels that are the light source of the stereoscopic image in order to compensate for the luminance of the stereoscopic image that is affected by the turning off of the pixels of the light source of the crosstalk light. The amount of increase in luminance by the luminance increasing unit 136 may be a value previously stored in the storage unit 120. That is, the amount of increase in luminance may be a predetermined value that is determined in advance, or may be a value calculated based on the positional relationship between the stereoscopic image and the pixels whose luminance is to be increased. To explain with reference to FIG. 9, when the green light emitter G1 is turned off, the luminance of the green light emitters G2 and G3 is increased.

[0065] Furthermore, the luminance increasing unit 136 may check whether the pixels causing crosstalk are of different colors, and only if the crosstalk light is of different colors, increase the luminance of pixels that are not turned off among the pixels that are the light source of the stereoscopic image in order to compensate for the luminance of the stereoscopic image affected by the turning off of the pixels of the light source of the crosstalk light. Since crosstalk is most noticeable when it is of different colors, the processing can be simplified by ignoring crosstalk of the same color.

[0066] The luminance increasing unit 136 may store the relationship between the line of sight direction and the pixels whose luminance is to be increased or the color tone in the storage unit 120, and may increase the luminance of the pixels that are not turned off among the pixels that are the light source of the stereoscopic image by reading the information from the storage unit 120. This can reduce the calculation load when performing real-time processing.

[0067] (Display Device Configuration) Next, the configuration of the display device according to the present disclosure will be described with reference to FIG. 13. FIG. 13 is a diagram showing a configuration example of the display device according to the present disclosure. As shown in FIG. 13, the display device 300 according to the present disclosure includes a flat panel display 310, a microlens array 320, a driving circuit 330, an I / F unit 340, and a shielding filter 350. Note that the shielding filter 350 is not an essential component. Below, these components will be described in order.

[0068] The flat panel display 310 is a display that displays various images. The flat panel display 310 may be realized by an organic EL (Electro Luminescence) display, a micro LED (Light Emitting Diode) display, or the like.

[0069] Microlens array 320 is an optical element in which many minute microlenses measuring in the order of μm to mm are integrated. Microlens array 320 is disposed in front of the light emission direction of pixels of flat panel display 310. Microlens array 320 reproduces the light emitted by an object by changing the direction of light emitted from flat panel display 310.

[0070] The driving circuit 330 provides control signals for driving the pixels of the flat panel display 310 and the shielding filter 350. The driving circuit 330 includes switching elements for driving the flat panel display 310 and the shielding filter 350. That is, the brightness of the pixels of the flat panel display 310 and the shielding filter 350 is controlled by the control signals sent from the driving circuit 330.

[0071] The I / F unit 340 receives a control signal from the control device 100. The I / F unit 340 may receive the control signal from the control device 100 via a wired or wireless connection, and in the wired connection, the I / F unit 340 may include a connection terminal such as a USB, SDI, or HDMI. In the wireless connection, the I / F unit 340 may be realized by a wireless LAN defined in IEEE802.11.

[0072] The shielding filter 350 is a filter that can switch between blocking and passing light rays by an electric signal. The shielding filter 350 is disposed, for example, between the flat panel display 310 and the microlens array 320. The shielding filter 350 may be, for example, a twisted nematic (TN) type liquid crystal in which a twisted liquid crystal (nematic liquid crystal) is sandwiched between two deflection filters with orthogonal deflection directions, via two glass plates. Note that the blocking and passing of light rays may be switched by controlling the orientation of the liquid crystal by applying an electric signal to the liquid crystal using a thin film transistor (TFT) corresponding to the RGB subpixels of each pixel.

[0073] The shielding filter 350 may be realized by an electrochromic element using an electrochromic (EC) material that exhibits a reversible change in optical properties in response to an electric signal. The electrochromic element changes its optical properties when a voltage is applied, and therefore can control the blocking and passing of light by an electric signal. The size of the pixel of the shielding filter 350 is preferably half the size of the pixel of the flat panel display 310.

[0074] (Image display method and program) Next, the image display method according to the present disclosure will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of the image display method according to the present disclosure. Below, the image display method according to the present disclosure will be described along the flow shown in Fig. 14.

[0075] First, the image display device 1 detects the line of sight of the user (step S101). Next, the image display device 1 causes the display device 300 to render a stereoscopic image according to the detected line of sight (step S102). Next, the image display device 1 identifies an unnecessary light source range indicating a range of candidates for pixels that become the light source of the crosstalk light (step S103). Next, the image display device 1 identifies light source pixels indicating pixels that become the light source of the crosstalk light (step S104). Next, the image display device 1 performs a predetermined process on the identified light source pixels or light rays from the light source pixels, and does not allow the light rays from the light source pixels to pass through the position of the stereoscopic image (step S105).

[0076] The image display method described above may be executed by the image display device 1, or may be executed by a program that causes the control unit 130 of the control device 100 of the image display device 1 to execute processing.

[0077] According to this, a candidate range of pixels that are the source of crosstalk light is specified according to the direction of the user's line of sight, pixels that are the source of crosstalk light are specified from within the range, and a predetermined process is performed on the specified light source pixels or light rays from the light source pixels, so that the light rays from the light source pixels do not pass through the position of the stereoscopic image. Therefore, an image display method and program that can appropriately suppress the occurrence of crosstalk can be provided.

[0078] (Composition and Effects) The image display device 1 according to the present disclosure includes a gaze detection unit 131 that detects the direction of a user's gaze, a rendering unit 132 that causes the display device 300 to render a stereoscopic image according to the direction of the user's gaze, a range identification unit 133 that identifies an unnecessary light source range that indicates a range of pixels that can become the source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and that pass through the position of the stereoscopic image, a pixel identification unit 134 that identifies light source pixels that indicate pixels that can become the source of crosstalk light among the pixels in the unnecessary light source range, and a light ray processing unit 135 that performs a predetermined process on the identified light source pixels or light rays from the light source pixels, and does not allow the light rays from the light source pixels to pass through the position of the stereoscopic image.

[0079] According to this configuration, a candidate range of pixels that become the source of crosstalk light is specified according to the direction of the user's line of sight, pixels that become the source of crosstalk light are specified from within the range, and a predetermined process is performed on the specified light source pixels or light rays from the light source pixels, so that the light rays from the light source pixels do not pass through the position of the stereoscopic image. Therefore, an image display device 1 that can appropriately suppress the occurrence of crosstalk can be provided.

[0080] The light processing unit 135 of the image display device 1 according to the present disclosure includes a light-off instruction unit 1351 that turns off the light emission of the identified light source pixel.

[0081] According to this configuration, a candidate range of pixels that become the source of crosstalk light is specified according to the direction of the user's line of sight, pixels that become the source of crosstalk light are specified from within the range, and the light emission of the specified light source pixels can be turned off. Therefore, it is possible to provide an image display device 1 that can appropriately suppress the occurrence of crosstalk.

[0082] The light processing unit 135 of the image display device 1 according to the present disclosure includes a flat panel display 310 and a shielding instruction unit 1352 that instructs a shielding filter 350 placed between the flat panel display 310 and the microlens array 320 to shield a pixel of the shielding filter 350 that corresponds to the position of a specified light source pixel.

[0083] According to this configuration, a candidate range of pixels that become the source of crosstalk light is specified according to the user's line of sight, pixels that become the source of crosstalk light are specified from within the range, and an instruction is given to block the pixel of the blocking filter 350 that corresponds to the position of the specified light source pixel, thereby preventing the light beam from the light source pixel from passing through the position of the stereoscopic image. Therefore, an image display device 1 that can appropriately suppress the occurrence of crosstalk can be provided.

[0084] The range specifying section 133 of the image display device 1 according to the present disclosure specifies the unnecessary light source range based on the size of the user's pupil and the positional relationship between the user's pupil, the stereoscopic image, and the flat panel display.

[0085] According to this configuration, the unnecessary light source range is specified based on the size of the user's pupil and the positional relationship between the user's pupil, the stereoscopic image, and the flat panel display, and pixels that are related to the occurrence of crosstalk are specified as pixels where a straight line connecting the center of the corresponding microlens passes through the stereoscopic image, and the emission of the specified light source pixels can be turned off. Therefore, it is possible to provide an image display device 1 that can appropriately suppress the occurrence of crosstalk.

[0086] The image display device 1 according to the present disclosure further includes a brightness increasing section 136 that increases the brightness of pixels that are not turned off among the pixels that serve as light sources for the stereoscopic image, so as to compensate for the brightness of the stereoscopic image that is affected by the turning off of pixels of the light source of the crosstalk light.

[0087] This configuration makes it possible to compensate for the luminance of a stereoscopic image that is affected by the turning off of pixels of the light source of the crosstalk light, thereby suppressing the occurrence of crosstalk and allowing the stereoscopic image to be displayed clearly.

[0088] The image display method according to the present disclosure includes the steps of detecting the direction of a user's gaze, causing the display device 300 to render a stereoscopic image in accordance with the direction of the user's gaze, identifying an unnecessary light source range indicating a range of candidate pixels that can be the source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and which pass through the position of the stereoscopic image, identifying light source pixels among the pixels in the unnecessary light source range that indicate pixels that can be the source of crosstalk light, and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels to prevent the light rays from the light source pixels from passing through the position of the stereoscopic image.

[0089] According to this configuration, a candidate range of pixels that are the source of crosstalk light is specified according to the direction of the user's line of sight, pixels that are the source of crosstalk light are specified from within the range, and a predetermined process is performed on the specified light source pixels or light rays from the light source pixels, so that the light rays from the light source pixels do not pass through the position of the stereoscopic image. Therefore, an image display method that can appropriately suppress the occurrence of crosstalk can be provided.

[0090] The program according to the present disclosure causes a computer to execute the steps of detecting the direction of a user's gaze, causing the display device 300 to render a stereoscopic image in accordance with the direction of the user's gaze, identifying an unnecessary light source range indicating a range of candidate pixels that can be the source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and which pass through the position of the stereoscopic image, identifying light source pixels among the pixels in the unnecessary light source range that indicate pixels that can be the source of crosstalk light, and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels to prevent the light rays from the light source pixels from passing through the position of the stereoscopic image.

[0091] According to this configuration, a candidate range of pixels that are the source of crosstalk light is specified according to the direction of the user's line of sight, pixels that are the source of crosstalk light are specified from within the range, and a predetermined process is performed on the specified light source pixel or a light beam from the light source pixel, so that the light beam from the light source pixel does not pass through the position of the stereoscopic image. Therefore, a program that can appropriately suppress the occurrence of crosstalk can be provided.

[0092] Although the embodiment of the present disclosure has been described above, the embodiment is not limited by the contents of this embodiment. In addition, the above-mentioned components include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-mentioned components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the above-mentioned embodiment. [Explanation of symbols]

[0093] 1 Image display device 100 Control device 110 I / F section 120 Storage section 121 Map information storage unit 122 Image storage unit for drawing 130 Control section 131 Line of Sight Detection Unit 1311 Acquisition Department 1312 Face detection unit 1313 Eye information detection unit 1314 Face direction detection unit 1315 Line of Sight Detection Unit 1316 Output section 132 Rendering Department 133 Range Specification Section 134 Pixel Identification Unit 135 Light Processing Unit 1351 Lights off indicator 1352 Shielding instruction part 136 Brightness Increaser 200 Cameras 300 display device 310 Flat Panel Display 320 Microlens Array 330 Drive circuit 340 I / F section 350 Shielding Filter

Claims

1. A gaze detection unit that detects the gaze direction of a user; a rendering unit that causes a display device to render a stereoscopic image in accordance with a user's line of sight; a range specifying unit that specifies an unnecessary light source range indicating a range of candidates for pixels that are a source of crosstalk light indicating light rays from a light source other than a light source of a drawn stereoscopic image and that pass through a position of the stereoscopic image; and a pixel specifying unit that specifies a light source pixel indicating a pixel that is a light source of crosstalk light among pixels in the unnecessary light source range; a light ray processing unit that performs a predetermined process on the identified light source pixel or a light ray from the light source pixel and does not allow the light ray from the light source pixel to pass through a position of the stereoscopic image; An image display device comprising:

2. The light ray processing unit includes a light-off instruction unit that turns off the light emission of the identified light source pixel. The image display device according to claim 1 .

3. the light processing unit includes a flat panel display and a shielding instruction unit that issues an instruction to a shielding filter placed between a microlens array to shield a pixel of the shielding filter corresponding to the identified position of the light source pixel, The image display device according to claim 1 .

4. the range specifying unit specifies the unnecessary light source range based on a size of the user's pupil and a positional relationship between the user's pupil, a stereoscopic image, and a flat panel display; The image display device according to claim 1 .

5. A brightness increasing unit that increases the brightness of pixels that are not turned off so as to compensate for the brightness of the stereoscopic image affected by the turning off of pixels of the light source of the crosstalk light.

3. The image display device according to claim 1 or 2.

6. Detecting a user's gaze direction; A step of causing a display device to render a stereoscopic image according to a line of sight of a user; A step of identifying an unnecessary light source range indicating a range of candidates for pixels that are the source of crosstalk light indicating light rays from a light source other than the light source of the drawn stereoscopic image and that pass through the position of the stereoscopic image; identifying light source pixels indicating pixels that are a source of crosstalk light among pixels in the unnecessary light source range; performing a predetermined process on the identified light source pixel or a light ray from the light source pixel, so as not to pass through the position of the stereoscopic image; 23. A method for displaying an image, comprising:

7. Detecting a user's gaze direction; A step of causing a display device to render a stereoscopic image according to a line of sight of a user; A step of identifying an unnecessary light source range indicating a range of pixels that are candidates for a light source of crosstalk light indicating a light ray from a light source other than a light source of a drawn stereoscopic image and passing through a position of the stereoscopic image; identifying light source pixels indicating pixels that are a source of crosstalk light among pixels in the unnecessary light source range; performing a predetermined process on the identified light source pixel or a light ray from the light source pixel, so as not to pass through the position of the stereoscopic image; A program that causes a computer to execute the following.

Images