Image display device, image display method, and program

Abstract

Provided are an image display device, an image display method, and a program with which it is possible to appropriately suppress occurrence of crosstalk. An image display device according to the present disclosure comprises: a line-of-sight detection unit that detects a user's line-of-sight direction; a rendering unit that causes the display device to render a stereoscopic image in accordance with the user's line-of-sight direction; a light source range identification unit that identifies an unnecessary light source range indicating a range of candidate pixels for serving as a light source of crosstalk light indicating a light beam from a light source other than the light source of the rendered stereoscopic image, the light beam passing through the position of the stereoscopic image; a pixel identification unit that identifies light source pixels that indicate the pixels serving as the light source of the crosstalk light, from among the pixels in the unnecessary light source range; and a light beam processing unit that performs prescribed processing on the identified light source pixels or the light beam from the light source pixels to prevent the light beam from the light source pixels from passing through the position of the stereoscopic image.

Description

Image display device, image display method, and program

[0001] The present disclosure relates to an image display device, an image display method, and a program.

[0002] For example, progress is being made in the development of light field displays that reproduce the light rays emitted from an object captured by a light field camera on a display.

[0003] The following Patent Document 1 discloses an eyeball projection type image display device that includes a field lens, a microlens array behind it, and an emissive display device such as an LED array as an image display means behind it for displaying an image.

[0004] Japanese Patent Application Publication No. 10-319342

[0005] However, in light field displays, crosstalk occurs, in which the stereoscopic image is affected by light emitted from pixels other than those that serve as light sources for generating the desired stereoscopic image, which can result in blurring, uneven color, and uneven brightness in the stereoscopic image.

[0006] In view of the above-described problems, an object of the present disclosure is to provide an image display device, an image display method, and a program that can appropriately suppress the occurrence of crosstalk.

[0007] In order to solve the above-mentioned problems and achieve the objectives, the image display device according to the present disclosure includes a gaze detection unit that detects the direction of the user's gaze, a rendering unit that causes the display device to draw a stereoscopic image according to the direction of the user's gaze, a range identification unit that identifies an unnecessary light source range that indicates a range of pixels that can become a source of crosstalk light, which indicates light rays from a light source other than the light source of the drawn stereoscopic image and that pass through the position of the stereoscopic image, a pixel identification unit that identifies light source pixels that indicate pixels that can become a source of crosstalk light among the pixels in the unnecessary light source range, and a light ray processing unit that performs predetermined processing on the identified light source pixels or light rays from the light source pixels, and does not allow light rays from the light source pixels to pass through the position of the stereoscopic image.

[0008] In order to solve the above-mentioned problems and achieve the objectives, the image display method of the present disclosure includes the steps of detecting a user's line of sight, causing a display device to render a stereoscopic image according to the user's line of sight, identifying an unnecessary light source range indicating a range of pixels that can be the source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and that pass through the position of the stereoscopic image, identifying light source pixels that indicate pixels that can be the source of crosstalk light from among the pixels in the unnecessary light source range, and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels, and preventing the light rays from the light source pixels from passing through the position of the stereoscopic image.

[0009] In order to solve the above-mentioned problems and achieve the objectives, the program of the present disclosure causes a computer to execute the following steps: detecting the user's line of sight; causing a display device to draw a stereoscopic image according to the user's line of sight; identifying an unnecessary light source range indicating a range of candidate pixels that can be the source of crosstalk light, which indicates light rays from a light source other than the light source of the drawn stereoscopic image and that pass through the position of the stereoscopic image; identifying light source pixels from among the pixels in the unnecessary light source range that indicate pixels that can be the source of crosstalk light; and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels, thereby preventing the light rays from the light source pixels from passing through the position of the stereoscopic image.

[0010] According to the present disclosure, an image display device, an image display method, and a program capable of appropriately suppressing the occurrence of crosstalk can be provided.

[0011] FIG. 1 is a diagram illustrating an example configuration of an image display device according to the present disclosure. FIG. 2 is a diagram illustrating an example configuration of a control device according to the present disclosure. FIG. 3 is a diagram illustrating an example of information stored in a map information storage unit of a control device according to the present disclosure. FIG. 4 is a diagram illustrating an example of information stored in a drawing image storage unit of a control device according to the present disclosure. FIG. 5 is a diagram illustrating an example configuration of a gaze detection unit of a control device according to the present disclosure. FIG. 6 is a diagram illustrating image rendering processing by an image display device according to the present disclosure. FIG. 7 is a diagram illustrating crosstalk that may occur in rendering by an image display device according to the present disclosure. FIG. 8 is a diagram illustrating a description of light rays using the Plenoptioc Function. FIG. 9 is a diagram illustrating processing by a range specification unit and a light-off instruction unit of an image display device according to the present disclosure. FIG. 10 is a diagram illustrating an example configuration of a light ray processing unit of a control device according to the present disclosure. FIG. 11 is a diagram illustrating processing by a light ray processing unit of an image display device according to the present disclosure. FIG. 12 is a diagram illustrating a shading filter of an image display device according to the present disclosure. FIG. 13 is a diagram illustrating an example configuration of a display device according to the present disclosure. FIG. 14 is a flowchart illustrating a flow of an image display method according to the present disclosure.

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. However, the present disclosure is not limited to the embodiments described below.

[0013] (Configuration of image display device) First, the configuration of an image display device 1 according to the present disclosure will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example configuration of an image display device according to the present disclosure. As shown in Fig. 1, the image display device 1 according to the present disclosure includes a control device 100, a camera 200, and a display device 300. Below, a brief description of these configurations will be given.

[0014] The control device 100 executes various types of arithmetic processing and processing for realizing functions, and controls the display device 300 to render a stereoscopic image. The control device 100 detects the user's line of sight based on the face image of the user captured by the camera 200, and causes the display device 300 to display a stereoscopic image according to the detected line of sight.

[0015] Camera 200 captures a facial image of a user. Camera 200 includes an optical element and an imaging element. The optical element is an element that constitutes an optical system, such as a lens, a mirror, a prism, or a filter. The imaging element is an element that converts light incident through the optical element into an image signal, which is an electrical signal. The imaging element may be, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0016] The display device 300 is a device that displays a stereoscopic image. The display device 300 may be a light field display. A light field display displays a stereoscopic image by refracting light rays emitted from the display by passing the light rays through microlenses and reproducing light rays emitted by an object. The light field display may be realized, for example, by an integral imaging method.

[0017] Note that image display device 1 does not need to be an integrated device including control device 100, camera 200, and display device 300, and each may be an independent device. For example, image display device 1 may be realized by connecting control device 100, camera 200, and display device 300 to a network and transmitting and receiving information to and from each other via the network. In this case, image display device 1 can be said to be an image display system comprising control device 100, camera 200, display device 300, and the network.

[0018] (Configuration of Control Device) Next, the configuration of the control device 100 according to the present disclosure will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example configuration of the control device according to the present disclosure. As shown in Fig. 2, the control device 100 according to the present disclosure includes an I / F unit 110, a storage unit 120, and a control unit 130. Below, these components will be described in order.

[0019] The I / F unit 110 is responsible for transmitting and receiving control signals, video signals, and the like, via wired or wireless connections, between the camera 200 and the display device 300. In the case of wired connections, the I / F unit 110 may be realized by, for example, a Universal Serial Bus (USB), a Serial Digital Interface (SDI), or a High-Definition Multimedia Interface (HDMI (registered trademark)). In the case of wireless connections, the I / F unit 110 may be realized by a wireless LAN (Local Area Network) defined by IEEE 802.11.

[0020] The storage unit 120 is a storage device that stores various types of information. The storage unit 120 includes a main storage device and an auxiliary storage device. The main storage device may be realized by a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash memory. The auxiliary storage device may be realized by a hard disk drive or an SSD (Solid State Drive), for example.

[0021] 2, the storage unit 120 includes a map information storage unit 121 and a drawing image storage unit 122. An example of information stored in these components will be described below in order.

[0022] The map information storage unit 121 stores information relating to the relationship between the line of sight direction and the unnecessary light source range. An example of the information stored in the map information storage unit 121 will now be described with reference to Fig. 3. Fig. 3 is a diagram illustrating an example of the information stored in the map information storage unit of the control device according to the present disclosure.

[0023] As shown in FIG. 3, the map information storage unit 121 stores information relating to the items "line of sight direction," "X coordinate range," and "Y coordinate range."

[0024] "Gaze direction" is information indicating the gaze direction of the user, and for example, the gaze direction of the user may be represented by the Pleoptioc function and stored. "X coordinate range" is information indicating the X coordinate value of the unnecessary light source range associated with the "gaze direction." "Y coordinate range" is information indicating the Y coordinate value of the unnecessary light source range associated with the "gaze direction." The X coordinate and Y coordinate are coordinates indicating a position on the surface of the flat panel display 310, and the two are perpendicular to each other.

[0025] That is, FIG. 3 shows an example in which the X coordinate range of the unnecessary light source range associated with the line of sight direction "LOS#1" is stored as "XCRR#1" and the Y coordinate is stored as "YCRR#1."

[0026] The map information storage unit 121 is not limited to information relating to the items "line of sight direction," "X coordinate range," and "Y coordinate range," and may store information relating to the relationship between any other line of sight direction and the unnecessary light source range.

[0027] The drawing image storage unit 122 stores information relating to an image to be drawn by the display device 300. An example of the information stored in the drawing image storage unit 122 will now be described with reference to Fig. 4. Fig. 4 is a diagram illustrating an example of information stored in the drawing image storage unit of the control device according to the present disclosure.

[0028] As shown in FIG. 4, the drawing image storage unit 122 stores information relating to the items "image ID" and "image data."

[0029] The "video ID" is an identifier for identifying the drawing video, and is represented by a character string, a number, etc. The "video data" is video data of the drawing video identified by the "video ID", and may be, for example, video data captured by a light field camera or video data generated by CG (Computer Graphics).

[0030] That is, FIG. 4 shows an example in which video data "VDDT#1" of a drawing video identified by a video ID "VDID#1" is stored.

[0031] The image-to-be-drawn storage unit 122 is not limited to information relating to the items "image ID" and "image data," and may store information relating to images to be drawn on any other display device 300.

[0032] The control unit 130 is a controller that executes various types of arithmetic processing and processing to realize functions. The control unit 130 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs stored in the storage unit 120 using RAM as a work area. The control unit 130 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0033] As shown in FIG. 2 , the control unit 130 includes a gaze detection unit 131, a rendering unit 132, a range identification unit 133, a pixel identification unit 134, a light processing unit 135, and a brightness enhancement unit 136. The control unit 130 realizes these functions and performs these processes by reading and executing a program (software) from the storage unit 120. Note that these functions of the control unit 130 may be realized by electronic circuits. Furthermore, the control unit 130 may perform these processes using a single CPU, or may include multiple CPUs that perform these processes in parallel. Each of these components will be described below in order.

[0034] The gaze detection unit 131 detects the user's gaze direction based on image data of the user's face. That is, the gaze detection unit 131 detects the user's gaze direction from the image data using image processing technology. Specifically, the gaze detection unit 131 may be provided with the configuration described below and detect the user's gaze direction.

[0035] The configuration of the gaze detection unit 131 will be described with reference to Fig. 5. Fig. 5 is a diagram illustrating an example configuration of the gaze detection unit of the control device according to the present disclosure. As shown in Fig. 5, the gaze detection unit 131 includes an acquisition unit 1311, a face detection unit 1312, an pupil information detection unit 1313, a face direction detection unit 1314, a gaze detection unit 1315, and an output unit 1316. Below, these components will be described in order.

[0036] The acquisition unit 1311 acquires image data captured by the camera 200. Here, the image data acquired by the acquisition unit 1311 includes the user's face. After acquiring the image data, the acquisition unit 1311 stores the acquired image data in the storage unit 120 and then outputs the image data to the face detection unit 1312.

[0037] The face detection unit 1312 detects a face area including at least a part of the user's face from the image data acquired by the acquisition unit 1311. For example, the face detection unit 1312 may detect the face area using a classifier such as a convolutional neural network (CNN) that has been trained in advance to detect the face area. Note that the face area is detected as, for example, a rectangular area large enough to include the entire face.

[0038] The pupil information detection unit 1313 detects pupil information indicating information related to the user's pupil based on the image data. The pupil information includes, for example, coordinate data indicating the outer edge of the user's pupil, or information indicating the length of the outer edge of the user's pupil, such as the radius or diameter, and coordinate data of the center of the user's pupil. First, the pupil information detection unit 1313 detects an eye region containing the user's eyes from the face region detected by the face detection unit 1312. The eye region may be detected using a classifier such as a CNN that has been trained in advance to detect face regions. Next, the pupil information detection unit 1313 detects the user's pupil. An area estimated to contain the pupil is detected by narrowing down the eye region based on predetermined criteria. Based on this, coordinate data indicating the outer edge of the pupil, or information indicating the length of the outer edge of the pupil, such as the radius or diameter, and coordinate data of the center of the pupil are detected.

[0039] The face direction detection unit 1314 detects facial feature points from the face area detected by the face detection unit 1312 and detects face direction information indicating the direction of the user's face from the arrangement pattern of the detected feature points. The process of extracting facial feature points from the face area may be realized by using a classifier such as a CNN that has learned facial features. Note that a facial feature point refers to one or more points located in characteristic positions on each of multiple parts that make up the face, such as the outer corners and inner corners of the eyes, the facial contour, the bridge of the nose, the corners of the mouth, and the eyebrows.

[0040] The gaze detection unit 1315 detects the gaze direction indicating the user's gaze based on the pupil information detected by the pupil information detection unit 1313 and the facial direction information detected by the facial direction detection unit 1314. The gaze detection unit 1315 may detect the gaze information by applying a known gaze detection process that detects the gaze using, for example, a three-dimensional eyeball model to the pupil information and the facial direction information. Here, the gaze information may include a vector that three-dimensionally indicates the direction of the user's gaze, or may include coordinate data of a gaze point on a predetermined target surface (for example, the display device 300).

[0041] The output unit 1316 outputs the user's gaze direction detected by the gaze detection unit 1315. Specifically, the output unit 1316 outputs the detected user's gaze direction to the rendering unit 132 and the range identification unit 133.

[0042] The rendering unit 132 causes the display device 300 to render a stereoscopic image according to the user's line of sight. That is, the rendering unit 132 issues a rendering instruction to the display device 300 so that the stereoscopic image is positioned in the user's line of sight. The rendering unit 132 determines pixel positions on the flat panel display 310 that generate light rays that can reproduce a stereoscopic image, in accordance with the user's eye position and line of sight, and causes the pixels, taking color into consideration, to emit light at a brightness that matches the image signal. Specifically, this may be achieved by performing the processing described below.

[0043] The processing of the rendering unit 132 will be described with reference to FIG. 6 . FIG. 6 is a diagram illustrating the image rendering processing by the image display device according to the present disclosure. FIG. 6 illustrates a flat panel display 310 (described below), a microlens array 320, a lens LN that enlarges the pupil of the user's eye, and the user's line of sight AX1. As shown in FIG. 6 , the rendering unit 132 provides a rendering instruction to the flat panel display 310 of the display device 300, causing the pixels of the flat panel display 310 to emit light. For example, the rendering unit 132 lights up the red light emitters R1, R2, and R3 shown in FIG. 6 . As a result, red light rays RL1, RL2, and RL3 intersect to form a red stereoscopic image RPLS1. Note that when the green light emitters G1, G2, and G3 shown in FIG. 6 are also illuminated in the same manner, green light rays GL1, GL2, and GL3 intersect to form a green stereoscopic image GPLS1.

[0044] Next, crosstalk will be described using FIG. 7 . FIG. 7 is a diagram illustrating crosstalk that may occur during rendering by an image display device according to the present disclosure. As shown in FIG. 7 , when red light emitters R1, R2, and R3 are turned on to form a red stereoscopic image RPLS2, green light emitters G1, G2, and G3 are also turned on. Then, as shown in FIG. 7 , crosstalk light CSTL, which is a green light beam emitted from green light emitter G1, passes through the position where the red stereoscopic image RPLS2 is formed, resulting in crosstalk in which green is mixed into the red stereoscopic image RPLS2. As in the example shown in FIG. 7 , when the colors of the crosstalk light are different, the colors are mixed into the stereoscopic image, resulting in a color different from the original. For example, a slightly bright yellow spot appears on a dark red background, which is particularly noticeable. Even if the crosstalk light is also light from the same red pixel, the reproduced stereoscopic image may be slightly blurred or some parts may be brighter than the original.

[0045] The range specifying unit 133 specifies an unnecessary light source range that indicates a range of candidate pixels that may be the source of crosstalk light, which indicates light rays from a light source other than the light source of the drawn stereoscopic image that pass through the position of the stereoscopic image. In other words, it can be said that the range specifying unit 133 specifies a range where there is a possibility that a pixel that may be the source of crosstalk light exists.

[0046] Specifically, the area specifying unit 133 specifies the unnecessary light source area based on the size of the user's pupil and the positional relationship between the user's pupil, the stereoscopic image, and the flat panel display. The processing of the area specifying unit 133 will be described with reference to FIGS. 8 and 9.

[0047] 8 is a diagram for explaining the description of a ray by the Plenoptioc Function. As shown in Fig. 8, in general, any ray L in space can be expressed by the Plenoptioc Function proposed by Adelson and Bergen using three dimensions (x, y, z) that represent the light emission position and two dimensions (θ, φ) that represent the direction.

[0048] This will be applied to the case of a light field display and examined using FIG. 9 . FIG. 9 is a diagram illustrating the processing of the range determination unit and the extinguishing instruction unit of the image display device according to the present disclosure. Let (x, y, z) be the position coordinates of the spatial three-dimensional image to be displayed, and (θ, φ) be the gaze direction toward the pupil detected by the gaze detection unit 131. Then, a gaze axis AX1 is determined, connecting the pupil to the spatial three-dimensional image and the central pixel of the flat panel display 310 used to display it. Next, a virtual pupil circle C0 is assumed to be centered on the pupil and slightly larger than the pupil size. This circle C is then reduced in proportion to the distance from the spatial three-dimensional image to the pupil and the distance from the spatial three-dimensional image to the flat panel display 310, and projected onto the flat panel display 310 to identify the unnecessary light source range RG. In other words, pixels within the unnecessary light source range RG other than the pixels used to display the desired three-dimensional image are candidates for pixels that may cause crosstalk.

[0049] For example, if the size of the circle on the pupil side is 4 mm in radius and you are viewing a stereoscopic image displayed 50 mm in front of, or 550 mm away from, the flat panel display 310 600 mm away, then proportionally the circle on the flat panel display 310 will have a radius of 0.37 mm, and any pixels within this circle other than those displaying the desired stereoscopic image can be determined to be pixels that may cause crosstalk.

[0050] Furthermore, the range specifying unit 133 may specify, as the unnecessary light source range, a range corresponding to a predetermined angle centered on the line of sight, taking into account the discriminative field of view, which is a range in which high visual acuity can be achieved without moving the eyes. That is, the range specifying unit 133 may set the unnecessary light source range, in which to check for the occurrence of crosstalk light, to approximately ±2.5 degrees centered on the line of sight of the discriminative field of view, which is a range in which high visual acuity can be achieved without moving the eyes, taking into account the visual field characteristics of the human eye. This makes it possible to efficiently prevent crosstalk in a range in which crosstalk is easily noticeable with high visual acuity, and to reproduce a high-quality stereoscopic image.

[0051] Specifically, the crosstalk check range is a radius of 24 mm + 0.37 mm, equivalent to ±2.5 degrees, centered on the line of sight around the stereoscopic image displayed 550 mm away. When viewing a range that is more than ±2.5 degrees away from the center of the visual field, the eye naturally moves in that direction, changing the direction of the line of sight and the rendering state. Therefore, there is no need to constantly be aware of and deal with crosstalk light at positions more than ±2.5 degrees away. Regarding human visual field characteristics, the range in which a person can achieve high visual acuity without moving their eyes (discrimination field of view) is approximately ±1.5 to ±2.5 degrees.

[0052] Furthermore, the range specifying unit 133 may specify the unnecessary light source range according to the gaze direction by reading the unnecessary light source range from the map information storage unit 121 based on the gaze direction. As described above, the map information storage unit 121 stores the unnecessary light source range according to the gaze direction in advance. Therefore, if the gaze detection unit 131 can detect the gaze direction of the user, it is possible to specify the unnecessary light source range by reading the unnecessary light source range according to the gaze direction from the map information storage unit 121.

[0053] In addition, when there are multiple users, the range identification unit 133 may identify the unnecessary light source range by performing the above-mentioned processing for each user according to the gaze direction detected by the gaze detection unit 131 for each user.

[0054] The pixel specifying unit 134 specifies light source pixels that indicate pixels that are light sources of crosstalk light from among the pixels in the unnecessary light source range. Specifically, the pixel specifying unit 134 specifies, from among the pixels in the unnecessary light source range, pixels whose straight lines connecting the pixels and the centers of their corresponding microlenses pass through the stereoscopic image as pixels related to the occurrence of crosstalk.

[0055] Specifically, the pixel specifying unit 134 specifies, among the pixels in the unnecessary light source range, those pixels where a straight line connecting the centers of the microlenses where the pixels are within the range of the viewing angle of the microlenses passes through the target three-dimensional image as pixels causing crosstalk. Explaining with reference to Figure 9, a green light ray G4 emitted from a green light emitter G1 located in the unnecessary light source range passes through the microlens and passes through the red three-dimensional image RPLS3, and is therefore specified as crosstalk light CSLT.

[0056] The light ray processing unit 135 performs predetermined processing on the identified light source pixels or light rays from the light source pixels, preventing the light rays from the light source pixels from passing through the position of the stereoscopic image. The configuration of the light ray processing unit 135 that performs such processing will be described with reference to FIG. 10 . FIG. 10 is a diagram illustrating an example configuration of the light ray processing unit of a control device according to the present disclosure. As shown in FIG. 10 , the light ray processing unit 135 according to the present disclosure includes at least one of a light-off instruction unit 1351 and a shield instruction unit 1352.

[0057] The light-off instruction unit 1351 turns off the light emission of the identified light source pixel. In other words, when the light ray processing unit 135 includes the light-off instruction unit 1351, it issues a light-off instruction to the pixel identified by the pixel identification unit 134, thereby turning off the light emission of the identified pixel, thereby preventing the light ray from the light source pixel from passing through the position of the stereoscopic image. Referring to FIG. 9 , if the green light emitter G1 is identified as the pixel that is the source of the crosstalk light, turning off the green light emitter G1 will turn off the crosstalk light CSLT and the green light ray GL1. This allows the light emission of the pixel that causes crosstalk to be turned off, thereby appropriately suppressing crosstalk. In general, in a display, pixels that emit unnecessary light due to defective lighting are more noticeable than black spots that do not emit light, so it is advantageous to not emit crosstalk light.

[0058] The light-off instruction unit 1351 may check whether the pixels causing crosstalk are of a different color from the stereoscopic image, and may turn off the pixels causing the crosstalk light only if the crosstalk light is of a different color. Since crosstalk is most noticeable when the colors are different, crosstalk of the same color can be ignored, simplifying the processing.

[0059] The shading instruction unit 1352 instructs the shading filter 350, which is placed between the flat panel display 310 and the microlens array 320, to shading the pixel of the shading filter 350 that corresponds to the position of the identified light source pixel. In other words, when the light ray processing unit 135 is configured to include the shading instruction unit 1352, by shading the shading filter 350 that corresponds to the position of the identified light source pixel, the shading filter 350 blocks the light rays from the light source pixel and prevents the light rays from passing through the position of the stereoscopic image. The processing of the shading instruction unit 1352 will be specifically described with reference to FIG. 11 .

[0060] 11 is a diagram illustrating the processing of the shading instruction unit of the image display device according to the present disclosure. As shown in FIG. 11, the display device 300 may include a shading filter 350 between the front of the flat panel display 310 and the microlens array 320. The configuration of the shading filter 350 will be described later.

[0061] 11 , the shading instruction unit 1352 issues an instruction to shade the pixel of the shading filter 350 corresponding to the position of the identified light source pixel, so that the light ray GL4 emitted by the identified light source pixel (green light emitter G1) does not pass through the position of the stereoscopic image. This shades the light ray emitted by the pixel that causes crosstalk and prevents it from passing through the position of the stereoscopic image, thereby making it possible to appropriately suppress crosstalk.

[0062] The size of one pixel of the shielding filter 350 may be half the size of a pixel of the flat panel display 310. An embodiment of such a shielding filter 350 will be described with reference to FIG. 12 . FIG. 12 is a diagram illustrating the shielding filter of the image display device according to the present disclosure. As shown in FIG. 12 , the size of one pixel of the shielding filter 350 may be half the size of a pixel of the flat panel display 310.

[0063] Furthermore, the shading instruction unit 1352 may issue a shading instruction to two pixels of the shading filter 350 corresponding to the position of the identified light source pixel, in other words, of the two pixels of the shading filter 350 perpendicularly facing the identified light source pixel, the pixel on the side of the stereoscopic image in which crosstalk occurs. In this way, in the example shown in Fig. 12 , the light ray GL1 from the light source G can pass through the shading filter 350 as usual to form a stereoscopic image, and therefore, a decrease in brightness of the stereoscopic image due to the light ray from the light source G can be suppressed compared to when the light source G is turned off. In other words, while suppressing crosstalk due to light emission from pixels that cause crosstalk, a decrease in brightness of other stereoscopic images in which crosstalk does not occur can also be suppressed.

[0064] The brightness increasing unit 136 increases the brightness of pixels that are not turned off among the pixels that serve as light sources for the stereoscopic image to compensate for the brightness of the stereoscopic image affected by the turning off of pixels that serve as light sources for crosstalk light. The amount of brightness increase by the brightness increasing unit 136 may be a value previously stored in the storage unit 120. That is, the amount of brightness increase may be a predetermined value, or may be a value calculated based on the positional relationship between the stereoscopic image and the pixels whose brightness is to be increased. Referring to FIG. 9 , when the green light emitter G1 is turned off, the brightness of the green light emitters G2 and G3 is increased.

[0065] Furthermore, the brightness increasing unit 136 may check whether the pixels causing crosstalk are of different colors, and only if the crosstalk light is of different colors, increase the brightness of the pixels that are not turned off among the pixels that serve as light sources for the stereoscopic image in order to compensate for the brightness of the stereoscopic image that is affected by the turning off of pixels that are the light sources of the crosstalk light. Because crosstalk is most noticeable when it is of different colors, it is possible to simplify processing by ignoring crosstalk when it is of the same color.

[0066] The brightness increasing unit 136 may store the relationship between the line of sight direction and the pixels whose brightness is to be increased or their color shades in the storage unit 120, and may increase the brightness of pixels that are not turned off among the pixels that serve as light sources for the stereoscopic image by reading out the information from the storage unit 120. This reduces the calculation load when performing real-time processing.

[0067] (Configuration of Display Device) Next, the configuration of a display device according to the present disclosure will be described with reference to FIG. 13 . FIG. 13 is a diagram showing an example configuration of a display device according to the present disclosure. As shown in FIG. 13 , a display device 300 according to the present disclosure includes a flat panel display 310, a microlens array 320, a drive circuit 330, an I / F unit 340, and a shielding filter 350. Note that the shielding filter 350 is not an essential component. These components will be described below in order.

[0068] The flat panel display 310 is a display that displays various images, and may be realized by an organic electroluminescence (EL) display, a micro light-emitting diode (LED) display, or the like.

[0069] Microlens array 320 is an optical element that integrates many minute microlenses measuring in the μm to mm range. Microlens array 320 is disposed in front of the flat panel display 310 in the light-emitting direction of the pixels. Microlens array 320 changes the direction of light emitted from flat panel display 310, thereby reproducing the light rays emitted by an object.

[0070] The driving circuit 330 provides control signals for driving the pixels of the flat panel display 310 and the shielding filter 350. The driving circuit 330 includes switching elements for driving the flat panel display 310 and the shielding filter 350. That is, the brightness of the pixels of the flat panel display 310 and the shielding filter 350 is controlled by the control signals sent from the driving circuit 330.

[0071] The I / F unit 340 receives a control signal from the control device 100. The I / F unit 340 may receive the control signal via a wired or wireless connection with the control device 100. If the I / F unit 340 receives the control signal via a wired connection, the I / F unit 340 may be provided with a connection terminal such as a USB, SDI, or HDMI. If the I / F unit 340 receives the control signal wirelessly, the I / F unit 340 may be realized by a wireless LAN defined by IEEE 802.11.

[0072] The shading filter 350 is a filter that can switch between blocking and passing light rays in response to an electric signal. The shading filter 350 is disposed, for example, between the flat panel display 310 and the microlens array 320. The shading filter 350 may be, for example, a twisted nematic (TN) liquid crystal in which a twisted liquid crystal (nematic liquid crystal) is sandwiched between two polarizing filters with orthogonal polarization directions, with two glass plates interposed therebetween. Note that the shading filter 350 may be switched between blocking and passing light rays by applying an electric signal to the liquid crystal to control the orientation of the liquid crystal using thin film transistors (TFTs) corresponding to the RGB subpixels of each pixel.

[0073] The shielding filter 350 may also be realized by an electrochromic element using an electrochromic (EC) material that exhibits reversible changes in optical properties in response to an electric signal. The optical properties of the electrochromic element change when a voltage is applied, so that the blocking or passing of light can be controlled by an electric signal. It is preferable that the size of the pixels of the shielding filter 350 is half the size of the pixels of the flat panel display 310.

[0074] (Image display method and program) Next, an image display method according to the present disclosure will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of the image display method according to the present disclosure. Below, the image display method according to the present disclosure will be described along the flow shown in Fig. 14.

[0075] First, the image display device 1 detects the user's gaze direction (step S101). Next, the image display device 1 causes the display device 300 to render a stereoscopic image according to the detected gaze direction (step S102). Next, the image display device 1 identifies an unnecessary light source range indicating a range of candidate pixels that may be sources of crosstalk light (step S103). Next, the image display device 1 identifies light source pixels that indicate pixels that may be sources of crosstalk light (step S104). Next, the image display device 1 performs a predetermined process on the identified light source pixels or light rays from the light source pixels, thereby preventing the light rays from the light source pixels from passing through the position of the stereoscopic image (step S105).

[0076] The image display method described above may be executed by the image display device 1, or may be executed by a program that causes the control unit 130 of the control device 100 of the image display device 1 to execute processing.

[0077] According to this, a candidate range of pixels that can be the source of crosstalk light is identified according to the direction of the user's line of sight, pixels that can be the source of crosstalk light are identified from within the range, and predetermined processing is performed on the identified source pixels or light rays from the source pixels to prevent the light rays from the source pixels from passing through the position of the three-dimensional image. Thus, an image display method and program that can appropriately suppress the occurrence of crosstalk can be provided.

[0078] (Configuration and Effects) The image display device 1 according to the present disclosure includes a gaze detection unit 131 that detects the gaze direction of a user, a rendering unit 132 that causes the display device 300 to render a stereoscopic image according to the gaze direction of the user, a range identification unit 133 that identifies an unnecessary light source range that indicates a range of pixels that can become the source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and that pass through the position of the stereoscopic image, a pixel identification unit 134 that identifies light source pixels that indicate pixels that can become the source of crosstalk light among the pixels in the unnecessary light source range, and a light ray processing unit 135 that performs predetermined processing on the identified light source pixels or light rays from the light source pixels, and does not allow light rays from the light source pixels to pass through the position of the stereoscopic image.

[0079] According to this configuration, a candidate range of pixels that can be the source of crosstalk light is identified in accordance with the user's line of sight, pixels that can be the source of crosstalk light are identified from within the range, and predetermined processing is performed on the identified source pixels or light rays from the source pixels, so that the light rays from the source pixels do not pass through the position of the three-dimensional image. Therefore, it is possible to provide an image display device 1 that can appropriately suppress the occurrence of crosstalk.

[0080] The light processing unit 135 of the image display device 1 according to the present disclosure includes a light-off instruction unit 1351 that turns off the light emission of the identified light source pixel.

[0081] According to this configuration, a candidate range of pixels that can be the source of crosstalk light can be identified in accordance with the direction of the user's line of sight, pixels that can be the source of crosstalk light can be identified from within that range, and the identified light source pixels can be turned off. Therefore, it is possible to provide an image display device 1 that can appropriately suppress the occurrence of crosstalk.

[0082] The light processing unit 135 of the image display device 1 according to the present disclosure includes a flat panel display 310 and a shielding instruction unit 1352 that instructs a shielding filter 350 placed between the flat panel display 310 and the microlens array 320 to shield the pixel of the shielding filter 350 that corresponds to the position of the identified light source pixel.

[0083] According to this configuration, a candidate range of pixels that can be the source of crosstalk light is identified in accordance with the user's line of sight, pixels that can be the source of crosstalk light are identified from within that range, and an instruction to block pixels of the blocking filter 350 that correspond to the positions of the identified source pixels is given, thereby preventing light rays from the source pixels from passing through the position of the three-dimensional image. Therefore, it is possible to provide an image display device 1 that can appropriately suppress the occurrence of crosstalk.

[0084] The range specifying unit 133 of the image display device 1 according to the present disclosure specifies the unnecessary light source range based on the size of the user's pupil and the positional relationship between the user's pupil, the stereoscopic image, and the flat panel display.

[0085] According to this configuration, the unnecessary light source range is identified based on the size of the user's pupil and the positional relationship between the user's pupil, the three-dimensional image, and the flat panel display, and pixels where a line connecting the pixel and the center of the corresponding microlens passes through the three-dimensional image are identified as pixels related to the occurrence of crosstalk, and the light emission of the identified light source pixels can be turned off. Therefore, it is possible to provide an image display device 1 that can appropriately suppress the occurrence of crosstalk.

[0086] The image display device 1 according to the present disclosure further includes a brightness increasing unit 136 that increases the brightness of pixels that are not turned off among the pixels that serve as light sources for the stereoscopic image, so as to compensate for the brightness of the stereoscopic image that is affected by the turning off of pixels that are the light sources of crosstalk light.

[0087] This configuration can compensate for the luminance of a stereoscopic image that is affected by the turning off of pixels of the light source of the crosstalk light, thereby suppressing the occurrence of crosstalk and allowing the stereoscopic image to be displayed clearly.

[0088] The image display method according to the present disclosure includes the steps of detecting the direction of the user's gaze, causing the display device 300 to render a stereoscopic image in accordance with the direction of the user's gaze, identifying an unnecessary light source range indicating a range of candidate pixels that may be the source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and which pass through the position of the stereoscopic image, identifying light source pixels from among the pixels in the unnecessary light source range that indicate pixels that may be the source of crosstalk light, and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels, thereby preventing the light rays from the light source pixels from passing through the position of the stereoscopic image.

[0089] According to this configuration, a candidate range of pixels that can be the source of crosstalk light is identified in accordance with the user's line of sight, pixels that can be the source of crosstalk light are identified from within the range, and predetermined processing is performed on the identified source pixels or light rays from the source pixels to prevent the light rays from the source pixels from passing through the position of the three-dimensional image. Thus, an image display method that can appropriately suppress the occurrence of crosstalk can be provided.

[0090] The program according to the present disclosure causes a computer to perform the following steps: detecting the direction of the user's gaze; causing the display device 300 to render a stereoscopic image according to the direction of the user's gaze; identifying an unnecessary light source range indicating a range of pixels that can be the source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and that pass through the position of the stereoscopic image; identifying light source pixels from among the pixels in the unnecessary light source range that indicate pixels that can be the source of crosstalk light; and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels, thereby preventing the light rays from the light source pixels from passing through the position of the stereoscopic image.

[0091] According to this configuration, a candidate range of pixels that can be the source of crosstalk light is identified according to the user's line of sight, pixels that can be the source of crosstalk light are identified from within the range, and predetermined processing is performed on the identified source pixels or light rays from the source pixels to prevent the light rays from the source pixels from passing through the position of the three-dimensional image. Therefore, a program that can appropriately suppress the occurrence of crosstalk can be provided.

[0092] Although the embodiments of the present disclosure have been described above, the embodiments are not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

[0093] The image display device, image display method, and program according to the present embodiment can be used, for example, in an image display device, image display method, and program that can appropriately suppress the occurrence of crosstalk.

[0094] 1 Image display device 100 Control device 110 I / F unit 120 Memory unit 121 Map information memory unit 122 Drawing image memory unit 130 Control unit 131 Gaze detection unit 1311 Acquisition unit 1312 Face detection unit 1313 Pupil information detection unit 1314 Face direction detection unit 1315 Gaze detection unit 1316 Output unit 132 Rendering unit 133 Range identification unit 134 Pixel identification unit 135 Light processing unit 1351 Light-off instruction unit 1352 Shielding instruction unit 136 Brightness increase unit 200 Camera 300 Display device 310 Flat panel display 320 Microlens array 330 Drive circuit 340 I / F unit 350 Shielding filter

Claims

1. An image display device comprising: a gaze detection unit that detects the direction of a user's gaze; a rendering unit that causes a display device to render a stereoscopic image in accordance with the direction of the user's gaze; a range identification unit that identifies an unnecessary light source range that indicates a range of candidates for pixels that may become a source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and which pass through the position of the stereoscopic image; a pixel identification unit that identifies light source pixels that indicate pixels that may become a source of crosstalk light among the pixels in the unnecessary light source range; and a light ray processing unit that performs a predetermined process on the identified light source pixels or the light rays from the light source pixels, and does not allow the light rays from the light source pixels to pass through the position of the stereoscopic image.

2. The image display device according to claim 1, wherein the light processing unit includes a turn-off instruction unit that turns off the light emission of the identified light source pixel.

3. The image display device according to claim 1, wherein the light processing unit comprises a flat panel display and a shielding instruction unit that instructs a shielding filter placed between a microlens array to shield a pixel of the shielding filter that corresponds to the position of the identified light source pixel.

4. The image display device according to claim 1, wherein the range determination unit determines the unnecessary light source range based on a size of the user's pupil and a positional relationship between the user's pupil, a stereoscopic image, and a flat panel display.

5. The image display device according to claim 1 or 2, further comprising a brightness increasing section that increases the brightness of pixels that are not turned off so as to compensate for the brightness of the stereoscopic image that is affected by the turning off of pixels of the light source of the crosstalk light.

6. An image display method comprising the steps of: detecting a user's line of sight; causing a display device to render a stereoscopic image in accordance with the user's line of sight; identifying an unnecessary light source range indicating a range of candidate pixels that can become a source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and which pass through a position of the stereoscopic image; identifying light source pixels indicating pixels that can become a source of crosstalk light among pixels in the unnecessary light source range; and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels, to prevent the light rays from the light source pixels from passing through the position of the stereoscopic image.

7. A program that causes a computer to execute the steps of: detecting the direction of a user's gaze; causing a display device to render a stereoscopic image in accordance with the direction of the user's gaze; identifying an unnecessary light source range indicating a range of candidate pixels that can become a source of crosstalk light, which indicates light rays from a light source other than the light source of the rendered stereoscopic image and which pass through the position of the stereoscopic image; identifying light source pixels that indicate pixels that can become a source of crosstalk light among the pixels in the unnecessary light source range; and performing a predetermined process on the identified light source pixels or the light rays from the light source pixels, thereby preventing the light rays from the light source pixels from passing through the position of the stereoscopic image.

Images