Display device

Abstract

To provide a display device that can be flexibly responded to by a relationship between the alignment direction of a plurality of view points and the display device, and can have the displayed image responded to with high accuracy by a positional relationship between the display device and the view points.SOLUTION: A display device 1 comprises: a display panel provided with a plurality of pixels; a light source provided with a plurality of emission points, for emitting light to the plurality of pixels of the display panel; an acquisition unit for acquiring information on the view points of a user visually recognizing the display panel; a storage unit for storing a plurality of pieces of image data provided to be selectable in accordance with the positional relationship between the display panel and the view points; and a control unit for controlling the display of an image by working of the plurality of pixels on the basis of the information. When image data is not stored in the storage unit for the positional relationship between the image display surface of the display panel and the view points of the user, the control unit generates image data corresponding to the positional relationship from two pieces of image data close to the positional relationship by interpolation, and selects the generated image data.SELECTED DRAWING: Figure 20

Description

[Technical Field]

[0001] The present disclosure relates to a display device. [Background technology]

[0002] BACKGROUND ART Display devices are known that can display and output individual images for a plurality of viewpoints using an image separator such as a parallax barrier (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3865762 Summary of the Invention [Problem to be solved by the invention]

[0004] Image separators such as parallax barriers have a fixed arrangement direction for multiple viewpoints that can output images individually. However, the relationship between the arrangement direction of the multiple viewpoints and the display device is not necessarily invariable. For example, the relationship between a display device provided in a mobile terminal such as a smartphone and the arrangement direction of the eyes of a user viewing the display device is not fixed. Image separators such as parallax barriers cannot display and output individual images for multiple viewpoints depending on the relationship between the arrangement direction of the multiple viewpoints and the display device. Furthermore, when displaying individual images for multiple viewpoints, pre-prepared images do not necessarily correspond to the positions of the viewpoints, so improvements are also needed in terms of the accuracy with which the displayed images correspond to the positional relationship between the display device and the viewpoints.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a display device that can flexibly respond to the relationship between the arrangement direction of multiple viewpoints and the display device, and can accurately match the displayed image to the positional relationship between the display device and the viewpoints. [Means for solving the problem]

[0006] A display device according to one aspect of the present disclosure includes a display panel having a plurality of pixels; a light source having a plurality of light-emitting points and irradiating light onto the plurality of pixels of the display panel; an acquisition unit that acquires information regarding a plurality of viewpoints of a user viewing the display panel; a memory unit that stores a plurality of image data that are selectable depending on a positional relationship between the display panel and the viewpoints; and a control unit that controls display of an image by operation of the plurality of pixels based on the information, wherein a ratio between a pitch of the plurality of pixels aligned in a predetermined direction and a pitch of the plurality of light-emitting points in the predetermined direction is 1:4n or 1:6n, where n is a natural number, and the information includes information regarding the positions of the plurality of viewpoints and information indicating an arrangement direction of the plurality of viewpoints. The control unit selects image data from the image data stored in the memory unit that corresponds to the relative positional relationship based on the relative rotation angle between the display panel and the alignment direction and the relative positional relationship between the viewpoint and each light-emitting point, and transmits light to pixels located on at least a straight line connecting each light-emitting point and each of the viewpoints in order to display and output the selected image data.If image data for the positional relationship between the image display surface of the display panel and the viewpoint is not stored in the memory unit, the control unit has an image generation unit that interpolatively generates image data corresponding to the positional relationship from two image data stored in the memory unit that are close to the positional relationship, and in this case selects the image data generated by the image generation unit. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing the main configuration of a display device. [Figure 2] FIG. 2 is a diagram showing an example of viewpoint-corresponding image data. [Figure 3] FIG. 3 is a schematic diagram showing a layered structure of the display panel. [Figure 4] FIG. 4 is a cross-sectional view showing an example in which the light-emitting point pitch is six times the pixel pitch. [Figure 5]FIG. 5 is a diagram showing various parameters related to determining the X-coordinate of a pixel located on the ray of light between the viewpoint and the light-emitting point of light from the (i+1)th light source in the X direction from the origin. [Figure 6] FIG. 6 is a schematic diagram showing examples A and B of the relative angular relationship between a human face and a display device including a display panel. [Figure 7] FIG. 7 is a schematic diagram showing an example of the angle difference between the face and the display panel with respect to the vertical line and the horizontal line. [Figure 8] FIG. 8 is a schematic diagram showing an example of the relationship between the relative angle and the feasibility of individually outputting images to multiple viewpoints by controlling the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD described with reference to FIGS. 3 and 4. [Figure 9] FIG. 9 is a schematic diagram showing another example of the relationship between the relative angle and the feasibility of individually outputting images to multiple viewpoints by controlling the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD described with reference to FIGS. 3 and 4. [Figure 10] FIG. 10 is a schematic diagram showing an example of pixel arrangement control when the relative angle is 45 degrees (°). [Figure 11] FIG. 11 is a schematic diagram showing an example of pixel arrangement control when the relative angle is 90 degrees (°). [Figure 12] FIG. 12 is a schematic diagram showing an example of the relationship between the relative angle and the feasibility of individual output of images to a plurality of viewpoints when the arrangement control described with reference to FIGS. 10 and 11 is reflected. [Figure 13] FIG. 13 is a schematic diagram showing an example of a change pattern of the relative positional relationship between the display panel 20A and the face HF. [Figure 14] FIG. 14 is a diagram showing an example of a change pattern of the display output content corresponding to the change in the relative positional relationship shown in FIG. [Figure 15] FIG. 15 is a schematic diagram showing an example of a change pattern of the relative positional relationship between the display panel 20A and the face HF. [Figure 16] FIG. 16 is a diagram showing an example of a change pattern of the display output content corresponding to the change in the relative positional relationship shown in FIG. [Figure 17] FIG. 17 is a schematic diagram showing an example of a change pattern of the selected image data in accordance with a change in the rotation angle relative to the viewpoint EL, the viewpoint ER, and the display panel 20A. [Figure 18] FIG. 18 is an explanatory diagram of parameters relating to interpolative image data generation from a planar perspective. [Figure 19] FIG. 19 is an explanatory diagram from a side viewpoint including parameters in the height direction related to interpolative image data generation. [Figure 20] FIG. 20 is a schematic diagram showing an example of interpolation processing that refers to two pieces of image data aligned in the X direction and two pieces of image data that refer to two pieces of image data aligned in the Y direction. [Figure 21] FIG. 21 is a schematic diagram showing an example of interpolation processing that refers to two pieces of image data arranged diagonally in a direction intersecting the X and Y directions. [Figure 22] FIG. 22 is a diagram showing the relationship between the relative angle rot and the median line CLY of the face HF in the Y direction of the display panel 20A. [Figure 23] FIG. 23 is a diagram showing an outline of the process of extracting a block BlockS from image data PicS that is treated as one of two pieces of image data referenced in the image data interpolation process. [Figure 24] Figure 24 is a schematic diagram showing blocks BlockF(-3) to BlockF(3) extracted from image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, and block-matched with block BlockS. [Figure 25] Figure 25 is a schematic diagram showing blocks BlockF(-3) to BlockF(3) extracted from image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, and block-matched with block BlockS. [Figure 26] FIG. 26 is a diagram showing an example of the flow of processing performed as diagonal interpolation processing when rot<0, as an interpolation example. DETAILED DESCRIPTION OF THE INVENTION

[0008] Each embodiment of the present disclosure will be described below with reference to the drawings. The disclosure is merely an example, and appropriate modifications that a person skilled in the art can easily conceive while maintaining the gist of the invention are naturally included within the scope of the present disclosure. Furthermore, to clarify the explanation, the drawings may show the width, thickness, shape, etc. of each part schematically compared to the actual embodiment, but these are merely examples and are not intended to limit the interpretation of the present disclosure. Furthermore, in this specification and each drawing, elements similar to those described above with reference to the previous drawings will be designated by the same reference numerals, and detailed descriptions may be omitted as appropriate.

[0009] 1 is a block diagram showing the main configuration of a display device 1. The display device 1 includes an imaging unit 2, a distance measurement unit 3, a signal processing unit 10, a display panel 20, and a light source 30. The display device 1 is an information processing device (information processing terminal) that combines an imaging function using the imaging unit 2, a distance measurement function using the distance measurement unit 3, and an image display output function using the signal processing unit 10, the display panel 20, and the light source 30, like a smartphone, for example.

[0010] The imaging unit 2 captures an image. Specifically, the imaging unit 2 has an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 2 generates image data based on an electrical signal output by the imaging element.

[0011] The distance measuring unit 3 measures the distance between the display device 1 and an image capture target facing the image capture unit 2. Specifically, the distance measuring unit 3 includes, for example, a light emitting unit and a light detecting unit constituting a ToF (Time of Flight) sensor. The distance measuring unit 3 including the ToF sensor measures the distance based on the time difference between the light emitting timing when the light emitting unit emits light and the detection timing when the laser light emitted by the light emitting unit is reflected by the image capture target and detected by the light detecting unit. The specific mechanism by which the distance measuring unit 3 measures the distance is not limited to this. For example, an AF function of the camera, such as so-called contrast AF (Auto Focus), may be used, and the distance measured by the distance measuring unit 3 may be the distance at which the image is in focus determined by the AF function of the image capture unit 2. In the embodiment, the image capture unit 2 and the distance measuring unit 3 cooperate to function as an acquisition unit that acquires information indicating the positions of two viewpoints (a first viewpoint E1 (right eye) and a second viewpoint E2 (left eye) described later) of the user facing the display panel 20.

[0012] The imaging unit 2 is provided to capture an image of a user viewing the image display surface of the display panel 20. The distance measuring unit 3 is provided to measure the distance between the image display surface of the display panel 20 and the user viewing the image display surface. Specifically, the imaging unit 2 and the distance measuring unit 3 are arranged, for example, on one side of the housing of the display device 1 where the image display surface of the display panel 20 is exposed.

[0013] The signal processing unit 10 has an eye gaze tracking unit 11 and an image output unit 12. The eye gaze tracking unit 11 acquires information about the position of the user's eye gaze relative to the display panel 20 based on the outputs of the imaging unit 2 and the distance measuring unit 3. Details of the information about the eye gaze position will be described later.

[0014] Based on information about the position of the viewpoint acquired by the gaze tracking unit 11, the image output unit 12 outputs image data corresponding to the position of the viewpoint to the display panel 20. The image data output by the image output unit 12 may be, for example, image data based on an image signal IP input to the display device 1 from an external information processing device, or may be image data stored in advance in a storage device included in the display device 1.

[0015] The image output unit 12 of the embodiment includes a storage unit 12a and an image generation unit 12b. The storage unit 12a stores viewpoint-corresponding image data OP. The image generation unit 12b references two pieces of image data included in the viewpoint-corresponding image data OP and performs interpolation processing on the two pieces of image data to generate image data. The image generation unit 12b of the embodiment outputs, to the display panel 20, image data included in the viewpoint-corresponding image data OP or image data generated by the image generation unit 12b as image data corresponding to the position of the viewpoint acquired by the gaze tracking unit 11 from the viewpoint-corresponding image data OP.

[0016] Fig. 2 is a diagram showing an example of viewpoint-corresponding image data OP. As shown in Fig. 2, the viewpoint-corresponding image data OP includes a plurality of image data. The plurality of image data included in the viewpoint-corresponding image data OP are image data corresponding to different viewpoints.

[0017] Specifically, the viewpoint-corresponding image data OP includes a plurality of image data arranged in a matrix along the X and Y directions. The X and Y directions, which are the arrangement directions of the plurality of image data included in the viewpoint-corresponding image data OP, correspond to the X and Y directions of the display panel 20. Each of the plurality of image data includes a plurality of pixel data arranged in a matrix along the bx and by directions (see FIG. 23, etc.). The bx direction corresponds to the X direction, and the by direction corresponds to the Y direction.

[0018] The viewpoint-corresponding image data OP is assigned coordinates corresponding to the magnitude of the angle between the image display surface 200 of the display panel 20 (see FIG. 3 ) and the user's viewpoint (e.g., first viewpoint E1, second viewpoint E2) when the vertical direction of the image display surface 200 is set to 0 degrees. In the embodiment, the coordinates are assigned separately for the X direction and the Y direction. In FIG. 2 and other figures, the tilt in the X direction when the angle of the user's line of sight with respect to the image display surface 200 is considered to be tilted to one side in the X direction, with the central image data CP1 as the center, is shown as coordinates in 1.5-degree increments, such as "+1.5," "+3," "+4.5," "+6," and "+7.5." Furthermore, the tilt in the X direction when the angle of the user's line of sight with respect to the image display surface 200 is considered to be tilted to the other side in the X direction, with the central image data CP1 as the center, is shown as coordinates in 1.5-degree increments, such as "-1.5," "-3," "-4.5," "-6," and "-7.5." Furthermore, when the angle of the user's line of sight with respect to the image display surface 200 is considered to be tilted to one side in the Y direction with respect to the center image data CP1 as the center, the tilt in the Y direction is shown as coordinates in 1.5-degree increments, such as "+1.5," "+3," "+4.5," "+6," and "+7.5." When the angle of the user's line of sight with respect to the image display surface 200 is considered to be tilted to the other side in the Y direction with respect to the center image data CP1 as the center, the tilt in the Y direction is shown as coordinates in 1.5-degree increments, such as "-1.5," "-3," "-4.5," "-6," and "-7.5." Note that the center image data CP1 is selected as image data corresponding to the user's viewpoint when the viewpoint is located in the vertical direction from a center point HP (see FIG. 18) that corresponds to the center of the image display surface 200 in the X and Y directions.

[0019] As described with reference to Fig. 2, in this embodiment, coordinates centered on one image data (center image data CP1) selected when the angle between the image display surface 200 and the user's viewpoint is 0 degrees are assigned to each of the multiple image data, and the angle between the image display surface 200 and the user's viewpoint, i.e., the magnitude of the line-of-sight angle with respect to the image display surface 200, is managed using these coordinates. In the following description, the unit of the line-of-sight angle, measured in increments of the coordinates in the X and Y directions in the viewpoint-corresponding image data OP, is referred to as angle Dltθ. In the example shown in Fig. 2, angle Dltθ = 1.5 (°).

[0020] 2 illustrates image data in PNG (Portable Network Graphics) format, but the format and the number of image data included in the viewpoint-corresponding image data OP are not limited to those illustrated in Fig. 2 and can be changed as appropriate. When interpolation processing by the image generation unit 12b is not required, the image output unit 12 outputs image data corresponding to the position of the viewpoint acquired by the gaze tracking unit 11 from the viewpoint-corresponding image data OP including multiple image data as illustrated in Fig. 2 to the display panel 20.

[0021] 1, the display panel 20 has a display panel driver circuit 21. The display panel driver circuit 21 has circuits, such as a DDIC (Display Driver Integrated Circuit), that perform various processes related to the image display output of the display panel 20. The display panel driver circuit 21 drives a plurality of pixels Pix included in the display panel 20 in accordance with image data output from the image output unit 12.

[0022] FIG. 3 is a schematic diagram showing the layered structure of the display panel 20. As shown in FIG. 3, the display panel 20 has a first substrate 22 and a second substrate 23. The first substrate 22 and the second substrate 23 are light-transmitting substrates such as glass substrates. The first substrate 22 and the second substrate 23 are layered with a liquid crystal layer sandwiched between them. The liquid crystal layer is sealed between the first substrate 22 and the second substrate 23. The display panel 20 is a so-called liquid crystal display panel.

[0023] Hereinafter, the opposing direction of first substrate 22 and second substrate 23 is referred to as the Z direction. One of the two directions perpendicular to the Z direction is referred to as the X direction, and the other is referred to as the Y direction. The X direction and the Y direction are perpendicular to each other.

[0024] The first substrate 22 has a multi-layer structure formed on its surface facing the second substrate 23. The multi-layer structure includes, for example, a first electrode layer on which multiple pixel electrodes are formed, a second electrode layer on which a common electrode to which a reference potential is applied to the multiple pixels Pix is ​​formed, a circuit-forming layer on which switching elements for transmitting signals individually to each of the multiple pixel electrodes and wiring connected to the switching elements are formed, and an insulating layer for insulating between these layers. The pixel electrodes are individually provided for the sub-pixels included in each of the multiple pixels Pix. The pixels Pix are driven under the control of the display panel driver circuit 21, so that the alignment direction of liquid crystal molecules overlapping the pixel electrodes from a planar viewpoint is controlled to correspond to the potential difference between the common electrode and the pixel electrode. The planar viewpoint refers to a viewpoint from the front, looking at a plane perpendicular to the Z direction (XY plane).

[0025] As shown in FIG. 16 (to be described later), each pixel Pix has a plurality of sub-pixels. For example, each pixel Pix has a sub-pixel provided with a color filter that transmits red (R) light, a sub-pixel provided with a color filter that transmits green (G) light, and a sub-pixel provided with a color filter that transmits blue (B) light. Note that it is not essential for each pixel Pix to have all of these sub-pixels. For example, one of two adjacent pixels Pix may have sub-pixels of some of the multiple colors of sub-pixels, and the other may have some of the other sub-pixels. Furthermore, some or all of the pixels Pix may have sub-pixels provided with color filters that transmit light of a color different from the colors exemplified here. Furthermore, sub-pixels with no colorless filters or color filters may be further provided, and light from these sub-pixels is recognized as, for example, white (W). Furthermore, when a pixel Pix includes the above three sub-pixels, the shape of the pixel Pix is ​​preferably a square (with the same length in the X direction and the Y direction), but a rectangular shape in which one of the sides in the X direction and the Y direction is longer than the other can also be used.

[0026] The second substrate 23 is provided with color filters individually provided for the sub-pixels included in each of the plurality of pixels Pix, a black matrix separating the color filters for each sub-pixel, etc. The common electrode may be provided on the second substrate 23 instead of the first substrate 22.

[0027] The pixel pitch PP shown in FIG. 3 is the width of one second pixel Pix2 in the X direction. Although the first pixel Pix1 and the second pixel Pix2 are depicted separately in FIG. 3, the first pixel Pix1 and the second pixel Pix2 share the same pixel Pix in terms of configuration, and there is no difference in configuration between the first pixel Pix1 and the second pixel Pix2. Therefore, the width of one pixel Pix in the X direction is the pixel pitch PP. Strictly speaking, the pixel pitch PP is the distance between the center line of one side of the black matrix surrounding one pixel Pix in the X direction, which is located at one end of the pixel Pix in the X direction, and the center line of the other side of the black matrix surrounding the pixel Pix in the X direction, which is located at the other end of the pixel Pix in the X direction. Alternatively, the pixel pitch may be the center-to-center distance between sub-pixels of the same color in adjacent pixels when viewed in the X direction.

[0028] The display panel 20 faces the light source 30 via the polarizing layer 24 and the spacer 40. The polarizing layer 24 is provided on the first substrate 22 side of the display panel 20 (the rear surface side of the display panel). The spacer 40 is a plate-like light-transmitting member, such as glass, arranged to face the first substrate 22 with the polarizing layer 24 in between. An adhesive layer 42 is interposed between the spacer 40 and the polarizing layer 24. The adhesive layer 42 bonds the polarizing layer 24 and the spacer 40. Note that if a support material can be provided to maintain the distance between the light source 30 and the polarizing layer 24, a configuration in which an air layer is provided between them can also be adopted.

[0029] As shown in FIG. 3 , the light source 30 includes a surface light source 31, light-emitting points 32, and a light-shielding member 33. The surface light source 31 is a surface light source in which at least the surface facing the display panel 20 emits light. To give a specific configuration example, the surface light source 31 includes, for example, a light guide plate facing the display panel 20 in the Z direction, and a light source element (e.g., an LED (Light Emitting Diode)) that introduces light into the light guide plate from a direction perpendicular to the Z direction. The arrangement of the surface light source 31 shown in FIG. 3 illustrates the arrangement of the light guide plate, and the light source element is not shown. The light-emitting points 32 are holes formed in the light-shielding member 33. The light-shielding member 33 covers the surface of the surface light source 31 facing the spacer 40, except for the area where the light-emitting points 32 are formed. An adhesive layer 43 is interposed between the light-shielding member 33 and the spacer 40. The adhesive layer 43 bonds the polarizing layer 24 and the spacer 40 together. The adhesive layers 42 and 43 are a light-transmitting functional film having double-sided adhesiveness, such as OCA (Optical Clear Adhesive). The light source 30 irradiates the display panel 20 with light generated by the surface light source 31 from a plurality of light-emitting points 32.

[0030] The light-emitting point pitch SpP shown in FIG. 3 is the distance between the center lines of the light-emitting points 32 adjacent to each other in the X direction. The light-emitting point pitch SpP is 4n or 6n times the pixel pitch PP. n is a natural number. n is, for example, 1, but may be 2 or more. FIG. 3 illustrates a case where the light-emitting point pitch SpP is four times the pixel pitch PP. The aperture diameter SS shown in FIG. 3 is the aperture diameter of each light-emitting point 32 when viewed from a planar perspective. The aperture diameter SS is equal to or smaller than the pixel pitch PP. More specifically, the planar shape of the light-emitting points 32 is preferably the same as or similar to the shape of each pixel Pix but smaller than each pixel Pix (see FIG. 11, etc.).

[0031] As described above, the image output unit 12 outputs image data corresponding to the position of the viewpoint acquired by the gaze tracking unit 11 from the viewpoint-corresponding image data OP to the display panel 20. The display panel 20 performs display output corresponding to the image data. Therefore, the display panel 20 displays an image corresponding to the position of the viewpoint acquired by the gaze tracking unit 11. FIG. 3 schematically shows the display panel 20 in a state in which images corresponding to the first viewpoint E1 and the second viewpoint E2 are displayed and output, respectively. The first pixel Pix1 is a pixel Pix controlled to display and output an image corresponding to the first viewpoint E1. The second pixel Pix2 is a pixel Pix controlled to display and output an image corresponding to the second viewpoint E2.

[0032] The first viewpoint E1 corresponds to the user's right eye. The second viewpoint E2 corresponds to the user's left eye. The midpoint CP is the midpoint on the line between the first viewpoint E1 and the second viewpoint E2. The position of the midpoint CP generally corresponds to the position of the user's nose in the arrangement direction of the first viewpoint E1 and the second viewpoint E2. FIG. 3 shows a case where the arrangement direction of the first viewpoint E1 and the second viewpoint E2 is the X direction. If the distance in the X direction between the first viewpoint E1 and the midpoint CP and the distance in the X direction between the second viewpoint E2 and the midpoint CP are distance D1, and the distance in the X direction between the first viewpoint E1 and the second viewpoint E2 is distance D2, then distance D2 is twice distance D1.

[0033] Coordinates indicating the position of the midpoint CP relative to the predetermined origin of the display panel 20 can be expressed as (pos_x, pos_y, pos_h). pos_x is the X-coordinate of the midpoint CP. pox_y is the Y-coordinate of the midpoint CP. pox_h is the Z-coordinate of the midpoint CP. The X- and Y-coordinates of the predetermined coordinates of the origin of the display panel 20 are, for example, one of the four vertices of a rectangular display area, as viewed from a planar perspective, in which multiple pixels Pix are arranged on the display panel 20. Alternatively, the center of the display area of ​​the display panel 20 may be the origin. The Z-coordinate of the predetermined coordinates of the origin of the display panel 20 is the position on the center line of the pixels Pix (for example, the first pixel Pix1 and the second pixel Pix2 shown in FIG. 3) in the Z direction. Specifically, the center line of pixel Pix in the Z direction is the center line of the liquid crystal layer sealed between the first substrate 22 and the second substrate 23 in the Z direction, and is preferably located at a height position of d / 2, where d is the cell gap in the display panel 20. The predetermined position of the origin of the display panel 20 is not limited to this and can be any position. Hereinafter, unless otherwise specified, the term "origin" refers to the predetermined origin of the display panel 20.

[0034] The gaze tracking unit 11 identifies the positions of the user's two eyes (right eye and left eye) included in the image captured by the imaging unit 2 within the captured image. This identification is performed, for example, based on pattern matching, but is not limited to this and may be performed, for example, based on image recognition using machine learning. Information indicating the relationship between each position within the imaging range of the captured image and its X-direction coordinate and Y-direction coordinate is held in advance by the signal processing unit 10 and prepared so that it can be referenced by the gaze tracking unit 11. The gaze tracking unit 11 considers the midpoint between the right eye and the left eye in the image captured by the imaging unit 2 as a midpoint CP and identifies the X-direction coordinate and the Y-direction coordinate of the midpoint CP. Note that the method of identifying the position of the midpoint CP is merely an example and is not limited thereto and can be changed as appropriate. For example, the gaze tracking unit 11 may identify the midpoint CP based on the positional relationship between the positions of the user's two eyes (right eye and left eye) included in the image captured by the imaging unit 2 and the position of the user's nose. Furthermore, the gaze tracking unit 11 acquires the value of the distance measured by the distance measuring unit 3 as the value of pos_h. The gaze tracking unit 11 regards the midpoint between the right eye and the left eye in the image captured by the imaging unit 2 as a midpoint CP, and sets the position of the midpoint CP in the Z direction as pos_h. In this way, the gaze tracking unit 11 derives information about the position of the viewpoint.

[0035] Light emitted from each light-emitting point 32 reaches a first viewpoint E1 and a second viewpoint E2. Here, a first pixel Pix1 is located on a ray L1 of light reaching the first viewpoint E1 from each light-emitting point 32. Furthermore, a second pixel Pix2 is located on a ray L2 of light reaching the second viewpoint E2 from each light-emitting point 32. The image output by the first pixel Pix1 and the image output by the second pixel Pix2 are different images. The image output by the first pixel Pix1 is based on image data corresponding to the position of the first viewpoint E1. The image output by the second pixel Pix2 is based on image data corresponding to the position of the second viewpoint E2. More specifically, for example, image data 09-12.png in FIG. 2 is used as the image for the right eye's viewpoint, and image data 09-08.png is used as the image for the left eye's viewpoint. These images are combined by a signal processing unit and displayed as a single image (stereoscopic display image). More specifically, the stereoscopic display image, 09-08.png, and 09-12.png have the same number of pixels. For example, if pixels (n, m+1) (the left side indicates the nth row, m+1st position; the same applies hereinafter in this paragraph), (n, m+2), (n, m+3), and (n, m+4) in a pixel row of the stereoscopic display image need to be displayed as left-eye images, left-eye images, right-eye images, and right-eye images, respectively, the pixels (signals) corresponding to (n, m+1) and (n, m+2) in the stereoscopic display image use pixel signals corresponding to (n, m+1) and (n, m+2) in 09-08.png, which is the left-eye image data. Similarly, the pixels (signals) corresponding to (n, m+3) and (n, m+4) in the stereoscopic display image use pixel signals corresponding to (n, m+3) and (n, m+4) in 09-12.png, which is the right-eye image data. Furthermore, depending on the relationship between the user's viewpoint position and each light source (light-emitting point 32), there may be a pixel Pix that does not need to display any image. In this case, the pixel Pix displays an image with the lowest brightness (e.g., a black image). The image output unit 12 outputs image data corresponding to each of a plurality of viewpoints (e.g., a first viewpoint E1 and a second viewpoint E2) to the display panel 20 so that such an image can be displayed.

[0036] The distance in the Z direction between the center line of pixel Pix in the Z direction and midpoint CP can be expressed as distance Ph. The magnitude of distance Ph corresponds to the magnitude of the value of pos_h described above. The distance in the Z direction between the center line of pixel Pix in the Z direction and the point where light starts to be emitted from light-emitting point 32 can be expressed as distance Th. Note that distance Th is significantly smaller than distance Ph. In consideration of this, the center line of pixel Pix in the Z direction may be defined on the same plane as the pixel electrode, or may be defined on the same plane as the rear surface or front surface of second substrate 23, or the front surface of a cover glass provided on display panel 20. In the embodiment, the Z direction position of the point where light starts to be emitted from light-emitting point 32 is the position of the boundary line between light-shielding member 33 and adhesive layer 43.

[0037] The relationship between the light-emitting point pitch (e.g., light-emitting point pitch SpP or light-emitting point pitch SpP2) between light-emitting points 32 adjacent to each other in the X direction, the pitch (pixel pitch PP) of pixels Pix aligned in the X direction, and the relationship between light emitted from each of the light-emitting points 32 to each of the plurality of viewpoints will be described below with reference to Fig. 4. Fig. 4 is a cross-sectional view showing a cut surface of the display panel 20A cut along a plane (XZ plane) perpendicular to the Y direction, similar to the display panel 20 in Fig. 3. The display panel 20 shown in Fig. 1 may be the display panel 20A shown in Fig. 4 and subsequent figures.

[0038] FIG. 4 is a cross-sectional view illustrating an example in which the light-emitting point pitch SpP2, unlike the light-emitting point pitch SpP shown in FIG. 3, is six times the pixel pitch PP. In FIG. 4 and other figures referred to in the description of the embodiment, the rays L(m)1 and L(m)2, connected via a dashed line, are actually a single light ray that shares a common ray. For example, ray L321, which is the ray L(m)1 when m = 32, indicates the ray of light emitted from a single light-emitting point 32. The light emitted by this ray reaches the first viewpoint EC as ray L322, which is the ray L(m)2 when m = 32. While similar expressions are used below, the distance between the display panel 20 and the viewpoint is significantly greater than the pixel pitch PP, etc., and these relationships should be understood as being shown in a single diagram. In FIG. 4, m is a natural number between 32 and 35 or a natural number between 42 and 45. When m is a natural number from 32 to 35, the light ray and ray L31 are light rays that pass through the first pixel PixC and reach the first viewpoint EC. When m is a natural number from 42 to 45, the light ray and ray L41 are light rays that pass through the second pixel PixD and reach the second viewpoint ED. In Figure 4, a line that runs along the Z direction and passes through the second viewpoint ED is shown as a dashed dotted line PS2.

[0039] The first viewpoint EC is either the first viewpoint E1 or the second viewpoint E2 (see FIG. 3). The second viewpoint ED is the other of the first viewpoint E1 or the second viewpoint E2. When the first viewpoint EC is the first viewpoint E1, the first pixel PixC is the first pixel Pix1 (see FIG. 3). When the second viewpoint ED is the second viewpoint E2, the second pixel PixD is the second pixel Pix2 (see FIG. 3).

[0040] 4, the display panel 20A has a configuration in which six pixels Pix are arranged within a light-emitting point pitch SpP2, which is the distance between the center lines of two light-emitting points 32 adjacent to each other in the X direction. The X-direction position of the center lines of each of the two light-emitting points 32 overlaps with the X-direction position of the boundary line between the two pixels Pix adjacent to each other in the X direction when viewed from a plan view.

[0041] For example, as shown in Fig. 4, two of the six pixels Pix are controlled as first pixels PixC. The other two of the six pixels Pix are controlled as second pixels PixD. Of the six pixels Pix, a pixel Pix that does not correspond to either the first pixel PixC or the second pixel PixD is set as a third pixel PixE. The third pixel PixE is a pixel Pix that has the lowest degree of light transmission (for example, black display).

[0042] As shown by ray L41, the ray of light that passes through a second pixel PixD located opposite the second viewpoint ED in the Z direction and reaches the second viewpoint ED, i.e., the ray of light from a light-emitting point 32 located opposite the second viewpoint ED in the Z direction, runs along the Z direction. In FIG. 4, a line that runs along the Z direction and passes through the second viewpoint ED is shown as a dashed-dotted line PS2. Using the light-emitting point 32 that emits ray L41 as a reference, light that reaches the second viewpoint ED from other light-emitting points 32 located farther away from the light-emitting point 32 in the X direction has a ray of light that has a greater inclination angle with respect to the Z direction, as shown by rays L42, L43, L44, L45, and L46, as the distance in the X direction from the light-emitting point 32 increases. Based on these light ray projections, it is determined which of the multiple pixels Pix is ​​to be controlled as the second pixel PixD. Using a similar concept, as shown by the relationship between rays L31, L32, L33, L34, L35, and L36 and the first pixel PixC, which of the multiple pixels Pix will be controlled as the first pixel PixC is determined based on the ray of light emitted from the light emitting point 32 and reaching the first viewpoint EC.

[0043] Depending on the difference in the inclination angles of the rays L42, L43, L44, L45, and L46 relative to the Z direction, the pixels Pix controlled as the second pixels PixD may not necessarily be arranged at equal intervals in the X direction in some locations. Similarly, the pixels Pix controlled as the first pixels PixC may not necessarily be arranged at equal intervals in the X direction in some locations. In response to this control of the arrangement of the first pixels PixC and the second pixels PixD, the third pixels PixE may be arranged as appropriate, or the degree of light transmission may be controlled on a sub-pixel basis, as will be described later with reference to FIG. 17 and other figures. Such arrangement control can more reliably reduce the possibility of crosstalk.

[0044] 4, the midpoint CP is located on the dashed dotted line PC that overlaps with the center line in the X direction of one light-emitting point 32, but it is not essential that the midpoint CP be located at a position that overlaps with the center line in the X direction of the light-emitting point 32. The correspondence between the light from each light-emitting point 32, the positional relationship between the first viewpoint EA and the second viewpoint EB according to the position of the midpoint CP, and the control of the pixels Pix, which are the first pixel PixA and the second pixel PixB, is as described with reference to FIG. 4, regardless of the position of the midpoint CP.

[0045] Next, the basic concept of drive control of the pixel Pix according to the relative positional relationship between the viewpoint and the light emission start point will be described with reference to FIG.

[0046] FIG. 5 is a diagram showing various parameters related to determining the X-direction coordinates R_x(i) and L_x(i) of a pixel Pix located on the ray of light between the light-emitting point LP(i) of light from the (i+1)th light source in the X direction from the origin and the viewpoints ER and EL.

[0047] The light-emitting point LP(0) shown in FIG. 5 indicates the emission start point of light from the light-emitting point (e.g., light-emitting point 32) located at the position (first) closest to the origin in the X direction. The light-emitting point LP(i) indicates the emission start point of light from the light-emitting point (e.g., light-emitting point 32) located at the (i+1)th closest position counting from the origin in the X direction. For example, when i=1, the light-emitting point LP(1) indicates the emission start point of light from the light-emitting point (e.g., light-emitting point 32) located at the next closest position (second closest) to the light-emitting point LP(0) counting from the origin in the X direction. Therefore, i is an integer greater than or equal to 0.

[0048] In FIG. 5, the distance in the X direction between the origin and the light-emitting point LP(0) is defined as offset. The distance in the X direction between the origin and the light-emitting point LP(i) can be expressed as offset+(pitch×i). The magnitude of the value of pitch corresponds to the magnitude of the light-emitting point pitch SpP or the magnitude of the light-emitting point pitch SpP2 described above. The offset and offset+(pitch×i) are values ​​determined in advance according to the design of the display device 1, and are parameters that can be referenced in calculations related to determining the coordinates R_x(i) and L_x(i) in the X direction.

[0049] The magnitude of the distance Ph, which will be explained with reference to FIG. 3 and is also shown in FIG. 5, corresponds to the magnitude of the value of pos_h. The magnitude of the distance Px, shown in FIG. 5, corresponds to the magnitude of the value of pos_x. The distance in the Z direction between the light-emitting point LP(0) and the light-emitting point LP(i) and the origin is the distance Th described above. pos_h and pos_x can be acquired by the imaging unit 2 and the distance measuring unit 3.

[0050] Hereinafter, the distance in the X direction between the origin and the coordinate R_x(i) is defined as shiftR_x(i). Also, the distance in the X direction between the coordinate R_x(i) and the viewpoint ER is defined as widthR(i). Also, the distance in the X direction between the light-emitting point LP(i) and the viewpoint ER is defined as widthR_LED(i). The viewpoint ER is the viewpoint of the user's right eye, and is either the first viewpoint E1, EC or the second viewpoint E2, ED.

[0051] Also, the distance in the X direction between the origin and the coordinate L_x(i) is defined as shiftL_x(i). Also, the distance in the X direction between the coordinate L_x(i) and the viewpoint EL is defined as widthL(i). Also, the distance in the X direction between the light-emitting point LP(i) and the viewpoint EL is defined as widthL_LED(i). The viewpoint EL is the viewpoint of the user's left eye, and is the other of the first viewpoint E1, EC or the second viewpoint E2, ED.

[0052] The widthR_LED(i) can be expressed by the following formula (1). D1 in formula (1) etc. is a value indicating the magnitude of the distance D1 described with reference to FIG. 3 and also shown in FIG. 5. The value indicating the magnitude of the distance D1 can be set to a predetermined value based on the average value of a typical user. In the embodiment, the distance D1 is, for example, 31.25 millimeters (mm), but is not limited to this and can be changed as appropriate. widthR_LED(i)=pos_x-D1-{offset+(pitch×i)}…(1)

[0053] The widthR(i) can be expressed by the following formula (2). In formula (2) etc., Th is a value indicating the magnitude of the distance Th. The distance Th is determined in advance according to the design of the display device 1. The idea behind determining the distance Th at the time of design will be described later. widthR(i)=widthR_LED(i)×pos_h / (pos_h+Th)…(2)

[0054] shiftR_x(i) can be expressed as the following equation (3). shiftR_x(i)=pos_x-D1-widthR(i)…(3)

[0055] R_x(i) can be expressed as in the following equation (4). PP in equation (4) etc. is a value indicating the magnitude of the pixel pitch PP. The pixel pitch PP is determined in advance according to the design of the display device 1. Also, int() in equation (4) etc. indicates that an integer value is obtained by truncating the decimal point of the value in parentheses. R_x(i)=int(shiftR_x(i) / PP)…(4)

[0056] The widthL_LED(i) can be expressed as the following equation (5). widthL_LED(i)=pos_x+D1-{offset+(pitch×i)}…(5)

[0057] The widthL(i) can be expressed as the following equation (6). widthL(i)=widthL_LED(i)×pos_h / (pos_h+Th)…(6)

[0058] shiftL_x(i) can be expressed as the following equation (7). shiftL_x(i)=pos_x+D1-widthL(i)…(7)

[0059] L_x(i) can be expressed as the following equation (8). L_x(i)=int(shiftL_x(i) / PP)…(8)

[0060] By designating the pixel Pix arranged corresponding to R_x(i) as the first pixel Pix1, PixC and the pixel Pix arranged corresponding to L_x(i) as the second pixel Pix2, PixD, it is possible to realize display output control according to the positions of the first viewpoint E1, EC and the second viewpoint E2, ED as described with reference to Figures 3 and 4.

[0061] Next, the relative relationship between the arrangement direction of the second human eye and the X and Y directions corresponding to the arrangement of the pixels Pix of the display panel 20A will be described with reference to FIGS.

[0062] 6 is a schematic diagram showing examples A and B of the relative angular relationship between a human face HF and a display device 1 including a display panel 20A. Note that the rectangular display panel 20A shown in FIG. 6 and other figures has its longitudinal direction in the Y direction.

[0063] In Example A of FIG. 6, a reference line CLX that runs along the second arrangement direction of the face HF and passes through the midpoint CP is parallel to the X direction of the display panel 20A. Also, both the midline CLY of the face HF (see "Example B," FIG. 7) and the line that bisects the display panel 20A in the X direction overlap with the line CL. Also, the Y direction and the line CL are parallel. In Example A, similar to the explanation with reference to FIGS. 3 and 4, the arrangement direction of the first viewpoints E1, EC and the second viewpoints E2, ED is along the X direction. Therefore, in Example A, the display output control based on the explanation with reference to FIGS. 3 and 4 can be applied as is.

[0064] On the other hand, in example B of Figure 6, neither the midline CLY of the face HF nor the line bisecting the display panel 20A in the X direction overlaps with the line CL. Furthermore, the angle pos_r, which is the angle of the face HF relative to the line CL, differs from the angle dev_rot, which is the angle of the display panel 20A relative to the line CL. The angle pos_r is the angle formed by the midline CLY and the line CL. The angle dev_rot is the angle formed by the Y direction and the line CL.

[0065] In example A, it can be considered that the angle pos_r and the angle dev_rot are both 0 degrees (°).

[0066] 7 is a schematic diagram showing an example of the angular difference between the face HF and the display panel 20A based on the vertical line H and the horizontal line V. The vertical line H is aligned with the direction of gravity of the Earth. The horizontal line V is aligned with a plane perpendicular to the vertical line H. The vertical line H shown in FIG. 7 corresponds to the line CL in FIG.

[0067] The face HF shown in Figure 7 includes coordinates P1, P2, and P3. Coordinate P1 indicates the position of the first viewpoint E1, EC. Coordinate P2 indicates the position of the second viewpoint E2, ED. Coordinate P3 indicates a predetermined position (for example, the position of the nose) that overlaps with the midline of the face HF. The coordinates indicating the positions of coordinates P1, P2, and P3 are acquired based on image data captured by the imaging unit 2 and the distance measurement unit 3.

[0068] As a specific example, the gaze tracking unit 11 can identify the X- and Y-coordinates of the positions of the two eyes and nose on a human face HF using image processing technology using OpenCV. The gaze tracking unit 11 performs processing to derive a reference line CLX passing through coordinates P1 and P2. The gaze tracking unit 11 also performs processing to derive a median line CLY as a straight line that is perpendicular to the reference line CLX and passes through coordinate P3. The gaze tracking unit 11 also determines the midpoint between coordinates P1 and P2 as a midpoint CP, and derives the coordinates (pos_x, pos_y, pos_z) of the midpoint CP from the coordinates (X1, Y1, Z1) of coordinate P1 and the coordinates (X2, Y2, Z2) of coordinate P2. Generally, the midpoint CP coincides with the intersection of the reference line CLX and the median line CLY. Among the coordinates P1, P2, and P3, the coordinates (Z1, Z2, Z3) in the Z direction are measured by the distance measurement unit 3. The coordinate (pos_z) in the Z direction of the midpoint CP is treated as the distance Ph.

[0069] Furthermore, the gaze tracking unit 11 acquires information (tilt information) indicating the tilt direction of the display panel 20A relative to the vertical line H and the horizontal line V from the gyro sensor 4 included in the display device 1. The gaze tracking unit 11 derives the angle dev_rot based on the tilt information. The gaze tracking unit 11 identifies the orientations of the display panel 20A in the X direction and the Y direction relative to the vertical line H and the horizontal line V based on the relationship between the vertical line H, the horizontal line V, and the angle dev_rot.

[0070] The gaze tracking unit 11 derives the relative angle rot formed between the reference line CLX and the X direction. In the following description, if the relative angle rot is a positive value, it means that the midline CLY of the face HF is oriented so as to form an angle in the clockwise direction with respect to the Y direction of the display panel 20A. If the relative angle rot is a negative value, it means that the midline CLY of the face HF is oriented so as to form an angle in the counterclockwise direction with respect to the Y direction of the display panel 20A. The relative angle rot can be expressed, for example, in the range from -180 degrees (°) to 180 degrees (°). The angle pos_r is the sum of the angle dev_rot and the relative angle rot.

[0071] The image output unit 12 performs various processes related to display output control for displaying the viewpoint-corresponding image data OP on the display panel 20A by referring to information indicating the coordinates (pos_x, pos_y, pos_z) of the midpoint CP and information indicating the relative angle rot (or angle pos_r and angle dev_rot) among the various pieces of information derived and identified by the gaze tracking unit 11. Details of these processes will be described below.

[0072] Depending on the relative angle rot, it may not be possible to achieve individual output of images to multiple viewpoints by controlling the pixels Pix along the X direction of the first pixels Pix1, PixC and the second pixels Pix2, PixD as described with reference to Figures 3 and 4. Below, we will explain the relationship between the relative angle rot and the feasibility of stereoscopic vision.

[0073] FIG. 8 is a schematic diagram showing an example of the relationship between the relative angle rot and the feasibility of individually outputting images to multiple viewpoints by controlling the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD described with reference to FIGS. 3 and 4.

[0074] In Figure 8, the "Person" column schematically shows the direction of the person's face HF relative to the display panel 20A shown in the "Device" column. The "Device" column shows the range in which the person's line of sight is directed toward the display panel 20A of the display device 1 as one of ranges Fo1, Fo2, or Fo3. The "Relationship between Output and Recognition (Planar Viewpoint)" column schematically shows the relationship between the light-emitting point 32 and the pixel Pix in a portion of the display panel 20A that includes one of the ranges Fo1, Fo2, or Fo3 shown in the "Device" column. The "Relationship between Output and Recognition (Cross-Sectional Viewpoint)" column shows a cross section at the position indicated by the dashed line and arrow in the "Relationship between Output and Recognition (Planar Viewpoint)."

[0075] When the relative angle rot is 0 degrees (°), as shown schematically in the range Fo1 of the "Relationship between Output and Recognition (Cross-Sectional Viewpoint)," light L3 that passes through the first pixel PixC reaches the first viewpoint EC, and light L4 that passes through the second pixel PixD reaches the second viewpoint ED, by controlling the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD, as described with reference to Figures 3 and 4. In other words, when the relative angle rot is 0 degrees (°), individual output of images to multiple viewpoints can be realized by controlling the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD, as described with reference to Figures 3 and 4.

[0076] On the other hand, when the relative angle rot is 45 degrees (°), as shown schematically in the range Fo2 of the "Relationship Between Output and Recognition (Cross-Sectional Viewpoint)," simply applying the control of the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD described with reference to FIGS. 3 and 4 does not establish a ray of light between the first pixel PixC and the first viewpoint EC and a ray of light between the second pixel PixD and the second viewpoint ED. In other words, when the relative angle rot is 45 degrees (°), simply applying the control of the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD described with reference to FIGS. 3 and 4 makes it difficult to achieve individual output of images to multiple viewpoints. The same applies when the relative angle rot is 90 degrees (°), as shown schematically in the range Fo3 of the "Relationship Between Output and Recognition (Cross-Sectional Viewpoint)."

[0077] Fig. 9 is a schematic diagram showing another example of the relationship between the relative angle rot and the feasibility of individually outputting images to multiple viewpoints by controlling the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD described with reference to Figs. 3 and 4. In the configuration shown in Fig. 9, a linear light source 32A is provided instead of the light-emitting point 32 described with reference to Fig. 8. The light-emitting point 32 shown in Fig. 8 is a hole or a light-emitting element that functions as a so-called point light source. On the other hand, the linear light source 32A shown in Fig. 9 is a slit or a light-emitting element that functions as a linear light source along the Y direction.

[0078] Even when a linear light source 32A is used instead of the light-emitting point 32, if the relative angle rot is 0 degrees (°), individual output of images to multiple viewpoints can be achieved by controlling the pixels Pix of the first pixel PixC and the second pixel PixD along the X direction as described with reference to FIGS. 3 and 4. Also, when a linear light source 32A is used instead of the light-emitting point 32, if the relative angle rot is 45 degrees (°), as shown in the "45°" column of FIG. 9, individual output of images to multiple viewpoints can sometimes be achieved by controlling the pixels Pix of the first pixel PixC and the second pixel PixD along the X direction as described with reference to FIGS. 3 and 4, but this is not guaranteed. On the other hand, if the relative angle rot is 90 degrees (°), it is difficult to achieve individual output of images to multiple viewpoints simply by applying the control of the pixels Pix of the first pixel PixC and the second pixel PixD along the X direction as described with reference to FIGS. 3 and 4, regardless of whether the light-emitting point 32 or the linear light source 32A is used.

[0079] As described with reference to Figures 8 and 9, simply applying the control of the pixels Pix along the X direction of the first pixels Pix1 and PixC and the second pixels Pix2 and PixD described with reference to Figures 3 and 4 may make it difficult to achieve individual output of images to multiple viewpoints depending on the relative angle rot. Therefore, in this embodiment, processing (relative angle corresponding processing) is performed to more appropriately control the arrangement of the first pixels Pix1 and PixC and the second pixels Pix2 and PixD in accordance with the relative angle rot. An overview of this processing will be described below with reference to Figures 10 to 12.

[0080] Fig. 10 is a schematic diagram showing an example of pixel arrangement control when the relative angle rot is 45 degrees (°). Fig. 11 is a schematic diagram showing an example of pixel arrangement control when the relative angle rot is 90 degrees (°). Note that the arrangement control of the first pixel PixC and the second pixel PixD in the "Processing Not Reflected" column of Figs. 10 and 11 is the same as that in the "45°" column of Fig. 8. In the embodiment, pixel arrangement control is performed according to the relative angle rot, as shown in the "Processing Reflected" column.

[0081] 10 and 11, the pixel corresponding to the first pixel PixC when the control of the pixels Pix along the X direction of the first pixels Pix1 and PixC and the second pixels Pix2 and PixD described with reference to Figures 3 and 4 is applied is designated as the first pixel PixCB, and the pixel corresponding to the second pixel PixD is designated as the first pixel PixDB. Furthermore, the pixel corresponding to the first pixel PixC determined by the relative angle correspondence process applied in the embodiment is designated as the first pixel PixCA, and the pixel corresponding to the second pixel PixD is designated as the second pixel PixDA.

[0082] For example, when the control of the pixels Pix along the X direction of the first pixels Pix1 and PixC and the second pixels Pix2 and PixD described with reference to FIGS. 3 and 4 is reflected, a first image GC and a second image GD are output so as to be aligned in the X direction within the display panel 20A, as shown in the “Whole” column of the “Unreflected Processing” column in FIGS. 10 and 11 . The first image GC is an image perceived by light passing through the first pixel PixC. The second image GD is an image perceived by light passing through the second pixel PixD. If the relative angle rot is 0 degrees (°), a stereoscopic view is achieved using the first image GC and the second image GD shown in the “Whole” column of the “Unreflected Processing” column. On the other hand, if the relative angle rot is 45 degrees (°) or 90 degrees (°), a stereoscopic view is difficult to achieve using the first image GC and the second image GD shown in the “Whole” column of the “Unreflected Processing” column.

[0083] Therefore, as shown in the "Overall" column of "Processing Reflection" in FIGS. 10 and 11, pixel arrangement control is performed according to the relative angle rot so that the first image GCA and the second image GDA correspond to the second arrangement direction of the face HF within the display panel 20A. The first image GCA is an image recognized by light passing through the first pixel PixCA. The second image GDA is an image recognized by light passing through the second pixel PixDA. As shown by the comparison between the first image GC and the first image GCA and the second image GD and the second image GDA, the first image GCA is an image similar to the first image GC. The second image GDA is an image similar to the second image GD. Meanwhile, the relative positional and angular relationship between the first image GCA and the second image GDA is different from the relative positional and angular relationship between the first image GC and the second image GD. The relative positional and angular relationship between the first image GCA and the second image GDA is shifted according to the relative angle rot so as to correspond to the second arrangement direction of the face HF. The process for controlling the arrangement of the first pixel PixCA and the second pixel PixDA is performed so that the relative positional and angular relationship between the first image GCA and the second image GDA is established.

[0084] Fig. 12 is a schematic diagram showing an example of the relationship between the relative angle rot and the feasibility of individual output of images to multiple viewpoints when the arrangement control described with reference to Figs. 10 and 11 is reflected. As shown in the "45°" and "90°" columns in Fig. 12, when the arrangement control described with reference to Figs. 10 and 11 is reflected, light L3 that passes through the first pixel PixCA reaches the first viewpoint EC, and light L4 that passes through the second pixel PixDA reaches the second viewpoint ED. In other words, the arrangement control described with reference to Figs. 10 and 11 makes it possible to realize individual output of images to multiple viewpoints.

[0085] 10 to 12 are described taking as an example the case where the relative angle rot is 45 degrees (°) or 90 degrees (°), but in the embodiment, regardless of the value of the relative angle rot, the arrangement of the first pixel PixCA and the second pixel PixDA with respect to the light-emitting point LP such as the light-emitting point 32 or the line light source 32A can be made to correspond to the second arrangement direction on the face HF. That is, in the embodiment, regardless of the relative angle rot, it is possible to realize individual output of images to multiple viewpoints by controlling the pixels Pix along the X direction of the first pixel PixC and the second pixel PixD described with reference to FIGS.

[0086] The relative positional relationship between a display device such as the display panel 20A and a user's viewpoint (e.g., first viewpoint E1, second viewpoint E2) viewing an image output by the display device may change after the image output by the display device is determined. The display device of the embodiment changes the display output content in response to such changes in the relative positional relationship. Among such changes in the relative positional relationship, changes in the direction along the XY plane will be described below with reference to FIGS. 13 and 14. Furthermore, among such changes in the relative positional relationship, changes in the Z direction will be described with reference to FIGS. 13 and 14.

[0087] Fig. 13 is a schematic diagram showing an example of a change pattern of the relative positional relationship between the display panel 20A and the face HF. Fig. 14 is a diagram showing an example of a change pattern of the display output content corresponding to the change of the relative positional relationship shown in Fig. 13.

[0088] For example, assume that before the relative positional relationship between the display panel 20A and the face HF changes, the position of the face HF is at position PB1 shown in Fig. 13. In this assumption, the position of the midpoint CP (see Fig. 3, etc.) corresponds to the position of the central image data CP1 shown in Fig. 14, the image data selected for the image output for the first viewpoint E1 is image data LP1 shown in Fig. 14, and the image data selected for the image output for the second viewpoint E2 is image data LP2 shown in Fig. 14.

[0089] Here, as shown in "Example C" in FIG. 13, it is assumed that the position of the face HF has moved to one side in the X direction. That is, it is assumed that the position of the face HF has changed from position PB1 shown in FIG. 13 to position PA1. Furthermore, it is assumed that, due to this change in positional relationship, the position of the midpoint CP has changed from a position corresponding to the central image data CP1 shown in FIG. 14 to a position corresponding to the image data CP2. The coordinates of the image data CP2 are on one side in the X direction with respect to the coordinates of the central image data CP1. In this case, the display output content of the display panel 20A is changed in response to the change in the relative positional relationship between the display panel 20A and the face HF. Specifically, the image data selected for the image output for the first viewpoint E1 is changed from image data LP1 shown in FIG. 14 to image data LP2, and the image data selected for the image output for the second viewpoint E2 is changed from image data RP1 shown in FIG. 14 to image data RP2. The coordinates of the image data LP2 are on one side in the X direction with respect to the coordinates of the image data LP1. The coordinates of the image data RP2 are on one side in the X direction with respect to the coordinates of the image data RP1.

[0090] Alternatively, as shown in "Example D" in FIG. 13, suppose the position of the face HF moves to one side in the Y direction. That is, suppose the position of the face HF changes from position PB1 shown in FIG. 13 to position PA2. Furthermore, suppose that due to this change in positional relationship, the position of the midpoint CP changes from a position corresponding to the central image data CP1 shown in FIG. 14 to a position corresponding to the image data CP3. The coordinates of the image data CP3 are on one side in the Y direction with respect to the coordinates of the central image data CP1. In this case, the display output content of the display panel 20A is changed in response to the change in the relative positional relationship between the display panel 20A and the face HF. Specifically, the image data selected for the image output for the first viewpoint E1 is changed from image data LP1 shown in FIG. 14 to image data LP3, and the image data selected for the image output for the second viewpoint E2 is changed from image data RP1 shown in FIG. 14 to image data RP3. The coordinates of the image data LP3 are on one side in the Y direction with respect to the coordinates of the image data LP1. The coordinates of the image data RP3 are on one side in the Y direction with respect to the coordinates of the image data RP1.

[0091] Fig. 15 is a schematic diagram showing an example of a change pattern of the relative positional relationship between the display panel 20A and the face HF. Fig. 16 is a diagram showing an example of a change pattern of the display output content corresponding to the change of the relative positional relationship shown in Fig. 15.

[0092] For example, assume that before the relative positional relationship between the first viewpoint E1, the second viewpoint E2, and the midpoint CP and the display panel 20A changes, the positions of the first viewpoint E1, the second viewpoint E2, and the midpoint CP with respect to the display panel 20 are at position SP1 shown in Fig. 15. In this assumption, it is assumed that the position of the midpoint CP corresponds to the position of center image data CP1 shown in Fig. 16, the image data selected as the image to be output for the first viewpoint E1 is image data LP1 shown in Fig. 16, and the image data selected as the image to be output for the second viewpoint E2 is image data LP2 shown in Fig. 16. At this point, it is assumed that the distance between the first viewpoint E1, the second viewpoint E2, and the midpoint CP and the center line of the display panel 20 in the Z direction is distance Ph1.

[0093] Now, suppose that the relative positional relationship between the display panel 20A and the face HF has changed so that the face HF is closer to the display panel 20A. Specifically, suppose that the positions of the first viewpoint E1, the second viewpoint E2, and the midpoint CP have moved from position SP1 to position SP2 in FIG. 15. When the face HF is at position SP2, the distance between the first viewpoint E1, the second viewpoint E2, and the midpoint CP and the center line of the display panel 20 in the Z direction is distance Ph2. Distance Ph2 is shorter than distance Ph1. In this case, the display output content of the display panel 20A is changed in response to the change in the relative positional relationship between the display panel 20A and the face HF. Specifically, the image data selected for the image output for the first viewpoint E1 is changed from image data LP1 to image data LP5 shown in FIG. 16, and the image data selected for the image output for the second viewpoint E2 is changed from image data RP1 to image data RP5 shown in FIG. 16. The coordinates of the image data LP4 are on the other side in the X direction with respect to the coordinates of the image data LP1. The coordinates of the image data RP4 are on one side in the X direction with respect to the coordinates of the image data RP1.

[0094] Alternatively, suppose the relative positional relationship between the display panel 20A and the face HF changes so that the face HF becomes farther away from the display panel 20A. Specifically, suppose the positions of the first viewpoint E1, the second viewpoint E2, and the midpoint CP move from position SP1 to position SP3 in FIG. 15. When the face HF is at position SP3, the distance between the first viewpoint E1, the second viewpoint E2, and the midpoint CP and the center line of the display panel 20 in the Z direction is distance Ph2. Distance Ph3 is longer than distance Ph1. In this case, the display output content of the display panel 20A is changed in response to the change in the relative positional relationship between the display panel 20A and the face HF. Specifically, the image data selected for the image output for the first viewpoint E1 is changed from image data LP1 to image data LP4 shown in FIG. 16, and the image data selected for the image output for the second viewpoint E2 is changed from image data RP1 to image data RP4 shown in FIG. 16. The coordinates of the image data LP5 are on one side in the X direction with respect to the coordinates of the image data LP1. The coordinates of the image data RP5 are on the other side in the X direction with respect to the coordinates of the image data RP1.

[0095] In the embodiment, as shown in the relationship between image data LP1 and image data LP5, the relationship between image data RP1 and image data RP5, the relationship between image data LP1 and image data LP4, and the relationship between image data RP1 and image data RP4, when the distance between the display panel 20A and the user is relatively far, image data with coordinates closer to the central image data CP1 is selected compared to when the distance between the display panel 20A and the user is relatively close.

[0096] Specific processing related to the change of the display output content described with reference to Fig. 13 to Fig. 16 is performed by the signal processing unit 10. The signal processing unit 10 of the embodiment further has a function of causing the image generation unit 12b to interpolatively generate image data and display and output the image data on the display panel 20 when image data is not included in the viewpoint-corresponding image data OP at the positions of the first viewpoint E1 and the second viewpoint E2 detected by the gaze tracking unit 11. This function is performed by the image generation unit 12b.

[0097] When image data corresponding to the positional relationship between an image display surface such as the image display surface 200 (see FIG. 3) of a display panel such as the display panel 20A and a user's viewpoint (e.g., a first viewpoint E1 or a second viewpoint E2) is not stored in the storage unit 12a, the image generation unit 12b causes the image generation unit 12b to generate image data. In the embodiment, "when image data corresponding to the positional relationship between the image display surface of the display panel and the user's viewpoint is not stored in the storage unit" refers to a case where image data corresponding to the positions of the first viewpoint E1 and the second viewpoint E2 detected by the gaze tracking unit 11 is not included in the viewpoint-corresponding image data OP. The image generation unit 12b interpolatively generates image data corresponding to the positional relationship between the image display surface of the display panel and the user's viewpoint from two pieces of image data that are close to the positional relationship between the image display surface of the display panel and the user's viewpoint. This process of interpolatively generating image data is referred to as image data interpolation processing. In such a case, the image generation unit 12b selects the image data generated by the image generation unit 12b. Below, functions performed by the image generation unit 12b in the embodiment will be described with reference to FIGS. 17 to 24.

[0098] FIG. 17 is a schematic diagram showing an example of a change pattern of selected image data in response to a change in the rotation angle relative to the viewpoint EL and the viewpoint ER and the display panel 20A. For example, assume that the position of the midpoint CP (see FIG. 5) between the viewpoint EL and the viewpoint ER overlaps with the image data CP2 shown in FIG. 17, and that the viewpoints EL and ER are aligned along the X direction, as an initial condition. Assume that image data L_00 shown in FIG. 17 is selected as the image data for the viewpoint EL and image data R_00 shown in FIG. 17 is selected as the image data for the viewpoint ER under the initial condition. In this case, when the relative rotation angle between the display panel 20A and the user's face is changed and the viewpoints EL and ER rotate clockwise around the image data CP2 relative to the display panel 20A, the selected image data for the viewpoint EL and the image data for the viewpoint ER also become images corresponding to the rotation around the image data CP2. Below, we will assume that the viewpoints EL and ER rotate clockwise by Q degrees around the image data CP2 relative to the display panel 20A, based on the initial condition, and will explain the case depending on the value of Q.

[0099] When Q=15, image data L_15 shown in FIG. 17 is selected as image data for the viewpoint EL, and image data R_15 shown in FIG. 17 is selected as image data for the viewpoint ER. When Q=45, image data L_45 shown in FIG. 17 is selected as image data for the viewpoint EL, and image data R_30 shown in FIG. 17 is selected as image data for the viewpoint ER. When Q=75, image data L_75 shown in FIG. 17 is selected as image data for the viewpoint EL, and image data R_75 shown in FIG. 17 is selected as image data for the viewpoint ER. When Q=90, image data L_90 shown in FIG. 17 is selected as image data for the viewpoint EL, and image data R_90 shown in FIG. 17 is selected as image data for the viewpoint ER.

[0100] In FIG. 17, the X and Y coordinates of image data L_00 are (-4.5, 0.0). The X and Y coordinates of image data L_15 are (-4.5, +1.5). The X and Y coordinates of image data L_45 are (-3.0, +3.0). The X and Y coordinates of image data L_75 are (-1.5, +4.5). The X and Y coordinates of image data L_90 are (0.0, +4.5). The X and Y coordinates of image data R_00 are (+4.5, 0.0). The X and Y coordinates of image data R_15 are (+4.5, -1.5). The X and Y coordinates of image data R_45 are (+3.0, -3.0). The X and Y coordinates of image data R_75 are (+1.5, -4.5). The X and Y coordinates of image data R_90 are (0.0, -4.5).

[0101] On the other hand, there may be values ​​of Q that do not directly correspond to the image data included in the viewpoint-corresponding image data OP, such as when Q=30 or Q=60. When the viewpoint-corresponding image data OP does not include image data for a value of Q in this way, the image generating unit 12b of the embodiment generates image data that corresponds to Q by interpolation based on two corresponding image data with values ​​close to Q.

[0102] FIG. 18 is an explanatory diagram of parameters related to interpolative image data generation from a planar perspective. Hereinafter, when describing the coordinates of the display panel 20A, it is assumed that the shape of the display panel 20A from a planar perspective is rectangular, and one of the four vertices of the rectangle (the upper left in FIG. 18) is the origin (x, y) = (0, 0) in the coordinate system. Furthermore, the coordinates of the diagonal opposite side of the rectangle from the origin (the lower right in FIG. 18) are (H, V). Furthermore, the center point HP shown in FIG. 18 is the center point of the display panel 20A in the X and Y directions. The coordinates of the center point HP are (H / 2, V / 2).

[0103] Here, it is assumed that the coordinates of the intermediate point CP are (pos_x, pos_y, pos_h), the coordinates of the viewpoint EL are (posL_x, posL_y, posL_h), and the coordinates of the viewpoint ER are (posR_x, posR_y, posR_h). It is also assumed that (pos_x, pos_y, pos_h) are known, but the coordinates of the viewpoint EL and the viewpoint ER are not known. Specifically, an assumption is made that the data output by the gaze tracking unit 11 only indicates the coordinates of the intermediate point CP (pos_x, pos_y, pos_h), and does not indicate the coordinates of the viewpoint EL and the viewpoint ER individually. Hereinafter, first, the parameters related to the viewpoint EL will be described.

[0104] Each of posL_x, posL_y, and posL_h included in the coordinates (posL_x, posL_y, posL_h) of the viewpoint EL is calculated using the following equations (9), (10), and (11). D1 in equations (9), (10), and (11) is the distance D1 described with reference to Figure 3. rot in equations (9), (10), and (11) is the relative angle rot described with reference to Figure 7. PosL_x=pos_x-D1×cos(rot)…(9) PosL_y=pos_y-D1×sin(rot)…(10) posL_h=pos_h…(11)

[0105] Fig. 19 is an explanatory diagram from a side viewpoint including height direction parameters related to interpolative image data generation. The X and Y coordinates of point CxL shown in Figs. 18 and 19 can be expressed as (posL_x, V / 2). The X and Y coordinates of point CyL shown in Fig. 18 can be expressed as (H / 2, posL_y). Furthermore, the straight line length in the X direction of tangent ta_L shown in Fig. 19 can be calculated from the X coordinate of viewpoint EL and the X coordinate of point CyL, and can be expressed as the following equation (12). ta_L=PosL_x-H / 2…(12)

[0106] Furthermore, the tangent ta_L shown in Fig. 19 corresponds to the tangent (tan) in trigonometric functions for the angle θL_x shown in Fig. 19. The angle θL_x is the angle between the vertical line and the line connecting the viewpoint EL and the point CyL. The vertical line is a line connecting the point at which the angle is formed (in this case, the point CyL) and the image display surface 200 along the Z direction, and is indicated in Fig. 19 by a dashed line marked "pos_h". Therefore, the following equation (13) holds true. tan(θL_x)=(PosL_x-H / 2) / posL_h…(13)

[0107] Here, the following equation (14) holds from equation (13): Note that arcTan in equation (14) and equation (15) described later represents the inverse trigonometric function of tangent (tan). θL_x= arcTan((posL_x-H / 2) / posL_h)…(14)

[0108] Although not shown, the length of the straight line in the Y direction between the Y coordinate of the viewpoint EL and the Y coordinate of the point CyL is defined as ta_L2. Also, the angle between the vertical line and the line connecting the viewpoint EL and point CL_x (see FIG. 18) is defined as angle θL_y. In this case, the relationship between ta_L2 and θL_y is the same as the relationship between the tangent ta_L and the angle θL_x described above. Therefore, the following equation (15) holds: θL_y= arcTan((posL_y-V / 2) / posL_h)…(15)

[0109] Also, the unit angle of the viewpoint-corresponding image data OP described with reference to Figure 2 is assumed to be angle Dltθ. In the example shown in Figure 2, Dltθ=1.5 as described above. Using equations (16) and (17) that refer to Dltθ, the values ​​imgNLx and imgNLy can be defined as follows: Int{} in equations (16) and (17) indicates that the fraction after the decimal point of the value resulting from the calculation in {} is truncated (integer type). imgNLx=int{θL_x / (Dltθ / 2)} / 2…(16) imgNLy=int{θL_y / (Dltθ / 2)} / 2…(17)

[0110] As an example, when θL_x=7.4°, θL_y=−5.1°, and Dltθ=1.5°, imgNLx=4.5 and imgNLy=−3.

[0111] The combination of the value imgNLx and the value imgNLy identifies two pieces of image data to be selected corresponding to the coordinates of the viewpoint EL and the relative angle rot, and to be referenced in the image data interpolation process. Below, the concept of selecting two pieces of image data to be referenced in the image data interpolation process will be described with reference to Figures 20 to 22.

[0112] FIG. 20 is a schematic diagram showing an example of interpolation processing that references two image data aligned in the X direction and two image data aligned in the Y direction. In FIG. 20 and FIG. 21, which will be described later, the XY coordinates of image data Fu1, which is one of the two image data referenced in the interpolation processing, are assumed to be (α, β). The solid-line rectangles in FIGS. 20 and 21 represent a portion of the image data included in the viewpoint-corresponding image data OP. The dashed-line rectangles in FIGS. 20 and 21 represent image data that can be generated by the interpolation processing. In FIGS. 20 and 21, the XY coordinates other than (α, β) are shown as plus or minus (±) values ​​relative to α, β.

[0113] For example, if the two image data referenced in the interpolation process are image data Fu1 and image data Fu2 (see FIG. 20) that are aligned on one side of the image data Fu1 in the X direction, image data Fu3 is generated by the interpolation process. The image data Fu3 is considered to be image data between the image data Fu1 and the image data Fu2. Also, if the two image data referenced in the interpolation process are image data Fu1 and image data Fu4 (see FIG. 20) that are aligned on one side of the image data Fu1 in the Y direction, image data Fu5 is generated by the interpolation process. The image data Fu5 is considered to be image data between the image data Fu1 and the image data Fu4.

[0114] 21 is a schematic diagram showing an example of interpolation processing that references two image data that are arranged diagonally in a direction that intersects the X and Y directions. When the two image data referenced in the interpolation processing are image data Fu1 and image data Fu6 (see FIG. 21), image data Fu7 is generated by the interpolation processing. The image data Fu7 is considered to be image data between the image data Fu1 and the image data Fu6. When the two image data referenced in the interpolation processing are image data Fu1 and image data Fu8 (see FIG. 21), image data Fu9 is generated by the interpolation processing. The image data Fu9 is considered to be image data between the image data Fu1 and the image data Fu8.

[0115] The value of imgNLx is applied to α. The value of imgNLy is applied to β. Whether the two image data referenced in the interpolation process are aligned in the X direction, the Y direction, or a diagonal direction depends on the combination of the values ​​of imgNLx and imgNLy calculated by the above-mentioned equations (16) and (17).

[0116] First, we will explain the case where the value of the value imgNLx in the first decimal place is 5 and the value of the value imgNLy in the first decimal place is 0. In this case, two image data aligned in the X direction are referenced in the interpolation process. Also, as described above, the XY coordinates (α, β) of the image data Fu1 are α=imgNLx, β=imgNLy. One of the two image data to be referenced is the image data Fu1 at (α, β) identified in this way, but in this case, whether the image data on one side or the other side of the image data Fu1 in the X direction is referenced as the second image data depends on the value of the value imgNLx.

[0117] When the value imgNLx is positive, the second image data is (α+1.5,β). That is, in this case, the second image data is image data Fu2, which is arranged on one side of the image data Fu1 in the X direction, relative to the first image data Fu1 of (α,β). On the other hand, when the value imgNLx is negative, the second image data is (α-1.5,β). That is, in this case, the second image data is arranged on the other side of the image data Fu1 in the X direction, relative to the first image data Fu1 of (α,β). The image data arranged on the other side is image data that is located in a position linearly symmetrical in the X direction relative to the image data Fu2, with the image data Fu1 in between.

[0118] When the value imgNLx is 0, the same processing as when the value imgNLx is a positive value may be performed, or the same processing as when the value imgNLx is a negative value may be performed. In the embodiment, when the value imgNLx is 0, the same processing as when the value imgNLx is a positive value is performed.

[0119] Next, we will explain the case where the value of the first decimal place of the value imgNLx is 0 and the value of the first decimal place of the value imgNLy is 5. In this case, two image data aligned in the Y direction are referenced in the interpolation process. Also, as described above, the XY coordinates (α, β) of the image data Fu1 are α=imgNLx, β=imgNLy. One of the two image data to be referenced is the image data Fu1 at (α, β) identified in this way, but in this case, whether the image data on one side or the other side of the image data Fu1 in the Y direction is referenced as the second image data depends on the value of the value imgNLy.

[0120] When the value of imgNLy is positive, the second image data is (α,β-1.5). That is, in this case, the second image data is image data Fu4, which is arranged on one side of the image data Fu1 in the Y direction, relative to the first image data Fu1 with (α,β). On the other hand, when the value of imgNLy is negative, the second image data is (α,β+1.5). That is, in this case, the second image data is arranged on the other side of the image data Fu1 in the Y direction, relative to the first image data Fu1 with (α,β). The image data arranged on the other side is image data that is located in a position linearly symmetrical in the Y direction relative to the image data Fu4, with the image data Fu1 in between.

[0121] When the value imgNLy is 0, the same processing as when the value imgNLy is a positive value may be performed, or the same processing as when the value imgNLy is a negative value may be performed. In the embodiment, when the value imgNLy is 0, the same processing as when the value imgNLy is a positive value is performed.

[0122] Next, we will explain the case where the value of the value imgNLx to the first decimal place is 5 and the value of the value imgNLy to the first decimal place is 5. In this case, two image data aligned diagonally intersecting the X and Y directions are referenced in the interpolation process. Here, whether the diagonal direction is along the alignment direction of the image data Fu1 and image data Fu6 shown in FIG. 21 or along the alignment direction of the image data Fu1 and image data Fu8 depends on whether the relative angle rot is positive or negative.

[0123] 22 is a diagram showing the relationship between the relative angle rot and the midline CLY of the face HF with respect to the Y direction of the display panel 20A. As described above, when the relative angle rot is a positive value, this indicates that the midline CLY of the face HF is oriented so as to form an angle in the clockwise direction with respect to the Y direction of the display panel 20A. When the relative angle rot is a negative value, this indicates that the midline CLY of the face HF is oriented so as to form an angle in the counterclockwise direction with respect to the Y direction of the display panel 20A.

[0124] First, we will explain the case where two image data items arranged diagonally in a direction intersecting the X and Y directions are referenced in the interpolation process and the relative angle rot is a positive value. In this case, when the value imgNLx is a positive value or 0 and the value imgNLy is a positive value or 0, one of the two image data items becomes (α, β + 1.5) and the other of the two image data items becomes (α + 1.5, β). In addition, when the value imgNLx is a positive value or 0 and the value imgNLy is a negative value, one of the two image data items becomes (α, β) and the other of the two image data items becomes (α + 1.5, β - 1.5). In addition, when the value imgNLx is a negative value and the value imgNLy is a positive value or 0, one of the two image data items becomes (α, β) and the other of the two image data items becomes (α - 1.5, β + 1.5). Also, in this case, when the value of imgNLx is negative and the value of imgNLy is negative, one of the two image data becomes (α-1.5, β) and the other of the two image data becomes (α, β-1.5).

[0125] Next, we will explain the case where two image data items arranged diagonally in a direction intersecting the X and Y directions are referenced in the interpolation process and the relative angle rot is a negative value. In this case, when the value imgNLx is a positive value or 0 and the value imgNLy is a positive value or 0, one of the two image data items becomes (α, β) and the other of the two image data items becomes (α + 1.5, β + 1.5). In addition, when the value imgNLx is a positive value or 0 and the value imgNLy is a negative value, one of the two image data items becomes (α + 1.5, β) and the other of the two image data items becomes (α, β - 1.5). In addition, when the value imgNLx is a negative value and the value imgNLy is a positive value or 0, one of the two image data items becomes (α, β + 1.5) and the other of the two image data items becomes (α - 1.5, β). Also, in this case, when the value of imgNLx is negative and the value of imgNLy is negative, one of the two image data becomes (α, β) and the other of the two image data becomes (α-1.5, β-1.5).

[0126] The above has described the interpolation process based on various parameters derived from the coordinates of the viewpoint EL (posL_x, posL_y, posL_h). Next, we will explain the case where the coordinates of the viewpoint EL are replaced with the coordinates of the viewpoint ER.

[0127] The coordinates of the viewpoint ER shown in FIG. 18 are (posR_x, posR_y, posR_h). The coordinates of the point CxR are (posR_x, V / 2). The coordinates of the point CyR are (H / 2, posR_y). These parameters can be derived in the same way as the various parameters related to the coordinates of the viewpoint EL described with reference to the above equations (9) to (17). However, D1×cos(rot), which was subtracted from pos_x when calculating PosL_x of the viewpoint EL, is added when calculating PosR_x of the viewpoint ER. Furthermore, D1×sin(rot), which was subtracted from pos_y when calculating PosL_y of the viewpoint EL, is added when calculating PosR_y of the viewpoint ER. Therefore, the following equations (18) to (26) hold. PosR_x=pos_x+D1×cos(rot)…(18) PosR_y=pos_y+D1×sin(rot)…(19) posR_h=pos_h…(20) ta_R=PosR_x-H / 2…(21) tan(θR_x)=(PosR_x-H / 2) / posR_h…(22) θR_x= arcTan((posR_x-H / 2) / posR_h)…(23) θR_y= arcTan((posR_y-V / 2) / posR_h)…(24) imgNRx=int{θR_x / (Dltθ / 2)} / 2…(25) imgNRy=int{θR_y / (Dltθ / 2)} / 2…(26)

[0128] The above description of the combination of the values ​​imgNLx and imgNLy can be applied as is by replacing imgNLx with imgNRx and imgNLy with imgNRy. Therefore, the combination of the values ​​imgNRx and imgNRy can identify two pieces of image data to be selected corresponding to the coordinates of the viewpoint ER and the relative angle rot, and to be referenced in the image data interpolation process.

[0129] Next, a specific example of image data interpolation processing will be described with reference to Figures 23 to 25. In image data interpolation processing, block matching is performed centered on each of a plurality of pixel data included in image data treated as one of two pieces of image data referenced in the interpolation processing. Such block matching will be described below with reference to Figures 23 to 25. As can be seen above, the two pieces of image data used in this interpolation processing are very similar image data that differ slightly in angle in one direction, and therefore the block matching that interpolates between them can also be processed specialized in only that one direction, thereby simplifying image processing.

[0130] FIG. 23 is a diagram illustrating a process for extracting a block Block S from image data PicS, which is treated as one of two pieces of image data referenced in the image data interpolation process. In the image data interpolation process, processing is performed individually for each pixel data included in one of the two pieces of image data referenced in the interpolation process. In this pixel-data-by-pixel processing, first, a block centered on that pixel data is extracted. In FIG. 23, pixel data F_S is shown as the pixel data to be individually processed among the multiple pixel data included in image data PicS, which is treated as one of the two pieces of image data referenced in the image data interpolation process and is arranged in a matrix along the bx and by directions. Furthermore, block Block S is shown as a block including 5 × 5 = 25 pixel data arranged within a range of ±2 in the bx direction and ±2 in the by direction, centered around pixel data F_S. In this embodiment, block Block S is extracted during processing of pixel data F_S. The sum of the product of the gradation values ​​of the pixel data included in block Block S can be expressed as follows:

number

[0131] Hereinafter, a case where image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is aligned in the X direction with respect to image data PicS, will be described with reference to FIG.

[0132] Figures 24 and 25 are schematic diagrams showing blocks BlockF(-3) to BlockF(3) extracted from image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, and block-matched with block BlockS.

[0133] In image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, pixel data F_F is first identified. The position of pixel data F_F in the image data PicF is the same as the position of pixel data F_S in the image data PicS. Note that, although not shown in Figure 24, the number of pixels arranged along the bx and by directions in the actual image data PicF is the same as the number of pixels arranged along the bx and by directions in the image data PicS.

[0134] Furthermore, a pixel area having the same number of pixels arranged in the bx and by directions as the block BlockS, with the pixel data F_F at the center, is treated as a block BlockF(0).

[0135] As an example, assume that the coordinates of pixel data F_S and pixel data F_F are treated as (bx, by) = (0, 0). In this case, the bx coordinate of the pixel data included in a pixel region of bx × by = 5 × 5 located on one side in the bx direction is "2," and the bx coordinate of the pixel data included in the pixel region located on the other side in the bx direction is "-2." Furthermore, the by coordinate of the pixel data included in the pixel region located on one side in the by direction is "2," and the by coordinate of the pixel data included in the pixel region located on the other side in the by direction is "-2." Therefore, block BlockF(0) in FIG. 24 is illustrated as a pixel region with bx coordinates ranging from "2" to "-2" and by coordinates ranging from "2" to "-2," with pixel data F_F of (bx, by) = (0, 0) at its center.

[0136] In addition to block F(0), six pixel regions having the same number of pixels arranged in the bx and by directions as block F(0) are extracted from a bx × by = 25 × 25 pixel region within a range of ±3 pixels in the bx direction with block F(0) at the center. Here, with block F(0) as the base, the pixel region positioned Λ pixels away in the bx direction is called block F(Sar), and the pixel region positioned Λ pixels away on the other side in the bx direction is called block F(-Λ). Λ is a natural number ranging from 1 to 3. Therefore, the six pixel regions extracted in addition to block F(0) can be represented as block F(1), block F(2), block F(3), block F(-1), block F(-2), and block F(-3).

[0137] Block F(1) is centered at coordinates (bx, by) = (1, 0), and differs from block F(0) in that the bx coordinate of pixel data included in a pixel region of bx × by = 5 × 5 located on one side in the bx direction is "3," and the bx coordinate of pixel data included in the pixel region located on the other side in the bx direction is "-1." This is equivalent to the fact that, when Λ = 1, block F(Λ) is centered at coordinates (bx, by) = (0 + Λ, 0), and differs from block F(0) in that the bx coordinate of pixel data included in the pixel region of bx × by = 5 × 5 located on one side in the bx direction is "2 + Λ," and the bx coordinate of pixel data included in the pixel region located on the other side in the bx direction is "-2 + Λ." This difference between block F(Λ) and block F(0) holds even when Λ = 2 or Λ = 3.

[0138] Block BlockF(-1) differs from block BlockF(0) in that, with its center at the coordinates (bx, by) = (-1, 0), the bx coordinate of pixel data contained in a pixel region of bx × by = 5 × 5 located on one side in the bx direction is "1," and the bx coordinate of pixel data contained in the pixel region located on the other side in the bx direction is "-3." This is equivalent to saying that, when Λ = 1, block BlockF(Λ) differs from block BlockF(0) in that its center is at the coordinates (bx, by) = (0-Λ, 0), the bx coordinate of pixel data contained in a pixel region of bx × by = 5 × 5 located on one side in the bx direction is "2-Λ," and the bx coordinate of pixel data contained in the pixel region located on the other side in the bx direction is "-2-Λ." The difference between Block F(Λ) and Block F(0) is valid even when Λ=2 or Λ=3.

[0139] Although the illustration of the blocks BlockF(±2) is omitted in FIG. 24, the blocks BlockF(±2) are also extracted.

[0140] Here, if Sar is an integer value within the range of -3 to 3 (-3≦Sar≦3), the sum of the products of the gradation values ​​of the pixel data included in block BlockF(Sar) can be expressed as in the following equation (28). When Sar is 0, it is block BlockF(0). When Sar is a positive integer, block BlockF(Sar) can be said to be the same as block BlockF(Λ) described above. When Sar is a negative integer, block BlockF(Sar) can be said to be the same as block BlockF(-Λ) described above.

number

[0141] When image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is arranged on one side of image data PicS in the X direction, the sum of products of equation (28) described above is calculated individually for each of the cases where Sar takes the values ​​of 3, 2, 1, and 0, and compared with the sum of products of equation (27). When image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is arranged on the other side of image data PicS in the X direction, the sum of products of equation (28) described above is calculated individually for each of the cases where Sar takes the values ​​of 0, -1, -2, and -3, and compared with the sum of products of equation (27). In either case, the sum of products of equation (28) is calculated, and the value of Sar that calculates the sum of products closest to the sum of products of equation (27) is selected as SarX. Based on the selected SarX and the following equation (29), pixel data generated by the image data interpolation process is determined based on the pixel data of one of the two pieces of image data referenced in the image data interpolation process (the above-mentioned block BlockS). Out(x,y) in equation (29) represents pixel data generated by the image data interpolation process. InS(x,y) in equation (29) represents pixel data of one of the two pieces of image data referenced in the image data interpolation process (the above-mentioned block BlockS). Therefore, it can be said that the pixel data (x,y) generated by the image data interpolation process is pixel data of one of the two pieces of image data referenced in the image data interpolation process (the above-mentioned block BlockS), and is pixel data shifted by "+SarX / 2" in the bx direction from (bx,by)=(x,y). If x+SarX / 2 is not an integer value, half of the sum of the product of the pixel data of the two coordinates with the two integer values ​​closest to x+SarX / 2, i.e., InS(x+0.5+SarX / 2, y) and InS(x-0.5+SarX / 2, y), is used as the calculation result of equation (29). Out(x,y)=InS(x+SarX / 2,y)…(29)

[0142] The case where image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is arranged on one side of image data PicS in the X direction has been described above with reference to Figure 24. In contrast, when image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is arranged on one side of image data PicS in the Y direction, the sum of the products of the gradation values ​​of the pixel data included in block BlockF (Sar) is expressed by the following equation (30). Equation (30) differs from equation (28) in that Sar is added to the by coordinate rather than the bx coordinate. In other words, equation (30) indicates that the arrangement direction of the seven pixel areas indicated by block BlockF (Sar) is the by direction, with Sar being an integer value within the range from -3 to 3 (-3≦Sar≦3).

number

[0143] When image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is located on one side of image data PicS in the Y direction, the sum of products of equation (30) described above is calculated for each of the cases where Sar takes the values ​​of 3, 2, 1, and 0, and compared with the sum of products of equation (27). When image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is located on the other side of image data PicS in the Y direction, the sum of products of equation (30) described above is calculated for each of the cases where Sar takes the values ​​of 0, -1, -2, and -3, and compared with the sum of products of equation (27). In either case, the sum of products of equation (30) is calculated, and the value of Sar for which the sum of products closest to the sum of products of equation (27) is calculated is selected as SarY. Based on the selected SarY, pixel data generated in the image data interpolation process is determined based on the pixel data of one of the two pieces of image data referenced in the image data interpolation process (the above-mentioned block BlockS) according to the following equation (31): If y+SarY / 2 is not an integer value, half the sum of the products of the pixel data of the two coordinates with the two integer values ​​closest to y+SarY / 2, i.e., InS(x, y+0.5+SarY / 2) and InS(x, y-0.5+SarY / 2), is used as the calculation result of equation (31). As described above, except for points noted otherwise, the concept of the image data interpolation process when image data PicF, which is treated as the other of the two pieces of image data referenced in the image data interpolation process, is arranged on one side of the image data PicS in the Y direction is the same as the concept of the image data interpolation process when image data PicF, which is treated as the other of the two pieces of image data referenced in the image data interpolation process, is arranged on one side of the image data PicS in the X direction, as described with reference to FIG. Out(x,y)=InS(x,y+SarY / 2)…(31)

[0144] Next, a case where image data PicF, which is treated as the other of the two image data referenced in the image data interpolation process, is arranged diagonally relative to image data PicS in a direction intersecting the X and Y directions, will be described with reference to FIG.

[0145] In the case of the diagonal direction, the product-sum equation is determined so that the coordinates in both the bx and by directions change for one variable Sar. When the diagonal direction, which is the arrangement direction of two image data referenced in the image data interpolation process, is the arrangement direction shown in Fig. 25, that is, when the arrangement direction of block BlockF(Sar) centered on block BlockF(0) can be expressed as a direction in which both the bx and by directions increase as the value of Sar increases, the product-sum equation can be expressed as the following equation (32).

number

[0146] For each integer value of Sar between -3 and 3, the sum of products of the above-mentioned equation (32) is calculated individually and compared with the sum of products of equation (27). The value of Sar that calculates the sum of products closest to the sum of products of equation (27) is selected as SarZ. Based on the selected SarZ and the following equation (33), pixel data to be generated in the image data interpolation process is determined based on the pixel data of one of the two image data (the above-mentioned block BlockS) referenced in the image data interpolation process. Note that if SarZ / 2 is not an integer value, half of the sum of products of the pixel data of the two coordinates with the two integer values ​​closest to the right-hand side, i.e., the two pixel data InS(x+0.5+SarZ / 2, y+0.5-SarZ / 2) and InS(x+0.5-SarZ / 2, y-0.5-SarZ / 2), is used as the calculation result of equation (33). Out(x,y)=InS(x+SarZ / 2,y-SarZ / 2)…(33)

[0147] On the other hand, if the diagonal direction, which is the arrangement direction of the two image data referenced in the image data interpolation process, is different from the arrangement direction shown in Figure 25, that is, if the arrangement direction of block BlockF(Sar) centered on block BlockF(0) can be expressed as a direction in which the bx direction increases and the by direction decreases as the value of Sar increases, the product-sum equation can be expressed as the following equation (34).

number

[0148] For each integer value of Sar between -3 and 3, the sum of products of the above-mentioned equation (34) is calculated individually and compared with the sum of products of equation (27). The value of Sar that calculates the sum of products closest to the sum of products of equation (27) is selected as SarZ. Based on the selected SarZ and the following equation (35), pixel data to be generated in the image data interpolation process is determined based on the pixel data of one of the two image data (the above-mentioned block BlockS) referenced in the image data interpolation process. Note that if SarZ / 2 is not an integer value, half of the sum of products of the pixel data of the two coordinates with the two integer values ​​closest to the right-hand side, i.e., the two pixel data InS(x+0.5-SarZ / 2, y+0.5+SarZ / 2) and InS(x-0.5-SarZ / 2, y-0.5+SarZ / 2), is used as the calculation result of equation (33). Out(x,y)=InS(x-SarZ / 2,y+SarZ / 2)…(35)

[0149] The formulas (32) and (33) are used when the relative angle rot is a negative value, and the formulas (34) and (35) are used when the relative angle rot is a positive value.

[0150] Note that, when the arrangement direction of two image data referenced in the image data interpolation process is a diagonal direction, in the product sum calculated by equations (27), (32), and (34), the product when bx = by = 0 may be treated as having a greater weight than the product when the values ​​of bx and by are other values. Specifically, when the arrangement direction of two image data referenced in the image data interpolation process is a diagonal direction, in the product sum calculated by equations (27), (32), and (34), the value of the product when bx = by = 0 may be replaced by "twice the value of the product when bx = by = 0" to calculate the product sum. In other words, the product when bx = by = 0 may be weighted twice as much as the product when the values ​​of bx and by are other values.

[0151] As described above, except for the points noted above, the concept of image data interpolation processing when image data PicF, which is treated as the other of the two image data referenced in the image data interpolation processing, is aligned diagonally relative to image data PicS, is the same as the concept of image data interpolation processing when image data PicF, which is treated as the other of the two image data referenced in the image data interpolation processing, is aligned to one side of image data PicS in the X direction, as described with reference to Figure 24.

[0152] The signal processing unit 10 performs the processing related to the display output control in accordance with the relative positional relationship between the display panel 20A and the face HF, which has been described with reference to FIGS.

[0153] FIG. 26 is a diagram showing an example of the flow of processing performed as diagonal interpolation processing when rot<0. FIG. 26 shows an example in which image data PicIP is generated by interpolation processing with reference to two image data, image data PicS and image data PicF, arranged diagonally. Hereinafter, with reference to FIG. 26, Out(x, y) of image data PicIP obtained based on one block BlockS in image data PicS and a block group BF in image data PicF at a position corresponding to the block BlockS will be described. The block group BF in image data PicF shown in "Phase 1" of FIG. 26 is a collection of blocks BlockF(Sar) when the arrangement direction of blocks BlockF(Sar) centered on block BlockF(0) can be expressed as a direction in which both the bx direction and the by direction increase as the value of Sar increases, as also shown in FIG. 25. Here, as described above, it is assumed that Sar takes integer values ​​between -3 and 3.

[0154] 26 shows an example in which Block(-2) is selected from among the blocks BlockF(Sar) shown as the block group BF in "phase 1." That is, the example shown in Fig. 26 shows a case in which the sum of products of the above-mentioned equation (32) is calculated individually for each of BlockF(Sar) included in the block group BF in "phase 1" and having Sar between -3 and 3, and compared with the sum of products of equation (27). As a result, the value of Sar that is calculated to be closest to the sum of products of equation (27) is -2. In other words, "phase 2" shows a case in which SarZ=-2.

[0155] "Phase 3" in FIG. 26 shows that Out(x, y) calculated by applying SarZ=-2 selected in "Phase 2" to the above-mentioned equation (33) is applied to the image data PicIP. The above explanation relates to Out(x, y) of the image data PicIP obtained based on one block BlockS and a group of blocks BF in the image data PicF at a position corresponding to the block BlockS. However, the image data PicIP is obtained by applying a similar process to all pixel data F_S included in block BlockS. "Phase 3" shows that, as a result, a cubic image CB3 intermediate between the cubic image CB1 drawn in block BlockS and the cubic image CB2 drawn in block BlockF is drawn in the image data PicIP.

[0156] According to the above-described embodiment, the display device 1 includes a display panel (e.g., display panel 20 or display panel 20A) having a plurality of pixels (e.g., pixel Pix), a light source (e.g., light source 30) having a plurality of light-emitting points (e.g., light-emitting point LP, specifically light-emitting point 32) for irradiating the pixels of the display panel with light, an acquisition unit (e.g., image capture unit 2, distance measurement unit 3, gyro sensor 4, and gaze tracking unit 11) for acquiring information about the viewpoint of a user viewing the display panel, a storage unit (e.g., storage unit 12a) for storing a plurality of image data selectable according to a positional relationship between the display panel and the viewpoint, and a control unit (e.g., image output unit 12) for controlling the display of an image by operating the plurality of pixels based on the information. The information includes information about the positions of the viewpoints (e.g., first viewpoint E1 and second viewpoint E2, first viewpoint EC and second viewpoint ED, etc.) (e.g., pos_x, pos_y, pos_h) and information indicating the arrangement direction of the viewpoints (relative angle rot). The control unit drives and transmits light to at least some or all of the pixels (pixels Pix including the passing point UP) located on a line connecting each light-emitting point and each viewpoint, based on the angle (relative angle rot) between a predetermined direction (e.g., the X direction) and the arrangement direction on the display panel and the positional relationship between the viewpoint and the light-emitting points. If image data for the positional relationship between the display panel and the viewpoint is not stored in the storage unit, the control unit includes an image generation unit (e.g., image generation unit 12b) that interpolatively generates image data corresponding to the positional relationship from two pieces of image data close to the positional relationship. In this case, the control unit selects the image data generated by the image generation unit. The ratio of the pitch of the multiple pixels arranged in the predetermined direction to the pitch of the multiple light-emitting points arranged in the predetermined direction is 1:4n or 1:6n (e.g., 1:6), where n is a natural number. This allows the control unit to display an image based on information related to the user's viewpoint, thereby enabling more flexible adaptation to the relationship between the arrangement direction of the viewpoint and the display device 1.Furthermore, if image data corresponding to the positional relationship between the display panel and the viewpoint is not stored in the storage unit, image data corresponding to the positional relationship is interpolated from two pieces of image data stored in the storage unit that are close to the positional relationship, so that the displayed image can be made to correspond with higher accuracy to the positional relationship between the display device 1 and the viewpoint.

[0157] Furthermore, a storage unit (e.g., storage unit 12a) stores multiple image data corresponding to the position of the user's viewpoint (e.g., first viewpoint E1 and second viewpoint E2, first viewpoint EC and second viewpoint ED, etc.) on the image display surface of a display panel (e.g., display panel 20 or display panel 20A) at a planar viewpoint, and a control unit (e.g., image output unit 12) selects image data corresponding to the position of the user's viewpoint at the planar viewpoint, thereby making it possible to use image data that is prepared in advance and corresponds to the position of the user's viewpoint. Therefore, by making such prepared image data correspond to the positional relationship between the display device 1 and the viewpoint with higher accuracy, the displayed image can be made to correspond to the positional relationship between the display device 1 and the viewpoint with higher accuracy.

[0158] Furthermore, a storage unit (e.g., storage unit 12a) stores a plurality of image data corresponding to the magnitude of the angle between the image display surface of a display panel (e.g., display panel 20 or display panel 20A) and the user's viewpoint when the vertical direction of the image display surface is set to 0 degrees, and a control unit (e.g., image output unit 12) selects image data corresponding to the angle between the image display surface and the user's viewpoint, thereby making it possible to use image data that has been prepared in advance and that corresponds to the angle between the image display surface and the user's viewpoint. Therefore, by making such prepared image data correspond to the positional relationship between the display device 1 and the viewpoint with higher accuracy, it is possible to make the displayed image correspond to the positional relationship between the display device 1 and the viewpoint with higher accuracy.

[0159] Furthermore, by having an image generation unit (e.g., image generation unit 12b) perform block matching of two of the multiple image data stored in a memory unit (e.g., memory unit 12a) to generate image data, image data that corresponds to the positional relationship with high accuracy can be generated through a clear process.

[0160] The acquisition unit also includes an imaging unit (for example, imaging unit 2) that images the user, and a processing unit (for example, gaze tracking unit 11) that, based on the captured image of the user, identifies the alignment direction of the right eye and the left eye of the user, the relative rotation angle and positional relationship between the display panel and the alignment direction, etc. This makes it possible to acquire user viewpoint information from the captured image of the user.

[0161] The acquisition unit also includes a distance measurement unit (e.g., distance measurement unit 3) that measures the distance between the display panel (e.g., display panel 20 or display panel 20A) and the user. This allows the distance between the display panel and the user to be included in the user's viewpoint information. Therefore, display output that corresponds to the viewpoint position with high accuracy can be performed.

[0162] Furthermore, the multiple image data stored in a memory unit (e.g., memory unit 12a) correspond to the magnitude of the angle between the image display surface of a display panel (e.g., display panel 20 or display panel 20A) and the user's viewpoint (e.g., first viewpoint E1 and second viewpoint E2, first viewpoint EC and second viewpoint ED, etc.) when the vertical direction of the image display surface of the display panel is set to 0 degrees, and coordinates centered on one image data (e.g., central image data CP1) selected when the angle between the image display surface and the viewpoint is 0 degrees are assigned to each of the multiple image data, and the magnitude of the angle between the image display surface and the viewpoint is managed using these coordinates.When the distance between the display panel and the user is relatively far, image data with coordinates closer to the center is selected compared to when the distance is relatively close, so that the relationship between the distance and the multiple image data can be managed using coordinates, and control over the display can be made clearer.

[0163] It should be noted that in the figures referenced in the above explanation, examples where the relative angle rot is 0 degrees (°), 45 degrees (°), and 90 degrees (°) are specifically noted, but the relative angle rot is not limited to these angles and can be any angle within the range of -180 degrees (°) to 180 degrees (°) depending on the relationship between the display panel 20A and the face HF.

[0164] 4 to 15, the correspondence between the pitch of the plurality of pixels Pix and the pitch of the plurality of light-emitting points LP is 1:6. However, the various controls described with reference to FIGS. 8 to 26 can also be applied to a case where the correspondence is 1:4, as shown in FIG. 3. The correspondence may be 1:6α or 1:4α, where α is a natural number. The center position of the light-emitting point LP is not limited to a position between two adjacent pixels Pix from a planar viewpoint. For example, the center position of the light-emitting point LP and the center position of the pixel Pix may overlap, or may have another positional relationship.

[0165] The shape of the sub-pixels provided in a pixel Pix is, for example, a rectangle with its longitudinal direction in the Y direction, and a plurality of such sub-pixels are arranged along the X direction to form a pixel Pix, but the specific shape of the sub-pixels that make up a pixel Pix is ​​not limited to this. Furthermore, the arrangement of the sub-pixels provided in one pixel Pix is ​​not limited to an arrangement along the X direction, but may also be an arrangement along the Y direction or a matrix. Furthermore, the shape of the sub-pixels in a plan view is not limited to a rectangular shape, but may be any shape. Furthermore, the number of sub-pixels provided in one pixel Pix is ​​arbitrary, and may be, for example, three, two or less, or four or more.

[0166] Furthermore, other effects and advantages brought about by the aspects described in this embodiment that are clear from the description in this specification or that can be appropriately thought of by a person skilled in the art are naturally understood to be brought about by the present disclosure. [Explanation of symbols]

[0167] 1 Display device 2. Imaging unit 3 Ranging section 4 Gyro sensor 10 Signal Processing Section 11 Eye tracking unit 12 Image output section 12a Storage section 12b Image generation unit 20,20A display panel Pix E1,EC First Viewpoint E2,ED Second Viewpoint ER,EL perspective OP viewpoint compatible image data

Claims

1. a display panel provided with a plurality of pixels; a light source provided with a plurality of light-emitting points and configured to irradiate light onto a plurality of pixels of the display panel; an acquisition unit that acquires information about a plurality of viewpoints of a user viewing the display panel; a storage unit that stores a plurality of image data that are selectable depending on the positional relationship between the display panel and the viewpoint; a control unit that controls display of an image by operation of the plurality of pixels based on the information, a ratio of a pitch of the plurality of pixels arranged in a predetermined direction to a pitch of the plurality of light-emitting points in the predetermined direction is 1:4n or 1:6n; n is a natural number, the information includes information relating to the positions of the plurality of viewpoints and information indicating an arrangement direction of the plurality of viewpoints, The control unit selecting image data corresponding to the relative positional relationship from the image data stored in the storage unit based on a relative rotation angle between the display panel and the arrangement direction and a relative positional relationship between the viewpoint and each light-emitting point, and transmitting light to pixels positioned on at least a straight line connecting each light-emitting point and each of the viewpoints in order to display and output the selected image data; an image generation unit that, when image data corresponding to the positional relationship between the image display surface of the display panel and the viewpoint is not stored in the storage unit, interpolatively generates image data corresponding to the positional relationship from two image data stored in the storage unit that are close to the positional relationship, and in this case selects the image data generated by the image generation unit; Display device.

2. the storage unit stores a plurality of image data corresponding to the position of the viewpoint on the image display surface of the display panel at a planar viewpoint; the control unit selects image data corresponding to the position of the viewpoint in the planar viewpoint. The display device according to claim 1 .

3. the storage unit stores a plurality of image data corresponding to magnitudes of angles between the image display surface of the display panel and the viewpoint when the vertical direction of the image display surface of the display panel is set to 0 degrees; the control unit selects image data corresponding to an angle between the image display surface and the viewpoint. The display device according to claim 1 or 2.

4. the image generation unit performs block matching of the two pieces of image data to generate image data. The display device according to claim 1 or 2.

5. The acquisition unit an imaging unit that images the user; a processing unit that identifies the arrangement direction, the relative rotation angle, and the positional relationship of the right eye and the left eye of the user based on the captured image of the user; The display device according to claim 1 or 2, comprising:

6. the acquisition unit includes a distance measurement unit that measures a distance between the display panel and the user. The display device according to claim 5 .

7. The plurality of image data includes: corresponds to the magnitude of the angle between the image display surface of the display panel and the user's viewpoint when the vertical direction of the image display surface of the display panel is set to 0 degrees, a coordinate centered on one image data item selected when the angle between the image display surface and the user's viewpoint is 0 degrees is assigned to each of the plurality of image data items, and the magnitude of the angle between the image display surface and the user's viewpoint is managed using the coordinate; When the distance between the display panel and the user is relatively far, image data of coordinates closer to the center is selected compared to when the distance between the display panel and the user is relatively close. The display device according to claim 6.

Images