Display device and display method

JPWO2023181602A5Pending Publication Date: 2024-12-03
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024509786
Authority / Receiving Office
JP · JP
Patent Type
Applications
Filing Date
2024-09-02
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Conventional display devices cause eye discomfort and fatigue due to a constant focal length, which mismatches the convergence angle and binocular parallax, leading to ineffective three-dimensional viewing in virtual spaces.

Method used

A display device with a variable focus lens that adjusts focal length based on image distance, paired with a high-speed display and control device to dynamically change focal lengths for distant and nearby images, allowing for comfortable three-dimensional viewing.

Benefits of technology

Reduces eye discomfort and fatigue by dynamically adjusting the focal length to match the image distance, enhancing the three-dimensional viewing experience in virtual spaces.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

A display device according to an embodiment of the present invention includes a varifocal lens capable of changing the focal length, a display disposed so as to face the varifocal lens, and a control device that controls the focal length of the varifocal lens and causes the display to display an image. The control device increases the focal length of the varifocal lens when causing the display to display a distant image, and decreases the focal length of the varifocal lens when causing the display to display a nearby image.
Need to check novelty before this filing date? Find Prior Art

Description

Display device and display method

[0001] The present disclosure relates to a display device and a display method that enable an object in a virtual space to be viewed three-dimensionally.

[0002] People can perceive the three-dimensionality and depth of an object through the convergence angle (the angle between the lines of sight of both eyes relative to an object) and binocular parallax (the difference between the images seen by the right and left eyes). By utilizing this convergence angle and binocular parallax, people can perceive objects displayed on a two-dimensional display as three-dimensional.

[0003] For example, in Patent Document 1, a virtual image with a sense of three-dimensionality and depth can be displayed by displaying images that provide a convergence angle and binocular parallax on a display for the left eye and a display for the right eye, respectively.

[0004] Japanese Patent Application Laid-Open No. 2001-148871

[0005] However, in the configuration of Patent Document 1, the distance from both eyes to the virtual image is always constant, which may cause discomfort to the eyes because the distance from both eyes to an object determined by the convergence angle and binocular parallax does not match the distance from both eyes to the virtual image.

[0006] In addition, people normally unconsciously alternate between looking at nearby and distant objects to exercise their eye muscles and prevent fatigue. However, in virtual space, the focal length of the eyes is not adjusted even when looking at nearby and distant objects, so gazing for a long period of time can easily cause fatigue.

[0007] Therefore, an object of the present disclosure is to provide a display device and a display method that allow objects in a virtual space to be viewed three-dimensionally, and that reduce discomfort and fatigue in the eyes.

[0008] In order to achieve the above object, a display device according to one embodiment of the present disclosure comprises a variable-focus lens capable of changing its focal length, a display positioned opposite the variable-focus lens, and a control device that controls the focal length of the variable-focus lens and displays an image on the display, wherein the control device increases the focal length of the variable-focus lens when the image is to display a distant image, and decreases the focal length of the variable-focus lens when the image is to display a close-up image.

[0009] According to the present disclosure, in a display device that allows objects in a virtual space to be viewed three-dimensionally, discomfort and fatigue in the eyes can be reduced.

[0010] Schematic diagram of a display device according to the first embodiment. Graph showing changes in focal length of a variable-focus lens according to the first embodiment. Diagram for explaining a method for generating a group of images to be stored in a control device according to the first embodiment. Diagram for explaining the operating principle of a display device according to the first embodiment. Flowchart showing processing when an image is displayed on a display according to the first embodiment. Schematic diagram of a display device according to a second embodiment. Graph showing changes in focal length of a variable-focus lens according to the second embodiment. Schematic diagram of a display device according to a third embodiment. Example images captured by a camera according to the third embodiment. Flowchart showing processing when an image is displayed on a display according to the third embodiment.

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following description of the preferred embodiment is essentially merely illustrative and is not intended to limit the present invention, its applications, or its uses. In the following description, the same parts will be designated by the same reference numerals, and detailed description will be omitted as appropriate.

[0012] First Embodiment FIG. 1 is a schematic diagram of a display device according to a first embodiment.

[0013] 1, the display device according to the first embodiment includes variable-focus lenses 1L and 1R, displays 2L and 2R, and a control device 3. As shown in Fig. 1, the variable-focus lens 1L (1R) and the display 2L (2R) are arranged in the line of sight of the user's left eye 4L (right eye 4R).

[0014] The variable-focus lens 1L (1R) is a variable-focus lens that can continuously change its focal length f. The variable-focus lens 1L (1R) changes its focal length f by deforming its surface shape by injecting or expelling liquid through physical shape changes, such as a piezoelectric element filled with liquid or an electromagnetic coil. Because the variable-focus lens 1L (1R) changes its focal length f through physical shape changes, the smaller the lens diameter, the faster the response. In the following description, the maximum focal length of the variable-focus lens 1L (1R) is defined as f1.

[0015] The variable-focus lens 1L (1R) is disposed in front of the right eye 4R (left eye 4L) of the user who uses the display device.

[0016] The display 2L (2R) is a two-dimensional display that displays images. The display 2L (2R) has, for example, a plurality of pixels, each configured by an arrangement of tiny LED elements each representing a set of the three primary colors of light: R, G, and B. As will be described in detail later, the display 2L (2R) switches images displayed in a short period of time, requiring faster screen updates than conventional two-dimensional displays. For example, a liquid crystal display or the like is used as the two-dimensional display for the display 2L (2R). Because the response speed of a liquid crystal display is several milliseconds, an image update rate of approximately 30 Hz is possible, but images updated faster than this are prone to afterimages. In order to display an image with a sense of depth using the display 2L (2R), an image update rate in the depth direction (approximately 30 Hz) is required. Therefore, the display 2L (2R) requires an image update rate of approximately 900 Hz, which is difficult to achieve with a liquid crystal display. For this reason, the display 2L (2R) preferably has a pixel structure configured by LED elements, and each LED element is preferably capable of a high-speed response of several microseconds or less and capable of image updates of 1 kHz or more.

[0017] 1, the display 2L (2R) is disposed at a position that is the maximum focal length f1 from the variable-focus lens 1L (1R). In other words, the display (2L) is disposed opposite the variable-focus lens 1L (1R).

[0018] If the display 2L (2R) is positioned farther than the maximum focal length f1 of the variable-focus lens 1L (1R), it will not be possible to focus on the display 2L (2R) even if the focal length f of the variable-focus lens 1L (1R) is set to the maximum focal length f1, resulting in a blurred distant image displayed on the display 2L (2R). Furthermore, if the display 2L (2R) is positioned closer to the variable-focus lens 1L (1R) than the focal length f of the variable-focus lens 1L (1R), the distant image displayed on the display 2L (2R) will not be blurred, but the adjustment range of the focal length f of the variable-focus lens 1L (1R) will not be used effectively. In other words, the resolution of the display device will be reduced, and the quality of the depth image will be reduced. On the other hand, the focal length of an optical element, for example, a lens, generally has a manufacturing error of 1 to 2%. Therefore, if the position of the display 2L (2R) is set to the maximum focal length f1 of the variable-focus lens 1L (1R), the focal length f of the variable-focus lens 1L (1R) relative to the display 2L (2R) may fall outside the adjustment range due to the influence of manufacturing errors in the optical element. Therefore, taking into account lens manufacturing errors and assembly errors, it is desirable to set the position of the display 2L (2R) to f1 × (0.9 ± 0.1) with respect to the maximum focal length f1 of the variable-focus lens, taking into account a 10% error. In this case, the adjustment resolution of the variable-focus lens 1L (1R) will be reduced by 10%. Note that if the error becomes larger, it may be possible to set f1 × (0.8 ± 0.2) with a 20% error taken into account, but this will further deteriorate the adjustment resolution of the variable-focus lens 1L (1R).

[0019] However, if a certain degree of blurring of the image displayed on display 2L (2R) is acceptable, display 2L (2R) may be positioned farther away from variable-focus lens f1. For example, if 5% blurring is acceptable, display 2L (2R) should be positioned in the range of f1 x 0.8 to f1 x 1.05. From the above, display 2L (2R) can be positioned within the range of +5% to -20% of the maximum focal length f of variable-focus lens 1L (1R) from variable-focus lens 1L (1R).

[0020] The control device 3 controls the variable-focus lenses 1L and 1R and the displays 2L and 2R. Specifically, the control device 3 controls the focal lengths of the variable-focus lenses 1L and 1R.

[0021] 2 is a graph showing changes in the focal length of the variable-focus lens according to the first embodiment. In FIG. 2, the vertical axis represents the focal length f of the variable-focus lens 1L (1R), and the horizontal axis represents time. The control device 3 controls the focal length f of the variable-focus lens 1L (1R) in accordance with the graph shown in FIG. 2.

[0022] Specifically, the focal length f of the variable-focus lens 1L (1R) is gradually shortened from the maximum focal length f1 to a predetermined focal length (the predetermined focal length can be set arbitrarily). Then, the control device 3 gradually lengthens the focal length f of the variable-focus lens 1L (1R) from the predetermined focal length to the maximum focal length f1. The control device 3 performs this control of the focal length f every scanning period t2. The change in the focal length f of the variable-focus lens 1L (1R) is performed every image update time t1, which is the interval at which the image on the display 2L (2R) is updated.

[0023] The control device 3 also outputs an image group consisting of a plurality of images to the display 2L (2R), causing the display 2L (2R) to display a three-dimensional image. The control device 3 stores the image group, sequentially outputs images included in the image group to the display 2L (2R) at each image update time t1, and updates the image displayed on the display 2L (2R) at each image update time t1. As will be described in more detail below, the control device 3 reads out the image group from an external storage device (not shown) in which it is stored, and stores it in itself.

[0024] In order to reduce flickering of images displayed on the display 2L (2R), the scanning period t2 is at least 30 Hz. That is, all images included in the image group are displayed on the display 2L (2R) in 1 / 30 seconds or less. For example, if the image group contains 30 images, the image update time t1 is 1 / 900 seconds or less.

[0025] FIG. 3 is a diagram illustrating a method for generating a group of images stored in the control device according to the first embodiment. For example, suppose that a user is desired to recognize that an object P1 is near the user and an object P2 is far away in a virtual space. In this case, as shown in FIG. 3( a), a variable-focus lens 1L (1R), objects P1 and P2, and an imaging plane 5 of a camera for generating images are arranged. Here, if the distance from the imaging plane 5 to the variable-focus lens 1L (1R) is a and the distance from the variable-focus lens 1L (1R) to the object P1 is b, the focal length f of the variable-focus lens 1L (1R) is expressed as 1 / f = 1 / a + 1 / b.

[0026] 3(b) shows an image captured by the camera (imaging surface 5) when the focal length f of the variable-focus lens 1L (1R) is set to a distance A1, and FIG. 3(c) shows an image captured by the camera (imaging surface 5) when the focal length f of the variable-focus lens 1L (1R) is set to a distance A2. As shown in FIG. 3(b), when the focal length f of the variable-focus lens 1L (1R) is set to a distance A1 near the imaging surface 5, the focus is on an object P1 located near the imaging surface 5, so the outline of the object P1 becomes clear, but the outline of an object P2 located far from the imaging surface 5 becomes blurred. On the other hand, as shown in FIG. 3(c), when the focal length f of the variable-focus lens 1L (1R) is set to a distance A2 far from the imaging surface 5, the focus is on an object P2 located far from the imaging surface 5, so the outline of the object P2 becomes clear, but the outline of an object P1 located nearby becomes blurred.

[0027] In this embodiment, multiple images are generated by capturing images including objects P1 and P2 using the camera (imaging surface 5) while changing the focal length f of the variable-focus lens 1L (1R). The multiple images generated in this manner are stored as an image group in an external storage device (and control device 3). Note that, as the focal length f of the variable-focus lens 1L (1R) becomes shorter, images are captured at smaller intervals, while as the focal length f of the variable-focus lens 1L (1R) becomes longer, images are captured at larger intervals. In other words, when the focal length f is set near the imaging surface 5, images are generated (captured) at small intervals, and when the focal length f is set far from the imaging surface 5, images are generated (captured) at large intervals.

[0028] 4A and 4B are diagrams illustrating the operating principle of the display device according to the first embodiment. Specifically, FIG. 4A shows a state in which the user is looking at a distant object, and FIG. 4B shows a state in which the user is looking at a nearby object. Note that θ indicates the parallax direction of the user's left eye 4L (right eye 4R).

[0029] As shown in Figure 4(a), when viewing an image S1 located at a distance, the light rays entering the eye are nearly parallel. At this time, the crystalline lens of the user's left eye 4L (right eye 4R) is pulled by the muscles and becomes thinner, lengthening its focal length and adjusting so that the image is formed on the retina.

[0030] Here, when the control device 3 sets the focal length f of the variable-focus lens 1L (1R) to the maximum focal length f1, the display 2L (2R) is positioned at the maximum focal length f1 of the variable-focus lens 1L (1R), so the left eye 4L (right eye 4R) focuses on the image displayed on the display 2L (2R) (specifically, the position of image S1'). At this time, light emitted from each pixel of the display 2L (2R) is converted into parallel light by the variable-focus lens 1L (1R). As a result, light similar to that seen when viewing an image S1 located at a distance enters the left eye 4L (right eye 4R). Note that in FIG. 4(a), the image S1 located in the parallax direction θ corresponds to the image S1' displayed on the display 2L (2R) and located at a position of f1 × sin(θ).

[0031] As shown in Figure 4(b), when a user is looking at a nearby image S2, the light rays entering the eye are diverging. At this time, the lens of the user's left eye 4L (right eye 4R) is adjusted so that the focal length is shortened by the muscle thickening the lens, and the image is formed on the retina.

[0032] Here, if the distance from the variable-focus lens 1L (1R) to the image S2 is b1 and the imaging position of the image S2 when viewed through the variable-focus lens 1L (1R) is a1, the focal length f of the variable-focus lens 1L (1R) can be expressed as 1 / f = 1 / a1 - 1 / b1. Rearranging this equation yields 1 / a1 = 1 / f + 1 / b1. Therefore, if the focal length f of the variable-focus lens 1L (1R) remains at the maximum focal length f1, the image will be focused at a position shifted toward the variable-focus lens 1L (1R) from the display 2L (2R) (specifically, the position of the image S2''). For this reason, in conventional display devices (such as those disclosed in Patent Document 1), the left eye 4L (right eye 4R) adjusts the lens to match the focal length of the lens to the image displayed on the display 2L (2R) (image 2S' in FIG. 4B). This causes eye fatigue and discomfort.

[0033] In contrast, in this embodiment, the control device 3 sets the focal length f of the variable-focus lens 1L (1R) to a focal length f'. The focal length f' is expressed as 1 / f' = 1 / f - 1 / b1. As a result, when the user looks through the variable-focus lens 1L (1R), the image is focused on the position of the display 2L (2R) (specifically, the position of image S2'). This eliminates the need for the left eye 4L (right eye 4R) to adjust the focal length of the crystalline lens, reducing eye fatigue and discomfort. Note that in FIG. 4B, image S2 positioned in the parallax direction θ corresponds to image S2' displayed on the display 2L (2R) and positioned at f1 × sin(θ).

[0034] FIG. 5 is a flowchart showing the processing when an image is displayed on a display according to the first embodiment.

[0035] First, a group of images to be displayed on a display is generated (step S1). Specifically, as described in Fig. 3, a plurality of images (group of images) including objects P1, P2, etc. are generated (captured) by the camera (image capture surface 5) while changing the focal length f of the variable-focus lens 1L (1R). Then, the group of images is stored in an external storage device (step S2).

[0036] The control device 3 reads the image group from the external storage device (step S3) and swaps the image group stored in the control device 3 with the image group stored in itself (step S4). For example, the control device 3 compares the images included in the image group with each other, and if it determines that the images are different, it swaps the images.

[0037] The control device 3 selects the next image to be displayed on the display 2L (2R) from the images included in the image group (step S5) and displays the selected image on the display 2L (2R). Specifically, the control device 3 selects an image from the image group stored therein that has a shorter (or longer) focal length than the previously selected image as the next image. In this case, the control device 3 selects the next image so that the focal lengths of the images displayed on the display 2L (2R) gradually become shorter (or longer). In other words, the control device 3 selects the next image so that the images in the image group are displayed sequentially on the display 2L (2R) in order of focal length. The control device 3 then sets the focal length f of the variable-focus lens 1L (1R) to the focal length when the selected image was captured (step S6).

[0038] The control device 3 determines whether the selected image is the last image to be displayed among the images included in the image group (step S7). If the control device 3 determines that the selected image is not the last image to be displayed among the images included in the image group (No in step S7), the control device 3 returns to step S5.

[0039] If the control device 3 determines that the selected image is the last image to be displayed among the images included in the image group (Yes in step S7), it sets the next image to be displayed on the display 2L (2R) to the first image to be displayed among the images included in the image group (step S8). After step S8, the control device 3 returns to step S3 and repeats the processes from step S3 onwards.

[0040] With the above configuration, the display device according to this embodiment includes a variable-focus lens 1L (1R) whose focal length f is variable, a display 2L (2R) disposed opposite the variable-focus lens 1L (1R), and a control device 3 that controls the focal length f of the variable-focus lens 1L and displays an image on the display 2L (2R). The control device 3 lengthens the focal length f of the variable-focus lens 1L (1R) when displaying a distant image 1S on the display 2L (2R), and shortens the focal length f of the variable-focus lens 1L (1R) when displaying a nearby image 2S on the display 2L (2R). Here, the nearby image 2S is an image of an object that is closer to the user in virtual space than the distant image 1S.

[0041] In the conventional configuration, the user views the display 2L (2R) through a lens with a fixed focal length, and the position of the display 2L (2R) does not change even if the focal length of the image displayed on the display 2L (2R) changes. Therefore, in order to view the image displayed on the display 2L (2R), the user needs to adjust the focal length of the crystalline lens of the eye, which causes eye fatigue and discomfort.

[0042] In contrast, in this embodiment, the control device 3 increases the focal length f of the variable-focus lens 1L (1R) when displaying a distant image 1S on the display 2L (2R), and decreases the focal length f of the variable-focus lens 1L (1R) when displaying a close image 2S on the display 2L (2R). In other words, the control device 3 adjusts the focal length f of the variable-focus lens 1L (1R) in accordance with the focal length of the image displayed on the display 2L (2R), thereby reducing eye fatigue and discomfort.

[0043] The control device 3 also causes the display 2L (2R) to sequentially display an image group consisting of multiple images set at different focal lengths in order of focal length. The control device 3 also sets the focal length of the variable-focus lens 1L (1R) to the focal length of the images displayed on the display 2L (2R). As a result, when an image group including multiple images set at different focal lengths is displayed sequentially, the images displayed on the display 2L (2R) and the focal length of the variable-focus lens 1L (1R) change gradually, thereby reducing eye strain.

[0044] The group of images displayed on the display 2L (2R) are captured by the camera (imaging surface 5) through the variable-focus lens 1L (1R) while changing the focal length f of the variable-focus lens 1L (1R), thereby generating (capturing) multiple images with different focal lengths.

[0045] Second Embodiment Fig. 6 is a schematic diagram of a display device according to a second embodiment. In Fig. 6, compared to Fig. 1, relay lenses 6L and 7L (6R and 7R) are arranged between the left eye 4L (right eye 4R) and the variable-focus lens 1L (1R). The relay lenses 6L and 6R correspond to the first relay lens, and the relay lenses 7L and 7R correspond to the second relay lens. The relay lens 6L has an incident surface 6L1 and an exit surface 6L2. The relay lens 7L has an incident surface 7L1 and an exit surface 7L2.

[0046] As shown in Fig. 6, the relay lens 6L (6R) is arranged so that the focal plane on the exit surface side overlaps the pupil position of the user's left eye 4L (right eye 4R). The relay lens 6L (6R) is also arranged so that the focal plane on the entrance surface side overlaps the focal plane on the entrance surface side of the relay lens 7L (7R). The relay lens 7L (7R) is also arranged so that the focal plane on the exit surface side overlaps the variable-focus lens 1L (1R). Here, the focal plane refers to the plane that includes the focal point of the lens.

[0047] By arranging the focal planes of the relay lenses 6L (6R) and 7L (7R) so that they overlap, the relay lenses 6L (6R) and 7L (7R) are in an imaging relationship, i.e., light emitted from the entrance surface of the relay lens 6L (6R) is focused to a single point on the focal plane on the exit surface side of the relay lens 7L (7R).

[0048] Here, if the focal length of the relay lens 6L (6R) is fa and the focal length of the relay lens 7L (7R) is fb, then the effective diameter of the variable-focus lens 1L (1R) is fa / fb times the diameter of the pupil of the left eye 4L (right eye 4R). In the first embodiment, a predetermined distance is required between the left eye 4L (right eye 4R) and the variable-focus lens 1L (1R), so the effective diameter of the variable-focus lens 1L (1R) needs to be large depending on the angle of view, i.e., the range of the field of view. In contrast, in the second embodiment, the relay lenses 6L, 7L (6R, 7R) make the distance between the pupil of the left eye 4L (right eye 4R) and the variable-focus lens 1L (1R) zero, so the effective diameter of the variable-focus lens 1L (1R) can be made small. Furthermore, because the pupil diameter size of the left eye 4L (right eye 4R) on the variable-focus lens 1L (1R) is fa / fb times larger, the effective diameter of the variable-focus lens 1L (1R) can be further reduced by making fa<fb. Also, the smaller the effective diameter of the variable-focus lens 1L (1R), the faster the response becomes, making it possible to reduce flickering of the image displayed on the display 2L (2R).

[0049] Next, the operation of the display device according to the second embodiment will be described. In the display device according to the first embodiment, the pixels of the three colors RGB (Red, Green, and Blue) are simultaneously illuminated when updating the image on the display 2L (2R). In contrast, in the display device according to the second embodiment, the image to be displayed on the display 2L (2R) is decomposed into three images corresponding to the RGB colors, and the three images are displayed on the display 2L (2R) in the order R, G, and B. In this case, the control device 3 changes the focal length f of the variable-focus lens 1L (1R) for each RGB color within one image update time t1 (see FIG. 7 ). Specifically, the image update time t1 includes an image update time t11 for displaying the image decomposed into R, an image update time t12 for displaying the image decomposed into G, and an image update time t13 for displaying the image decomposed into B.

[0050] As described above, the relay lenses 6L, 6R, 7L, and 7R are Fresnel lenses, which can be thin and lightweight. However, they are prone to chromatic aberration due to the wavelength of light, i.e., focal length deviation for each color. While chromatic aberration can be corrected by combining multiple lenses, this approach results in a complex, large, and heavy structure, and assembly and adjustment are difficult. Therefore, in the second embodiment, the focal length deviation that occurs for each color can be reduced by controlling the focal length f of the variable-focus lens 1L (1R), thereby reducing the impact of color shift in the image due to chromatic aberration. The focal length control amount for each RGB color can be determined by measuring in advance the amount that minimizes color shift in the image due to chromatic aberration.

[0051] With the above configuration, the display device according to the second embodiment includes relay lenses 6L and 6R whose focal planes on the exit surface side overlap the pupil position of the user's left eye 4L (right eye 4R), and relay lenses 7L and 7R whose focal planes on the entrance surface side overlap the focal plane on the entrance surface side of the first relay lens, with the variable-focus lens disposed on the focal plane on the exit surface side. This allows the relay lenses 6L and 7L (6R, 7R) to zero the distance between the pupil of the left eye 4L (right eye 4R) and the variable-focus lens 1L (1R), thereby reducing the effective diameter of the variable-focus lens 1L (1R). Furthermore, since the pupil diameter size of the left eye 4L (right eye 4R) on the variable-focus lens 1L (1R) is multiplied by fa (the focal length of the relay lens 6L (6R)) / fb (the focal length of the relay lens 7L (7R)), satisfying fa < fb allows the effective diameter of the variable-focus lens 1L (1R) to be further reduced.

[0052] The relay lenses 6L, 6R, 7L, and 7R are Fresnel lenses, which allows the size of the display device to be reduced in the direction of the optical axis of the lenses.

[0053] The control device 3 also separates the image to be displayed on the display 2L (2R) into RGB colors, displays the separated images on the display 2L (2R), and sets the focal length f of the variable-focus lens 1L (1R) to the focal length of the image displayed on the display 2L (2R). This makes it possible to correct the deviation in focal length for each color of the image that occurs when the relay lenses 6L, 6R, 7L, 7R are Fresnel lenses.

[0054] Although the relay lenses 6L, 6R, 7L, and 7R are described as Fresnel lenses, the present invention is not limited to this and may be an achromatic lens or a plurality of lenses, etc. In this case, the achromatic lens or a plurality of lenses may be used to reduce chromatic aberration, and the image to be displayed on the display 2L (2R) may be separated into RGB colors, or the focal length f of the variable-focus lens 1L (1R) may not be controlled.

[0055] Furthermore, the display order of the color-separated images does not have to be R, G, B, and they may be displayed in any order.

[0056] Third Embodiment Fig. 8 is a schematic diagram of a display device according to a third embodiment. As shown in Fig. 8, the display device according to the third embodiment includes a camera 8L (8R) and a sensor 9. The camera 8L (8R) corresponds to a first sensor, and the sensor 9 corresponds to a second sensor.

[0057] The camera 8L (8R) is, for example, a near-infrared camera. The camera 8L (8R) captures an image of the left eye 4L (right eye 4R) via a half mirror 10L (10R) provided between the variable-focus lens 1L (1R) and the relay lens 7L (7R). The camera 8L (8R) is arranged superimposed on the focal plane (focal length fb) on the exit surface side of the relay lens 7L (7R) via the half mirror 10L (10R). The half mirror 10L (10R) is, for example, a near-infrared reflecting mirror that transmits visible light and reflects only near-infrared light.

[0058] The sensor 9 is, for example, an acceleration sensor, etc. The sensor 9 is attached to the user's head and detects the direction in which the user's head is facing (head direction).

[0059] As described above, the relay lenses 6L (6R) and 7L (7R) are arranged so that their focal planes overlap. Furthermore, the focal plane on the exit surface side of the relay lens 6L (6R) overlaps with the pupil position of the user's left eye 4L (right eye 4R). Furthermore, the focal plane on the exit surface side of the relay lens 7L (7R) overlaps with the imaging plane of the camera 8L (8R). This creates an imaging relationship between the camera 8L (8R) and the left eye 4L (right eye 4R), allowing the camera 8L (8R) to capture the left eye 4L (right eye 4R).

[0060] FIG. 9 shows an example of an image captured by a camera according to the third embodiment. As shown in FIG. 9, the image captured by the camera 8L (8R) includes an image of the left eye 4L (or the right eye 4R). When the control device 3 receives the image from the camera 8L (8R), it detects, for example, the pupil 41, inner corner 42, and outer corner 43 of the left eye 4L (or the right eye 4R) from the image. The control device 3 calculates the position of the pupil 41 (vertical position v_L and horizontal position h_L) using the inner corner 42 as a reference. The control device 3 then calculates the user's line of sight based on the relationship between the reference position of the pupil 41 and the calculated position of the pupil 41. The reference position of the pupil 41 is the position of the pupil 41 when a predetermined mark is displayed at an appropriate position on the display 2L (2R) and the user gazes at the mark.

[0061] FIG. 10 is a flowchart showing the processing when an image is displayed on a display according to the third embodiment.

[0062] First, the camera 8L (8R) captures an image of the left eye 4L (right eye 4R). Then, the control device 3 calculates the user's gaze direction from the image of the pupil of the left eye 4L (right eye 4R) included in the captured image (step S11). The control device 3 detects the user's head direction based on the output from the sensor 9 (step S12). The control device 3 calculates the gaze direction based on the gaze direction and head direction (step S13).

[0063] Next, the distance Z to the object closest to the user in the calculated line of sight in the virtual space is calculated, and an image focused on the object is generated (step S14). The image generation method at this time is the method described with reference to FIG.

[0064] The control device 3 displays the image generated in step S3 on the display 2L (2R), and sets the focal length f of the variable-focus lens 1L (1R) to the distance Z (step S15).

[0065] With the above configuration, the display device according to the third embodiment includes cameras 8L and 8R (first sensors) that detect the position of the user's pupils and a sensor 9 (second sensor) that detects the orientation of the user's head. The control device 3 calculates the user's gaze direction based on the detection results of the camera 8L (8R) and the sensor 9, and displays on the display 2L (2R) an image generated so that the focus is on the object in the virtual space closest to the gaze direction. The control device 3 also sets the focal length f of the variable-focus lens 1L (1R) to the focal length of the image. This allows the focal length f of the variable-focus lens 1L (1R) and the image on the display 2L (2R) to be changed only when the user changes the gaze direction, significantly reducing flickering of the image displayed on the display 2L (2R).

[0066] Other Embodiments As described above, the embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate.

[0067] Although the display 2L (2R) uses LEDs as pixels, the present invention is not limited to this and organic EL may be used as pixels.

[0068] Furthermore, the pixels included in the display 2L (2R) are not limited to the three colors of RGB, but may be two colors or less, or four colors or more.

[0069] Furthermore, the image displayed on the display 2L (2R) may be calculated and displayed if the control device 3 has a high computing power, or an image that has been generated in advance may be displayed.

[0070] In addition, in FIG. 4 and other figures, the number of objects captured by the camera is not limited to two, and there are no restrictions on the arrangement of the objects.

[0071] In addition, when generating an image in FIG. 4 etc., a real image may be captured, or the image may be generated based on 3D data of a virtual space.

[0072] Furthermore, steps S1 and S2 may be prepared in advance before the image is displayed, or the image may be generated in parallel with the image display.

[0073] The number of images included in the image group is not limited to 30, as long as the image group includes at least two images with different focal lengths. The more images included in the image group, the higher the display resolution in the depth direction. However, this requires a faster display update speed on display 2L (2R).

[0074] Alternatively, a half mirror may be placed between the left eye 4L (right eye 4R) and the variable focus lens 1L (1R) to superimpose a real image and a virtual image of depth display on a two-dimensional display.

[0075] Furthermore, the display devices according to the first to third embodiments can be applied to, for example, head-mounted displays having the respective configurations of the display devices according to the first to third embodiments, head-up displays, wearable devices such as wearable cameras, and the like.

[0076] The display device of the present disclosure allows objects in a virtual space to be viewed three-dimensionally, and can therefore be used in head-up displays, wearable devices, and the like.

[0077] 1L, 1R Variable focus lens 2L, 2R Display 3 Control device 4L, 4R User's right eye, left eye 6L, 6R Relay lens (first relay lens) 7L, 7R Relay lens (second relay lens) 8L, 8R Camera (first sensor) 9 Sensor (second sensor)

Claims

1. a variable-focus lens capable of changing the focal length; a display disposed opposite the variable focus lens; a control device that controls the focal length of the variable-focus lens and displays an image on the display; The display device is characterized in that the control device increases the focal length of the variable-focus lens when displaying a distant image in the image, and decreases the focal length of the variable-focus lens when displaying a close image in the image.

2. 2. The display device according to claim 1, The control device causes the display to sequentially display a group of images consisting of a plurality of images set at a plurality of different focal lengths in order of focal length, and sets the focal length of the variable-focus lens to the focal length of the images displayed on the display.

3. 3. The display device according to claim 2, 10. A display device, comprising: a display unit configured to display a group of images through a variable-focus lens while changing the focal length of the variable-focus lens;

4. 2. The display device according to claim 1, a first relay lens having an entrance surface and an exit surface; a second relay lens having an entrance surface and an exit surface; a focal plane on the exit surface side of the first relay lens overlaps with the position of the pupil of the user, a focal plane on the incident surface side of the second relay lens overlaps with a focal plane on the incident surface side of the first relay lens, A display device characterized in that the variable focus lens is disposed on a focal plane on the exit surface side of the second relay lens.

5. 5. The display device according to claim 4, 10. A display device, wherein each of the first and second relay lenses is a Fresnel lens.

6. 6. The display device according to claim 5, The display device is characterized in that the control device decomposes the image into three images corresponding to RGB colors, displays the three images on the display, and sets the focal length of the variable-focus lens to the focal length of each of the three images displayed on the display.

7. 2. The display device according to claim 1, a first sensor for detecting the position of the user's pupil; a second sensor that detects a head orientation of the user; The control device calculates the user's line of sight based on the detection results of the first and second sensors, displays on the display an image generated so that the focus is on the object closest to the user in the virtual space in the line of sight, and sets the focal length of the variable-focus lens to the focal length of the image.

8. A display method using a display device including a variable-focus lens capable of changing a focal length, a display disposed opposite the variable-focus lens, and a control device that controls the focal length of the variable-focus lens and displays an image on the display, comprising: a first step of generating an image group consisting of a plurality of images set at a plurality of different focal lengths; a second step of displaying the group of images on the display in order of focal length of the images, and setting the focal length of the variable-focus lens to a focal length corresponding to the images displayed on the display; a third step, after the second step, of regenerating the group of images and performing the second step again.

9. A display method using a display device including a variable-focus lens capable of changing a focal length, a display disposed opposite the variable-focus lens, and a control device that controls the focal length of the variable-focus lens and displays an image on the display, comprising: detecting a gaze direction of the user from the positions of the user's pupils and the orientation of the user's head; generating an image in a virtual space set to focus on an object closest to the user in the line of sight; a step of displaying the image on the display and setting the focal length of the variable-focus lens to the focal length of the image.