Image display device

Abstract

The present invention comprises: first and second image display units respectively displaying first and second images to the right eye and the left eye of a user; and first and second display optical elements respectively corresponding to the first and second image display units. The present invention further comprises an actuator for changing the relative position of at least one of the display optical elements and the corresponding image display units. The actuator is driven in accordance with depth information supplied by an image displayed in a region which is a portion of the first and second image display units and crosses the optical axis of the display optical elements in the first and second image display units. In addition, the relative positions of the display optical elements and the corresponding image display units are changed by driving the actuator.

Description

Video display device

[0001] The present invention relates to a video display device.

[0002] There are image display devices that are worn on the user's head, and image display devices that are worn like glasses. A display unit is placed near the user's eyes, and parallax images are displayed for each of the user's eyes. The parallax images contain depth information indicating whether the user is looking at something far away or close up, and by viewing the displayed parallax images, the user can obtain a three-dimensional image of the object displayed in the parallax images.

[0003] With this image display device, the user's line of sight is adjusted to the position of the 3D image created by the images displayed for each eye, but the focal point is fixed to each of the left and right images on the display screen, which creates an unnatural state that would not occur when viewing a real object.

[0004] For example, when a user views an image of an object displayed as a parallax image, the user's convergence angle changes depending on the position of the object. At this time, the focal length of the crystalline lens changes depending on the magnitude of the convergence angle based on experience in real space. In this case, the focal length of the crystalline lens may not match the fixed diopter of the image display device, which may result in the displayed parallax image being out of focus and difficult to see clearly, thereby preventing good visibility. Therefore, when adjusting the diopter of an image display device, it is desirable to change the diopter according to depth information in the parallax image that determines the user's convergence angle, for example, by driving a lens within the device.

[0005] In Patent Document 1, the visual acuity is adjusted to depth information according to the position of the user's point of gaze when viewing an image, taking into account individual differences and the usage conditions of the image display device, thereby making it possible to reduce the sense of discomfort felt when viewing a stereoscopic image.

[0006] Furthermore, in Patent Document 2, a gaze sensor is provided to adjust visibility based on depth information of an image at the user's gaze position, resulting in a system configuration that makes it easier to see images in the gaze direction.

[0007] The optical system of Patent Document 1 will now be described with reference to FIG.

[0008] FIG. 9 is a diagram showing a schematic configuration of a video display device 10 disclosed in Patent Document 1.

[0009] The image data acquisition unit 100, display processing unit 101, diopter change instruction unit 106, object position calculation unit 108, user information recording unit 109, and operation mode recording unit 110 are realized by one or more processors, such as a CPU, reading and executing programs. The image display device 10 can be used by being worn on the user's head or by being worn like glasses. In either form, the image display device 10 can be fixed near the user's left eye 2a and right eye 2b. Note that the symbols a and b are added to the reference numerals of components related to the user's left eye 2a and right eye 2b, respectively, to distinguish between them.

[0010] The image display device 1 includes an image data acquisition unit 100 and a display processing unit 101. The image data acquisition unit 100 acquires image data for display via an external device, a network, or the like. The display processing unit 101 performs processing such as adjusting the display magnification of the acquired image data. The processed image data is sent to two display units 102a and 102b and displayed thereon. For example, the image data may be divided into separate data for the two display units 102a and 102b and displayed thereon. This configuration is not limiting, and the screen of one display unit may be divided into two and the image data may be displayed on the divided screen. This type of image processing tailored to the right and left eyes allows the user to view the image without feeling uncomfortable.

[0011] Here, the image display device 1 includes first and second display optical elements corresponding to the left eye 2 a and the right eye 2 b, respectively. The first display optical element has a lens 103 a, and the second display optical element has a lens 103 b. Images displayed on the display units 102 a and 102 b are presented to the left eye 2 a and the right eye 2 b through the corresponding lenses 103 a and 103 b, respectively.

[0012] The diopter change drive units 104a and 104b have drive sources such as motors and drive the lenses 103a and 103b, respectively. That is, the lenses 103a and 103b are moved by the diopter change drive units 104a and 104b in directions along their optical axes (arrows 103a1 to 104a1, and 103b1 to 104b1).

[0013] In Patent Document 2, an eye-gaze sensor is provided in the above configuration, the eye-gaze position is calculated from the detected eye-gaze direction, and the display is driven in accordance with the depth information of the image at that eye-gaze position.

[0014] In Patent Document 1, the depth is adjusted by driving a lens, but in Prior Art Example 2, a similar effect is obtained by moving the display.

[0015] 10A and 10B are explanatory diagrams illustrating the state when the lenses 103a and 103b are moved by the diopter change drive units 104a and 104b. FIG. 10A illustrates the state when the lenses 103a and 103b are moved near the display units 102a and 102b, respectively. FIG. 10B illustrates the state when the lenses 103a and 103b are moved near the eyes 2a and 2b, respectively. When the observer (user) views the display units 102a and 102b through the lenses 103a and 103b with their eyes 2a and 2b, they view virtual images 105a and 105b. The positions of the virtual images 105a and 105b in the direction of the optical axes 103a1 and 103b1, based on the positions of the observer's eyes 2a and 2b, are defined as the virtual image formation position i. 10A and 2B, the virtual image formation position i can be changed by changing the positions of the lenses 103a and 103b. For example, as the lenses 103a and 103b move closer to the display units 102a and 102b, the virtual image formation position i moves closer to the eyes 2a and 2b. Conversely, as the lenses 103a and 103b move closer to the eyes 2a and 2b, the virtual image formation position i moves farther away from the eyes 2a and 2b.

[0016] Therefore, by moving the lenses 103a and 103b using the diopter changing drive units 104a and 104b, the viewer can see the image of a distant object or a nearby object without blurring or the like. The image has depth information indicating whether the point of gaze is near or far. When this depth information matches the viewer's diopter, the viewer will see the image clearly. Figure 11 shows the relationship between the line of sight and the image position in a conventional image display device.

[0017] The same parts as in the conventional configuration are designated by the same reference numerals.

[0018] In FIG. 11, the user's line of sight is directed leftward and the user is looking at the left portion of the display 102.

[0019] In this drawing, the focus is drawn on 102 to make the problem easier to understand, but in reality, as mentioned above, the virtual image plane changes as the lens moves, and the focus position (focal position) also changes.

[0020] Here, when a user looks at the left edge of the display 102, the optical system sees the image through the edge of the lens. When considering optical performance, looking at an object through the edge of the lens increases optical aberration and detection errors of the line-of-sight sensor, and the user's line of sight becomes unstable, resulting in a deterioration in the image quality of the image seen by the user. In other words, the image becomes distorted, making it impossible to obtain a clear image.

[0021] Furthermore, in a configuration that detects where the viewer is looking across the entire area of ​​the display 102 and obtains depth information for that image position, it becomes necessary to extract the depth information from the depth information of the image of the entire display, which creates the problem of increasing the processing time required to obtain the depth information.

[0022] JP 2023-32278 A JP 2023-32250 A

[0023] The present invention provides an image display device that is less affected by the optical characteristics of optical elements and can display clear images to a user.

[0024] The image display device of the present invention is an image display device comprising: first and second image display units that display first and second images to the user's right and left eyes, respectively; first and second display optical elements that correspond to the first and second image display units, respectively; and an actuator that changes the relative positions of at least one of the display optical elements and the corresponding image display units, characterized in that the position of the display optical element and the position of the corresponding image display unit are relatively changed by driving the actuator in accordance with depth information provided by an image that is part of the first and second image display units and is displayed in an area that intersects with the optical axis of the display optical element in the first and second image display units.

[0025] According to the present invention, it is possible to provide an image display device that is less affected by the optical characteristics of optical elements and can display clear images to a user.

[0026] 1 is a block diagram showing the schematic configuration of an image display device according to a first embodiment; 2 is a diagram showing an embodiment of the image display device according to the first embodiment; 3 is a flow chart showing the algorithm of the first embodiment and the display contents; 4 is a diagram showing the optical center area of ​​the first embodiment; 5 is a diagram showing the optical center area of ​​the first embodiment; 6 is a block diagram showing the schematic configuration of an image display device according to a second embodiment; 7 is a diagram showing the relationship between depth by a user and lens position in the second embodiment; 8 is a block diagram showing the schematic configuration of an image display device according to a third embodiment; 9 is a diagram showing the relationship between depth information change and lens drive position in the third embodiment; 10 is a block diagram showing the schematic configuration of a conventional image display device; 11 is a diagram showing the relationship between lens position and virtual image position in a conventional image display device;

[0027] The image display device of this embodiment includes the following elements: first and second image display units that display first and second images to the right and left eyes of a user, respectively, and first and second display optical elements that correspond to the first and second image display units, as well as an actuator that changes the relative positions of at least one or more of the display optical elements and the corresponding image display units.

[0028] Furthermore, the actuator is configured to be driven in response to depth information provided by an image displayed in a region that is part of the first and second image display units and intersects with the optical axis of the display optical element in the first and second image display units, and by driving the actuator, the position of the display optical element and the position of the corresponding image display unit are changed relatively.

[0029] This configuration allows the user to see images with minimal optical aberration and good optical performance, providing clear images. It also has the effect of reducing user nausea and fatigue. Furthermore, limiting the image processing area reduces the amount of image processing, enabling high-speed processing.

[0030] The present invention will be described in detail below with reference to the drawings. In addition, although the following description will explain an embodiment, the present invention may be configured by combining various elements as long as it does not deviate from the spirit of the invention.

[0031] First, the terms used to describe this embodiment are defined as follows.

[0032] Optical center: The area that intersects with the optical axis of the optical system. An area that includes not only the point of intersection but also the location of the intersection. Gaze direction: Indicates the direction in which the user's eyes are facing. Depth information: An indicator that indicates whether the image's gaze point is close or far. Diopter adjustment: Moving the lens according to the image's depth information to adjust to the position where the image is in focus. Diopter: The position where the image is in focus.

[0033] An embodiment of the present invention will now be described with reference to Figures 1 to 4. Figure 1 shows the configuration of a video display device according to this embodiment. The same components as those in the configuration of a conventional video display device are assigned the same reference numerals. The device may also include a posture change detection unit for detecting head rotation, etc.

[0034] In this embodiment, for example, in addition to the function of diopter adjustment, a posture change detection unit 111 is provided for detecting head rotation, etc., and processing is performed so that the position of the image changes in response to posture changes. Figure 2 shows the image display device of this embodiment before and after diopter adjustment, with 1 indicating the user, 10 indicating the image display unit within the image display device, and 102 indicating how the display appears. Figure 2(a) shows a state in which user 1 is looking at a triangular mark, which is an image displayed in the center of the display, and the + mark indicates the user's line of sight. Here, the triangular mark is an object at a great depth (far away), and the object is in focus.

[0035] The circle on the left side of the display is an object at a small depth (nearby), and in the state of Figure 2(a) it is at the edge of the display and the focus adjustment is also set to a large depth (farther), so it is out of focus and blurred. Furthermore, as shown in the issues of this proposal, when looking at the edge of the display, you are shifted from the intersection of the optical axes, and the effects of lens aberrations, which are the display optical elements, are also superimposed.

[0036] Figure 2(b) shows the state in which, as user 1 moves his / her head, the image display unit of the device moves the circle mark at the edge of the display to the optical center position, and further, by adjusting the diopter, the focus changes from large depth (far) to small depth (near).

[0037] Here, since the object has been moved to the optical center position, it is no longer subject to the effects of optical aberrations and the like, and it is possible to view the object as a clear image.

[0038] 3 shows a flowchart of this embodiment and the corresponding display image states during the process. The flow of the change from (a) to (b) in FIG. 2 will be explained using this flowchart.

[0039] First, the user is looking at a deep (far) triangle mark at the center of the display. At this point, the lens driver 104 attached to the lens 103 has moved the lens 103 to a position where the triangle mark is in focus (F-01, D-01).

[0040] Next, the user moves his head to the left in an attempt to look at the circle (small depth (near)) on the left side of the display (F-02).

[0041] When the head moves, the posture change detection unit 111 (e.g., a detection unit configured with a head tracking sensor) shown in Figure 1 detects the amount of change in the direction of the posture change. For example, a posture change can be detected by detecting the posture of the image display device using a detection unit installed in the image display device. Furthermore, the image information to be displayed on the image display unit can be determined according to the posture detection results, such as the detected posture or posture change.

[0042] In FIG. 1, the image center calculation unit 112 moves the image center position in accordance with the posture change detection result.

[0043] At this time, the diopter adjustment position at the center of the display is set to the deepest (farther) position, so the circle remains blurred (F-03, D-02).

[0044] As the circle moves to the center of the display, the depth information of the circle is acquired (F-04).

[0045] Since the depth information has changed from the previous state of large depth (far) to the circled small depth (near), the lens (motor) position is moved so that the focus is on the small depth (near) (F-05).

[0046] The lens moves to the optimal position and stops. In this state, the circle in the center of the display is in focus, and the image becomes clear (F-06, D-03).

[0047] If the depth at the center of the display changes due to head movement, the lens is ready to move again. In this example, the image moves with head movement, and the depth information at the center of the display changes. However, even if the image changes and the depth changes without head movement, the lens can be driven to adjust the diopter, so that the image can be seen clearly (F-07).

[0048] By adjusting the diopter at the optical center as in this embodiment, there is no need to install a line-of-sight sensor, and since viewing is done at the optical center, there is the advantage that the areas at the edges of the lens, which have poor optical performance, do not need to be used. Furthermore, it is desirable to use a vibration-type motor that has a holding force when stopped to drive the motor in this embodiment, so that the lens does not move due to inertial force when posture changes due to head rotation, etc. The vibration-type motor is a vibration-type motor that has a vibrating body and a contact body that comes into contact with this vibrating body, and can be configured to hold the display optical element using the holding torque of the vibration-type motor.

[0049] 4A and 4B are diagrams showing the optical central region. The entire area of ​​the display 102 is taken as 100%, and the central 50% or less is taken as the region that appears optically clear, as the range that is least affected by lens aberration. This central region may be the region that includes the point where the display 102 intersects with the optical axis of the lens and occupies 50% or less of the area of ​​the display area of ​​the display 102, assuming that the area is 100%. Alternatively, it may be the region that includes the point where the display 102 intersects with the optical axis of the lens and corresponds to a width of 50% or less of the area of ​​the display area, assuming that one width of the display area is 100%, when the display 102 is substantially rectangular.

[0050] 4A shows the state where the diopter is adjusted to a small depth (nearby), and FIG. 4B shows the state where the diopter is adjusted to a large depth (farther), with the virtual image positions 105a and 105b. Although the angle of view is different, the diopter adjustment area is the center of the range that the user can see, so that the processing of depth information and lens drive control only needs to be performed on the center of the image, which not only produces a clear image but also enables high-speed, highly responsive processing.

[0051] In this embodiment, the appropriate area is set to 50% or less of the display area, but it is possible to make the area wider depending on the optical performance and information processing capacity. Furthermore, in this embodiment, the diopter adjustment is performed by moving the lens, but a similar effect can be obtained with a device in which the display moves in the optical axis direction, as shown in Patent Document 2. That is, it is only necessary to configure the device so that the positions of at least one of the first and second display optical elements and the positions of the first and second image display units can be changed relatively by driving the actuator.

[0052] A second embodiment of the present invention will be described with reference to FIG.

[0053] FIG. 5 is a diagram showing the configuration of a video display device according to the second embodiment.

[0054] The same components as those in the configuration of the conventional image display device are assigned the same reference numerals. In this embodiment, a left eye correction unit 106a and a right eye correction unit 106b are provided between the diopter change instruction unit 106 and the lens drive actuator, respectively.

[0055] The left eye correction unit 106a and the right eye correction unit 106b store data indicating the relationship between the characteristic data of the user's eyes and the amount of lens movement, and the amount of lens movement is set according to the user.

[0056] 6 is a graph showing the relationship between the depth information (horizontal axis) and the lens position (vertical axis) for each user. Lines (A), (B), and (C) in FIG. 6 show the relationship between the depth information (horizontal axis) and the lens position (vertical axis) for each user.

[0057] Generally, a user's diopter changes depending on the object distance. When a person is looking at an object at a close distance, the diopter adjusts to a closer position, and conversely, when a person is looking at an object at a farther distance, the diopter adjusts to a farther position. As a result, a person can see an object clearly, whether it is close or far away.

[0058] The depth information of the image displayed on the display has the same meaning as the distance information (object distance) from the position corresponding to the real reference position to the image in the virtual space projected on the display. Therefore, even if the position of the display does not change, the object distance appears to change when the lens moves.

[0059] In FIG. 6, when the depth information on the display changes, it is equivalent to a change in the object distance, and it is necessary to change the lens position so that the focus is adjusted at that position.

[0060] Line (B) in Figure 6 shows the lens position for improving the decline in the focusing ability of the crystalline lens of the eye due to, for example, aging. For middle-aged and elderly people, the range of object distances at which they can see clearly is narrow, and they often find it difficult to see objects that are far or close clearly.

[0061] In this embodiment, the range of lens movement is increased for the aging user, widening the visible range. By increasing the range of movement in this way, objects can be clearly seen regardless of whether they are far or close.

[0062] In the case of line (C) in Figure 6, the relationship between the depth information and the change in lens position is the same as in the case of line (A), but the lens position is different at the same object distance. This occurs when the object distance at which the image appears clear differs due to the influence of myopia, hyperopia, etc. Such people often use eyeglasses to correct their vision so that the relationship between object distance and diopter is as shown by line (A).

[0063] By setting the relationship between the object distance and the lens position as shown by line (C) in this embodiment, the same effect as wearing glasses can be obtained, and it becomes possible to see the image clearly without wearing glasses.

[0064] Although FIG. 6 shows the relationship between depth and lens position for a user who has the same information such as visual acuity for both eyes, it is also possible to accommodate a user whose left and right eyes have different visual acuity information.

[0065] FIG. 5 shows the relationship between lens positions for a user whose left eye has poorer vision than his right eye.

[0066] 5, the left eye correction unit 106a determines the position of the left eye lens so that the right eye is focused at a distance farther away, and the right eye correction unit 106b sets the position of the right eye lens so that the right eye is focused at a distance closer than the left eye. For example, the relationship is as shown in FIG. 6(A) for the left eye and FIG. 6(B) for the right eye.

[0067] In this way, in this embodiment, by providing the left and right correction units, the left and right lens positions can be set independently.

[0068] Furthermore, when performing the above correction, the correction data differs depending on the gaze direction, i.e., whether the user is looking at a position away from the optical center or at the optical center. In other words, in addition to data on the user's visual acuity, it is necessary to store data on the product of the gaze direction and the lens drive amount, which can lead to problems such as an increase in data volume and an increase in the time required to process that data. Furthermore, in this embodiment, as in the first embodiment, only the depth information at the position of the optical center is used, so the amount of data to be processed can be significantly reduced, and processing corresponding to the user's left and right eyes can be fully performed.

[0069] In this way, the image display device can respond to information such as the user's eyesight, and by moving the lenses, the user can view the image with the same sensation as if wearing glasses, even without glasses. Because the user can wear the image display device without glasses, there is no need for space for glasses, and the image display device can also be made smaller.

[0070] A third embodiment of the present invention will be described with reference to FIGS.

[0071] FIG. 7 is a diagram showing the configuration of a video display device according to the third embodiment.

[0072] The same components as those in the conventional image display device are designated by the same reference numerals.

[0073] In this embodiment, a variation processing unit 113 is provided between the image center calculation unit 112 and the display processing unit 101 .

[0074] In the first embodiment, the diopter adjustment was performed by moving the lens according to the depth information at the center of the image, but in this embodiment, the change in the depth information at the center of the image is calculated by the variation processing unit 113. Conventionally, even if one wanted to see the change in depth information, it was necessary to determine which information in the entire image to calculate after detecting the gaze direction with an eye sensor, which required a long processing time.

[0075] In this embodiment, the detection position is determined to be the optical center position, so that changes in depth information can be easily detected, and information on the changes can be easily obtained.

[0076] FIG. 8 is a diagram showing the change in depth of the image center and the lens movement over time.

[0077] Here, changes in depth information at the optical center can be detected in real time, but because the detection is done in accordance with the optical center in this proposal, the amount of data to be processed is reduced, allowing changes in depth information to be detected almost in real time.

[0078] The variation processing section performs, for example, integration processing on the changes in the depth information, so as not to respond to sudden changes, but to respond slowly to any changes that occur.

[0079] The variation processing section is provided with a threshold value for the amount of variation from the previous value, and when the threshold value is exceeded, a command to drive the lens is issued to the diopter change instruction section.

[0080] For example, if there is a sudden change in depth information such as at time t1, where the change returns to the original state in an instant, the result of integration does not exceed the threshold value, and therefore the lens is not driven.

[0081] From time t2 to time t3, the depth information changes to a larger value.

[0082] In such a case, the result of integration in the fluctuation processing section exceeds the threshold value, and a command to drive the lens is sent to the diopter change instruction section, and the lens is driven.

[0083] As described above, in this embodiment, after the depth information of the optical center is processed at high speed and obtained, the variation processing section determines whether lens movement is necessary and controls the lens movement.

[0084] By operating as in this embodiment, the lens does not have to adapt to sudden changes, and the user can see smooth changes in the image, making it possible to watch the image without experiencing discomfort or nausea.

[0085] The proposed image display device can be used in devices such as head-mounted displays and image display glasses. Recently, there has been a demand for higher-definition images, and this device, which can provide clear images, can be deployed in products with a wide range of applications. It can also be used without corrective glasses, which can contribute to the miniaturization of devices.

[0086] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention.

[0087] This application claims priority based on Japanese Patent Application No. 2024-040947, filed March 15, 2024, the entire contents of which are incorporated herein by reference.

[0088] 1 User 10 Image display device 100 Image acquisition unit 101 Display processing unit 102a, b Image display unit 103a, b Lens 104a, b Actuator 104a, b Lens movement range 105a, b Virtual image 106 Diopter change instruction unit 106a Left eye correction unit 106b Left eye correction unit 108 Object position calculation unit 109 User information storage unit 110 Operation mode recording unit 111 Posture change detection unit 112 Image center calculation unit 113 Variation processing unit 201a Left eye 201b Right eye

Claims

1. An image display device comprising: first and second image display units that display first and second images to the user's right and left eyes, respectively; first and second display optical elements that correspond to the first and second image display units, respectively; and an actuator that changes the relative positions of at least one of the display optical elements and the corresponding image display units, wherein the image display device is a part of the first and second image display units, and the position of the display optical element and the position of the corresponding image display unit are changed relatively by driving the actuator in accordance with depth information provided by an image displayed in an area that intersects with the optical axis of the display optical element in the first and second image display units.

2. The image display device according to claim 1, wherein said area is 50% or less of the display area of ​​said image display unit.

3. The image display device according to claim 2, wherein said area occupies 50% or less of the area of ​​said display area, where the area of ​​said display area is 100%.

4. The image display device according to claim 2, wherein said area is an area corresponding to a width of 50% or less when one width of said rectangular image display section is 100%.

5. An image display device as described in claim 1 or 2, characterized in that the depth information of the image in the area intersecting the optical axis of the display optical element also includes distance information of the object displayed on the first and second image display units.

6. A video display device according to claim 1 or 2, further comprising a detection unit for detecting the orientation of said video display device, and determining the video information to be displayed in said area according to the result of said orientation detection.

7. The image display device according to claim 6, wherein the actuator is a vibration type motor having a vibrating body and a contact body that contacts the vibrating body, and when detecting the posture, the display optical element is held by the holding torque of the vibration type motor.

8. The image display device according to claim 1, further comprising a correction section for changing the position of said display optical element based on the visual acuity information of said user.

9. An image display device according to claim 8, wherein said correction section changes the position of said display optical element based on depth information provided by said image in addition to visual acuity information of said user.

10. An image display device according to claim 1 or 2, further comprising a variation processing section that calculates a change in depth information provided by the image, and that drives the actuator when the change exceeds a threshold value.

11. An image display device according to claim 1 or 2, wherein said actuator is an actuator for changing the position of at least one of said display optical elements.

Images