Image display device, method for driving the same, and program

Abstract

The image display device according to this application includes: a first image display unit and a second image display unit that respectively display a first image and a second image for a user's right eye and left eye; a first display optical element and a second display optical element respectively corresponding to the first image display unit and the second image display unit; and an actuator for changing the position of the display optical element. The image display device is characterized in that it drives the actuator based on the detection result of the physiological or psychological state of the user using the image display device to change the position of at least one of the first display optical element and the second display optical element.

Description

Technical Field

[0001] This invention relates to a technique for adjusting diopter in image display devices. Background Technology

[0002] There are image display devices that users wear on their heads, such as head-mounted displays or those worn like glasses. The display unit is positioned near the user's eyes and performs parallax image display processing for the user's left and right eyes. By visually recognizing the displayed parallax image, the user obtains a stereoscopic effect of the objects displayed in the parallax image.

[0003] In this image display device, the user's gaze is directed to the position of a stereoscopic image generated based on the images displayed for the corresponding left and right eyes, while the focal point is adjusted to each of the left and right images on the display screen. This results in an unnatural state that does not occur when observing real objects.

[0004] For example, when a user observes an image of an object displayed as a parallax image, the user's angle of convergence changes depending on the position of the displayed object. At this time, based on experience in real space, the focal length of the lens varies according to the size of the angle of convergence. In this situation, the focal length of the lens may not match the refractive power of the image display device, resulting in an inability to ensure the desired visibility, such as difficulty clearly observing the displayed parallax image due to the inability to focus on it. Therefore, in adjusting the refractive power of the image display device, it is desirable, for example, to change the refractive power based on depth information in the parallax image that determines the user's angle of convergence, using a lens in the driving device.

[0005] PTL 1 reduces discomfort during stereoscopic viewing by adjusting the diopter information corresponding to the user's gaze point position during image viewing in a way that adapts to individual differences and the usage status of the image display device.

[0006] Reference List

[0007] Patent documents

[0008] PTL 1: Japanese Patent Publication No. 2023-32278 Summary of the Invention

[0009] Technical issues

[0010] However, even if refractive adjustment reduces discomfort during stereoscopic viewing, continuous image viewing over long periods can lead to motion sickness or cumulative fatigue due to changes in the user's physical condition, deviations in refractive adjustment, and other factors. In particular, motion sickness or fatigue is known to frequently occur when an object displayed in a parallax image is viewed for an extended period in an image placed near the user's side. Users of image display devices often have opportunities for prolonged viewing, such as in games and live performances, and experiencing motion sickness or fatigue is unpleasant.

[0011] Therefore, the object of the present invention is to provide an image display device that can provide users with stereoscopic images through parallax while reducing the burden on users during long-term image viewing.

[0012] Solution to the problem

[0013] To achieve the above objective, an image display device includes:

[0014] A first image display unit is configured to display a first image for the user's right eye;

[0015] A second image display unit is configured to display a second image for the user's left eye;

[0016] A first display optical element, the first display optical element corresponding to the first image display unit;

[0017] A second display optical element, the second display optical element corresponding to the second image display unit; and

[0018] An actuator, configured to change the position of the display optical element, wherein,

[0019] The image display device further includes a state detection unit configured to detect the physiological or psychological state of a user during use of the image display device, and the actuator is driven based on the detection result of the state detection unit to change at least one of the position of the first display optical element relative to the first image display unit and the position of the second display optical element relative to the second image display unit.

[0020] Advantages of the present invention

[0021] According to the image display device of the present invention, the burden on the user during the display of stereoscopic images via parallax can be reduced based on the user's motion sickness or fatigue during long-term image viewing. Attached Figure Description

[0022] [Figure 1 ] Figure 1 This is an explanatory diagram illustrating a schematic configuration of an image display device according to a first embodiment.

[0023] [ Figure 2A ] Figure 2A This is an explanatory diagram illustrating the state in which the lens position is changed in an image display device.

[0024] [ Figure 2B ] Figure 2B This is an explanatory diagram illustrating the state in which the lens position is changed in an image display device.

[0025] [ Figure 3A ] Figure 3A This is an explanatory diagram illustrating an example of stereoscopic image display using parallax.

[0026] [ Figure 3B ] Figure 3B This is an explanatory diagram illustrating an example of stereoscopic image display using parallax.

[0027] [ Figure 3C ] Figure 3C This is an explanatory diagram illustrating an example of stereoscopic image display using parallax.

[0028] [ Figure 4A ] Figure 4A It is an illustration showing the convergence angle and refractive power of the eye when focusing on a target object.

[0029] [ Figure 4B ] Figure 4B It is an illustration showing the convergence angle and refractive power of the eye when focusing on a target object.

[0030] [ Figure 4C ] Figure 4C It is an illustration showing the convergence angle and refractive power of the eye when focusing on a target object.

[0031] [ Figure 5 ] Figure 5 This is a flowchart illustrating the operation of the image display device according to the first embodiment.

[0032] [ Figure 6 ] Figure 6 It is an explanatory diagram illustrating the changes in physiological or psychological state and lens position according to the first embodiment.

[0033] [ Figure 7 ] Figure 7 This is an explanatory diagram illustrating a modified example of a schematic configuration of an image display device according to the first embodiment.

[0034] [ Figure 8 ] Figure 8 This is an explanatory diagram illustrating the changes in physiological or psychological state and lens position in a modified example of the image display device according to the first embodiment.

[0035] [ Figure 9 ] Figure 9 This is a flowchart illustrating the operation of the image display device according to the second embodiment.

[0036] [ Figure 10 ] Figure 10 It is an explanatory diagram illustrating the changes in physiological or psychological state and maximum driving speed according to the second embodiment.

[0037] [ Figure 11 ] Figure 11 This is a flowchart illustrating the operation of a modified example of the image display device according to the second embodiment.

[0038] [ Figure 12 ] Figure 12 This is an explanatory diagram illustrating the changes in physiological or psychological state and maximum driving speed in a modified example of the image display device according to the second embodiment. Detailed Implementation

[0039] In the following description, embodiments of the invention will be illustrated in detail with reference to the accompanying drawings. In the embodiments, a head-mounted display will be described as an example of an image display device capable of performing stereoscopic display via a plurality of display optics by having a plurality of image display units respectively display a pair of images with parallax.

[0040] The image display device according to the present invention includes a first image display unit and a second image display unit, the first image display unit and the second image display unit being configured to display a first image and a second image for the user's right eye and left eye, respectively. The image display device also includes a first display optical element and a second display optical element corresponding to the first image display unit and the second image display unit, respectively, and an actuator configured to change the position of the display optical element. Further, in the image display device, the actuator is driven based on the detection result of the user's physiological or psychological state during use of the image display device, to change at least one of the positions of the first display optical element relative to the first image display unit and the second display optical element relative to the second image display unit.

[0041] The following embodiments provide representative example configurations, and the invention is not limited to the description of the embodiments. The image display device, the method for controlling the image display device, and the procedure thereof can be configured in desired combinations without departing from the spirit of the invention.

[0042] <First Embodiment>

[0043] Figure 1 This is a schematic diagram illustrating the configuration of an image display device 100 according to a first embodiment of the present invention.

[0044] The image display device 100 includes an image acquisition unit 101 and a display processing unit 102. The image acquisition unit 100 acquires a display image to be displayed on the image display unit 103 via an external device (not shown), a network, or the like. The display processing unit 102 performs display magnification adjustment processing, etc., on the acquired image. The processed image is sent to and displayed on the image display units 103a and 103b. For example, the image is divided for display on each of the image display units 103a and 103b; however, this configuration is not limited to this. A single screen can be divided into two areas to display the image corresponding to the divided image display unit.

[0045] The image display device 100 includes a first display optical element and a second display optical element corresponding to the left eye 201a and the right eye 201b, respectively. For example, the first display optical element may have a lens 104a, and the second display optical element may have a lens 104b. Images displayed on the image display units 103a and 103b are presented to the left eye 201a and the right eye 201b through the corresponding lenses 104a and 104b, respectively. Figure 1 The convex lens shown can be used as a lens.

[0046] Lenses 104a and 104b are driven by actuators 105a and 105b and are movable in the direction along the optical axis of the lens (arrows 107a and 107b). This embodiment is described based on the assumption that the optical axes 202a and 202b of lenses 104a and 104b pass through the centers of image display units 103a and 103b and the centers of the left eye 201a and right eye 201b, respectively; however, it is not limited thereto.

[0047] Figure 2A and Figure 2B This is a schematic diagram illustrating the states in which lenses 104a and 104b are moved by actuators 105a and 105b, respectively. Figure 2A This is a diagram illustrating the state in which lenses 104a and 104b are moved closer to image display units 103a and 103b, respectively.

[0048] Figure 2BThis diagram illustrates the state in which lenses 104a and 104b are moved closer to the left eye 201a and right eye 201b, respectively. When the image display units 103a and 103b are viewed through lenses 104a and 104b using the left eye 201a and right eye 201b, the user of the image display device 100 visually recognizes virtual images 203a and 203b. Here, the position of the virtual images 203a and 203b in the direction of the optical axes 202a and 202b, with reference to the positions of the left eye 201a and right eye 201b, is defined as the virtual image forming position X. The virtual image forming position X can be changed by changing the positions of lenses 104a and 104b. For example, as... Figure 2A As shown, as lenses 104a and 104b approach image display units 103a and 103b, the virtual image forming position X moves closer to the left eye 201a and right eye 201b. Conversely, as... Figure 2B As shown, as lenses 104a and 104b approach the left eye 201a and right eye 201b, the virtual image forming position X moves away from the left eye 201a and right eye 201b. In this way, moving lenses 104a and 104b via actuators 105a and 105b allows the refractive power to be changed.

[0049] While this embodiment describes a configuration where the virtual image forming position X moves closer to the left eye 201a and right eye 201b as lenses 104a and 104b approach image display units 103a and 103b, it is not limited to this configuration. In another configuration, the virtual image forming position X can move away from the left eye 201a and right eye 201b as lenses 104a and 104b approach image display units 103a and 103b. Furthermore, in the above configuration, lenses 104a and 104b are moved by actuators 105a and 105b to change the refractive power; however, the configuration is not limited to the movement of optical components. As an alternative configuration for the movement of lenses 104a and 104b, the following embodiment can be adopted: for example, a liquid lens is used, and the position of the interface formed by water and oil, etc., is changed by an electrical signal applied from actuators 105a and 105b. Furthermore, since the image display device 100, represented by a head-mounted display, is used when positioned close to the user's ear, it is preferable to use a quiet actuator for changing the diopter, i.e., an ultrasonic motor or an electromagnetic motor such as a voice coil motor, which serves as a vibration actuator. Additionally, although lenses 104a and 104b are... Figure 1 , Figure 2A and Figure 2B The image is shown as a single convex lens, but each of lenses 104a and 104b may include multiple lenses configured such that the position of a predetermined lens is moved to change the diopter.

[0050] Reference Figures 3A to 3CThe parallax image 300 to be displayed on the image display units 103a and 103b is described. Figures 3A to 3C This is an illustrative diagram of an example image. For example, the parallax image 300 includes a left-eye image 301a to be displayed on an image display unit 103a corresponding to the left eye 201a and a right-eye image 301b to be displayed on an image display unit 103b corresponding to the right eye 201b.

[0051] While this embodiment describes an example of a parallax image 300 including a left-eye image 301a and a right-eye image 301b, it is not limited thereto. For example, a parallax image to be displayed on each of the two image display units 103a and 103b can be generated by processing based on three-dimensional data by the display processing unit 102. Alternatively, an image captured by a camera mounted on the image display device 100 can be disposed of as at least a portion of the parallax image 300. This configuration allows the image display device 1 to display images in an augmented reality spatial representation. For example, an image captured by a camera mounted on the image display device 1 is superimposed on an image based on pre-created three-dimensional data to create the parallax image 300. As a result, a composite image is displayed.

[0052] For example, such as Figure 3A As shown, when the user's gaze converges on a position overlapping with the displayed image, images 301a and 301b in the displayed image are perceived as being located at the same position as the image display unit 103. Figure 3B As shown, when the user's gaze is focused on a position in front of the image display unit 103, images 301a and 301b in the displayed image are perceived as protruding relative to the image display unit 103. Figure 3C As shown, when the user's gaze is focused on the area behind the image display unit 103, images 301a and 301b in the displayed image are perceived as being located at a distance behind the image display unit 103.

[0053] Although this embodiment illustrates the case where images 301a and 301b are each a single image, it is not limited to this. For example, multiple objects may be displayed in the image captured by the camera. In this case, it is possible to detect the gaze direction of the user's left eye 201a and right eye 201b, as... Figure 1 The gaze detection components 106a and 106b shown are mounted on the device. Furthermore, the user's gaze point is processed as images 301a and 301b. The gaze detection components may include, for example, an infrared illumination unit and a gaze detection camera.

[0054] The refractive power adjustment calculation unit 111 calculates the refractive power adjustment based on depth information at the location of the fixation point determined from the detection results of the gaze detection units 106a and 106b. Based on the calculated refractive power adjustment, the control unit 108 outputs a drive command to drive the actuators 105a and 105b, thereby performing the refractive power adjustment. The corresponding distances between the image display unit 103a and the lens 104a, and between the image display unit 103b and the lens 104b, are changed to adjust the refractive power. For example, whenever the user's fixation point changes, the refractive power adjustment is performed in real time by the refractive power detection units 106a and 106b.

[0055] The control unit 108 is a so-called microcomputer and may include electrical components such as an arithmetic unit (CPU), a memory for storing programs, and a memory serving as a work area on which the programs are loaded. In this case, the control unit 108 is used to generate signals with information for controlling the drive of the actuator 105.

[0056] Figures 4A to 4C It is a diagram illustrating the convergence angle and refractive power of the eye when focusing on a target object. Figure 4A The illustration depicts the state during gaze in the real world, and Figure 4B The illustration shows the state during gaze in the image display device 100.

[0057] exist Figure 4A In the real world, the user is focused on a target object 401 located at a distance A. The lines of sight from the user's left eye 201a and right eye 201b converge on the target object 401 to form an angle α. Angle α corresponds to the convergence angle when the user is focused on the target object 401. Therefore, in the real world, the distance corresponding to the convergence angle α and the distance A corresponding to the refractive power are always the same.

[0058] On the other hand, Figure 4B In the image display device 100, the user is in a state of focusing on an object 401 located at a distance A. The convergence angle α and... Figure 4A The convergence angles shown are the same as in the real world. Virtual image position 402 is the position of the virtual image formed when the user observes image display units 103a and 103b through lenses 104a and 104b (not shown), respectively. Virtual image position 402 is located at a distance B from the left eye 201a and right eye 201b.

[0059] Assigning the convergence angle α to the virtual image position 402 allows the user to visually recognize the virtual image as if it were located at the same position as the target object 401. However, in the image display device 100, unlike the real world, the distance corresponding to the convergence angle α and the distance B corresponding to the refractive power are not the same. This is called vergence-accommodation conflict, and the persistence of this state increases motion sickness or fatigue for the user.

[0060] Therefore, as in Figure 4C In this context, changing the spacing between image display units 103a and 103b and lenses 104a and 104b (not shown) can alter the virtual image position 402. For example, when the gaze detection component detects that a user is looking at target object 401, the spacing between the display units and the lenses changes, causing the virtual image position 402, located at distance B, to move to distance A. Therefore, as... Figure 4A As shown, the distance corresponding to the convergence angle and the distance corresponding to the refractive power are aligned to provide a more natural state, thereby reducing motion sickness or fatigue.

[0061] To change the spacing between the display unit and the lens, it is sufficient to drive at least the display unit or the lens.

[0062] Figure 1 The state detection unit 109 shown is configured to detect the user's physiological or psychological state, and to detect, for example, the user's heart rate, electrocardiogram, respiration, electrooculogram, skin potential, and center of gravity.

[0063] Regarding heart rate, the user's heart rate is measured to determine the average instantaneous heart rate, the respiratory component of heart rate variability, and the magnitude of the Mayer wave component of heart rate variability, and at least one of these magnitudes is used as a detection value.

[0064] Regarding electrocardiograms (ECGs), a user's ECG is measured to determine the high-frequency (HF) and low-frequency (LF) components of heart rate variability, or the ratio of HF to LF, and at least one of these is used as a detection value. HF and LF are the magnitudes of specific frequency components of the baseline fluctuations of the ECG.

[0065] Regarding breathing, the frequency, magnitude, and irregularity of the user's breathing are measured, and at least one of these is used as a detection value.

[0066] Regarding electrooculography, the user's blink count, cumulative frequency, cumulative count, rate of change, and blink interval are measured, and at least one of these is used as a detection value.

[0067] In addition, the user's center of gravity can be measured, and the magnitude of the center of gravity swing can be used as a detection value.

[0068] The detected value from the state detection unit 109 is output to the state determination unit 110 and compared with a pre-recorded normal value (i.e., a predetermined value) to determine whether the user is experiencing motion sickness or fatigue. Physiological or psychological states can be determined by combining multiple factors, such as electrocardiogram, respiration, electrooculogram, skin potential, and center of gravity, rather than using only one of them.

[0069] When using skin potential, determination can be performed based on the magnitude of its resistive component. When using center of gravity, determination can be performed based on the oscillation of the center of gravity.

[0070] If motion sickness or fatigue is detected in the user, the control unit 108 drives actuators 105a and 105b to move lenses 104a and 104b toward the image display unit, thereby changing the diopter. The status detection unit 109 and status determination unit 110 do not need to be part of the image display device and can be configured to wirelessly transmit and receive signals, allowing for signal exchange. Conversely, integrating the status detection unit 109 and status determination unit 110 with the image display device allows the user of the head-mounted display to immediately use the status detection function without the intervention of remote communication.

[0071] Reference Figure 5 The flowchart shown illustrates the operation according to the first embodiment. Figure 5 The program corresponding to the flowchart shown is stored in the storage unit of the arithmetic processing unit. The control unit 108 includes a CPU, memory, etc., and Figure 5 Each process shown in the flowchart is implemented by the CPU loading a predetermined program stored in memory. The control unit 108 may or may not be integrated with the image display device.

[0072] First, in S501, the user's physiological or psychological state is detected using the state detection unit 109. Then, in S502, the user's motion sickness or fatigue state is determined based on the detected value. If the detected value is greater than or equal to the expected value, in S503, a movement amount is calculated to move lenses 104a and 104b in a direction away from image display units 103a and 103b, and in S504, the control unit 108 drives actuators 105a and 105b. The movement amount can be determined based on the detected value, or the movement amount can increase as the detected value increases and decrease as the detected value decreases. This prevents the displayed parallax image from being focused and clearly observed, reducing the amount of information entering the left eye 201a and right eye 201b, thereby relaxing the eye muscles and reducing motion sickness or fatigue.

[0073] If the detected value in S502 is less than or equal to the expected value, then in S505, the gaze detection components 106a and 106b are used to detect the gaze of the user's left eye 201a and right eye 201b, respectively.

[0074] Then, in S506, the control unit 107 performs the process of determining the fixation point from the line of sight of the left eye 201a and the line of sight of the right eye 201b. The fixation point can be calculated by using the average of the lines of sight of the left eye 201a and the right eye 201b.

[0075] Then, in S507, the refractive power adjustment calculation unit 111 performs a process to calculate the refractive power adjustment at the position of the fixation point of the images to be displayed on the image display units 103a and 103b. Specifically, it calculates the disparity between the left-eye image and the right-eye image at the fixation point, and derives the refractive power adjustment from the depth information corresponding to the disparity. For example, it provides tabular data relating disparity and refractive power adjustment to each other to calculate the refractive power adjustment value corresponding to the disparity. Alternatively, the refractive power adjustment calculation unit 111 can calculate the refractive power adjustment based on the disparity using a mathematical formula indicating the relationship between disparity and refractive power adjustment.

[0076] Then, in S508, the control unit 108 drives the actuators 105a and 105b based on the refractive power adjustment calculated by the refractive power adjustment calculation unit 111. Therefore, an operation is performed to change the distance between the image display units 103a and 103b and the lenses 104a and 104b.

[0077] Because the above operation is performed whenever the fixation point is changed, diopter adjustment can be performed in real time to change the diopter.

[0078] Figure 6 The diagram illustrates the measured values ​​of the user's physiological or psychological state, as well as... Figure 1 The lens positions in the configuration shown. For example... Figure 6 As shown, during the time period from time t0 to t1, the gaze detection unit calculates the fixation point based on the gaze, and performs real-time diopter adjustment in the directions of arrows 107a and 107b based on the depth information of the fixation point. When the state detection value detected by the state detection unit exceeds a predetermined value at time t1, actuators 105a and 105b are operated to move lenses 104a and 104b closer to the user side, causing lenses 104a and 104b to move away from image display units 103a and 103b, respectively. Subsequently, when the detection value drops below the desired value at time t2, the gaze detection unit again calculates the fixation point based on the gaze, and performs real-time diopter adjustment in the directions of arrows 107a and 107b based on the depth information of the fixation point.

[0079] The following configuration can be used: the reciprocal of the state detection value is set as the evaluation target, and the case where the reciprocal of the state detection value drops below a certain value is used as a reference; this is synonymous with the state detection value "exceeding the predetermined value".

[0080] In this example, since the lens 104a corresponding to the left eye 201a and the lens 104b corresponding to the right eye 201b can be driven by independent actuators 105a and 105b respectively, this can even support users with different vision in their left and right eyes. That is, a configuration in which the position of at least one of the first and second display optical elements is changed can be adopted.

[0081] According to this example, refractive power adjustment is performed based on depth information of the gaze point, and actuators are driven to move lenses 104a and 104b away from image display units 103a and 103b, depending on the user's motion sickness or fatigue. This results in a blurred perception of the image displayed on the image display unit. Reducing the amount of information the user receives from the image can reduce motion sickness or fatigue accumulated from prolonged use.

[0082] While the above example describes operating the actuator to move the image display unit and the lens away from each other, it is not limited to this. The actuator can be operated to bring the image display unit and the lens closer together, achieving an equivalent effect. In this case, when the value detected by the state detection unit 109 exceeds a predetermined value, lenses 104a and 104b are moved by actuators 105a and 105b to move closer to image display units 103a and 103b.

[0083] While the above description assumes a configuration for performing refractive power adjustment based on depth information of the fixation point detected by gaze detection components 106a and 106b, it is not limited thereto. Other configurations may be employed, such as... Figure 7 The configuration shown is for manually performing diopter adjustment. Figure 8 The diagram illustrates the measured values ​​of the user's physiological or psychological state, as well as... Figure 7 The lens position in the configuration shown. In this configuration, as... Figure 8 As shown, at time t0, the user manually adjusts the diopter to a position suitable for viewing the image and sets this position as the lens reference position. When the value detected by the state detection unit exceeds a predetermined value at time t1, the actuator is activated to move the lens closer to the user side, causing the lens to move away from the display unit. Subsequently, when the detected value drops below the desired value at time t2, the actuator is activated to return the lens to the lens reference position manually adjusted at time t0.

[0084] <Second Embodiment>

[0085] The operation method of actuators 105a and 105b driven by control unit 108 using the value detected by state detection unit 109 in the first embodiment described above will be described. Since the principle, configuration, etc. are similar to those of the first example, their description is omitted.

[0086] Reference Figure 9 The flowchart shown illustrates the operation according to the second embodiment. Figure 9 The program corresponding to the flowchart shown is stored in the storage unit of the arithmetic processing unit. The control unit 108 includes a CPU, memory, etc., and Figure 9 Each process shown in the flowchart is implemented by the CPU loading a predefined program stored in memory.

[0087] First, in S1001, the user's physiological or psychological state is detected using the state detection unit 109. Then, in S1002, the user's motion sickness or fatigue state is determined based on the detected value. If the detected value is less than or equal to the expected value, in S1003, the control unit 108 sets the maximum drive speed of actuators 105a and 105b to A (e.g., 100 mm / s). If the detected value is greater than or equal to the expected value, in S1004, the control unit 108 sets the maximum drive speed of actuators 105a and 105b to B (e.g., 50 mm / s), which is less than A. Changing the maximum drive speed in this way can suppress the amount of change in the user's focus on the image, thereby reducing the tendency for the eyes to follow sudden changes in focus. Therefore, even during long periods of image viewing, reducing the burden on the user can reduce the increase or accumulation of motion sickness or fatigue.

[0088] Then, in S1005, gaze detection components 106a and 106b are used to detect the gaze of the user's left eye 201a and right eye 201b, respectively.

[0089] Then, in S1006, the control unit 108 performs the process of determining the fixation point from the line of sight of the left eye 201a and the line of sight of the right eye 201b. The fixation point can be calculated by using the average of the lines of sight of the left eye 201a and the right eye 201b.

[0090] Then, in S1007, the refractive power adjustment calculation unit 111 performs a process to calculate the refractive power adjustment at the position of the fixation point of the images to be displayed on the image display units 103a and 103b. Specifically, it calculates the disparity between the left-eye image and the right-eye image at the fixation point, and derives the refractive power adjustment from the depth information corresponding to the disparity. For example, it provides tabular data relating disparity and refractive power adjustment to each other to calculate the refractive power adjustment value corresponding to the disparity. Alternatively, the refractive power adjustment calculation unit 111 can calculate the refractive power adjustment based on the disparity using a mathematical formula indicating the relationship between disparity and refractive power adjustment.

[0091] Then, in S1008, the control unit 108 drives the actuators 105a and 105b based on the refractive power adjustment calculated by the refractive power adjustment calculation unit 111. Therefore, an operation is performed to change the distance between the image display units 103a and 103b and the lenses 104a and 104b. Since this operation is performed whenever the fixation point changes, refractive power adjustment can be performed in real time to change the refractive power.

[0092] Figure 10 The diagram illustrates the measured values ​​of the user's physiological or psychological state, as well as the maximum driving speeds of actuators 105a and 105b. (For example...) Figure 10 As shown, during the period from time t0 to t1 when the value detected by the state detection unit 109 is less than a predetermined value, the maximum drive speed of actuators 105a and 105b is set to A. During the period from time t1 to t2 when the value detected by the state detection unit 109 is greater than or equal to the predetermined value, a maximum drive speed B, which is less than the maximum drive speed A, is set. During the period from time t2 to t3 when the value detected by the state detection unit 109 is less than the predetermined value, the maximum drive speed A is set again.

[0093] While the maximum drive speed is changed in this example, it is not a limitation. The control gain or control cycle can be changed, and similar effects can be achieved in this case. Control gain refers to the ratio of the control output required to change the actuator by a certain amount. Lowering this ratio can suppress responsiveness and the amount of change in the user's focus on the image, thereby reducing the tendency of the eye to follow sudden changes in focus. Therefore, even during long periods of image viewing, reducing the burden on the user can reduce the increase or accumulation of motion sickness or fatigue. Control cycle refers to the time interval between issuing commands to control the actuator. Increasing this time interval can suppress the update frequency and the amount of change in the user's focus on the image, thereby reducing the tendency of the eye to follow sudden changes in focus. Therefore, even during long periods of image viewing, reducing the burden on the user can reduce the increase or accumulation of motion sickness or fatigue. Alternatively, if the value detected by the state detection unit 109 is greater than or equal to a predetermined value, two or more of the maximum drive speed, control gain, and control cycle can be changed simultaneously.

[0094] The above describes an example of changing the maximum drive speed, control gain, or control cycle based on whether the value detected by the state detection unit 109 exceeds a predetermined value. However, the invention is not limited to this example. Figure 11 In the flowchart shown, the maximum drive speed, control gain, or control cycle can be changed based on the detected values. In this case, firstly, in S1201, the user's physiological or psychological state is detected using the state detection unit 109. Then, in S1202, the control unit 108 sets the maximum drive speed of actuators 105a and 105b based on the detected values. For example, tabular data relating the detected values ​​of physiological or psychological states to the maximum drive speed can be provided to calculate the maximum drive speed based on the detected values ​​of physiological or psychological states. Alternatively, the maximum drive speed can be calculated using a mathematical formula indicating the relationship between the detected values ​​of physiological or psychological states and the maximum drive speed. Changing the maximum drive speed in this way can suppress the amount of change in the user's focus on the image, thereby reducing the tendency for the eyes to follow sudden changes in focus. Therefore, even during long periods of image viewing, reducing the burden on the user can reduce the increase or accumulation of motion sickness or fatigue, thereby achieving an effect similar to that in the example above.

[0095] Figure 11 The operations from S1203 to S1206 are the same as those described above. Figure 9 The operations from S1005 to S1008 in the flowchart are similar, so their description is omitted.

[0096] Figure 12 The diagram illustrates the measured values ​​of the user's physiological or psychological state, as well as the maximum driving speeds of actuators 105a and 105b. (For example...) Figure 12As shown, the maximum driving speed of actuators 105a and 105b is gradually set according to the value detected by the state detection unit 109.

[0097] While the maximum drive speed has been changed in this example, it is not a limitation. The control gain or control cycle can be changed, and a similar effect can be achieved in this case. Alternatively, if the value detected by the state detection unit 109 is greater than or equal to a predetermined value, two or more of the maximum drive speed, control gain, and control cycle can be changed simultaneously.

[0098] While the present invention has been described in detail with reference to preferred embodiments, it is not limited to these specific embodiments and can be modified in various ways without departing from the scope of the invention. Such modifications also fall within the scope of the invention. A control method for performing the above steps and a non-transitory computer-readable program for performing the control method can also be provided.

[0099] The present invention can also be implemented by the following process: a program for implementing one or more functions of the above embodiments is supplied to a system or device via a network or storage medium, and one or more processors in the computer of the system or device read and execute the program. Furthermore, the present invention can be implemented by a circuit system (e.g., an ASIC) that implements one or more functions.

[0100] This invention is not limited to the embodiments described above, and can be changed and modified in various ways without departing from the spirit and scope of the invention. Therefore, the following claims are made to inform the public of the scope of the invention.

[0101] This application is based on and claims priority to Japanese Patent Application No. 2023-201980, filed on November 29, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0102] Reference Symbol List

[0103] 100 Image display devices

[0104] 101 Image Acquisition Unit

[0105] 102 Display Processing Unit

[0106] 103 Image Display Unit

[0107] 104 Lens

[0108] 105 Actuator

[0109] 106 Gaze Detection Components

[0110] 107 Drive Direction

[0111] 108 Control Unit

[0112] 109 Status Detection Unit

[0113] 110 State Determination Unit

[0114] 111 Depth Information Computing Unit

[0115] 201a Left eye

[0116] 201b Right eye

[0117] 202 optical axes

[0118] 203 Virtual Images

Claims

1. An image display device, comprising: A first image display unit is configured to display a first image for the user's right eye; A second image display unit is configured to display a second image for the user's left eye; A first display optical element, the first display optical element corresponding to the first image display unit; A second display optical element, which corresponds to the second image display unit; as well as An actuator, configured to change the position of the display optical element, wherein, The image display device further includes a state detection unit configured to detect the physiological or psychological state of a user during use of the image display device, and the actuator is driven based on the detection result of the state detection unit to change at least one of the position of the first display optical element relative to the first image display unit and the position of the second display optical element relative to the second image display unit.

2. The image display device according to claim 1, wherein, When the detection result of the physiological state or the psychological state is greater than or equal to a predetermined value, the actuator is driven such that at least one pair of the first image display unit and the first display optical element, and the second image display unit and the second display optical element, are positioned far apart from each other.

3. The image display device according to claim 1, wherein, When the detection result of the physiological state or the psychological state is greater than or equal to a predetermined value, the actuator is driven such that at least one pair of the first image display unit and the first display optical element, and the pair of the second image display unit and the second display optical element, are positioned close to each other.

4. The image display device according to any one of claims 1 to 3, comprising a first gaze detection component and a second gaze detection component, wherein the first gaze detection component is configured to perform gaze detection on the first image display unit, and the second gaze detection component is configured to perform gaze detection on the second image display unit.

5. The image display device according to claim 4, wherein, The gaze detection is performed without using the first display optics and the second display optics.

6. The image display apparatus according to any one of claims 1 to 3, further comprising a calculation unit configured to calculate a refractive power adjustment corresponding to the parallax between the first image and the second image at the location of the fixation point obtained from the first gaze detection unit and the second gaze detection unit.

7. The image display apparatus of claim 6, further comprising a control unit configured to output drive commands, wherein, The control unit is configured to issue a drive command to the actuator corresponding to the diopter adjustment amount.

8. The image display device according to any one of claims 1 to 3, wherein, The control gain of the actuator is further changed based on the value of the detection result of the physiological state or the psychological state.

9. The image display device according to any one of claims 1 to 3, wherein, The control cycle of the actuator is further changed according to the value of the detection result of the physiological state or the psychological state.

10. The image display device according to any one of claims 1 to 3, wherein, The maximum driving speed of the actuator is further changed based on the value of the detection result of the physiological state or the psychological state.

11. The image display device according to claim 4, wherein, The first gaze detection component and the second gaze detection component each include an infrared illumination unit and a gaze detection camera.

12. The image display device according to any one of claims 1 to 3, wherein, The state detection unit is configured to obtain the detection result of the physiological state or the psychological state based on at least one of blink count, cumulative frequency, rate of change, and time interval between blinks derived from the electrooculogram of the user of the measured image display device.

13. The image display device according to any one of claims 1 to 3, wherein, The state detection unit is configured to obtain the detection result of the physiological state or the psychological state based on the magnitude or ratio of the frequency components of the baseline fluctuations of the user's electrocardiogram measured by the image display device.

14. The image display device according to any one of claims 1 to 3, wherein, The state detection unit is configured to obtain the detection result of the physiological state or the psychological state based on the magnitude of the resistive component of the skin potential of the user of the image display device.

15. The image display device according to any one of claims 1 to 3, wherein, The state detection unit is configured to obtain the detection result of the physiological state or the psychological state based on at least one of the frequency, magnitude, and irregularity of the user's breathing measured on the image display device.

16. The image display device according to any one of claims 1 to 3, wherein, The state detection unit is configured to obtain the detection result of the physiological state or the psychological state based on the measured sway of the user's center of gravity of the image display device.

17. The image display device according to any one of claims 1 to 3, wherein, The state detection unit is configured to measure the heart rate of the user of the image display device being measured, and to obtain the detection result of the physiological state or the psychological state based on the average value of the instantaneous heart rate, the respiratory component in the heart rate variability, and the Mayer wave component in the heart rate variability.

18. The image display device according to any one of claims 1 to 3, wherein, The first display optical element and the second display optical element are lenses.

19. The image display device according to any one of claims 1 to 3, wherein, The actuator is an electromagnetic motor or a vibration actuator.

20. A method for controlling an image display device, the image display device comprising: A first image display unit is configured to display a first image for the user's right eye; A second image display unit is configured to display a second image for the user's left eye; A first display optical element, the first display optical element corresponding to the first image display unit; A second display optical element, which corresponds to the second image display unit; as well as An actuator configured to change the position of the display optical element. The method includes the following steps: The control unit obtains the detection results of the user's physiological or psychological state during the use of the image display device; as well as The control unit drives the actuator based on the detection result to change the position of at least one of the first display optical element and the second display optical element.

21. A non-transitory computer-readable program, said non-transitory computer-readable program being used to cause a control unit to perform the steps of claim 20.

Images