Video display device, method for driving video display device, and program therefor

Abstract

A video display device according to the present application comprises: first and second video display units that respectively display first and second videos to the right eye and the left eye of a user; first and second display optical elements that respectively correspond to the first and second video display units; and an actuator that changes the position of the display optical element. The video display device is characterized by changing the position of at least one of the first and second display optical elements by driving the actuator on the basis of the detection result of the physiological state or psychological state of the user who is using the video display device.

Description

Video display device, video display device driving method, and program therefor

[0001] The present invention relates to a technology for adjusting visibility in a video display device.

[0002] There are image display devices such as head-mounted displays that are worn by the user on the head, and image display devices that are worn by the user like glasses. A display unit is placed near the user's eyes, and parallax images are displayed for each of the user's left and right eyes. When the user views the displayed parallax images, they can get a three-dimensional impression of the object displayed in the parallax images.

[0003] With this image display device, the user's line of sight is adjusted to the position of the 3D image created by the images displayed for each eye, but the focal point is adjusted to the left and right images on the display screen, which creates an unnatural state that would not occur when viewing a real object.

[0004] For example, when a user views an image of an object displayed as a parallax image, the user's convergence angle changes depending on the position of the displayed object. At this time, the focal length of the crystalline lens changes depending on the magnitude of the convergence angle based on experience in real space. In this case, the focal length of the crystalline lens may not match the diopter of the image display device, which may result in the displayed parallax image being out of focus and difficult to see clearly, thereby preventing good visibility. Therefore, when adjusting the diopter of an image display device, it is desirable to change the diopter according to depth information in the parallax image that determines the user's convergence angle, for example, by driving a lens within the device.

[0005] In Patent Document 1, the visual acuity is adjusted to depth information according to the position of the user's point of gaze when viewing an image, taking into account individual differences and the usage conditions of the image display device, thereby making it possible to reduce the sense of discomfort felt when viewing a stereoscopic image.

[0006] JP 2023-32278 A

[0007] However, even if adjusting the diopter reduces the discomfort felt during stereoscopic viewing, continued viewing of images for a long period of time can lead to the accumulation of motion sickness and fatigue due to an accumulation of changes in the user's physical condition and misalignment of the diopter adjustment. In particular, it is known that motion sickness and fatigue often occur when viewing images for a long period of time in which objects displayed in parallax images are positioned in front of the user. Users of image display devices often have the opportunity to view images for long periods of time, such as playing games or attending live performances, and feeling motion sickness or fatigue is unpleasant for users.

[0008] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image display device that can provide a user with a stereoscopic image using parallax while reducing the burden on the user when viewing an image for a long period of time.

[0009] An image display device that achieves the above-mentioned object comprises: a first image display unit that displays a first image to the right eye of a user; a second image display unit that displays a second image to the left eye of the user; a first display optical element corresponding to the first image display unit; a second display optical element corresponding to the second image display unit; and an actuator that changes the position of the display optical elements, and further comprises a state detection unit that detects the physiological state or psychological state of the user while using the image display device, and is characterized in that the actuator is driven based on the detection result of the state detection unit to change at least one of the position of the first display optical element relative to the first image display unit and the position of the second display optical element relative to the second image display unit.

[0010] According to the image display device of the present invention, it is possible to reduce the burden on the user when displaying a stereoscopic image due to parallax, depending on the state of sickness or fatigue of the user when viewing images for a long period of time.

[0011] FIG. 1 is an explanatory diagram showing a schematic configuration of an image display device according to a first embodiment; FIG. 2 is an explanatory diagram showing a state when a lens position is changed in the image display device; FIG. 3 is an explanatory diagram showing a state when a lens position is changed in the image display device; FIG. 4 is an explanatory diagram showing an example of a stereoscopic image displayed by parallax; FIG. 5 is an explanatory diagram showing an example of a stereoscopic image displayed by parallax; FIG. 6 is an explanatory diagram showing an example of a stereoscopic image displayed by parallax; FIG. 7 is an explanatory diagram showing the convergence angle and diopter of the eyeballs when gazing at an object; FIG. 8 is an explanatory diagram showing the convergence angle and diopter of the eyeballs when gazing at an object; FIG. 9 is an explanatory diagram showing the convergence angle and diopter of the eyeballs when gazing at an object; FIG. 10 is a flowchart explaining the operation of an image display device according to a first embodiment; FIG. 11 is an explanatory diagram showing changes in physiological state or psychological state and lens position in a modification of the image display device according to the first embodiment; FIG. 12 is a flowchart explaining the operation of an image display device according to a second embodiment; FIG. 13 is an explanatory diagram showing changes in physiological state or psychological state and maximum drive speed in a second embodiment. 10 is a flowchart illustrating the operation of a modified example of the image display device of the second embodiment.FIG. 11 is an explanatory diagram illustrating changes in physiological or psychological states and maximum drive speeds of the modified example of the image display device of the second embodiment.

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the embodiment, a head-mounted display will be described as an example of an image display device in which a pair of images having parallax are displayed on a plurality of image display units, respectively, and stereoscopic display is possible via a plurality of display optical elements.

[0013] The image display device of the present invention includes first and second image display units that display first and second images to the right and left eyes of a user, respectively. The image display device further includes first and second display optical elements corresponding to the first and second image display units, respectively, and an actuator that changes the positions of the display optical elements. The image display device then changes the position of at least one of the first display optical element relative to the first image display unit and the second display optical element relative to the second image display unit by driving the actuator based on a detection result of a physiological or psychological state of the user while using the image display device.

[0014] Furthermore, the embodiments of the present invention shown below are representative configuration examples, and the present invention is not limited to these descriptions. As long as they do not deviate from the spirit of the present invention, a video display device, a control method for a video display device, and a program thereof can be configured in any desired combination.

[0015] First Embodiment FIG. 1 is a diagram showing a schematic configuration of an image display device 100 according to a first embodiment of the present invention.

[0016] The video display device 100 includes a video acquisition unit 101 and a display processing unit 102. The video acquisition unit 100 acquires video to be displayed on the video display unit 103 via an external device, a network, or the like (not shown). The display processing unit 102 performs processing such as adjusting the display magnification of the acquired video. The processed video is sent to the video display units 103a and 103b and displayed thereon. For example, the video is divided and displayed on each of the video display units 103a and 103b, but the present invention is not limited to this configuration. Alternatively, a single screen may be divided into two regions and a corresponding video may be displayed on each of the divided video display units.

[0017] The image display device 100 includes first and second display optical elements corresponding to the left eye 201 a and the right eye 201 b, respectively. For example, the first display optical element may include a lens 104 a, and the second display optical element may include a lens 104 b. Images displayed on the image display units 103 a and 103 b are presented to the left eye 201 a and the right eye 201 b, respectively, through the corresponding lenses 104 a and 104 b. The lenses may be convex lenses as shown in FIG. 1 .

[0018] The lenses 104a and 104b are driven by actuators 105a and 105b, and can move in directions along the optical axes of the lenses (arrows 107a and 107b). In this embodiment, the optical axes 202a and 202b of the lenses 104a and 104b are described as passing through the centers of the image display units 103a and 103b and the left eye 201a and right eye 201b, respectively, but the present invention is not limited to this.

[0019] 2A and 2B are explanatory diagrams showing the state when the lenses 104a and 104b are moved by the actuators 105a and 105b, respectively. Fig. 2A shows the state when the lenses 104a and 104b are moved near the image display units 103a and 103b, respectively.

[0020] FIG. 2B illustrates the state when the lenses 104a and 104b are moved closer to the left eye 201a and the right eye 201b, respectively. When the user of the image display device 100 views the image display units 103a and 103b with the left eye 201a and the right eye 201b through the lenses 104a and 104b, the user perceives virtual images 203a and 203b. Here, the positions of the virtual images 203a and 203b in the direction of the optical axes 202a and 202b, based on the positions of the left eye 201a and the right eye 201b, are defined as virtual image formation position X. The virtual image formation position X can be changed by changing the positions of the lenses 104a and 104b. For example, as shown in FIG. 2A, as the lenses 104a and 104b move closer to the image display units 103a and 103b, the virtual image formation position X moves closer to the left eye 201a and the right eye 201b. 2B, when the lenses 104a and 104b move closer to the left eye 201a and the right eye 201b, the virtual image formation position X moves farther away from the left eye 201a and the right eye 201b. In this way, the diopter can be changed by moving the lenses 104a and 104b with the actuators 105a and 105b.

[0021] In the present embodiment, the virtual image formation position X approaches the left eye 201a and the right eye 201b as the lenses 104a and 104b approach the image display units 103a and 103b, respectively. However, the present invention is not limited to this. Alternatively, the virtual image formation position X may be moved away from the left eye 201a and the right eye 201b as the lenses 104a and 104b approach the image display units 103a and 103b. Furthermore, while the present invention is not limited to moving optical components, the present invention is not limited to moving optical components. Alternatively, the present invention may employ a liquid lens to change the position of an interface between water and oil, for example, using an electrical signal applied from the actuators 105a and 105b. Furthermore, since the image display device 100, typified by a head-mounted display, is used by being placed near the user's ears, it is preferable to use an electromagnetic motor, such as an ultrasonic motor or a voice coil motor, which is a vibration-type actuator that is highly quiet, as the actuator that changes the diopter. Furthermore, although the lenses 104a and 104b are represented as a single convex lens in Figures 1, 2A, and 2B, each may be represented as a multiple lens, and the position of a specific lens may be moved when the diopter is changed.

[0022] 3A to 3C, a parallax image 300 displayed on the image display units 103a and 103b will be described. Figures 3A to 3C are explanatory diagrams of an example of an image. The parallax image 300 is composed of, for example, a left-eye image 301a displayed on the image display unit 103a corresponding to the left eye 201a and a right-eye image 301b displayed on the image display unit 103b corresponding to the right eye 201b.

[0023] In this embodiment, an example is shown in which the parallax image 300 is composed of an image 301a for the left eye and an image 301b for the right eye, but the present invention is not limited to this. For example, a configuration may be adopted in which processing is performed by the display processing unit 102 based on 3D data, and parallax images are generated to be displayed on the two image display units 103a and 103b, respectively. Furthermore, an image captured by a camera mounted on the image display device 100 may be treated as at least a part of the parallax image 300. With such a configuration, it becomes possible for the image display device 1 to display an image in an augmented reality space representation format. For example, the parallax image 300 is created by superimposing an image captured by a camera mounted on the image display device 1 on an image based on 3D data created in advance, and a composite image is displayed.

[0024] For example, as shown in Fig. 3A, when the location where the user's line of sight intersects is exactly the same as the screen of the displayed image, images 301a and 301b in the displayed image appear to be in the same position as image display unit 103. Also, as shown in Fig. 3B, when the location where images 301a and 301b in the displayed image intersect with the user's line of sight is in front of image display unit 103, the images appear to jump out relative to image display unit 103. Furthermore, as shown in Fig. 3C, when the location where images 301a and 301b in the displayed image intersect with the user's line of sight is behind image display unit 103, the images appear to be located far behind the screen of image display unit 103.

[0025] In this embodiment, the image 301a and the image 301b are shown as being one each, but the present invention is not limited to this. For example, a configuration in which multiple objects are displayed in an image captured by a camera is also possible. In this case, the device is equipped with gaze detection means 106a and 106b as shown in FIG. 1 that can detect the gaze direction of the user's left eye 201a and right eye 201b, and processing is performed to treat the gaze points at which the user's gaze is directed as the images 301a and 301b. The gaze detection means can use an infrared illumination unit, a gaze detection camera, or the like.

[0026] The diopter adjustment amount calculation unit 111 calculates the diopter adjustment amount based on depth information at the position of the gaze point determined from the detection results by the line-of-sight detection units 106a and 106b. Based on this calculated diopter adjustment amount, the control unit 108 outputs a drive command to drive the actuators 105a and 105b to perform diopter adjustment. The diopter is adjusted by changing the respective distances between the image display units 103a and 103b and the lenses 104a and 104b. For example, the diopter detection units 106a and 106b allow the user to adjust the diopter in real time each time the gaze point is changed.

[0027] The control unit 108 is a so-called microcomputer and may be composed of electrical components such as a central processing unit (CPU), a memory for storing programs, and a memory as a work area where the programs are expanded. In this case, the control unit 108 plays a role in generating signals having information for controlling the driving of the actuator 105.

[0028] 4A to 4C are diagrams illustrating the convergence angle and visibility of the eyeballs when gazing at an object. Fig. 4A shows the state when gazing at an object in the real world, and Fig. 4B shows the state when gazing at an object on image display device 100.

[0029] 4A , in the real world, the user is gazing at an object 401 located a distance A away. The lines of sight of the user's left eye 201a and the lines of sight of the user's right eye 201b intersect on the object 401, forming an angle α. The angle α corresponds to the angle of convergence when the user gazes at the object 401. Therefore, in the real world, the distance corresponding to the angle of convergence α and the distance A corresponding to the diopter always coincide with each other.

[0030] On the other hand, in Fig. 4B, the user is gazing at an object 401 located at a distance A away on the image display device 100. The convergence angle α is the same as that in the real world shown in Fig. 4A. The virtual image position 402 is the position of a virtual image when the user views the image display units 103a and 103b through the lenses 104a and 104b (not shown), respectively. The virtual image position 402 is a position that is a distance B away from the left eye 201a and the right eye 201b.

[0031] By providing a convergence angle α for the virtual image position 402, the user visually recognizes it as if it were the object 401. However, unlike the real world, the image display device 100 does not match the distance corresponding to the convergence angle α and the distance B corresponding to the diopter. This is called the contradiction between convergence and diopter adjustment, and if this state continues, the user will experience increased dizziness and fatigue.

[0032] Therefore, as shown in Fig. 4C , the virtual image position 402 can be changed by changing the distance between the image display units 103a and 103b and the lenses 104a and 104b (not shown). For example, when the line of sight detection means detects that the user is gazing at the object 401, the distance between the display units and the lenses is changed so that the virtual image position 402, which is at distance B, becomes distance A. As a result, as shown in Fig. 4A , the distance corresponding to the convergence angle and the distance corresponding to the diopter coincide, creating a more natural state and reducing motion sickness and fatigue.

[0033] In addition, when the distance between the display unit and the lens is changed, it is only necessary to drive at least one of the display unit and the lens.

[0034] The state detection unit 109 in FIG. 1 detects the physiological or psychological state of the user, for example, the heart rate, electrocardiogram, respiration, electrooculography, electrodermal potential, and center of gravity of the user.

[0035] As for the heart rate, the user's heart rate is measured, and the average value of the instantaneous heart rate, the respiratory component in the heart rate fluctuation, and the magnitude of the Mayer wave component in the heart rate fluctuation are determined, and at least one of these is used as the detected value.

[0036] As for the electrocardiogram, the user's electrocardiogram is measured, and the high frequency component (HF), low frequency component (LF), or ratio of HF and LF of the heart rate fluctuation, which is the magnitude of a specific frequency component of the baseline fluctuation of the electrocardiogram, is determined, and at least one of these is used as the detection value.

[0037] As for respiration, the user's breathing frequency, breathing volume, and breathing irregularity are measured, and at least one of these is used as a detection value.

[0038] As the electrooculogram, the number of blinks of the user, the cumulative number, the cumulative number, the rate of change, and the blink interval are measured, and at least one of these is used as the detected value.

[0039] Alternatively, the center of gravity of the user may be measured, and the magnitude of the sway of the center of gravity may be used as the detected value.

[0040] The detection value detected by the state detection unit 109 is output to the state determination unit 110, and the detection value is compared with a predetermined normal value recorded in advance to determine whether the user is experiencing sickness or fatigue. Note that the physiological state or psychological state may be determined using not only one of the electrocardiogram, respiration, electrooculography, electrodermal potential, and center of gravity, but also a combination of two or more of them.

[0041] In the case of skin potential, the determination may be made based on the magnitude of the resistance component, and in the case of center of gravity, the determination may be made based on the fluctuation of the center of gravity.

[0042] If it is determined that the user is experiencing motion sickness or fatigue, the control unit 108 drives the actuators 105a and 105b to move the lenses 104a and 104b toward the image display unit, thereby changing the visibility. The state detection unit 109 and state determination unit 110 do not necessarily have to constitute part of the image display device, and can also be configured to be able to send and receive signals wirelessly. Conversely, by adopting an integrated configuration with the image display device, the user of the head-mounted display can immediately use the state detection function without the need for remote communication.

[0043] The operation of the first embodiment will be described with reference to the flowchart of Fig. 5. A program corresponding to the flowchart of Fig. 5 is stored in a storage unit within the arithmetic processing unit. The control unit 108 represents a CPU, memory, etc., and each process shown in the flowchart of Fig. 5 is realized by the CPU expanding a predetermined program stored in the memory. The control unit 108 may or may not be configured as an integral part of the video display device.

[0044] First, in S501, the state detection unit 109 detects the physiological or psychological state of the user. Next, in S502, the state of motion sickness or fatigue of the user is determined based on the detected value. If the detected value is equal to or greater than a desired value, in S503, a movement amount is calculated to move the lenses 104a and 104b in a direction away from the image display units 103a and 103b. In S504, the control unit 108 drives the actuators 105a and 105b. This movement amount may be determined based on the detected value, or may be increased as the detected value increases and decreased as the detected value decreases. In this way, the displayed parallax image is not focused on and does not appear clear, reducing the amount of information entering the left eye 201a and right eye 201b and relaxing the eye muscles, thereby reducing motion sickness and fatigue.

[0045] If the detected value is equal to or less than the desired value in S502, the gaze detection means 106a and 106b are used to detect the gaze of each of the user's left eye 201a and right eye 201b in S505.

[0046] Next, in S506, the control unit 107 performs processing to determine the gaze point from the line of sight of the left eye 201a and the line of sight of the right eye 201b. In calculating the gaze point, an average value of the line of sight of the left eye 201a and the right eye 201b can be used.

[0047] Next, in S507, the diopter adjustment amount calculation unit 111 performs a process of calculating the diopter adjustment amount at the position of the gaze point of the image displayed on the image display units 103a and 103b. Specifically, the parallax between the image at the gaze point position of the left eye and the image at the gaze point position of the right eye is calculated, and the diopter adjustment amount is derived from depth information corresponding to this parallax. For example, by providing table data that associates the parallax with the diopter adjustment amount, it is possible to calculate the diopter adjustment amount corresponding to the parallax. Alternatively, the diopter adjustment amount calculation unit 111 may calculate the diopter adjustment amount from the parallax using a formula that indicates the relationship between the parallax and the diopter adjustment amount.

[0048] Next, in S508, the control unit 108 drives the actuators 105a and 105b based on the diopter adjustment amount calculated by the diopter adjustment amount calculation unit 111. This causes an operation to change the distance between the image display units 103a and 103b and the lenses 104a and 104b.

[0049] The above operation is performed every time the gaze point is changed, so that the diopter can be adjusted in real time and changed.

[0050] Figure 6 shows the detected values ​​of the user's physiological or psychological state and the lens position in the configuration of Figure 1. As shown in Figure 6, during the period from time t0 to t1, the gaze point is calculated from the gaze detected by the gaze detection means, and diopter adjustment is performed in real time in the directions of arrows 107a and 107b according to depth information of the gaze point. When the state detection value detected by the state detection unit exceeds a predetermined value at time t1, actuators 105a and 105b are moved to move lenses 104a and 104b toward the user and away from image display units 103a and 103b. Thereafter, when the detected value falls below the desired value at time t2, the gaze point is again calculated from the gaze detected by the gaze detection means, and diopter adjustment is performed in real time in the directions of arrows 107a and 107b according to depth information of the gaze point.

[0051] It is also possible to adopt a configuration in which the inverse of this state detection value is used as the evaluation target and the case in which the inverse of the state detection value falls below a certain value is used as the standard, which is also synonymous with "exceeding a predetermined value."

[0052] In this embodiment, the lens 104a corresponding to the left eye 201a and the lens 104b corresponding to the right eye 201b can be driven by independent actuators 105a and 105b, so that it is possible to accommodate users with different visual acuity in the left and right eyes. In other words, it is possible to adopt a configuration in which the position of at least one of the first and second display optical elements is changed.

[0053] According to this embodiment, the diopter is adjusted according to the depth information of the gaze point, and the actuator is driven to move the lenses 104a and 104b in a direction away from the image display units 103a and 103b according to the user's state of sickness or fatigue. This causes the image displayed on the image display unit to appear blurred, reducing the amount of information the user obtains from the image, thereby reducing sickness and fatigue accumulated due to long-term use.

[0054] In the above example, the actuators are moved so that the image display unit and the lens are separated from each other, but the present invention is not limited to this. The actuators may be moved so that the image display unit and the lens are closer to each other, and the same effect can be obtained. In this case, when the value detected by the state detection unit 109 exceeds a predetermined value, the lenses 104a and 104b are moved by the actuators 105a and 105b so that they are closer to the image display units 103a and 103b.

[0055] Note that the above description has been based on a configuration in which diopter adjustment is performed based on depth information of the gaze point detected by the gaze detection means 106a and 106b. However, this is not limiting, and a configuration in which diopter adjustment is performed manually, such as the configuration shown in FIG. 7, may also be used. FIG. 8 shows the detected values ​​of the user's physiological or psychological state and the lens position in the configuration shown in FIG. 7. In this configuration, as shown in FIG. 8, the user manually adjusts the diopter to a position that matches the image being viewed at time t0, and this position is used as the reference position for the lens. When the value detected by the state detection unit exceeds a predetermined value at time t1, the actuator is moved to move the lens toward the user and away from the display unit. Thereafter, when the detected value falls below a desired value at time t2, the actuator is moved to return the lens to the same reference position that was manually adjusted at time t0.

[0056] Second Embodiment A description will be given of the operation method of the actuators 105a and 105b driven by the control unit 108 using the values ​​detected by the state detection unit 109 in the first embodiment described above. The principles and configuration are the same as those in the first example, and therefore will be omitted.

[0057] The operation of the second embodiment will be described with reference to the flowchart of Fig. 9. A program corresponding to the flowchart of Fig. 9 is stored in a storage unit within the arithmetic processing unit. The control unit 108 represents a CPU, memory, etc., and each process shown in the flowchart of Fig. 9 is realized by the CPU expanding a predetermined program stored in the memory.

[0058] First, in S1001, the state detection unit 109 detects the physiological or psychological state of the user. Next, in S1002, the state of motion sickness or fatigue of the user is determined based on the detected value. If the detected value is equal to or less than the desired value, in S1003, the control unit 108 sets the maximum drive speed of the actuators 105a and 105b to A (e.g., 100 mm / s). If the detected value is equal to or greater than the desired value, in S1004, the control unit 108 sets the maximum drive speed of the actuators 105a and 105b to B (e.g., 50 mm / s), which is smaller than A. By changing the maximum drive speed in this way, the amount of change in the user's focus on the image can be suppressed, and the eyes can be prevented from trying to follow sudden changes in focus. This reduces the burden on the user, even when watching images for long periods of time, thereby reducing the increase or accumulation of motion sickness and fatigue.

[0059] Next, in S1005, the gaze detection means 106a and 106b are used to detect the gaze of the left eye 201a and the right eye 201b of the user.

[0060] Next, in S1006, the control unit 108 performs processing to determine a gaze point from the line of sight of the left eye 201 a and the line of sight of the right eye 201 b. The gaze point can be calculated using an average value of the line of sight of the left eye 201 a and the right eye 201 b.

[0061] Next, in S1007, the diopter adjustment amount calculation unit 111 performs a process of calculating the diopter adjustment amount at the position of the gaze point of the image displayed on the image display units 103a and 103b. Specifically, the parallax between the image at the gaze point position of the left eye and the image at the gaze point position of the right eye is calculated, and the diopter adjustment amount is derived from depth information corresponding to this parallax. For example, by providing table data that associates the parallax with the diopter adjustment amount, it is possible to calculate the diopter adjustment amount corresponding to the parallax. Alternatively, the diopter adjustment amount calculation unit 111 may calculate the diopter adjustment amount from the parallax using a mathematical formula that indicates the relationship between the parallax and the diopter adjustment amount.

[0062] Next, in S1008, the control unit 108 drives the actuators 105a and 105b based on the diopter adjustment amount calculated by the diopter adjustment amount calculation unit 111. This changes the distance between the image display units 103a and 103b and the lenses 104a and 104b. The above operation is performed every time the gaze point is changed, so the diopter can be adjusted in real time to change the diopter.

[0063] 10 shows the detected values ​​of the physiological state or psychological state of the user and the maximum drive speed of the actuators 105a, 105b. As shown in Fig. 10, from time t0 to t1, when the value detected by the state detection unit 109 is less than a predetermined value, the maximum drive speed of the actuators 105a, 105b is set to A. From time t1 to t2, when the value detected by the state detection unit 109 is equal to or greater than the predetermined value, the maximum drive speed is set to B, which is smaller than A. From time t2 to t3, when the value detected by the state detection unit 109 is less than the predetermined value, the maximum drive speed is again set to A.

[0064] While the example shown here is one in which the maximum drive speed is changed, the present invention is not limited to this. Alternatively, the control gain or the control period may be changed, and similar effects can be achieved. The control gain refers to the ratio of the control output required to change the actuator by a certain amount. Reducing this ratio reduces responsiveness, suppressing the amount of change in the user's focus on the image, and reducing the tendency of the user's eyes to follow sudden changes in focus. This reduces the burden on the user, even when watching videos for long periods of time, thereby reducing the increase or accumulation of motion sickness and fatigue. The control period refers to the time interval between commands issued in controlling the actuator. Extending this time interval reduces the update frequency, suppressing the amount of change in the user's focus on the image, and reducing the tendency of the user's eyes to follow sudden changes in focus. This reduces the burden on the user, even when watching videos for long periods of time, thereby reducing the increase or accumulation of motion sickness and fatigue. Furthermore, when the value detected by the state detection unit 109 is equal to or greater than a predetermined value, multiple of the maximum drive speed, control gain, and control period may be changed simultaneously.

[0065] Up to this point, an example has been shown in which the maximum drive speed, control gain, and control period are changed depending on whether the detection value of the state detection unit 109 exceeds a predetermined value. However, this is not limiting, and the maximum drive speed, control gain, and control period may be changed depending on the detection value, as shown in the flowchart of FIG. 11 . In this case, first, in S1201, the state detection unit 109 detects the user's physiological or psychological state. Next, in S1202, the control unit 108 sets the maximum drive speed of the actuators 105a and 105b based on the detected value. For example, by providing table data correlating the detected values ​​of the physiological or psychological state with the maximum drive speed, the maximum drive speed can be calculated according to the detected values ​​of the physiological or psychological state. Alternatively, the maximum drive speed may be calculated using a formula showing the relationship between the detected values ​​of the physiological or psychological state and the maximum drive speed. By changing the maximum drive speed in this manner, the amount of change in the user's focus on the image can be reduced, reducing the tendency for the eyes to follow sudden changes in focus. This reduces the burden on the user even when watching videos for long periods of time, making it possible to reduce the increase or accumulation of motion sickness and fatigue, and achieving the same effect as in the example described above.

[0066] Note that the operations from S1203 to S1206 in FIG. 11 are the same as the operations from S1005 to S1008 in the flowchart of FIG. 9, and therefore a description thereof will be omitted.

[0067] 12 shows the detected values ​​of the physiological or psychological state of the user and the maximum drive speed of the actuators 105a, 105b. As shown in FIG. 12, the maximum drive speed of the actuators 105a, 105b is set in stages according to the values ​​detected by the state detection unit 109.

[0068] Although the example shown here is one in which the maximum drive speed is changed, the present invention is not limited to this, and the control gain or control period may be changed to obtain the same effect. Furthermore, when the value detected by the state detection unit 109 is equal to or greater than a predetermined value, more than one of the maximum drive speed, control gain, and control period may be changed simultaneously.

[0069] While the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various modifications within the scope of the present invention are also included. It is also possible to provide a control method for executing each of the above-described steps, and a computer-readable non-transitory program for executing the control method.

[0070] For example, the present invention can be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions.

[0071] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention.

[0072] This application claims priority based on Japanese Patent Application No. 2023-201980, filed November 29, 2023, the entire contents of which are incorporated herein by reference.

[0073] REFERENCE SIGNS LIST 100 Image display device 101 Image acquisition unit 102 Display processing unit 103 Image display unit 104 Lens 105 Actuator 106 Line-of-sight detection means 107 Driving direction 108 Control unit 109 State detection unit 110 State determination unit 111 Depth information calculation unit 201a Left eye 201b Right eye 202 Optical axis 203 Virtual image

Claims

1. An image display device comprising: a first image display unit that displays a first image to a user's right eye; a second image display unit that displays a second image to the user's left eye; a first display optical element corresponding to the first image display unit; a second display optical element corresponding to the second image display unit; and an actuator that changes the position of the display optical elements, further comprising a state detection unit that detects a physiological state or psychological state of a user while using the image display device, and by driving the actuator based on the detection result of the state detection unit, changes at least one of the position of the first display optical element relative to the first image display unit and the position of the second display optical element relative to the second image display unit.

2. The image display device of claim 1, characterized in that when the detection result of the physiological state or psychological state reaches or exceeds a predetermined value, the actuator is driven so that at least one of the first image display unit and the first display optical element, and the second image display unit and the second display optical element, are positioned away from each other.

3. The image display device of claim 1, characterized in that when the detection result of the physiological state or psychological state reaches or exceeds a predetermined value, the actuator is driven so that at least one of the first image display unit and the first display optical element, and the second image display unit and the second display optical element, are positioned closer to each other.

4. An image display device as described in any one of claims 1 to 3, characterized in having a first gaze detection means for performing gaze detection related to the first image display unit, and a second gaze detection means for performing gaze detection related to the second image display unit.

5. The image display device according to claim 4, which is configured to perform the line of sight detection without using the first display optical element and the second display optical element.

6. An image display device as described in any one of claims 1 to 3, further comprising a calculation means for calculating a visibility adjustment amount corresponding to the parallax between the first image and the second image at the position of the gaze point obtained from the first gaze detection means and the second gaze detection means.

7. The image display device according to claim 6, further comprising a control unit which outputs a drive command, and the control unit issues the drive command corresponding to the visibility adjustment amount to the actuator.

8. An image display device according to any one of claims 1 to 3, further comprising: a control gain of said actuator being changed in accordance with a value of the detection result of said physiological state or psychological state.

9. An image display device according to any one of claims 1 to 3, further comprising: a control period of said actuator that is changed in accordance with the value of the detection result of said physiological state or psychological state.

10. An image display device according to any one of claims 1 to 3, further comprising: a maximum drive speed of the actuator that is changed in accordance with the value of the detection result of the physiological state or psychological state.

11. An image display device according to claim 4, wherein the first line-of-sight detection means and the second line-of-sight detection means each include an infrared illumination unit and a line-of-sight detection camera.

12. An image display device as described in any one of claims 1 to 3, characterized in that the state detection unit obtains a detection result of the physiological state or psychological state based on at least one of the number of blinks, cumulative number of blinks, rate of change, and time interval between blinks, which are measured based on the electrooculography of the user of the image display device.

13. An image display device as claimed in any one of claims 1 to 3, characterized in that the state detection unit obtains detection results of the physiological state or psychological state based on the magnitude or ratio of frequency components of baseline fluctuations in the measured electrocardiogram of the user of the image display device.

14. An image display device as described in any one of claims 1 to 3, characterized in that the state detection unit obtains a detection result of the physiological state or psychological state based on the magnitude of the resistance component due to the measured skin potential of the user of the image display device.

15. An image display device as described in any one of claims 1 to 3, characterized in that the state detection unit obtains a detection result of the physiological state or psychological state based on at least one of the measured respiratory frequency, respiratory volume, and respiratory irregularity of the user of the image display device.

16. An image display device as claimed in any one of claims 1 to 3, characterized in that the state detection unit obtains a detection result of the physiological state or psychological state based on the measured center of gravity fluctuation of the center of gravity of the user of the image display device.

17. An image display device as described in any one of claims 1 to 3, characterized in that the state detection unit measures the measured heart rate of the user of the image display device and obtains a detection result of the physiological state or psychological state based on the average instantaneous heart rate, the respiratory component in the heart rate fluctuations, and the Mayer wave component in the heart rate fluctuations.

18. An image display device according to any one of claims 1 to 3, wherein the first display optical element and the second display optical element are lenses.

19. An image display device according to any one of claims 1 to 3, wherein the actuator is an electromagnetic motor or a vibration type actuator.

20. A control method for an image display device comprising a first image display unit that displays a first image to a user's right eye, a second image display unit that displays a second image to the user's left eye, a first display optical element corresponding to the first image display unit, a second display optical element corresponding to the second image display unit, and an actuator that changes the position of the display optical element, wherein the control unit executes a step of obtaining a detection result of a physiological state or psychological state of a user while using the image display device, and a step of driving the actuator based on the detection result to change the position of at least one of the first display optical element and the second display optical element.

21. A non-transitory computer readable program causing the control unit to execute each of the steps recited in claim 20.

Images