Variable focus display device and control method thereof

Abstract

To provide a variable focus display device with which it is possible to move the position of the virtual image to be presented in a depth direction at high speed with low delay, as well as move it smoothly at low speed.SOLUTION: A variable focus display device 1 comprises display means 102a, 102b for displaying a video, and display optical systems 103a, 103b that guide light from the display means to eyes 2a, 2b of an observer and that include a plurality of variable focus elements LCL1-LCLn those focal distances change in accordance with an electric conduction state. The variable focus display device 1 also includes drive means 104a, 104b that move the display means and at least one of the plurality of variable focus elements in the optical axis direction of the display optical systems, and control means 106, 108 that exercise first control for controlling the respective electric conduction states of the plurality of variable focus elements and second control for controlling the drive means. The control means performs a diopter adjustment process that combines the first and second controls, in accordance with an acquired diopter target value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a varifocal image display device such as a VR / AR / MR device. [Background technology]

[0002] The above-mentioned image display devices display images rendered with binocular parallax on left-eye and right-eye displays, thereby presenting three-dimensional (3D) images to a user who views these images through the left-eye and right-eye display optical systems. However, such 3D imaging techniques using binocular parallax can place a strain on the user due to a vergence accommodation conflict (VAC) caused by a mismatch between the convergence distance and focal length of the two eyes. To solve this problem, various variable-focus image display devices have been proposed, which allow the focal length of the display optical system to be changed.

[0003] Patent Document 1 discloses a variable-focus image display device that uses a lens assembly including multiple liquid crystal lenses. This image display device controls the focal lengths (i.e., diopter) of the left-eye and right-eye lens assemblies by turning on and off the power to the liquid crystal lenses in the left-eye and right-eye lens assemblies. This allows the user to adjust the positions of the virtual images seen by the left and right eyes through the lens assemblies. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 10,852,619 Summary of the Invention [Problem to be solved by the invention]

[0005] FIG. 3 shows an example of the diopter adjustment range in the image display device disclosed in Patent Document 1. The current pattern numbers on the horizontal axis are current pattern numbers representing combinations of ON and OFF current to multiple liquid crystal lenses. The number of combinations increases as the number of liquid crystal lenses increases. The vertical axis shows the focal length (diopter) for each current pattern. The current pattern numbers on the horizontal axis are arranged in ascending order of the focal length on the vertical axis. Note that FIG. 3 shows the results of a simulation performed by the inventor of the present invention assuming the number of liquid crystal lenses to be six.

[0006] As can be seen from this diagram, the image display device disclosed in Patent Document 1 uses a mechanism for discretely adjusting diopter by combining the ON / OFF of power to multiple liquid crystal lenses. Therefore, in principle, it is not possible to adjust the focal length with high resolution. As a result, it is conceivable that a solution would be used in which a virtual image that appears in a position different from its intended location is displayed as if it were in the correct position using display processing. However, this solution may result in either the convergence distance or the focal length not being at an ideal value, which could cause the user to feel uncomfortable.

[0007] Furthermore, the image display device disclosed in Patent Document 1 can electrically and sequentially switch between adjacent focal length values ​​quickly and efficiently, enabling the position of the virtual image of a virtual object viewed by the user to move in the depth direction at high speed and with low latency. However, it is difficult to smoothly move the position of the virtual image at low speed. This is because if the speed at which the virtual image's position is moved is slow, the virtual image's step-like movement in the depth direction becomes noticeable, jumping over the difference in focal length between the current conduction patterns of adjacent numbers. According to the simulation results shown in Figure 3, it is difficult to completely eliminate the large difference (gap) in focal length between the current conduction patterns of numbers 16 and 17 through design.

[0008] The present invention provides a variable-focus display device that is capable of moving the position of a presented virtual image in the depth direction at high speed and with low delay, and also capable of moving it smoothly at low speed. [Means for solving the problem]

[0009] A variable-focus display device according to one aspect of the present invention includes a display means for displaying an image, a display optical system that guides light from the display means to the viewer's eye and includes multiple variable-focus elements whose focal length changes depending on the state of conduction, a drive means for moving the display means and at least one of the multiple variable-focus elements along the optical axis of the display optical system, and a control means for performing a first control that controls the state of conduction of each of the multiple variable-focus elements and a second control that controls the drive means. The control means performs a diopter adjustment process that combines the first control and the second control in accordance with an acquired diopter target value.

[0010] Another aspect of the present invention is a control method applied to a variable-focus display device having a display means for displaying an image, a display optical system that guides light from the display means to the viewer's eye and includes multiple variable-focus elements whose focal length changes depending on their energization state, and drive means for moving the display means and at least one of the multiple variable-focus elements along the optical axis of the display optical system. The control method includes the steps of causing the variable-focus display device to perform a first control that controls the energization state of each of the multiple variable-focus elements, and a second control that controls the drive means, and is characterized in that the control method causes the variable-focus display device to perform a diopter adjustment process that combines the first control and the second control in accordance with an acquired diopter target value. Note that a program for causing a computer to execute processing in accordance with the control method also constitutes another aspect of the present invention. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a variable focus display device that can move the position of a virtual image presented through a display optical system in the depth direction at high speed and with low delay, and can also move it smoothly at low speed. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram showing the configuration of a video display device according to an embodiment; [Figure 2] 1 is a diagram showing the configuration of a video display device according to a first embodiment. [Figure 3] FIG. 10 is a diagram showing an example of a diopter adjustment range in a conventional image display device. [Figure 4] FIG. 3 is a diagram showing a diopter adjustment range of the image display device in the first embodiment. [Figure 5] 10 is a flowchart showing a fixed-position diopter adjustment process in the second embodiment. [Figure 6] 10 is a flowchart showing a fixed-position diopter adjustment process as a modified example of the second embodiment. [Figure 7] 10A and 10B are diagrams for explaining a control algorithm for moving the virtual image position in the depth direction at a low speed. [Figure 8] 11 is a flowchart showing a low-speed diopter adjustment process in the third embodiment. [Figure 9] FIG. 10 is a diagram showing a case where there are a plurality of control command values ​​for a diopter target value. [Figure 10] 1 is a diagram showing the configuration of a display optical system in an image display device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] 1 shows the configuration of an image display device 1 as a variable-focus display device used in an AR (Augmented Reality) device, an MR (Mixed Reality) device, a VR (Virtual Reality) device, etc., which is an embodiment of the present invention. The image display device 1 is worn by a user as an observer on the head or worn like glasses, and display optical systems 103a and 103b can be placed near the user's left eye 2a and right eye 2b. In FIG. 1, the reference numerals of components provided for the left eye 2a are marked with a, and the reference numerals of components provided for the right eye 2b are marked with a b.

[0015] The image display device 1 includes an image data acquisition unit 100, a display processing unit 101, right-eye and left-eye displays (display means) 102a, 102b, display optical systems 103a, 103b, line-of-sight detection units 105a, 105b, an object position calculation unit 108, and a diopter change instruction unit 106. The object position calculation unit 108 and the diopter change instruction unit 106 correspond to control means, and are each implemented by separate computers or a single computer.

[0016] The video data acquisition unit 100 acquires video data for display from an external device or via a network. The display processing unit 101 performs processing such as adjusting the display magnification of the acquired video data. The processed video data is sent to the left-eye and right-eye displays 102a and 102b for display. The display processing unit 101 displays a left-eye image and a right-eye image having mutual parallax on the left-eye and right-eye displays 102a and 102b. The display optical systems 103a and 103b guide light from the displays 102a and 102b to the user's left eye 2a and right eye 2b, presenting virtual images of objects in the left-eye and right-eye images to the left eye 2a and right eye 2b.

[0017] The gaze detection units 105a and 105b detect the gaze directions of the left eye 2a and the right eye 2b as gaze information using a method such as corneal reflex. The object position calculation unit 108 estimates (calculates) the gaze target in the image displayed on the display units 102a and 102b from the gaze information of the left eye 2a and the right eye 2b. Then, the object position calculation unit 108 calculates information on the distance (target diopter value) at which the virtual object being gazed at appears as a virtual image, and sends a diopter target command including this information to the diopter change instruction unit 106. The gaze target is estimated, for example, by calculating gaze point coordinates on the display units 102a and 102b from the gaze directions of the left eye 2a and the right eye 2b obtained by the gaze detection units 105a and 105b. The distance information is calculated, for example, by comparing the gaze point coordinates with depth map information for each frame of the image to find the distance.

[0018] The image display device 1 can be used by connecting it to an external computer (not shown) via a wired or wireless connection. In this case, the image display device 1 does not necessarily need to include components other than the displays 102a and 102b and the display optical systems 103a and 103b, and calculations and image editing can be performed by a computer (not shown). In this case, the display and display optical system unit and a separate computer constitute a variable-focus display device.

[0019] The diopter change instruction unit 106 controls the driving units (driving means) 104a and 104b and the display optical systems 103a and 103b based on a diopter target command corresponding to the distance at which the virtual image is to appear.

[0020] In FIG. 1, the display optical systems 103a and 103b are fixed on optical axes 103a1 and 103b1, respectively. Although the display optical systems 103a and 103b are shown as a single lens in the figure, they actually include multiple liquid crystal lenses LCL1 to LCLn (n=2, 3, 4, ...) as shown in FIG. 10. The liquid crystal lenses are variable-focus elements whose focal length changes when their energized state (power ON / OFF) is switched. Changing the focal length of each liquid crystal lens can change the focal length of the display optical systems 103a and 103b. Note that the display optical systems 103a and 103b may include at least one optical lens in addition to the multiple liquid crystal lenses.

[0021] 1, the display devices 102a and 102b are moved by the drive units 104a and 104b in the optical axis directions (indicated by arrows 104a1 and 104b1) along which the optical axes 103a1 and 103b1 of the display optical systems 103a and 103b extend. By controlling the drive units 104a and 104b, the distances between the display devices 102a and 102b and the display optical systems 103a and 103b on the optical axes 103a1 and 103b1 can be changed, thereby changing the focal lengths of the left and right display optical systems 103a and 103b. The drive units 104a and 104b may drive only parts of the display optical systems 103a and 103b, or may drive the display devices 102a and 102b and parts of the display optical systems 103a and 103b together. In the following, an example will be described in which the displays 102a and 102b are moved in the optical axis direction by the driving units 104a and 104b, but the focal length may also be changed by moving at least one of the multiple liquid crystal lenses and the displays included in the display optical system in the optical axis direction. In such a configuration, the multiple liquid crystal lenses may be unitized and moved collectively by the same driving unit, or may be moved independently by individual driving units. [Example]

[0022] Fig. 2 shows the configuration of the image display device of Example 1. In Fig. 2, the diopter change instruction unit 106, object position calculation unit 108, and display processing unit 101 are the same as in Fig. 1. In addition, the displays 102a and 102b and the drivers 104a and 104b shown in Fig. 1 are collectively shown as the display 102 and the driver 104, respectively, in Fig. 2. Furthermore, a lens current control unit 109 and a drive control unit 110, which were not shown in Fig. 1, have been added to Fig. 2.

[0023] The lens current control unit 109 controls ON / OFF of current to each of the plurality of liquid crystal lenses LCL1 to LCLn included in the display optical systems 103a and 103b shown in Fig. 10. The drive control unit 110 controls the driving of the drive unit 104 (movement of the display 102). The lens current control unit 109 and the liquid crystal lenses LCL1 to LCLn form a first adjustment unit 201, and the drive control unit 110 and the drive unit 104 form a second adjustment unit 202. The control of the first adjustment unit 201 (liquid crystal lenses LCL1 to LCLn) corresponds to first control, and the control of the second adjustment unit 202 (drive unit 104) corresponds to second control.

[0024] The diopter change instruction unit 106 receives a diopter target command including distance information indicating the distance at which a virtual image is to appear from the object position calculation unit 108, and obtains a power supply command and a drive command corresponding to the distance information from the control table 107. Combinations of power supply commands and drive commands for a plurality of different distances (diopter target values) are stored in advance in the control table 107. The diopter change instruction unit 106 transmits the power supply command to the lens power supply control unit 109 and transmits the drive command to the drive control unit 110. The power supply command includes information on a combination of liquid crystal lenses to be powered on among the liquid crystal lenses LCL1 to LCLn. The drive command includes information on the speed and target position of the drive unit 104. The lens power supply control unit 109 turns on power to the liquid crystal lenses to be powered on in accordance with the received power supply command. The drive control unit 110 controls the drive unit 104 to the target position at a speed according to the received drive command. When the diopter adjustment is completed in this manner, the diopter change instruction unit 106 transmits an image display command to the display processing unit 101. This causes the display device 102 to start displaying the image.

[0025] FIG. 4 shows the diopter adjustment range of the image display device in Example 1. FIG. 4 shows the results of a simulation in which the number of liquid crystal lenses is six. The horizontal axis indicates the current pattern number, which is a combination of ON and OFF current to multiple liquid crystal lenses stored in the control table 107. The vertical axis indicates the focal length for each current pattern. The current pattern numbers are arranged in ascending order of the focal length on the vertical axis. FIG. 4 shows the focal length distribution obtained by adding mechanical movement of the display device 102 by the second adjustment unit 202 to the focal length distribution obtained by the first adjustment unit 201. The black dots in the figure indicate the focal lengths of both eyes obtained by the first adjustment unit 201. The solid lines extending vertically from the black dots in the figure indicate the range of focal lengths adjustable by the second adjustment unit 202. In other words, the figure shows the change in focal length when the display device 102 is moved from one end of its movable stroke to the other end under the control of the drive unit 104.

[0026] Point A is the focal length obtained by the energization pattern of number 17 and one end of the drive stroke of the drive unit 104 (the movement stroke of the display 102). Point B is the focal length obtained by the energization pattern of number 16 and the other end of the movement stroke of the display 102. In Figure 4, point B is above point A. Therefore, the difference (gap) in focal length between the adjacent energization patterns of numbers 16 and 17 is filled. In other words, by adding the second adjustment unit to the first adjustment unit, it is possible to set all focal lengths from 10 mm to 1000 mm shown in Figure 3.

[0027] Next, the relationship between the first adjustment unit 201 and the second adjustment unit 202 will be described. As described above, the first adjustment unit 201 has a liquid crystal lens as an electrically variable-focus lens controlled by the applied electric signal, which is binary, ON or OFF. A liquid crystal lens is an element capable of switching the focal length by controlling the orientation of liquid crystal molecules by applying a voltage to a liquid crystal cell in which the liquid crystal molecules are oriented. For example, a polarized direct flat lens can be used as the liquid crystal lens, which functions as a convex lens or a concave lens depending on the direction of circular polarization of incident light. Polarized direct flat lenses are thin and lightweight, and multiple polarized direct flat lenses can be stacked to form a compact unit. Increasing the number of variable-focus lenses increases the number of combinations of ON and OFF states of the electric current to the lenses, thereby increasing the focal length options of the display optical system 103. Note that liquid crystal lenses other than polarized direct flat lenses may also be used.

[0028] On the other hand, in the second adjustment unit 202, the driver 104 is an actuator that can be controlled with high response speed, such as a piezoelectric actuator or vibration motor that can be driven linearly using piezoelectric ceramics, an electromechanical energy conversion element. These actuators utilize the inverse piezoelectric effect, in which inorganic materials deform and generate displacement when an electric field is applied to piezoelectric ceramics, resulting in a fast response time until activation. In particular, a multilayer piezoelectric actuator, in which piezoelectric ceramic sheets are stacked in the thickness direction, can shorten the total drive time until the driven member is precisely positioned. However, even the drive time of a multilayer piezoelectric actuator is often longer than the control time of the first adjustment unit 201, which uses electrical orientation control. In other words, if the controlled displacement amount of the driver 104 is small, the control time of the first adjustment unit 201 may be shorter, but the control time of the second adjustment unit 202 is generally longer.

[0029] The first adjustment unit 201, which uses a liquid crystal lens, is suitable for greatly changing the focal length in a short period of time within the focal length range shown in FIG. 3. On the other hand, the second adjustment unit 202, which uses a mechanical drive, takes longer to control than the first adjustment unit, but is suitable for changing the focal length with higher resolution (i.e., for finer diopter adjustment). For this reason, in the variable focus system of this embodiment, focal length adjustment is performed by electrical coarse adjustment using the first adjustment unit 201 and mechanical fine adjustment using the second adjustment unit 202. This allows the drive stroke of the second adjustment unit 202 to be shortened, thereby minimizing the difference between the control time of the first adjustment unit 201 and the control time of the second adjustment unit 202.

[0030] The following is a design consideration, and explains the balance between the number of liquid crystal lenses in first adjustment unit 201 and the drive stroke length of drive unit 104 in second adjustment unit 202. In principle, it is possible to increase the focal length resolution if a large number of liquid crystal lenses can be used in the electrical focal length adjustment performed by first adjustment unit 201. However, liquid crystal lenses are generally very expensive, and there is a limit to the number of liquid crystal lenses that can be used.

[0031] On the other hand, in the mechanical focal length adjustment performed by the second adjustment unit 202, if the total drive time required to control the positioning of the display 102, which is the driven member, is as long as 100 milliseconds, this is insufficient in terms of performance. Generally, the image display frequency at which a user perceives a clear display screen is about 30 Hz. Therefore, when displaying images while continuously changing the focal length, it is preferable that the control of the drive unit 104 be completed within the time when the image display frequency is 30 Hz or higher. Taking this display frequency into consideration, the drive stroke length of the drive unit 104 should be limited.

[0032] Therefore, it is desirable to balance the number of liquid crystal lenses in the first adjustment unit 201 and the drive stroke length of the second adjustment unit 202 so as to keep the time required for mechanical fine adjustment of the focal length by the second adjustment unit 202 short. [Example]

[0033] A second embodiment will be described with reference to FIG. 5. FIG. 5 is a flowchart showing a fixed-position diopter adjustment process until a virtual image is moved to a certain position and an image is displayed. The diopter change instruction unit 106 executes this process according to a computer program. In this embodiment, the lens current control unit 109 and the drive control unit 110 control the liquid crystal lens and the drive unit 104 in two control modes (modes 1 and 2). Mode 1 (first mode) is a drawing mode in which a virtual image is displayed at a position where the convergence distance and focal length of both eyes are completely aligned. Mode 2 (second mode) is a drawing mode in which the display of a virtual image is allowed to start from a state in which there is a slight discrepancy between the convergence distance and focal length of both eyes.

[0034] In step S1, the diopter change instruction unit 106 acquires a diopter target value (diopter target command) from the object position calculation unit 108. The diopter change instruction unit 106 acquires, from the control table 107, control command values ​​(energization command and drive command) corresponding to the diopter target value.

[0035] Next, in step S2, the diopter change instruction unit 106 calculates the amount of mechanical fine adjustment (amount of displacement) to be performed by the second adjustment unit 202. Specifically, the current position of the drive unit 104 (display 102) is detected, and the amount of displacement is calculated from the difference between the current position and the target position indicated by the control command value.

[0036] Next, in step S3, if the displacement amount calculated in step S2 is smaller than a predetermined value and the control time until the mechanical fine adjustment is completed is sufficiently short, the diopter change instruction unit 106 selects mode 1. If the displacement amount is equal to or greater than a predetermined value and the control time until the mechanical fine adjustment is completed is long, the diopter change instruction unit 106 selects mode 2. Note that the mode may be selected based on factors other than the magnitude of the mechanical displacement amount; for example, the user may select either mode 1 or mode 2 depending on the content of the video content, or mode 1 or 2 may be automatically selected as an application setting.

[0037] In step S4, the diopter change instruction unit 106 performs control in mode 1. In mode 1, the diopter change instruction unit 106 activates both the electrical coarse adjustment by the first adjustment unit 201 and the mechanical fine adjustment by the second adjustment unit 202 in parallel (simultaneously). When the first adjustment unit 201 is activated, the diopter change instruction unit 106 sends a power-on command to the lens current control unit 109, which turns on an electrical signal to the liquid crystal lens to be powered. When the second adjustment unit 202 is activated, the diopter change instruction unit 106 sends a drive command to the drive control unit 110, which causes the drive unit 104 to move the display unit 102. The electrical coarse adjustment by the first adjustment unit 201 is completed before the mechanical fine adjustment by the second adjustment unit 202. Even when the diopter change instruction unit 106 receives a signal indicating the completion of the electrical coarse adjustment from the first adjustment unit 201, it goes into a standby state in which it keeps the liquid crystal lens energized. When the mechanical fine adjustment by the second adjustment unit 202 is completed, the convergence distance and focal length of both eyes become the same. When the adjustments by both the first and second adjustment units 201, 202 are completed, the diopter change instruction unit 106 allows the display processing unit 101 to display on the display 102 (video display ON), and ends processing in mode 1.

[0038] As described above, mode 2 is selected when the control time for mechanical fine adjustment by second adjustment unit 202 is long. Specifically, mode 2 is selected when the control time does not satisfy the conditions of the video display frequency. In step S5, diopter change instruction unit 106 performs control in mode 2. In mode 2, as in mode 1, diopter change instruction unit 106 activates both electrical coarse adjustment by first adjustment unit 201 and mechanical fine adjustment by second adjustment unit 202 in parallel (simultaneously).

[0039] In mode 2, too, the electrical coarse adjustment by the first adjustment unit 201 is completed before the mechanical fine adjustment by the second adjustment unit 202. However, in mode 2, when the diopter change instruction unit 106 receives a signal indicating the completion of the electrical coarse adjustment from the first adjustment unit 201, it waits for a predetermined time before allowing the display device 102 to display an image. That is, the image display starts while the mechanical fine adjustment by the second adjustment unit 202 (control of the drive unit 104) continues. The reason for waiting for a predetermined time is to advance the mechanical fine adjustment by the second adjustment unit 202 as much as possible during the waiting time, as long as it is within a range in which the user does not perceive a display delay, before displaying the image. This waiting time makes it possible to display the image after the convergence distance and focal length of both eyes are approximately equal.

[0040] Note that when the image display starts after the waiting time, the user may notice a slight shift in the focal length in the depth direction and feel uncomfortable. To avoid this, the display processing unit 101 may perform processing such as editing the image so as to offset the slight change in focal length after the image display starts.

[0041] When the process of step S4 or step S5 is completed, the diopter change instruction unit 106 ends this process.

[0042] (Variation) 6 is a flowchart showing a fixed-position diopter adjustment process as a modified example of Example 2, and shows a process that can provide a quicker diopter switching function than Example 2. Steps S1 to S4 are the same as in FIG. 5, and step S5′ is step S5 of FIG. 5 with one additional determination process.

[0043] That is, in this modified example, the diopter change instruction unit 106 determines whether a new diopter target value has been acquired while the second adjustment unit 202 is continuing its mechanical fine adjustment in step S5′ (before the fine adjustment is completed). The new diopter target values ​​are successively inserted at random timings according to the movement of the user's line of sight. When the new diopter target value is acquired, the diopter change instruction unit 106 forcibly terminates the processing of mode 2 without confirming that the processing of mode 2 has been completed to the end (i.e., before the fine adjustment is completed). Then, the process returns to step S2 to acquire a control command value corresponding to the new diopter target value, the next control mode is selected in step S3, and control of the first and second adjustment units 201 and 202 is started in step S4 or S5′.

[0044] Furthermore, after step S4 is completed and if there is no interruption of a new diopter target value in step S5', the diopter change instruction unit 106 determines whether or not a new diopter target value has been acquired in step S6. If a new diopter target value has been acquired, the process returns to step S2; if not, the process ends. [Example]

[0045] Before describing the processing in the third embodiment, a control algorithm for moving a virtual image in the depth direction at a slow speed will be described with reference to FIG. 7. FIG. 7 shows a control method for increasing the focal length of both eyes from 50 mm to 200 mm. The black circles and the solid lines extending from them in the figure have the same meanings as in FIG. 4. The arrows in the figure indicate the route by which the focal length increases from 50 mm (sp in the figure) to 200 mm (e5 in the figure). Specifically, in the change in focal length indicated by the upward arrow, the driver 104 of the second adjustment unit 202 is driven at a slow speed. For example, the driver 104 is controlled in the energization pattern numbered 14 to increase the focal length from sp to e1. ​​During this time, the user sees a slowly moving virtual image. The same applies to the control indicated by the upward arrow in the energization patterns numbered 15 and onward.

[0046] On the other hand, the right-pointing arrow indicates a case where both the control of the liquid crystal lens by the first adjustment unit 201 and the control of the drive unit 104 by the second adjustment unit 202 are executed. The focal length is the same before and after the right-pointing arrow, but the control command value (drive command) for the drive unit 104 is different between the focal length e1 obtained by controlling the drive unit 104 in the current conduction pattern of number 14 and the focal length s2 obtained by controlling the drive unit 104 in the current conduction pattern of number 15. In this embodiment, while the control indicated by this right-pointing arrow is being executed, the image display is temporarily turned off. Therefore, the user basically does not see a virtual image during this time. The same applies to the control indicated by the right-pointing arrow in the current conduction patterns from number 16 onwards.

[0047] These controls indicated by the horizontal and vertical arrows are alternately combined to form one subsequence. By repeating the above subsequence within the overall main sequence of (previous diopter adjustment) → sp → e1 → s2 → ... → s5 → ep, the focal length can be changed at a slow speed, and the virtual image seen by the user can be moved slowly in the depth direction.

[0048] Fig. 8 shows the slow diopter adjustment process in this embodiment, in which a virtual image is moved in the depth direction at a slow speed while an image is displayed. The symbols indicating the focal lengths shown in Fig. 7 are used here. The diopter change instruction unit 106 executes this process in accordance with a computer program.

[0049] In step S11, the diopter change instruction unit 106 acquires information on the movement start point (focal length sp) and movement end point (focal length ep) of the virtual image as diopter target values.

[0050] Next, in steps S12 and S13, the diopter change instruction unit 106 determines a route for changing the focal length by repeating the subsequences of steps S14 to S16, which will be described later, based on the movement start point (sp) and movement end point (ep). Specifically, it determines the number of times (predetermined number of times) N the subsequence will be repeated, and the diopter target value of the movement start point (si: i=2, 3, . . . , N) in each subsequence.

[0051] When the preparations up to this point are complete, in step S14, the diopter change instruction unit 106 acquires a control command value corresponding to the diopter target value of the initial movement start point (sp) from the control table 107. Then, the image display is temporarily turned off and the first and second adjustment units 201 and 202 are activated. As a result, the virtual image moves at high speed from its position before the start of this process to the movement start point (sp). After the position of the virtual image reaches the movement start point (sp), the diopter change instruction unit 106 transmits a signal to the display device 102 to permit image display, causing the image to be displayed. This allows the user to begin observing the virtual image.

[0052] Next, in steps S15 and S16, the diopter change instruction unit 106 causes the second adjustment unit 202 to start controlling the drive unit 104 to change the focal length at a low speed. While the virtual image is moving at a low speed, the display processing unit 101 causes the display device 102 to display an image that has been rendered so that the convergence distance always matches the focal length of both eyes. When the position of the virtual image reaches the movement end point (e1), the second adjustment unit 202 notifies the diopter change instruction unit 106 that the control is complete.

[0053] When the subsequence from step S14 to step S16 is completed, if it is determined in step S17 that the number of subsequences i is smaller than the number of repetitions N determined in step S12, the process returns to step S14 and the next subsequence is performed. For example, in step S14 of the second subsequence, the diopter change instruction unit 106 acquires a control command value corresponding to the diopter target value at the movement start point (s2), and when the movement end point (e2) is reached in step S16, the diopter change instruction unit 106 performs the determination in step S17 again. Then, when the number of subsequences i reaches the number of repetitions N, the diopter change instruction unit 106 terminates this process. [Example]

[0054] FIG. 9 shows the diopter adjustment range of the image display device of Example 4, with the horizontal axis representing the current pattern number and the vertical axis representing the focal length. The current pattern numbers are arranged in ascending order of the focal length on the vertical axis. The meanings of the black circles and the solid lines extending from them in the figure are the same as those in FIG. 4.

[0055] 9, the focal length L indicated by the dotted line is within the range of focal lengths that can be obtained by mechanical fine adjustment by the second adjustment unit 202 in the current conduction patterns numbered 15 and 16. This means that there are two possible control command values ​​(i.e., two combinations of the control by the first adjustment unit 201 and the control by the second adjustment unit 202) that correspond to the focal length L.

[0056] At this time, the diopter change instruction unit 106 calculates the amount of displacement due to the mechanical fine adjustment by the second adjustment unit 202. Specifically, the diopter change instruction unit 106 acquires the current position of the drive unit 104 and calculates the amount of displacement from the difference between the current position and the target position of the drive unit 104 corresponding to the new control command value. In the example of FIG. 9, the amounts of displacement by the second adjustment unit 202 in the current conduction patterns numbered 15 and 16 are calculated. The diopter change instruction unit 106 then automatically selects the smaller of the calculated amounts of displacement, i.e., the control of the second adjustment unit 202 with the shorter control time for the mechanical fine adjustment, and the current conduction pattern of the first adjustment unit 201 combined with this. This makes it possible to reduce the delay in the completion of the mechanical fine adjustment by the second adjustment unit 202 relative to the completion of the electrical coarse adjustment by the first adjustment unit 201. (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0057] The embodiments described above are merely representative examples, and various modifications and alterations are possible to each embodiment when implementing the present invention. [Explanation of symbols]

[0058] 1. Video display device 100 Video data acquisition unit 101 Display processing unit 102a,102b Display 103a,103b Display optical system 104a, 104b Drive unit 106 Diopter change instruction section 108 Object position calculation section

Claims

1. a display means for displaying an image; a display optical system that guides light from the display means to the viewer's eye, the display optical system including a plurality of variable focus elements whose focal lengths change depending on the energized state; a driving means for moving the display means and at least one of the plurality of variable-focus elements in the direction of the optical axis of the display optical system; a control unit that performs a first control to control a current-carrying state of each of the plurality of variable-focus elements and a second control to control the driving unit, The variable focus display device is characterized in that the control means performs diopter adjustment processing by combining the first control and the second control in accordance with the acquired diopter target value.

2. The variable focus display device according to claim 1, characterized in that, in the first control, the control means selects a combination to be used from among a plurality of combinations of the energization states of each of the plurality of variable focus elements according to the diopter target value.

3. 3. The variable focus adjustment device according to claim 1, wherein the control means performs the second control to adjust the diopter more precisely than the first control.

4. 4. The variable-focus display device according to claim 1, wherein the control means starts the first control and the second control in parallel in the diopter adjustment process.

5. A variable focus display device as described in any one of claims 1 to 4, characterized in that the control means has, as modes of the visibility adjustment process, a first mode in which an image is displayed on the display means after both the first control and the second control are completed, and a second mode in which an image is displayed on the display means after the first control is completed but before the second control is completed.

6. 6. A variable focus display device according to claim 5, wherein when the control means acquires a new diopter target value before the second control is completed in the second mode, the control means starts the first and second controls anew before the second control is completed.

7. 7. The variable-focus display device according to claim 1, wherein the control means repeatedly executes the process of performing the first and second controls and then the second control a predetermined number of times.

8. A variable focus display device as described in any one of claims 1 to 7, characterized in that when there are multiple combinations of the first control and the second control for the diopter target value, the control means uses a combination that includes the second control with a shorter control time of the drive means.

9. The display means and the display optical system are provided for the right and left eyes, 9. The variable-focus display device according to claim 1, wherein the control means performs the first and second controls on the display means and the display optical systems for the right and left eyes.

10. 10. The variable focus display device according to claim 9, wherein the right eye display means and the left eye display means display images having a parallax therebetween.

11. a detection means for detecting the gaze direction of the eye, 11. The variable-focus display device according to claim 1, wherein the control means acquires the diopter target value according to the detected line-of-sight direction.

12. 12. The variable-focus display device according to claim 1, wherein the plurality of variable-focus elements are liquid crystal lenses.

13. 12. The variable focus display device according to claim 1, wherein the plurality of variable focus elements are polarized direct flat lenses.

14. 14. A variable-focus display device according to claim 1, wherein the driving means is a vibration motor.

15. A control method for a variable-focus display device having a display means for displaying an image, an optical system for guiding light from the display means to the eye of a viewer, the display optical system including a plurality of variable-focus elements whose focal length changes depending on the energization state, and a drive means for moving the display means and at least one of the plurality of variable-focus elements in the direction of an optical axis of the display optical system, The variable focus display device includes: a step of performing a first control for controlling a current-carrying state of each of the plurality of variable-focus elements; and performing a second control for controlling the driving means, A control method comprising: performing a diopter adjustment process that combines the first control and the second control in accordance with the acquired diopter target value.

16. A program causing a computer to execute a process according to the control method of claim 15.

Images