Focus adjustment for multi-plane head mounted displays
Through the dynamic focal length adjustment technology of multi-plane head-mounted display, visual fatigue and nausea caused by conflicts in the existing technology are solved, and a more realistic virtual reality experience and user comfort is achieved.
Patent Information
- Application Number
- CN202210141252.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2017-02-21
- Filing Date
- 2018-02-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2038-02-20
AI Technical Summary
When simulating a virtual environment, existing head-mounted displays are prone to visual fatigue and nausea caused by the conflict of convergence and regulation, and cannot correctly compensate for vergence and focal length conflict.
Using a multi-plane head-mounted display, dynamically adjusts the distance between the displays and the lens shape through at least one processor, two or more electronic displays and optical blocks, combined with an eye tracking system and a zoom actuator block to match the The user's gaze and depth of radiating enables the focal length adjustment of the virtual scene.
It effectively reduces visual fatigue and nausea, provides a more realistic virtual reality experience, and improves user comfort and immersion by dynamically adjusting the focal length and depth of field blur.
Smart Images

Figure CN114326128B_ABST
Abstract
Description
[0001] This application is a divisional application of an application filed on February 20, 2018, with application number 201880013205.X, and invention name “Focus-adjustable multi-plane head-mounted display”.
[0002] background
[0003] The present disclosure relates generally to enhancing images from electronic displays, and in particular to multi-plane displays having two or more display planes located at different optical distances that can be dynamically adjusted based on position within a scene presented by a head-mounted display.
[0004] Head-mounted displays can be used to simulate artificial or virtual environments. For example, stereoscopic images can be displayed on an electronic display inside a headset to simulate the illusion of depth, and head tracking sensors can be used to estimate which part of the artificial environment the user is looking at. However, such simulations can lead to visual fatigue and nausea caused by the inability of existing headsets to properly provide or otherwise compensate for vergence and accommodation conflicts.
[0005] Overview
[0006] A multi-plane head mounted display (HMD) includes two or more virtual display planes located at optical distances for each eye, which can be dynamically adjusted based on the position of the user viewing a scene presented by the HMD. For example, the scene is presented on two or more electronic display elements (e.g., screens) of the HMD. A zoom system (e.g., an element that mechanically changes the distance between a lens system in an optics block and the electronic display element, an element that changes the shape of one or more lenses in a lens system in the optics block, etc.) is used to adjust the focal length of the optics block that directs image light from the electronic display element to the user's eyes based on the position or object in the scene the user is looking at.
[0007] In one embodiment, a head mounted display (HMD) includes (for each eye of a user) a first electronic display located at a first optical distance from an optical block of the HMD and a second electronic display located at a second optical distance from the optical block. Each electronic display is configured to display a different virtual plane of a virtual scene to a user wearing the HMD. In one embodiment, objects in the virtual scene are mapped to scene geometry or depth information provided by a content creator, and each electronic display is mapped to a different subset of objects based on their scene geometry. For example, a first electronic display displaying a first virtual display plane located at a first optical distance displays objects within the virtual scene that are mapped to scene geometry at or near the first virtual display plane at the first optical distance. Thus, each subsequent electronic display displays different objects based on the location of the virtual display plane and the scene geometry. Thus, virtual scene content from each virtual display plane is combined for viewing by a viewing user. Thus, at least one of the electronic displays is at least translucent to allow image light of the virtual scene to be combined for viewing by a user wearing the HMD.
[0008] In an optional embodiment, a head mounted display (HMD) includes a single electronic display (for each eye of the user) that sequentially displays each virtual display plane. As described above, objects in the virtual scene are mapped to the scene geometry of the virtual scene, and each of the two or more virtual planes displays a different subset of the objects based on the scene geometry, but sequentially rather than simultaneously as previously described.
[0009] In addition, the HMD automatically adjusts its focus based on the position within the virtual scene presented by the virtual reality headset that the user is viewing. The virtual scene is presented on one or more electronic displays, and a focal length of an optical block that directs image light from the electronic display to the user's eyes is adjusted based on the position or object in the virtual scene that the user is viewing using a zoom element (e.g., an element that mechanically changes the distance between a lens system in the optical block and an electronic display element, an element that changes the shape of one or more lenses in the lens system in the optical block, etc.). For example, the HMD tracks the user's eyes to approximate a gaze line and determines a gaze point including a convergence depth as an estimated intersection of the gaze lines. The gaze point identifies an object or focal plane of a particular frame of the virtual scene presented to the user by the HMD.
[0010] Embodiments according to the invention are disclosed in the attached claims, in particular for HMDs and headphones, wherein any feature mentioned in one claim category, such as HMD, may also be claimed in another claim category, such as headphones, systems, methods, storage media or computer program products. Dependencies or reverse references in the attached claims are selected for formal reasons only. However, any subject matter resulting from a prudent reverse reference (in particular multiple dependencies) to any previous claim may also be claimed, so that any combination of claims and their features is disclosed and may be claimed, regardless of the dependencies selected in the attached claims. The subject matter that may be claimed includes not only the combination of features as set out in the attached claims, but also any other combination of features in the claims, wherein each feature mentioned in a claim may be combined with any other feature in the claim or a combination of other features. In addition, any of the embodiments and features described or depicted herein may be claimed in a separate claim and / or in any combination with any embodiment and feature described or depicted herein or with any feature of the attached claims.
[0011] In an embodiment according to the present invention, a head mounted display (HMD) may include:
[0012] at least one processor;
[0013] a first electronic display configured to display a first virtual plane within the virtual scene to a user wearing the HMD;
[0014] a second electronic display configured to display a second virtual plane of the virtual scene to a user wearing the HMD, the first virtual plane being located at a different optical depth relative to the second virtual plane;
[0015] an optical block configured to direct light from the first electronic display and the second electronic display to an exit pupil of the HMD;
[0016] an eye tracking system including an image capture element, the eye tracking system configured to determine an eye position of each eye of a user and a gaze line of each eye of the user;
[0017] A memory comprising instructions that, when executed by at least one processor, cause the at least one processor to:
[0018] determining a vergence depth of the user based on an estimated intersection of the gaze lines of each of the user's eyes; and
[0019] A zoom actuator block is configured to change a focal length of the optical block based at least in part on the depth of convergence.
[0020] In an embodiment according to the present invention, the HMD may include: one or more optical combination elements, wherein a first electronic display is positioned perpendicular to a second electronic display, and the one or more optical combination elements combine a first virtual plane of a virtual scene with a second virtual plane of the virtual scene for viewing by a user wearing the HMD.
[0021] Objects in the virtual scene may be mapped to scene geometry or depth information provided by a content creator, a first electronic display may be mapped to a first set of scene geometries, and a second electronic display may be mapped to a second, different set of scene geometries.
[0022] At least one of the first electronic display and the second electronic display can be at least translucent to allow image light of the virtual scene to be combined for viewing by a user wearing the HMD.
[0023] The zoom actuation block may be configured to change a focal length of the optical block by changing a distance between the optical block and the first electronic display and the second electronic display.
[0024] The zoom actuation block may be configured to change a focal length of the optical block by changing a shape or an optical path length of a lens included in the optical block.
[0025] Changing the shape or optical path length of a lens of an optical block may include using at least one selected from the group consisting of: shape changing polymer lenses, liquid lenses and electrowetting, Alvarez-Lohmann lenses, deformable membrane mirrors, liquid crystal (electroactive) lenses, phase-only spatial light modulators (SLMs), and any combination of these.
[0026] In an embodiment according to the present invention, the HMD may include:
[0027] at least one processor;
[0028] an electronic display configured to sequentially display two or more virtual planes of a virtual scene, wherein objects in the virtual scene are mapped to a scene geometry of the virtual scene, and each of the two or more virtual planes displays a different subset of the objects based on the scene geometry;
[0029] an optical block configured to direct light from the electronic display to an exit pupil of the HMD;
[0030] An eye tracking system configured to determine:
[0031] the eye position of each eye of the user,
[0032] the gaze line of each eye based on the eye position of each eye, and
[0033] the vergence depth corresponding to the intersection of the gaze lines; and
[0034] A zoom actuation block is configured to change a focal length of the optical block based at least in part on a vergence depth of each eye of a user corresponding to a viewing position of the user within the virtual scene.
[0035] The zoom actuator block may be configured to oscillate in a fixed focus pattern, and electronic display refresh between sequentially displaying the two or more virtual planes may be timed to create the two or more virtual display planes at the optical depth based on at least one of the scene geometries.
[0036] One of the two or more virtual display planes may be presented at a viewer's vergence depth based on the user's viewing position in the virtual scene.
[0037] The zoom actuator block may be configured to change the focal length of the optical block by changing the distance between the optical block and the electronic display.
[0038] The zoom actuation block may be configured to change a focal length of the optical block by changing a shape or an optical path length of a lens included in the optical block.
[0039] Changing the shape or optical path length of a lens of an optical block may include using at least one selected from the group consisting of: shape changing polymer lenses, liquid lenses and electrowetting, Alvarez-Lohmann lenses, deformable membrane mirrors, liquid crystal (electroactive) lenses, phase-only spatial light modulators (SLMs), and any combination thereof.
[0040] In an embodiment according to the present invention, the headset may include:
[0041] at least one processor;
[0042] a first electronic display configured to display a first virtual plane within a virtual scene to a user wearing the headset;
[0043] a second electronic display configured to display a second virtual plane of the virtual scene to a user wearing the headset, the first virtual plane being located at a different optical depth relative to the second virtual plane;
[0044] an optical block configured to direct light from the first electronic display and the second electronic display to an exit pupil of the headset;
[0045] an eye tracking system including an image capture element, the eye tracking system configured to determine an eye position of each eye of a user, a gaze line of each eye of the user based on the eye position, and a vergence depth corresponding to a viewing position of the user within the virtual scene; and
[0046] A zoom actuator block is configured to change a focal length of the optical block based at least in part on the depth of convergence.
[0047] The first electronic display may be positioned perpendicular to the second electronic display, and the one or more optical combining elements may combine the first virtual plane of the virtual scene with the second virtual plane of the virtual scene for viewing by a user wearing the headset.
[0048] Objects in the virtual scene may be mapped to scene geometry or depth information provided by a content creator, a first electronic display may be mapped to a first set of scene geometries, and a second electronic display may be mapped to a second, different set of scene geometries.
[0049] At least one of the first electronic display and the second electronic display can be at least translucent to allow image light of the virtual scene to be combined for viewing by a user wearing the headset.
[0050] The zoom actuation block may be configured to change a focal length of the optical block by changing a distance between the optical block and the first electronic display and the second electronic display.
[0051] The zoom actuation block may be configured to change a focal length of the optical block by changing a shape or an optical path length of a lens included in the optical block.
[0052] Changing the shape or optical path length of a lens of an optical block may include using at least one selected from the group consisting of: shape changing polymer lenses, liquid lenses and electrowetting, Alvarez-Lohmann lenses, deformable membrane mirrors, liquid crystal (electroactive) lenses, phase-only spatial light modulators (SLMs), and any combination of these. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 An example multi-plane display is shown in which the position of the focal plane can be adjusted by a zoom system according to one embodiment.
[0054] Figure 2 An example plane sequential zoom multi-plane display is shown according to one embodiment.
[0055] Figure 3 An example planar synchronized zoom multi-planar display including a beam splitter to direct light from two vertically opposed electronic displays is shown according to one embodiment.
[0056] Figure 4 An example planar synchronized zoom multi-planar display with two electronic displays optically connected in series is shown in accordance with one embodiment.
[0057] Figure 5An example zoom system is shown according to one embodiment.
[0058] Figure 6 A cross section of a multi-plane head mounted display including cameras for tracking eye position is shown according to one embodiment.
[0059] Fig. 7A Shown is the relationship between vergence and eye focal length in the real world.
[0060] Figure 7B Shown is the conflict between vergence and eye focus in a three-dimensional display.
[0061] Fig. 8A An optical block of a multi-planar head mounted display is shown in a first position relative to an electronic display according to one embodiment.
[0062] Figure 8B An example of using a zoom element to move a Fig. 8A The optical block is in the second position.
[0063] The accompanying drawings depict embodiments of the present disclosure for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods described herein may be employed without departing from the principles or benefits of the present disclosure described herein. Detailed Description
[0064] Overview
[0065] A multi-plane head mounted display (HMD) includes two or more artificial display planes located at an optical distance for each eye that can be dynamically adjusted based on the position of the user viewing a scene presented by the HMD. For example, the scene is presented on two or more electronic display elements (e.g., screens) of the HMD. A focal length of the optical block that directs image light from the electronic display element to the user's eyes is adjusted based on the position or object in the scene the user is viewing using a zoom system (e.g., an element that mechanically changes the distance between a lens system in the optical block and the electronic display element, an element that changes the shape of one or more lenses in the lens system in the optical block, etc.).
[0066] Figure 1An example multi-plane display system 100 is shown that includes four separate electronic displays 102a, 102b, 102c, and 102d (collectively referred to as "electronic displays 102") located at different distances relative to an optical block 104 and an exit pupil 110 of the multi-plane display system 100. In one embodiment, each electronic display 102 displays a different depth component or focal plane of a scene (e.g., a virtual reality scene) that together present a complete image of the scene to a user. Different electronic displays 102 may have different resolutions, refresh rates, number of color channels, brightness capabilities, or some combination thereof.
[0067] The electronic display 102 generates multiple image planes. For example, each image of the artificial scene may include a z value corresponding to the distance value of each pixel currently displayed. When one pixel is illuminated, the visually aligned pixels of other electronic displays 102 remain transparent.
[0068] The optical block 104 presents content from one or more of the electronic displays 102 at a particular focal plane based in part on the position of the electronic displays 102 relative to the optical block. Thus, each electronic display 102 has a corresponding focal plane 108 relative to the position of the optical block 104. The zoom system can change the position of the focal plane 108 based on a position within the scene corresponding to where the user is looking, for example, by changing the position 512 or other properties of the optical block 104 relative to the electronic display 102. Instead of mechanically moving the optical block 104 to change the position of the focal plane 108, as Figure 1 As shown, properties of optical block 104 that affect focal length may be changed to change the location of focal plane 108 , or the position of one or more of electronic displays 102 may be changed relative to each other and / or exit pupil 110 .
[0069] In order to determine the location or object in the scene that the user is looking at, the multi-plane display system 100 of the HMD includes an eye tracking system configured to detect convergence and / or accommodation and other viewing parameters (e.g., position, orientation, and pupil size). Thus, the multi-plane display system 100 tracks the user's eyes and approximates the gaze point and the depth of convergence based on the detected convergence and other viewing parameters, and the zoom system adjusts the optical power of the optical block to provide accommodation for the user's eyes at the depth of convergence.
[0070] Embodiments of the present invention may include an artificial reality system or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that is adjusted in some way before being presented to a user, which may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), mixed reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content or generated content combined with captured (e.g., real world) content. Artificial reality content may include video, audio, tactile feedback, or some combination thereof, and any of which may be presented in a single channel or in multiple channels (e.g., stereoscopic video that produces a three-dimensional effect to the viewer). In addition, in some embodiments, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof for, for example, creating content in artificial reality and / or otherwise using it in artificial reality (e.g., performing activities in artificial reality). An artificial reality system that provides artificial reality content may be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system, an independent HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.
[0071] Plane Sequential Zoom Multi-Plane Display
[0072] Figure 2 An example multi-plane display system 200 is shown, which in one embodiment includes a single electronic display 102 located at a distance from an optical block 104, an eye tracking system configured to detect convergence and / or accommodation and other viewing parameters (e.g., position, orientation, and pupil size), and a zoom system to adjust the focus of the optical block 104 to a position corresponding to the detected convergence. When illuminated, light emitted from the single electronic display 102 is presented as an artificial display plane at a specific focal depth. The location of the artificial focal plane is based in part on the focus setting of the optical block 104 set by the zoom system. The electronic display 102 can display multiple artificial display planes by illuminating the electronic display 102 (e.g., emitting image light) multiple times while using the zoom system to change the fixed focus of the optical block 104. Therefore, in one embodiment, the electronic display 102 is a high refresh rate display (e.g., 90 Hz multiplied by the number of display planes). The zoom element can then be oscillated in a fixed focus pattern (e.g., a sinusoidal oscillation from 0 to 3 diopters), and the refresh of the electronic display 102 can be timed to create an artificial display plane at an arbitrary optical depth. The optical depth of the artificial display plane is then adjusted based on the vergence, position, orientation, pupil size, and other measurements received from the eye tracking system.
[0073] At least one artificial display plane is presented at a plane close to the estimated accommodation plane of the viewer based on the viewing position of the user in the artificial scene. Thus, images of the artificial scene are displayed as layers using each artificial display plane to maximize the visual quality metric using a multi-plane decomposition algorithm. Furthermore, the artificial planes on which objects in the artificial scene are displayed can be based on scene geometry provided by the content creator, depth information, or information from an eye tracking system.
[0074] Plane Synchronous Zoom Multi-Plane Display
[0075] Figure 3 An example multi-plane display system 300 is shown, which in one embodiment includes two electronic displays 102a and 102b, an optical combination element 302 (e.g., a beam splitter), and an eye tracking system and a zoom system, as described similarly above. In this example, the electronic displays 102a and 102b are positioned vertically relative to each other, and the optical combination element 302 is oriented to be at an angle relative to the two electronic displays 102a and 102b to combine the images of the two displays. The two electronic displays 102a and 102b can generate multiple artificial display planes within the scene, and the optical depth of each of the electronic displays 102a and 102b can be adjusted by the zoom system. For example, the electronic display 102a can display a portion of the scene image that is located in an artificial plane closer to the user relative to the portion of the scene image displayed by the electronic display 102b.
[0076] Plane Synchronous Zoom Multi-Plane Display
[0077] Figure 4 An example multi-plane display system 400 is shown, which in one embodiment includes two electronic displays 102a and 102b and an eye tracking system and a zoom system, as similarly described above. In this example, the two electronic displays 102a and 102b are optically positioned in series at different distances relative to the optical block 104, and thus can be semi-transparent (e.g., a stack of TOLEDs or LCDs). In addition, the electronic displays 102a and 102b can not only emit light, but can also absorb it (e.g., LCDs).
[0078] In one embodiment, electronic displays 102a and 102b can each display a different set of artificial planes of a scene that together present a complete image of the scene to the user. Different sets of artificial planes can also correspond to artificial planes that do not need to be presented at the highest resolution, and thus electronic displays 102a and 102b can each have different resolutions, refresh rates, number of color channels, brightness capabilities, etc. Thus, multiple artificial display planes or layers of a scene are generated by two electronic displays 102a and 102b, and the optical depth of each display is adjusted using a zoom system as described above to keep the user's eye 106 in accommodation at the user's convergence depth as detected by the eye tracking system.
[0079] Multi-plane zoom system
[0080] Figure 5 A multi-plane zoom system 500 is shown in which a head mounted display (HMD) 501 operates. Zoom system 500 can be used as a virtual reality (VR) system, an augmented reality (AR) system, a mixed reality (MR) system, or some combination thereof. In this example, zoom system 500 includes HMD 501, imaging device 560, and I / O interface 570, each of which is coupled to console 550. Although Figure 1 A single HMD 501, a single imaging device 560, and a single I / O interface 570 are shown, but any number of these components may be included in the system in other embodiments. For example, there may be multiple HMDs 501, each with an associated I / O interface 570 and monitored by one or more imaging devices 560, each HMD 501, I / O interface 570, and imaging device 560 communicating with a console 550. In alternative configurations, different and / or additional components may also be included in the system environment.
[0081] HMD 501 presents content to the user. Example content includes images, videos, audio, or some combination thereof. Audio content may be presented via a separate device (e.g., speaker and / or headset) external to HMD 501 that receives audio information from HMD 501, console 550, or both. HMD 501 includes electronic display 102, optical block 104, zoom actuation block 506, focus prediction module 508, eye tracking module 510, convergence processing module 512, one or more localizers 514, internal measurement unit (IMU) 516, head tracking sensor 518, and scene rendering module 520.
[0082] The optical block 104 directs light from the electronic display 102 to an exit pupil for viewing by a user using one or more optical elements such as an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, etc., and may include a combination of different optical elements. In some embodiments, one or more optical elements in the optical block 104 may have one or more coatings, such as an anti-reflective coating. The amplification of the image light by the optical block 104 allows the electronic display 102 to be physically smaller, lighter, and consume less power than larger displays. In addition, the amplification of the image light can increase the field of view of the displayed content. For example, the field of view of the displayed content enables the displayed content to be presented using almost all of the user's field of view (e.g., 550 degrees diagonal) and in some cases the entire field of view.
[0083] Optical block 104 may be designed to correct one or more optical errors. Examples of optical errors include: barrel distortion, pincushion distortion, longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, coma, field curvature, astigmatism, etc. In some embodiments, the content provided to electronic display 102 for display is pre-distorted, and optical block 104 corrects the distortion when it receives image light generated based on the content from electronic display 102.
[0084] The zoom actuator block 506 includes a zoom element that causes the optical block 104 to change the focal length (or optical power) of the HMD 501 so as to keep the user's eyes in a comfort zone as the convergence and accommodation change. In one embodiment, the zoom actuator block 506 physically changes the distance between the electronic display 102 and the optical block 104 by moving the electronic display 102 or the optical block 104 (or both). Optionally, the zoom actuator block 506 changes the focal length of the optical block 104 by adjusting one or more properties of one or more lenses. Example properties of the lenses adjusted by the zoom actuator block include: optical path length, refractive index of the lens medium, shape of the lens, etc. For example, the zoom actuator block 506 changes the focal length of one or more lenses using a shape-changing polymer lens, an electrowetting method using a liquid lens, an Alvarez-Lohmann lens, a deformable membrane mirror, a liquid crystal (electroactive) lens, or a pure phase spatial light modulator (SLM), or any other suitable component. Additionally, moving or translating two or more lenses relative to each other may also be used to change the focal length of the HMD 501. Thus, the zoom actuation block 506 may include an actuator or motor that moves one or more of the electronic display 102 and / or the optical block 104 on a track to change the distance therebetween, or may include actuators and other components or mechanisms for changing the properties of one or more lenses included in the optical block 104. In various embodiments, the zoom actuation block 506 may be separate from the optical block 104 or integrated into the optical block 104.
[0085] Each state of the optical block 104 corresponds to a focal length of the HMD 501 or a combination of an eye position and a focal length relative to the optical block 104 (as discussed further below). In operation, the optical block 104 can be moved within a range of ˜5 mm with a positional accuracy of ˜5 μm for a granularity of approximately 1000 focal lengths corresponding to the 1000 states of the optical block 104. Any number of states may be provided; however, the limited number of states accommodates the sensitivity of the human eye, allowing some embodiments to include fewer focal lengths. For example, a first state may correspond to a focal length of theoretically infinite meters (0 diopters), a second state may correspond to a focal length of 2.0 meters (0.5 diopters), a third state may correspond to a focal length of 1.0 meters (1 diopters), a fourth state may correspond to a focal length of 0.5 meters (1 diopters), a fifth state may correspond to a focal length of 0.333 meters (3 diopters), and a sixth state may correspond to a focal length of 0.250 meters (4 diopters). Thus, the zoom actuator block 506 sets and changes the state of the optics block 104 to achieve a desired focal length.
[0086] The focus prediction module 508 is an encoder that includes logic to track the state of the optical block 104 to predict one or more future states or positions of the optical block 104. For example, the focus prediction module 508 accumulates historical information corresponding to previous states of the optical block 104 and predicts future states of the optical block 104 based on the previous states. Because the presentation of the virtual scene by the HMD 501 is adjusted based on the state of the optical block 104, the predicted state allows the scene rendering module 520, which is further described below, to determine the adjustments to apply to the virtual scene for a particular frame. Therefore, the focus prediction module 508 passes information describing the predicted state of the optical block 104 for a frame to the scene rendering module 520. The adjustments to the different states of the optical block 104 performed by the scene rendering module 520 are further described below.
[0087] The eye tracking module 510 tracks the eye position and eye movement of the user of the HMD 501. In one embodiment, a camera or other optical sensor inside the HMD 501 captures image information of the user's eyes, and the eye tracking module 110 uses the captured information to determine the interpupillary distance, the interocular distance, the three-dimensional (3D) position of each eye relative to the HMD 501 (e.g., for the purpose of distortion adjustment), including the amplitude of the twist and rotation (i.e., roll, pitch, and yaw) of each eye, and the gaze direction. In one example, infrared light is emitted within the HMD 501 and reflected from each eye. The reflected light is received or detected by the camera and analyzed to extract the eye rotation from the changes in the infrared light reflected by each eye. Many methods for tracking the user's eyes can be used by the eye tracking module 510. Thus, the eye tracking module 510 can track up to six degrees of freedom (i.e., 3D position, rotation, pitch, and yaw) for each eye, and at least a subset of the tracked quantities can be combined from the user's two eyes to estimate the gaze point (i.e., the 3D location or position in the virtual scene that the user is looking at). For example, the eye tracking module 510 integrates information from past measurements, measurements that identify the position of the user's head, and 3D information describing the scene presented by the electronic display element 102. Thus, information about the position and orientation of the user's eyes is used to determine the gaze point in the virtual scene presented by the HMD 501 that the user is looking at.
[0088] Furthermore, as the eyes move to look in different directions, the 3D position of the pupil relative to the optical block 104 changes. As the viewing direction changes, the changing 3D position of the pupil relative to the optical block 104 contributes to distortion perceived by the user as "pupil swim". Thus, measuring distortion at different 3D eye positions relative to the optical block 104 and generating distortion corrections for different positions and distances allows for the distortion caused by "pupil swim" to be mitigated by tracking the 3D position of the user's eyes and applying distortion corrections corresponding to the 3D position of each of the user's eyes at a given point in time. Thus, knowing the 3D position of each of the user's eyes allows for the distortion caused by changes in the distance between the pupil of the eye and the optical block 104 to be mitigated by applying distortion corrections for each 3D eye position. Methods for reducing pupil swim are further described in U.S. Provisional Patent Application No. 62 / 302,675, filed on March 2, 2016, which is hereby incorporated by reference in its entirety.
[0089] In addition to contributing to distortions perceived by the user as "wandering pupils," the changing 3D position of the pupils can also cause the user to perceive parallax in a multi-plane display system. For example, the changing 3D position of the pupil within the eyebox relative to an electronic display that displays an image at different depths relative to the eyebox can cause problems with how the final image (or composite image) looks to the user. Therefore, by tracking the 3D position of each of the user's eyes, parallax correction can be applied to one or more frames of content displayed by multiple electronic displays to account for parallax based on the current location of the user's 3D eye positions.
[0090] The convergence processing module 512 determines the convergence depth of the user's gaze based on the estimated intersection or gaze point of the gaze lines determined by the eye tracking module 510. Convergence is the simultaneous movement or rotation of two eyes in opposite directions to maintain binocular single vision, which is performed naturally and automatically by the human eye. Therefore, the position where the user's eyes are tended to is the position where the user is looking, and is usually also the position where the user's eyes are focused on. For example, the convergence processing module 512 triangulates the gaze line to estimate the distance or depth from the user associated with the intersection of the gaze lines. Then, the depth associated with the intersection of the gaze lines can be used as an approximation of the adjustment distance, which identifies the distance from the user to which the user's eyes are directed. Therefore, the convergence distance allows the determination of the position where the user's eyes should be focused and the depth from the user's eyes where the eyes are focused, thereby providing information such as a focus object or a focus plane for providing adjustments to the virtual scene.
[0091] In some embodiments, instead of providing accommodation for the eyes at a determined depth of convergence, accommodation may be determined directly by a wavefront sensor, such as a Shack-Hartmann wavefront sensor; thus, the state of optical block 104 may be a function of the depth of convergence or accommodation and the 3D position of each eye, so that optical block 104 brings objects in a scene presented by electronic display element 102 into focus for the user to view the scene. Furthermore, the vergence and accommodation information may be combined to focus optical block 104 and provide a synthetic depth of field blur.
[0092] The locators 514 are objects that are located at specific locations on the HMD 501 relative to each other and relative to specific reference points on the HMD 501. The locators 514 can be light emitting diodes (LEDs), three right-angle cone reflectors, reflective markers, a type of light source that contrasts with the environment in which the HMD 501 operates, or some combination thereof. Active locators 514 (i.e., LEDs or other types of light emitting devices) can emit light in the visible light band (~380nm to 750nm), in the infrared (IR) band (~750nm to 1mm), in the ultraviolet band (10nm to 380nm), some other part of the electromagnetic spectrum, or some combination thereof.
[0093] The locators 514 may be located below an outer surface of the HMD 501 that is transparent to, or thin enough not to significantly attenuate, the wavelengths of light emitted or reflected by the locators 514. Additionally, the outer surface or other portions of the HMD 501 may be opaque in the visible band of wavelengths of light. Thus, the locators 514 may emit light in the IR band while below an outer surface of the HMD 501 that is transparent in the IR band but opaque in the visible band.
[0094] IMU 516 is an electronic device that generates fast calibration data based on measurement signals received from one or more of head tracking sensors 518, which generate one or more measurement signals in response to the movement of HMD 501. Examples of head tracking sensors 518 include accelerometers, gyroscopes, magnetometers, other sensors suitable for detecting movement, correcting errors associated with IMU 516, or some combination thereof. Head tracking sensors 518 can be located external to IMU 516, internal to IMU 516, or some combination thereof.
[0095] Based on the measurement signals from the head tracking sensor 518, the IMU 516 generates fast calibration data indicating the estimated position of the HMD 501 relative to the initial position of the HMD 501. For example, the head tracking sensor 518 includes a plurality of accelerometers that measure translational motion (forward / backward, up / down, left / right) and a plurality of gyroscopes that measure rotational motion (e.g., pitch, yaw, and roll). For example, the IMU 516 can quickly sample the measurement signals and calculate the estimated position of the HMD 501 from the sampled data. For example, the IMU 516 integrates the measurement signals received from the accelerometers over time to estimate a velocity vector, and integrates the velocity vector over time to determine the estimated position of a reference point on the HMD 501. A reference point is a point that can be used to describe the position of the HMD 501. Although a reference point can generally be defined as a point in space, in various embodiments the reference point is defined as a point within the HMD 501 (e.g., the center of the IMU 130). Optionally, the IMU 516 provides sampled measurement signals to the console 550, which determines fast calibration data.
[0096] In addition, the IMU 516 may receive one or more calibration parameters from the console 550. As discussed further below, the one or more calibration parameters are used to maintain tracking of the HMD 501. Based on the received calibration parameters, the IMU 516 may adjust one or more IMU parameters (e.g., sampling rate). In some embodiments, certain calibration parameters cause the IMU 516 to update the initial position of the reference point to correspond to the next calibration position of the reference point. Updating the initial position of the reference point to the next calibration position of the reference point helps reduce the cumulative error associated with determining the estimated position. The cumulative error, also known as drift error, causes the estimated position of the reference point to "drift" away from the actual position of the reference point over time.
[0097] The scene rendering module 520 receives content of the virtual scene from the engine 556 and provides the content for display on the electronic display 102. In addition, the scene rendering module 520 can adjust the content based on information from the focus prediction module 108, the convergence processing module 512, the IMU 516, and the head tracking sensor 518. For example, when receiving the content from the engine 556, the scene rendering module 520 adjusts the content by adding corrections or pre-distortions to the rendering of the virtual scene to compensate for or correct for distortion caused by the predicted state of the optical block 104 received from the focus prediction module 508 (i.e., eye position and focal length). The scene rendering module 520 can also add depth of field blur based on the user's gaze, the depth of convergence (or accommodation depth) received from the convergence processing module 512, or measured properties of the user's eyes (e.g., the 3D position of the eyes, etc.). Additionally, the scene rendering module 520 determines a portion of the content to be displayed on the electronic display 102 based on one or more of the tracking module 554, the head tracking sensor 518, or the IMU 516, as further described below.
[0098] The imaging device 560 generates slow calibration data based on the calibration parameters received from the console 550. The slow calibration data includes one or more images showing the observed position of the locator 514 detectable by the imaging device 560. The imaging device 560 may include one or more cameras, one or more video cameras, other devices capable of capturing images including one or more locators 514, or some combination thereof. In addition, the imaging device 560 may include one or more filters (e.g., for increasing the signal-to-noise ratio). The imaging device 560 is configured to detect light emitted or reflected from the locator 514 in the field of view of the imaging device 560. In embodiments where the locator 514 includes a passive element (e.g., a retroreflector), the imaging device 560 may include a light source that illuminates some or all of the locator 514, and the locator 514 retroreflects light toward the light source in the imaging device 560. Slow calibration data is communicated from imaging device 560 to console 550, and imaging device 560 receives one or more calibration parameters from console 550 to adjust one or more imaging parameters (e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.).
[0099] The I / O interface 570 is a device that allows a user to send an action request to the console 550. An action request is a request to perform a specific action. For example, an action request can be to start or end an application or to perform a specific action within an application. The I / O interface 570 may include one or more input devices. Example input devices include a keyboard, a mouse, a game controller, or any other suitable device for receiving an action request and passing the received action request to the console 550. The action request received by the I / O interface 570 is passed to the console 550, which performs the action corresponding to the action request. In some embodiments, the I / O interface 570 can provide tactile feedback to the user according to the instruction received from the console 550. For example, when an action request is received, or when the console 550 performs an action, the console 550 passes an instruction to the I / O interface 570 and the I / O interface 570 generates tactile feedback, and the tactile feedback is provided by the I / O interface 570.
[0100] The console 550 provides content to the HMD 501 for presentation to the user based on information received from the imaging device 560, the HMD 501, or the I / O interface 570. Figure 1 In the example shown, the console 550 includes an application memory 552, a tracking module 554, and a virtual reality (VR) engine 556. Figure 1 Some embodiments of console 550 have different or additional modules than those described herein. Similarly, the functionality described further below may be distributed among the components of console 550 in a manner different than that described here.
[0101] The application cache 552 stores one or more applications for execution by the console 550. An application is a set of instructions that, when executed by the processor, generates content for presentation to a user. The content generated by the application may be responsive to input received from the user via the movement or interface device 570 of the HMD 501. Examples of applications include gaming applications, conferencing applications, video playback applications, or other suitable applications.
[0102] The tracking module 554 calibrates the system using one or more calibration parameters, and may adjust one or more calibration parameters to reduce errors in determining the position of the HMD 501. For example, the tracking module 554 adjusts the focus of the imaging device 560 to obtain a more accurate position of the observed localizers 514 on the HMD 501. In addition, the calibration performed by the tracking module 554 also takes into account information received from the IMU 516. In addition, if tracking of the HMD 501 is lost (e.g., the imaging device 560 loses sight of at least a threshold number of localizers), the tracking module 554 recalibrates some or all of the system components.
[0103] In addition, the tracking module 554 uses the slow calibration information from the imaging device 560 to track the motion of the HMD 501, and uses the observed localizers to determine the positions of reference points on the HMD 501 based on the slow calibration information and the model of the HMD 501. The tracking module 554 also uses the position information from the fast calibration information of the IMU 516 on the HMD 501 to determine the positions of the reference points on the HMD 501. In addition, the tracking module 554 can use portions of the fast calibration information, the slow calibration information, or some combination thereof to predict the future position of the HMD 501, which is provided to the engine 556.
[0104] The engine 556 executes applications within the system and receives position information, acceleration information, velocity information, predicted future position, or some combination thereof of the HMD 501 from the tracking module 554. Based on the received information, the engine 556 determines the content to be provided to the HMD 501 for presentation to the user, such as a virtual scene. For example, if the received information indicates that the user has looked to the left, the engine 556 generates content for the HMD 501 that reflects or tracks the user's movement in the virtual environment. In addition, the engine 556 performs actions within the application executed on the console 550 in response to an action request received from the I / O interface 570, and provides feedback to the user that the action is performed. The feedback provided may be visual or auditory feedback via the HMD 501 or tactile feedback via the I / O interface 570.
[0105] Focus adjustment method
[0106] As discussed above, multi-plane zoom system 500 can dynamically change its focus to bring the image presented to a user wearing HMD 501 into focus, which keeps the user's eyes in a comfort zone as vergence and accommodation vary.
[0107] Therefore, as above combined Figure 5 As described, the position, orientation, and / or motion of the HMD 501 is determined by a combination of the localizer 514, the IMU 516, the head tracking sensor 518, the imaging device 560, and the tracking module 554. Portions of the virtual scene presented by the HMD 501 are mapped to various positions and orientations of the HMD 501. Thus, a portion of the virtual scene currently viewed by the user is determined based on the position, orientation, and motion of the HMD 501. After determining the portion of the virtual scene viewed by the user, the system can then determine a location or object within the determined portion that the user is viewing to adjust the focus of the location or object accordingly.
[0108] In order to determine locations or objects within the determined portion of the virtual scene that the user is looking at, the HMD 501 tracks the position and location of the user's eyes. Thus, the HMD 501 determines the eye position of each of the user's eyes. For example, the HMD 501 tracks at least a subset of the 3D position, roll, pitch, and yaw of each eye and uses these quantities to estimate the 3D gaze point of each eye. Additionally, in various embodiments, information from past eye positions, information describing the position of the user's head, and information describing the scene presented to the user may also be used to estimate the 3D gaze point of the eyes. For example, referring to Figure 6 , the HMD 501 includes a camera 612 for tracking the position of each eye 610. In this example, the camera 102 captures images of the user's eyes, and the eye tracking module 510 determines an output for each eye 610 and a gaze line 614 corresponding to a gaze point or location at which the user is looking based on the captured images.
[0109] Figure 6 Further shown is the process of determining the convergence depth (d ) of the user's gaze point based on the estimated intersection of the gaze lines 614. v )608. Figure 6 As shown, the gaze line 614 is at the position d where the object 606 is located. v 608. Because the virtual distance within the virtual scene is known to the system, the convergence depth 608 can be filtered or verified to determine a more accurate convergence depth for the virtual scene. For example, the convergence depth 608 is an approximation of the intersection of the gaze lines 614, which are themselves approximations based on the position of the user's eyes 610. The gaze lines 614 do not always appear to intersect accurately. Therefore, the virtual distance within the virtual scene can be compared to the convergence depth of the portion of the virtual scene to generate a convergence depth. Determining a more accurate convergence depth or gaze point enables the virtual scene to more accurately determine the user's focused object or focal plane, allowing the scene rendering module 520 to add depth of field blur to the appropriate depths and / or objects in the virtual scene, or otherwise modify the virtual scene to appear more realistic.
[0110] Thus, the state of the optical block 104 is determined for a frame of the virtual scene based on the state of the optical block 104 during the rendering of the previous frame of the virtual scene. For example, the focus prediction module 508 tracks the state of the optical block 104 for each frame of the virtual scene to predict the future state of the optical block 104 for subsequent frames of the virtual scene. The predicted state of the optical block 104 (e.g., the predicted position of the optical block 104) allows the scene rendering module 514 to determine an adjustment to apply to the frame of the virtual scene so that the distortion caused by the predicted state of the optical block 104 corrects or cancels the applied adjustment instead of distorting the frame. Thus, based on the state of the optical block 104, a distortion correction is determined for application to the frame of the virtual scene, thereby correcting the optical error introduced by the state of the optical block 104.
[0111] Fig. 7A An example of how the human eye experiences vergence and accommodation in the real world is shown. Vergence is the simultaneous movement or rotation of the two eyes in opposite directions to obtain or maintain binocular single vision, and is related to the accommodation of the eyes. Under normal conditions, changing the focus of the eyes to see objects at different distances automatically causes vergence and accommodation. Fig. 7A In the example of , the user is looking at the real object 700A (i.e., the user's eyes tend to the real object 700A, and the gaze line from the user's eyes intersects the real object 700A). Fig. 7A As the real object 700A moves closer to the user, each eye 702 rotates inward to keep looking at the real object 700A, as indicated by the arrows in FIG. 7A . As the real object 700A gets closer, the eye 702 must "adapt" to the closer distance by changing its shape to reduce the optical power or focal length of the eye 702. Thus, under normal conditions in the real world, the depth of convergence (d v ) is equal to the focal length (d f ).
[0112] However, Figure 7B An example conflict between convergence and accommodation that may occur for some three-dimensional displays is shown. In this example, the user is looking at a virtual object 700B displayed on a 3D electronic screen 704; however, the user's eyes tend to virtual object 700B, and the gaze line from the user's eyes intersects at virtual object 700B, which is at a greater distance from the user's eyes than from the 3D electronic screen 704. When virtual object 700B is presented on the 3D electronic display 704 to appear closer to the user, each eye 702 rotates inward again to keep tending to virtual object 700B, but the optical power or focal length of each eye is not reduced; therefore, the user's eyes do not appear to be moving. Fig. 7AThus, instead of reducing the optical power or focal length to accommodate a closer depth of convergence, eye 702 remains accommodated at the distance associated with 3D electronic display 704. Thus, for an object displayed on the 3D electronic display, the depth of convergence (d v ) is often not equal to the focal length of the human eye (d f ). This difference between vergence depth and focal length is called "vergence-accommodation conflict". Users who experience only vergence or accommodation but not both will eventually experience some degree of fatigue and nausea, which is undesirable for virtual reality system creators. Changes in vergence of a 3D electronic screen can be accommodated by the headset dynamically adjusting the optical power of the optical block according to the vergence depth (or predicted vergence depth).
[0113] Thus, the focal length (or optical power) of the optical block 104 is adjusted for the rendered frame of the virtual scene to provide adjustment for the generated depth of convergence. Fig. 8A and Figure 8B An example process for adjusting the focal length of optical block 104 by changing the distance between electronic display 802 and optical block 104 using zoom element 804 is shown. Fig. 8A and Figure 8B In the example of , the zoom actuator block 506 includes a zoom element 804, such as an actuator or motor and a track 806, but may also include other components that enable the optical block 104, the electronic display 802 (e.g., 802a, 802b), or both to move along the track 806 to dynamically adjust the optical focal length of the optical block 104.
[0114] Fig. 8A An example of an HMD 501 providing focus adjustment for a frame n of a virtual scene is shown. In this example, the virtual scene includes an object 808 displayed on an electronic display 802a, to which the gaze of the user 800 is directed (i.e., directed). The virtual image of the object 808 is located at a virtual distance d behind the electronic display 802a from the exit pupil 810. i At. Fig. 8A In the example, the optical block 104 is at position P i This provides the distance d i to achieve comfortable viewing for object 806.
[0115] Figure 8BHMD 501 is shown providing focus adjustment for a subsequent frame n+1 of a virtual scene. In this example, user 800 may have repositioned his eyes when object 808 quickly moves toward user 800 in the virtual scene. As a result, the virtual image of object 808 is relatively close to or positioned on electronic display 802b. In response to the closer position of object 808, user 800's eyes rotate inward to converge toward object 808, causing convergence processing module 512 to determine a new convergence depth for frame n+1 and provide the new convergence depth to zoom actuator block 506. Based on the new convergence depth, zoom element 804 moves optical block 104 from position p to position n+1. i Move to new position p f To adapt the user 800 to the new convergence depth d of the closer object 808 f Place.
[0116] In one example, each state of the optical block 104 corresponds to a combination of focal length and eye position, provides adjustment for a range of depths of convergence, and is associated with a specific position of the optical block 104. Thus, the depths of convergence can be mapped to the positions of the optical block 104 and stored in a lookup table. Thus, when the depth of convergence is received from the convergence processing module 512, the zoom actuator block 506 automatically moves the optical block 104 to a position corresponding to the received depth of convergence based on the lookup table.
[0117] In many instances, virtual reality systems are intended to present a user with a virtual environment that closely simulates a real-world environment or provides the user with content that causes the user to get lost in an illusion created by the virtual reality system. In order to provide a realistic or fascinating virtual environment to the user, the virtual reality system implements multiple systems and methods discussed herein to operate together at an efficiency that is imperceptible to the user. For example, transition delays are particularly costly to the user's experience of the virtual reality system. If the user is waiting for the virtual scene presented by the headset to catch up to what the user's brain has already expected, the illusion is broken and / or the user may become nauseous. However, processing speeds and commercially available actuators are currently faster than the coordination of the human eye changing its lens shape and the human brain registering what the new shape of the lens is focused on, allowing the disclosed systems and methods to provide a high-quality virtual environment to the user.
[0118] Come back for reference Fig. 8A and Figure 8B In order to accommodate the new depth of convergence while also allowing time to perform additional calculations without the user noticing a delay, the speed at which zoom element 804 moves optical block 104 is limited by the rate at which the human eye can perform accommodation. For example, assuming that the human eye has a peak accommodation speed of 10 diopters / sec, a peak accommodation speed of 100 diopters / sec, and a maximum accommodation speed of 100 diopters / sec. 2The peak acceleration and changing the distance between the electronic display 802 and the optical block 104 moves the virtual image by approximately 0.5 diopters / mm, with the zoom element 804 moving at a minimum speed of 10 / 0.5=20 mm / sec and 100 / 0.5=200 mm / sec. 2 The minimum acceleration of the acceleration operates to prevent the user from perceiving the repositioning of the optical block 104 relative to the electronic display 102. There are commercially available actuators that meet the above values.
[0119] In addition, the depth blur of the scene can be determined based on the viewing position of the user. In order to determine the depth blur, the point to which the user's gaze is directed within the scene presented to the user by the HMD 501 is determined, and the optical block 104 is configured as a state in which the point within the scene to which the user's gaze is directed is in focus of the user. Then, the depth blur is determined relative to the point to which the user's gaze is directed in the scene. In one example, the depth within the scene geometry corresponding to the convergence depth (e.g., the distance within the virtual scene) is determined as the focal plane of the frame of the virtual scene. Therefore, objects or features of the virtual environment having a distance greater than or less than the distance of the focal plane from the user's eyes within the virtual scene can be presented with synthetic blur. In another example, the depth blur is determined based on the object (i.e., the "focused object") on which the user's gaze is focused in the scene at a depth corresponding to the convergence depth. Therefore, the focused object, rather than the focal plane, provides a reference point to identify other objects in the scene that are presented with depth blur, even if the other objects have a similar depth in the scene to the focused object.
[0120] The blur can be progressive, with a level of blur applied to an object or feature based on the distance of the object or feature from the focal plane (or focused object), or a generally uniform level of blur can be applied to objects or features in a virtual scene. Depth of field blur is a natural result of binocular vision, and thus including depth of field blur in a virtual scene promotes the illusion of a virtual scene by providing the user with expected depth cues, which can enhance the user's experience of the virtual scene. In addition, the blur can be based at least in part on measured properties of the user's eyes. For example, the wavefront aberrations of the user's eyes can be measured by a wavefront aberrometer, and the depth of field blur is based at least in part on the measured wavefront aberrations. Example wavefront aberrations of the user's eyes can include higher-order aberrations that are not typically corrected by glasses, contact lenses, or refractive surgery. Taking into account the properties of the user's eyes when determining the depth of field blur can improve the user's comfort when viewing the scene.
[0121] A frame of the virtual scene corresponding to the portion of the virtual scene being viewed by the user is displayed on the electronic display 102 with distortion correction to correct for optical errors caused by the determined state of the optical block 104 and with depth of field blur based on the depth of convergence. In addition, the zoom actuation block 506 changes the focus of the optical block 104 to provide focus and accommodation to the position in the portion of the virtual scene to which the user's eyes are directed.
[0122] Additional configuration information
[0123] The foregoing description of the embodiments is presented for the purpose of illustration; it is not intended to be exhaustive or to limit the patent rights to the precise form disclosed. Those skilled in the relevant art will recognize that many modifications and variations are possible in light of the above disclosure.
[0124] The language used in the specification is selected primarily for readability and teaching purposes, and it may not be selected to describe or limit the inventive subject matter. Therefore, it is intended that the scope of the present patent right is not limited by this detailed description, but rather by any claims issued on an application based thereon. Therefore, the disclosure of the embodiments is intended to be illustrative, but not limiting, of the scope of the patent right.
Claims
1. A system for generating a three-dimensional image for a user, the system include: at least one processor; a single electronic display configured to sequentially display a first virtual plane and a second virtual plane of a virtual scene, wherein the first virtual plane is associated with a first set of depth values within the virtual scene and the second virtual plane is associated with a second set of depth values that are different relative to the first set of depth values; an optical block configured to direct light from the electronic display to an exit pupil; an eye tracking system configured to determine a vergence depth of a user viewing the virtual scene; as well as a zoom actuator block configured to change a focal length of the optical block based on the vergence depth of each eye of the user corresponding to a viewing position of the user within the virtual scene, Wherein a refresh of the electronic display between sequentially displaying two or more virtual planes is timed to generate the first virtual plane and the second virtual plane at the optical depth based on at least one of the scene geometries.
2. The system according to claim 1, in, The first virtual plane and the second virtual plane are presented based on the viewing position of the user in the virtual scene and based on the convergence depth of the user.
3. The system according to claim 1, in, The first virtual plane is generated with respect to a first optical depth of the virtual scene, and the second virtual plane is generated with respect to a second optical depth within the virtual scene, the second optical depth being different relative to the first optical depth.
4. The system according to claim 1, in, The zoom actuation block is configured to change a focal length of the optical block by changing a distance between the optical block and the electronic display.
5. The system according to claim 1, in, The zoom actuation block is configured to change a focal length of the optical block by changing a shape or an optical path length of a lens included in the optical block.
6. The system according to claim 5, in, Changing the shape or optical path length of the lens of the optical block includes using at least one selected from the group consisting of: shape changing polymer lenses, liquid lenses and electrowetting, Alvarez-Lohmann lenses, deformable membrane mirrors, liquid crystal (electroactive) lenses, pure phase spatial light modulators and any combination of these items.
7. A head mounted display, include: at least one processor; a single electronic display configured to sequentially display two or more virtual planes of a virtual scene, wherein each virtual plane displays a different set of objects in the virtual scene based on scene geometry data associated with each object; an optical block configured to direct light from the electronic display to an exit pupil of the head mounted display; an eye tracking system configured to determine an eye position of each eye of a user and a vergence depth based on the eye position of each eye of the user; as well as a zoom actuator block configured to change a focal length of the optical block based at least in part on the vergence depth for each eye of the user corresponding to a viewing position of the user within the virtual scene, wherein the zoom actuator block is further configured to oscillate in a fixed focus mode and electronic display refresh between sequentially displaying the two or more virtual planes is timed to create two or more virtual display planes at optical depth based on at least one of the scene geometries.
8. The head mounted display according to claim 7, in, A first virtual plane of the two or more virtual planes is generated with respect to a first optical depth of the virtual scene, and a second virtual plane of the two or more virtual planes is generated with respect to a second optical depth within the virtual scene, the second optical depth being different relative to the first optical depth.
9. The head mounted display according to claim 7, in, One of the two or more virtual display planes is presented at the convergence depth of a viewer based on the viewing position of the user in the virtual scene.
10. The head mounted display according to claim 7, in, The zoom actuation block is configured to change a focal length of the optical block by changing a distance between the optical block and the electronic display.
11. The head mounted display according to claim 7, in, The zoom actuation block is configured to change a focal length of the optical block by changing a shape or an optical path length of a lens included in the optical block.
12. The head mounted display according to claim 11, in, Changing the shape or optical path length of the lens of the optical block includes using at least one selected from the group consisting of: shape changing polymer lenses, liquid lenses and electrowetting, Alvarez-Lohmann lenses, deformable membrane mirrors, liquid crystal (electroactive) lenses, pure phase spatial light modulators and any combination of these items.
13. A system for generating a three-dimensional image for a user, the system include: at least one processor; a single electronic display configured to sequentially display a first virtual plane and a second virtual plane of a virtual scene, wherein the first virtual plane is associated with a first set of depth values within the virtual scene and the second virtual plane is associated with a second set of depth values that are different relative to the first set of depth values; an optical block configured to direct light from the electronic display to an exit pupil; an eye tracking system configured to determine a vergence depth of the user based on an estimated intersection of gaze lines determined for each eye of the user viewing the virtual scene; as well as a zoom actuator block configured to change a focal length of the optical block based on the vergence depth of each eye of the user corresponding to a viewing position of the user within the virtual scene, Wherein a refresh of the electronic display between sequentially displaying two or more virtual planes is timed to generate the first virtual plane and the second virtual plane at the optical depth based on at least one of the scene geometries.
14. The system according to claim 13, in, The first virtual plane and the second virtual plane are presented based on the viewing position of the user in the virtual scene and based on the convergence depth of the user.
15. The system according to claim 13, in, The first virtual plane is generated with respect to a first optical depth of the virtual scene, and the second virtual plane is generated with respect to a second optical depth within the virtual scene, the second optical depth being different relative to the first optical depth.
16. The system according to claim 13, in, The zoom actuation block is configured to change a focal length of the optical block by changing a distance between the optical block and the electronic display.
