Monovision Display for Wearable Devices

Abstract

Wearable and optical display systems incorporating mono-vision display techniques and methods of operation thereof are disclosed. The wearable device may include left and right optical stacks configured to switch between displaying virtual content in a first focal plane or a second focal plane. The wearable device may determine whether an activation condition is satisfied. In response to determining that the activation condition is satisfied, a mono-vision display mode associated with the wearable device may be activated, which may include causing the left optical stack to display virtual content in the first focal plane and causing the right optical stack to display virtual content in the second focal plane.

Description

Technical Field

[0001] This application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 030,249, filed on May 26, 2020, entitled "MONOVISION DISPLAY FOR WEARABLE DEVICE", the entire contents of which are incorporated herein by reference for all purposes.

Background Art

[0002] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or is perceived to be real. Virtual reality, i.e., "VR" scenarios, typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs. Augmented reality, i.e., "AR" scenarios, typically involve the presentation of digital or virtual image information as an augmentation to the visualization of the actual world surrounding the user.

[0003] Despite the progress made in these display technologies, there remains a need in the art for improved methods, systems, and devices related to augmented reality systems, particularly display systems.

Summary of the Invention

Means for Solving the Problems

[0004] The present disclosure generally relates to techniques for improving the performance and user experience of optical systems. More specifically, embodiments of the present disclosure provide techniques for operating a fixed focal plane optical system to reduce vergence-accommodation conflict (VAC), which is masked by the user. The present invention is described with reference to optical systems such as augmented reality (AR) devices, but the present disclosure is applicable to various applications in computer vision and image display systems.

[0005] Descriptions of various embodiments of the present invention are provided below as a list of examples. As used below, any reference to a series of examples is to be understood as a disjunctive reference to each of those examples (e.g., "Examples 1-4" is to be understood as "Example 1, 2, 3, or 4").

[0006] Example 1 is a wearable device comprising a left optical stack including a left eyepiece lens configured to receive left virtual image light and output the left virtual image light toward a user side of the wearable device, a left focus adjustment lens disposed between the left eyepiece lens and the user side of the wearable device, and a left compensation lens disposed between the left eyepiece lens and a world side of the wearable device; and a right optical stack including a right eyepiece lens configured to receive right virtual image light and output the right virtual image light toward the user side of the wearable device, a right focus adjustment lens disposed between the right eyepiece lens and the user side of the wearable device, and a right compensation lens disposed between the right eyepiece lens and the world side of the wearable device, wherein a refractive power of the left focus adjustment lens is equal in magnitude to a refractive power of the left compensation lens, a refractive power of the right focus adjustment lens is equal in magnitude to a refractive power of the right compensation lens, and the refractive power of the left focus adjustment lens and the refractive power of the right focus adjustment lens differ by a certain offset amount.

[0007] Example 2 is the wearable device according to Example 1, wherein the left focus adjustment lens is a diverging lens and the left compensation lens is a converging lens.

[0008] Example 3 is the wearable device according to Example 1, wherein the right near and far adjustment lens is a diverging lens, and the right compensation lens is a converging lens.

[0009] Example 4 is the wearable device according to Example 1, wherein the offset amount exceeds a certain threshold value.

[0010] Example 5 is the wearable device according to Example 4, wherein the threshold value is one of 0.1D, 0.2D, 0.3D, 0.4D, 0.5D, 0.6D, 0.7D, 0.8D, 0.9D, or 1.0D.

[0011] Example 6 is the wearable device according to Example 1, wherein the refractive power of the left near and far adjustment lens is -1.0D, the refractive power of the left compensation lens is +1.0D, the refractive power of the right near and far adjustment lens is -1.65D, and the refractive power of the right compensation lens is +1.65D.

[0012] Example 7 is an optical system, comprising: a left optical stack configured to output left virtual image light toward the user side of the optical system; a right optical stack configured to output right virtual image light toward the user side of the optical system, wherein the left optical stack and the right optical stack are each configured to switch between displaying virtual content on a first focal plane or a second focal plane; a step of determining whether an activation condition is satisfied; and a step of activating a monovision display mode associated with the optical system in response to a determination that the activation condition is satisfied, the step of activating the monovision display mode including: a step of causing the left optical stack to display virtual content on the first focal plane; and a step of causing the right optical stack to display virtual content on the second focal plane, and a processing module configured to perform operations including these steps.

[0013] Example 8 is the optical system described in Example 7, in which the refractive power associated with the first focal plane and the refractive power associated with the second focal plane differ by a certain offset amount.

[0014] Example 9 is the optical system described in Example 8, in which the offset amount exceeds a certain threshold value.

[0015] Example 10 is the optical system described in Example 9, in which the threshold value is one of 0.1D, 0.2D, 0.3D, 0.4D, 0.5D, 0.6D, 0.7D, 0.8D, 0.9D, or 1.0D.

[0016] Example 11 is the optical system described in Example 7, including steps of determining whether activation conditions are met by using one or more eye-tracking cameras of the optical system to capture eye-tracking data corresponding to one or both eyes of the user of the optical system, determining whether the convergence / divergence movement distance can be determined based on the eye-tracking data, and determining that the convergence / divergence movement distance cannot be determined based on the eye-tracking data.

[0017] Example 12 is the optical system described in Example 7, in which the step of determining whether activation conditions are met includes a step of determining that the eye-tracking data is unavailable.

[0018] Example 13 is the optical system described in Example 7, in which the step of determining whether activation conditions are met includes a step of determining whether the virtual content to be displayed is represented on both the first focal plane and the second focal plane, and a step of determining that the virtual content to be displayed is represented on both the first focal plane and the second focal plane.

[0019] Example 14 is an optical system according to Example 7, wherein the operation further includes deactivating the monovision display mode in response to a determination that the activation condition is no longer satisfied, and the step of deactivating the monovision display mode includes causing the left optical stack and the right optical stack to display virtual content on a first focal plane, or causing the left optical stack and the right optical stack to display virtual content on a second focal plane.

[0020] Example 15 is an optical system according to Example 7, wherein the operation further includes modifying the monovision display mode after a predetermined amount of time, thereby causing the left optical stack to switch from displaying virtual content on a first focal plane to displaying virtual content on a second focal plane, and causing the right optical stack to switch from displaying virtual content on a second focal plane to displaying virtual content on a first focal plane.

[0021] Example 16 is a method including providing a wearable device including a left optical stack and a right optical stack, wherein the left optical stack and the right optical stack are each configured to switch between displaying virtual content on a first focal plane or a second focal plane; receiving virtual content to be displayed on the left optical stack and the right optical stack; determining whether an activation condition is satisfied; and activating a monovision display mode associated with the wearable device in response to a determination that the activation condition is satisfied, wherein the step of activating the monovision display mode includes causing the left optical stack to display virtual content on a first focal plane and causing the right optical stack to display virtual content on a second focal plane.

[0022] Example 17 is the method described in Example 16, where the refractive power associated with the first focal plane and the refractive power associated with the second focal plane differ by a certain offset amount.

[0023] Example 18 is the method described in Example 17, where the offset amount exceeds a certain threshold value.

[0024] Example 19 is the method described in Example 18, where the threshold value is one of 0.1D, 0.2D, 0.3D, 0.4D, 0.5D, 0.6D, 0.7D, 0.8D, 0.9D, or 1.0D.

[0025] Example 20 is the method described in Example 16, including steps of determining whether the activation condition is satisfied by using one or more eye-tracking cameras of the wearable device to capture eye-tracking data corresponding to one or both eyes of the user of the wearable device, determining whether the convergence / divergence movement distance can be determined based on the eye-tracking data, and determining that the convergence / divergence movement distance cannot be determined based on the eye-tracking data.

[0026] Example 21 is the method described in Example 16, where the step of determining whether the activation condition is satisfied includes the step of determining that the eye-tracking data is unavailable.

[0027] Example 22 is the method described in Example 16, where the step of determining whether the activation condition is satisfied includes the step of determining whether the virtual content to be displayed is represented on both the first focal plane and the second focal plane, and the step of determining that the virtual content to be displayed is represented on both the first focal plane and the second focal plane.

[0028] Example 23 further includes deactivating the monovision display mode in response to a determination that the activation conditions are no longer met, and the step of deactivating the monovision display mode includes causing the left and right optical stacks to display virtual content on a first focal plane, or causing the left and right optical stacks to display virtual content on a second focal plane, the method according to Example 16.

[0029] Example 24 further includes modifying the monovision display mode after a certain amount of time, thereby causing the left optical stack to switch from displaying virtual content on a first focal plane to displaying virtual content on a second focal plane, and causing the right optical stack to switch from displaying virtual content on a second focal plane to displaying virtual content on a first focal plane, the method according to Example 16.

[0030] Example 25 is a wearable system configured to perform any of the methods according to Examples 16-24.

[0031] Example 26 is a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods according to Examples 16-24.

[0032] A number of benefits over conventional techniques are achieved by the method of the present disclosure. For example, the embodiments described herein reduce the VAC within a defined operating range for an eyeglass AR / VR headset device by extending the depth of field through monovision. This allows each user's eye to receive different refractive powers for virtual content while the real world remains unmodified. This encourages the user to focus more clearly, regardless of which eye, essentially reducing the VAC and increasing the perceived sharpness of the virtual and / or real world images.

Brief Description of the Drawings

[0033]

Figure 1

[0034]

Figure 2

[0035]

Figure 3

[0036]

Figure 4A

Figure 4B

[0037]

Figure 5

[0038]

Figure 6

[0039]

Figure 7

[0040]

Figure 8

[0041]

Figure 9

[0042]

Figure 10

[0043]

Figure 11

[0044]

Figure 12

[0045]

Figure 13

DETAILED DESCRIPTION OF THE INVENTION

[0046] Detailed description of specific embodiments An augmented reality (AR) wearable display can present virtual content to a user by utilizing a focal plane. Many of these displays afflict the user with visual discomfort due to vergence-accommodation conflict (VAC), where they are occluded differently at different depths. During natural vision in the real world, the inward rotation (vergence) of the user's eyes and focus control or accommodation are neutrally coupled. When the user suffers from VAC, the user's brain receives an inconsistency cue between the distance of the virtual object and the convergence distance required for the eye to focus on that object. VAC leads to visual fatigue, headaches, nausea, and eye strain and remains a significant source of discomfort for the user. Thus, to maintain user comfort, modern AR and mixed reality (MR) wearable displays may consider the VAC budget allowance when delivering virtual content over a certain depth range, which can result in a significantly reduced depth range.

[0047] Various approaches for reducing VAC can be implemented. One approach involves adding a second focal plane and a variable focus switch based on an eye-tracking display system. Many of these systems cannot render information at different depths simultaneously, and thus, the entire scene is rendered using one of the two focal planes. The focal plane on which the entire scene is rendered may be selected based on the line-of-sight information calculated using an eye-tracking camera. Another approach is to add an eyepiece focal plane with the ability to sweep across a wide range to a variable focus element. This approach may entail increased volume through the integration of an additional eyepiece layer in the form of and / or a liquid-fillable adjustable lens pair spanning the eyepiece, and increased complexity due to a complex illumination scheme.

[0048] The embodiments described herein include wearable systems and devices that incorporate monovision display techniques in which each user's eye perceives virtual content having different refractive powers. These systems and devices generally overcome many of the problems associated with conventional wearable devices of displaying an entire scene on a single focal plane for both optical stacks, which are generally associated with significant VAC. In some embodiments, the wearable device may activate and deactivate a monovision display mode based on whether activation conditions are met. The activation conditions may include, among other possibilities, the availability of eye tracking information, the depth of the virtual content to be displayed, and user preferences.

[0049] Figures 1A - 1C illustrate examples of the convergence / divergence movement and accommodation of a user's eyes during natural vision. As used herein, convergence / divergence movement refers to the rotation of the eyes to fixate on an object and maintain a single fused image, and accommodation refers to the adjustment of the refractive power of the eye's lens to maintain a sharp image on the retina. The distance of the fixed object from the eyes to which the convergence / divergence movement is directed is referred to as the convergence / divergence movement distance, and the distance from the eyes at which accommodation is adjusted is referred to as the accommodation distance. Convergence / divergence movement and accommodation are neutrally coupled during natural vision in the real world such that the convergence / divergence movement distance and the accommodation distance are equal.

[0050] As shown in Figures 1A - 1C, as the object being fixated is placed at various distances from the eyes, the convergence / divergence movement distance and the accommodation distance are matched to that distance. As the object becomes closer in Figures 1B and 1C, the user's eye accommodation is adjusted by increasing the thickness of the lens (and thus its refractive power). This clearly projects the object onto the retina while causing natural depth of field blur to other objects within the user's field of view at different distances from the user's eyes.

[0051] Figures 2A-2C illustrate examples of the convergence / divergence movement and accommodation of the user's eyes when using an AR / MR display that utilizes a fixed focal plane. When viewing a virtual object in a set at the same distance as the real-world object shown in Figures 1A-1C, the virtual object appears to the user to be located at the correct set of distances. However, the light associated with the virtual object originates from the fixed distance of the display, which does not change the refractive power of the eye's lens. Thus, the convergence / divergence movement distance and the accommodation distance generally do not match because the eye accommodates to the fixed distance of the display while converging to the distance of the virtual object. Additionally, other virtual objects within the user's field of view at different distances from the user's eyes may not suffer from natural depth-of-field blur and may instead appear sharp.

[0052] Figure 3 illustrates an AR scene 300 as viewed through a wearable AR device, according to some embodiments of the present invention. The AR scene 300 depicts, for a user of AR technology, a real-world park-like setting 306 featuring various real-world objects 330 such as people, trees, buildings in the background, and a real-world concrete platform 320. In addition to these items, the user of AR technology also "sees" various virtual objects 302 such as a robotic image 302-2 standing on the real-world concrete platform 320 and a comic-like avatar character 302-1 flying beside it that appears anthropomorphic like a bumblebee. However, these elements (character 302-1 and image 302-2) do not exist in the real world.

[0053] Figure 4A illustrates an AR device 400A having a single fixed focal plane according to some embodiments of the present invention. During operation, the projector 414 of the AR device 400A may project virtual image light 423 (i.e., light associated with virtual content) onto the eyepiece lens 402-1, which projects a light field (i.e., an angular representation of the virtual content) onto the user's retina in such a manner that the user perceives the corresponding virtual content to be located at several places within the user's environment. For example, the virtual image light 423 externally coupled by the eyepiece lens 402-1 may cause the user to perceive the image 302-2 to be located on the second virtual depth plane 410-2 and the character 302-1 to be located on the first virtual depth plane 410-1. The user perceives the virtual content together with world light 432 corresponding to one or more world objects 430 such as the platform 320.

[0054] In some embodiments, the AR device 400A includes a first lens assembly 405-1 positioned on the user side of the eyepiece lens 402-1 (the side of the eyepiece lens 402-1 closest to the user's eye) and a second lens assembly 405-2 positioned on the world side of the eyepiece lens 402-1. The lens assemblies 405-1, 405-2 may each be configured to apply a refractive power to the light passing therethrough.

[0055] Figure 4B illustrates an AR device 400B having two fixed focal planes, according to some embodiments of the present invention. During operation, the projector 414 may project virtual image light 423 onto the first eyepiece lens 402-1 and the second eyepiece lens 402-2, which may project a light field onto the user's retina in a manner such that the user perceives the corresponding virtual content to be located at several places within the user's environment. For example, the virtual image light 423 externally coupled by the first eyepiece lens 402-1 may cause the user to perceive the character 102-1 to be positioned on the first virtual depth plane 410-1, and the virtual image light 423 externally coupled by the second eyepiece lens 402-2 may cause the user to perceive the image 102-2 to be positioned on the second virtual depth plane 410-2.

[0056] Figure 5 illustrates a schematic diagram of an exemplary wearable system 500, according to some embodiments of the present invention. The wearable system 500 may include a wearable device 5A and at least one remote device 503 remote from the wearable device 501 (e.g., separate hardware but communicatively coupled). The wearable device 501, as described with reference to Figure 5, may correspond to the AR device 400 as described above with reference to Figures 4A and 4B. While the wearable device 501 is being worn by the user (generally as a headset), the remote device 503 may be held by the user (e.g., as a handheld controller), or fixedly attached to a helmet or hat worn by the user, fixedly attached to a frame, incorporated within headphones, or otherwise removably attached to the user (e.g., in a backpack configuration, in a belt-coupled configuration, etc.) and may be mounted in various configurations.

[0057] The wearable device 501 may include a left eyepiece 502A and a left lens assembly 505A that are arranged in a juxtaposed configuration and constitute a left optical stack. The left lens assembly 505A may include a focusing lens on the user side of the left optical stack and a compensating lens on the world side of the left optical stack. Similarly, the wearable device 501 may include a right eyepiece 502B and a right lens assembly 505B that are arranged in a juxtaposed configuration and constitute a right optical stack. The right lens assembly 505B may include a focusing lens on the user side of the right optical stack and a compensating lens on the world side of the right optical stack.

[0058] In some embodiments, the wearable device 501 includes one or more sensors including, but not limited to, a left front-facing world camera 506A directly attached to or near the left eyepiece 502A, a right front-facing world camera 506B directly attached to or near the right eyepiece 502B, a left side-facing world camera 506C directly attached to or near the left eyepiece 502A, a right side-facing world camera 506D directly attached to or near the right eyepiece 502B, a left eye tracking camera 526A directed towards the left eye, a right eye tracking camera 526B directed towards the right eye, and a depth sensor 528 attached between the eyepieces 502. The wearable device 501 may also include one or more image projection devices such as a left projector 514A optically coupled to the left eyepiece 502A and a right projector 514B optically coupled to the right eyepiece 502B.

[0059] The wearable system 500 may include a processing module 550 for collecting, processing, and / or controlling data within the system. The components of the processing module 550 may be distributed between the wearable device 501 and the remote device 503. For example, the processing module 550 may include a local processing module 552 on the wearable portion of the wearable system 500 and a remote processing module 556 that is physically separate from and communicatively coupled to the local processing module 552. The local processing module 552 and the remote processing module 556 may each include one or more processing units (e.g., a central processing unit (CPU), a graphics processing unit (GPU), etc.) and one or more storage devices such as non-volatile memory (e.g., flash memory).

[0060] The processing module 550 may collect data captured by various sensors of the wearable system 500 such as a camera 506, an eye-tracking camera 526, a depth sensor 528, a remote sensor 530, an ambient light sensor, a microphone, an inertial measurement unit (IMU), an accelerometer, a compass, a global navigation satellite system (GNSS) unit, a wireless device, and / or a gyroscope. For example, the processing module 550 may receive an image 520 from the camera 506. Specifically, the processing module 550 may receive a left front image 520A from the world camera 506A facing left front, a right front image 520B from the world camera 506B facing right front, a left side image 520C from the world camera 506C facing left side, and a right side image 520D from the world camera 506D facing right side. In some embodiments, the image 520 may include a single image, a pair of images, a video including a stream of images, a video including a stream of paired images, and the like. The image 520 may be generated periodically and transmitted to the processing module 550 while the wearable system 500 is powered on, or may be generated in response to an instruction transmitted by the processing module 550 to one or more of the cameras.

[0061] The camera 506 may be configured at various positions and orientations along the outer surface of the wearable device 501 to capture an image of the user's surroundings. In some instances, the cameras 506A, 506B may be positioned to capture images that substantially overlap the FOVs of the user's left and right eyes, respectively. Thus, the cameras 506 may be located near the user's eyes, but not so close as to obscure the user's FOV. Alternatively, or in addition, the cameras 506A, 506B may be positioned to align with the internal conjugation locations of the virtual image lights 522A, 522B, respectively. The cameras 506C, 506D may be positioned to capture images of the user's sides, such as within or outside the user's peripheral vision. The images 520C, 520D captured using the cameras 506C, 506D need not necessarily overlap with the images 520A, 520B captured using the cameras 506A, 506B.

[0062] In some embodiments, the processing module 550 may receive ambient light information from the ambient light sensor. The ambient light information may indicate a brightness value or a spatially resolved range of brightness values. The depth sensor 528 may capture a depth image 532 in a direction facing the front of the wearable device 501. Each value of the depth image 532 may correspond to the distance between the depth sensor 528 and the closest detected object in a particular direction. As another example, the processing module 550 may receive eye tracking data 534 from the eye tracking camera 526, which may include images of the left and right eyes. As another example, the processing module 550 may receive projected image brightness values from one or both of the projectors 514. The remote sensor 530 located within the remote device 503 may include any of the sensors described above with similar functionality.

[0063] Virtual content is delivered to the user of the wearable system 500 using the projectors 514 and the eyepieces 502, along with other components within the optical stack. For example, the eyepieces 502A, 502B may each comprise a transparent or translucent waveguide configured to direct and out-couple light generated by the projectors 514A, 514B, respectively. Specifically, the processing module 550 may cause the left projector 514A to output left virtual image light 522A onto the left eyepiece 502A, and the right projector 514B to output right virtual image light 522B onto the right eyepiece 502B. In some embodiments, the projector 514 may include a microelectromechanical systems (MEMS) spatial light modulator (SLM) scanning device. In some embodiments, the eyepieces 502A, 502B may each comprise a plurality of waveguides corresponding to different colors. In some embodiments, the lens assemblies 505A, 505B may be coupled to and / or integrated with the eyepieces 502A, 502B. For example, the lens assemblies 505A, 505B may be incorporated within a multi-layer eyepiece and form one or more layers that constitute one of the eyepieces 502A, 502B.

[0064] Figure 6 illustrates the relationship between the distance of the VAC and the virtual depth plane for each of the AR devices 400A, 400B, described with reference to FIGS. 4A and 4B respectively. For the AR device 400B, the two - focal - plane system provides switchable focal planes at 1.95 diopters (0.51 meters) and 0.65 diopters (1.54 meters), with a switching point at 1.3 diopters (0.77 meters), a near - content limit (clipping plane) at 2.7 diopters (0.37 meters), and the ability to provide an imaging that never exceeds 1.0 diopter VAC between that plane and infinity. For the AR device 400A, the single fixed - focal - plane system has a focal - plane location at 1.5 diopters (0.6 meters), a near - content limit of 2.5 diopters (0.4 meters), and a far - content limit of 0.31 diopters (3.2 meters), assuming a maximum allowable VAC of 1.0 diopter. Such a configuration has an available range of 0.4 to 3.2 meters where the content falls outside that range, and would require some solution to mitigate the VAC limit exceedance.

[0065] Figures 7A and 7B illustrate graphical representations of the perceived sharpness of virtual content under different conditions. Figure 7A illustrates the perceived sharpness degradation of virtual content (left - hand graphic) or real - world objects (right - hand graphic) for a single - focal - plane solution where a large amount of VAC is present. Figure 7B illustrates the perceived sharpness of both virtual content and real - world objects when monovision is employed by a wearable system.

[0066] FIG. 8 illustrates a wearable device 800 that utilizes a monovision solution according to some embodiments of the present invention. The wearable device 800 includes a set of inner refractive lenses that provide a desired refractive power of virtual content to the user's eyes. The wearable device 800 may be calibrated such that all distortions and associated magnifications are not distinguishable between the left virtual image and the right virtual image, and only significant artifacts between the left virtual image and the right virtual image are associated with the depth / parallax adjustment plane. The wearable device 800 also includes a set of outer refractive lenses that compensate for real-world distortions and magnifications for each eye.

[0067] Specifically, the wearable device 800 includes a left optical stack 802A and a right optical stack 802B, each of which includes an inner lens, an eyepiece lens, and an outer lens. The eyepiece lenses may each receive virtual image light from a projector of the wearable device 800 and output the virtual image light toward one of the user's eyes. The inner lenses, which may also be referred to as focusing lenses and / or diverging lenses, may each impart a negative refractive power to light passing therethrough, which may include virtual image light and / or world light. The outer lenses, which may also be referred to as compensating lenses and / or converging lenses, may each impart a positive refractive power to light passing therethrough, which may include only world light.

[0068] The near and far accommodation of the eye is promoted by the near and far accommodation demand of the eye, which is optimally corrected to view the convergence / divergence movement distance. This concept encourages the user to accommodate near and far so that any eye has a sharper image and less VAC. This essentially reduces the VAC and thus improves visual comfort over the target operating range. If the desired operating range of the virtual content is defined to be 2.65 to 0 diopters (37 cm to infinity) and the acceptable VAC is defined to be <1.0 diopter, this will place the nominal depth plane at approximately 1.3 diopters and there will be a refractive power difference of + / -0.3 diopters between the left and right eyes. The difference between the refractive powers of the left and right inner lenses may be referred to as the offset amount, which in the illustrated embodiment is equal to 0.65 diopters.

[0069] Figures 9A and 9B illustrate graphs showing VAC versus content position measured from the user's eye plane, according to some embodiments of the present invention. Figure 9A illustrates a single depth plane solution centered at 1.3 diopters, and Figure 9B illustrates an equivalent solution using the wearable device 800. As shown, the VAC experienced by the user is equal to the minimum VAC between the two focal planes. For example, the VAC experienced by the user over a distance closer than about 0.8 meters is equal to the VAC for the near focal plane, and the VAC experienced by the user over a distance farther than about 0.8 meters is equal to the VAC for the far focal plane. The + / -0.3 diopter anisometropia (or 0.6 diopter offset amount) in this example can be replaced by any other amount of anisometropia to achieve different levels of VAC.

[0070] Figure 10 illustrates examples of different operating states of a wearable device according to some embodiments of the present invention. The wearable device includes left and right optical stacks 1002, each with the ability to switch between displaying virtual content at a near focal plane and a far focal plane. According to the first operating state 1000-1, the optical stacks 1002 each cause virtual content to be displayed at the near focal plane. According to the second operating state 1000-2, the optical stacks 1002 each cause virtual content to be displayed at the far focal plane. According to the third operating state 1000-3, the left optical stack 1002A causes virtual content to be displayed at the far focal plane, and the right optical stack 1002B causes virtual content to be displayed at the near focal plane. According to the fourth operating state 1000-4, the left optical stack 1002A causes virtual content to be displayed at the near focal plane, and the right optical stack 1002B causes virtual content to be displayed at the far focal plane.

[0071] The wearable device can switch between different operating states 1000 based on various factors. For example, eye-tracking data captured by an eye-tracking camera can be analyzed to determine that the user is fixating on the near focal plane or the far focal plane, and thus the wearable device may switch to the operating state 1000-1 or the operating state 1000-2, respectively. In some embodiments, the virtual content to be displayed can be analyzed to determine that the virtual content is represented only at the near focal plane or the far focal plane, and thus the wearable device may switch to the operating state 1000-1 or the operating state 1000-2, respectively.

[0072] In some embodiments, it may be determined that activation conditions associated with the monovision display mode are met, and thus, the wearable device may activate the monovision display mode by switching to either operating state 1000-3 or operating state 1000-4. The activation conditions may include that eye tracking data is unavailable (e.g., the eye tracking camera is unable to capture an image of the eye), the vergence / accommodation movement distance cannot be determined based on the eye tracking data (e.g., the eye tracking data is too unreliable or inconsistent), the virtual content to be displayed is represented on both the near and far focal planes, the user provides an input indicating user discomfort, the user provides an input indicating a preference for monovision, or any combination thereof.

[0073] After a predetermined amount of time from the activation of the monovision display mode, the wearable device may re-evaluate the activation conditions and determine whether they remain met. If the activation conditions are no longer met (e.g., the eye tracking data becomes available, the virtual content to be displayed is represented on a single focal plane, etc.), the wearable device may deactivate the monovision display mode by switching to either operating state 1000-1 or operating state 1000-2.

[0074] In some embodiments, after a predetermined amount of time from the activation of the monovision display mode, the wearable device may switch from operating state 1000-3 to operating state 1000-4 or from operating state 1000-4 to operating state 1000-3. This can prevent the same anisometropia from being imposed on the user's eyes over a long period of time. By periodically switching the focal plane, this situation can be prevented from occurring.

[0075] FIG. 11 illustrates a method 1100 for operating a wearable device according to some embodiments of the present invention. One or more steps of method 1100 may be omitted during the implementation of method 1100, and the steps of method 1100 need not be performed in the order shown. One or more steps of method 1100 may be performed or facilitated by one or more processors, such as those included within processing module 550.

[0076] In step 1102, virtual content to be displayed in each of the optical stacks is received at the wearable device. In step 1104, it is determined whether the virtual content to be displayed is represented on both a first focal plane and a second focal plane. If so, method 1100 proceeds to step 1108. If not, method 1100 proceeds to step 1106. In step 1106, a first focal plane or a second focal plane is selected for displaying virtual content for both optical stacks based on the focal plane on which the virtual content is represented. In step 1108, it is determined whether eye-tracking data is available. If so, method 1100 proceeds to step 1110. If not, method 1100 proceeds to step 1112.

[0077] In step 1110, a first focal plane or a second focal plane is selected for displaying virtual content for both optical stacks based on the focal plane on which the user is fixating. In step 1112, a monovision display mode is activated by the wearable device by causing the virtual content to be displayed on a first focal plane in the left optical stack and on a second focal plane in the right optical stack. The optical stacks may display virtual content on different focal planes simultaneously.

[0078] FIG. 12 illustrates a method 1200 for operating a wearable device according to some embodiments of the present invention. One or more steps of method 1200 may be omitted during the implementation of method 1200, and the steps of method 1200 need not be performed in the order shown. One or more steps of method 1200 may be performed or facilitated by one or more processors, such as those included within processing module 550.

[0079] In step 1202, virtual content to be displayed on each of the optical stacks is received at the wearable device. For example, an application launched on the wearable device may render a left frame and a right frame (or a series of left frames and right frames) to be displayed and deliver the frames to the left projector and the right projector of the wearable device.

[0080] In step 1204, the left optical stack and the right optical stack are caused to display the virtual content on the same focal plane. For example, both optical stacks may display the virtual content on a first focal plane of the wearable device, or both optical stacks may display the virtual content on a second focal plane of the wearable device. In some examples, the first focal plane is a near focal plane and the second focal plane is a far focal plane. In some examples, the first focal plane is a far focal plane and the second focal plane is a near focal plane. The refractive power associated with the first focal plane and the refractive power associated with the second focal plane may differ by some offset amount.

[0081] In step 1206, it is determined that the activation conditions are satisfied. The activation conditions may include that the eye-tracking data is unavailable (for example, the eye-tracking camera is unable to capture an image of the eye), the convergence / divergence movement distance cannot be determined based on the eye-tracking data (for example, the eye-tracking data is too unreliable or inconsistent), the virtual content to be displayed is represented on both the first focal plane and the second focal plane, the user provides an input indicating user discomfort, the user provides an input indicating a preference regarding monovision, or any combination thereof.

[0082] In step 1208, in response to the determination that the activation conditions are satisfied, the monovision display mode associated with the wearable device is activated. The step of activating the monovision display mode may include, in step 1210, the step of displaying the virtual content on the first focal plane on the left optical stack, and, in step 1212, the step of displaying the virtual content on the second focal plane on the right optical stack.

[0083] In step 1214, it is determined that the activation conditions are no longer satisfied. The step of determining that the activation conditions are no longer satisfied may include determining that the eye-tracking data is available, the virtual content is represented on a single focal plane, the user provides an input indicating user comfort, or the provision of an input indicating user discomfort has stopped, the user provides an input for deactivating monovision, or any combination thereof.

[0084] In step 1216, in response to the determination that the activation condition is no longer satisfied, the monovision display mode is deactivated. The step of deactivating the monovision display mode may include, in step 1218, the step of causing the left and right optical stacks to display virtual content on the same focal plane. For example, both optical stacks may display virtual content on the first focal plane or the second focal plane of the wearable device.

[0085] FIG. 13 illustrates a simplified computer system 1300 according to an embodiment described herein. The computer system 1300 as illustrated in FIG. 13 may be incorporated within a device as described herein. FIG. 13 provides a schematic illustration of one embodiment of a computer system 1300 that may perform some or all of the steps of a method provided by various embodiments. It should be noted that FIG. 13 is intended only to provide a generalized illustration of various components and that any or all of them may be utilized as necessary. FIG. 13 thus illustrates, in a broad sense, situations in which individual system elements may be implemented in a relatively separated manner or in a relatively more integrated manner.

[0086] The computer system 1300 is shown to include hardware elements that can be electrically coupled via a bus 1305 or communicate in other ways as necessary. The hardware elements may include one or more processors 1310 including, but not limited to, one or more general-purpose processors and / or one or more special-purpose processors such as digital signal processing chips, graphics acceleration processors, and / or the like; one or more input devices 1315 including, but not limited to, a mouse, keyboard, camera, and / or the like; and one or more output devices 1320 including, but not limited to, a display device, printer, and / or the like.

[0087] The computer system 1300 can further include, without limitation, local and / or network-accessible storage devices, and / or one or more non-transitory storage devices 1325 including, without limitation, disk drives, drive arrays, optical storage devices, solid state storage devices such as random access memory (“RAM”), and / or read only memory (“ROM”) that can be programmable, flash updatable, and / or the like, and / or communicate therewith. Such storage devices may be configured to implement any suitable data storage including, without limitation, various file systems, database structures, and / or the like.

[0088] Computer system 1300 may also include a communication subsystem 1319 including, but not limited to, modems, network cards (wireless or wired), infrared communication devices, wireless communication devices, and / or chip sets such as Bluetooth® devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication equipment, and / or the like. The communication subsystem 1319 includes one or more input and / or output communication interfaces and may enable data to be exchanged with a network such as, by way of example for one embodiment, the networks described hereinafter, namely other computer systems, televisions, and / or any other devices described herein. Depending on the desired functionality and / or other implementation considerations, a portable electronic device or similar device may communicate images and / or other information via the communication subsystem 1319. In other embodiments, a portable electronic device, e.g., a first electronic device, may be incorporated as an input device 1315 within a computer system 1300, e.g., an electronic device. In some embodiments, computer system 1300 further comprises a working memory 1335, which may include a RAM or ROM device as described above.

[0089] The computer system 1300 may also include a computer program provided by various embodiments and / or implement the methods provided by other embodiments as described herein and / or be designed to configure the system, including an operating system 1340, device drivers, executable libraries, and / or other code, such as one or more application programs 1345, shown as being currently located within the working memory 1335. Merely by way of example, one or more procedures described with respect to the methods discussed above may be implemented as code and / or instructions executable by a computer or a processor within a computer, and in some aspects, such code and / or instructions may then be used to configure and / or adapt a general-purpose computer or other device to perform one or more operations in accordance with the described methods.

[0090] These sets of instructions and / or code may be stored on a non-transitory computer-readable storage medium, such as the storage device 1325 described above. In some cases, the storage medium may be incorporated within a computer system, such as the computer system 1300. In other embodiments, the storage medium is separate from the computer system, for example, a removable medium such as a compact disc, and / or may be provided within an installation package such that the storage medium can be used to program, configure, and / or adapt a general-purpose computer with the instructions / code stored thereon. These instructions may take the form of executable code executable by the computer system 1300 and / or may take the form of source and / or installable code that, upon compilation and / or installation onto the computer system 1300 using, for example, any of various generally available compilers, installation programs, compression / decompression utilities, etc., then takes the form of executable code.

[0091] It will be apparent to those skilled in the art that substantial variations may be made and yet remain within the scope of the specific requirements. For example, customized hardware may also be used and / or particular elements may be implemented in software, or both, including portable software such as applets, hardware, etc. Further, connections to other computing devices, such as network input / output devices, may also be employed.

[0092] As described above, in one aspect, some embodiments may employ a computer system, such as computer system 1300, to perform methods according to various embodiments of the present technology. According to one set of embodiments, some or all of the procedures of such methods may be performed by computer system 1300 in response to one or more sequences of one or more instructions that may be incorporated within operating system 1340 and / or other code, such as application program 1345, contained within working memory 1335, where the processor 1310 may execute the instructions. Such instructions may be read into working memory 1335 from another computer-readable medium, such as one or more of storage devices 1325. Merely by way of example, execution of a sequence of instructions contained within working memory 1335 may cause processor 1310 to perform one or more procedures of the methods described herein. Additionally, or alternatively, some of the methods described herein may be performed through special hardware.

[0093] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any medium involved in providing data that causes a machine to operate in a specific manner. In certain embodiments implemented using computer system 1300, various computer-readable media may be involved in providing instructions / codes for execution to processor 1310 and / or may be used to store and / or carry such instructions / codes. In many implementations, the computer-readable medium is a physical and / or tangible storage medium. Such a medium may take the form of a non-volatile medium or a volatile medium. Non-volatile media includes, for example, optical and / or magnetic disks such as storage device 1325. Volatile media includes, but is not limited to, dynamic memory such as work memory 1335.

[0094] Common forms of physical and / or tangible computer-readable media include, for example, a floppy (registered trademark) disk, flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read instructions and / or codes.

[0095] Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions for execution to processor 1310. Merely by way of example, the instructions may first be carried on a magnetic disk and / or optical disk of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit the instructions as a signal via a transmission medium that is received and / or executed by computer system 1300.

[0096] The communication subsystem 1319 and / or its components generally receive signals, and then the bus 1305 can convey the signals and / or the data, instructions, etc. carried by the signals to the working memory 1335, from where the processor 1310 reads and executes the instructions. The instructions received by the working memory 1335 may optionally be stored on the non-transitory memory device 1325 either before or after execution by the processor 1310.

[0097] The methods, systems, and devices discussed above are examples. Various configurations may omit, substitute, or add various procedures or components as needed. For example, in alternative configurations, the method may be performed in a different order than that described, and / or various steps may be added, omitted, and / or combined. Also, features described with respect to one configuration may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Also, technology advances, and thus many of the elements are examples and do not limit the scope or claims of the present disclosure.

[0098] Specific details are given in the description to provide a complete understanding of the exemplary configurations, including implementations. However, the configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail to avoid obscuring the configurations. This description provides only exemplary configurations and does not limit the scope, usability, or configurations of the claims. Rather, the foregoing description of the configurations will provide an effective description for those skilled in the art to implement the techniques described. Various changes may be made to the functions and arrangements of the elements without departing from the spirit or scope of the present disclosure.

[0099] Also, the architecture can be described as a process, depicted as a schematic flowchart or block diagram. Each can be described as an iterative process, although many of the operations can be performed in parallel or simultaneously. Additionally, the order of operations can be rearranged. The process may have additional steps not included in the figures. Further, embodiments of the present method may be implemented by hardware, software, firmware, middleware, microcode, a hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments for performing the required tasks may be stored in a non-transitory computer-readable medium such as a storage medium. A processor may perform the described tasks.

[0100] Although some exemplary architectures have been described, various modifications, alternative structures, and equivalents may be used without departing from the spirit of the present disclosure. For example, the elements described above may be components of a larger system, and other rules may take precedence over or otherwise modify the use of this technology. Also, some steps may be performed before, during, or after the elements described above are considered. Thus, the foregoing description does not limit the scope of the claims.

[0101] As used in this specification and the appended claims, the singular forms "a", "an", and "the" include plural references unless the context clearly dictates otherwise. Thus, for example, reference to "a user" includes reference to a plurality of such users, and reference to "a processor" includes reference to one or more processors and equivalents thereof known to those skilled in the art.

[0102] Also, the words "comprise", "comprising", "contains", "containing", "include", "including", and "includes", when used in this specification and the following claims, are intended to specify the presence of the stated features, integers, components, or steps, but they do not preclude the presence or addition of one or more other features, integers, components, steps, acts, or groups.

[0103] Also, the examples and embodiments described herein are for illustrative purposes only, and in light of this, it should be understood that various modifications or changes may be suggested to those skilled in the art and are included within the spirit and scope of the present application and the appended claims.

Claims

1. An optical system, a left optical stack configured to output left virtual image light toward a user side of the optical system and having a first left eyepiece lens and a second left eyepiece lens respectively associated with a first focal plane and a second focal plane, a right optical stack configured to output right virtual image light toward the user side of the optical system and having a first right eyepiece lens and a second right eyepiece lens respectively associated with the first focal plane and the second focal plane, wherein the left optical stack and the right optical stack are each configured to switch between displays of virtual content at the first focal plane or the second focal plane, a right optical stack, a processing module, determining whether an activation condition is satisfied, in response to determining that the activation condition is satisfied, activating a monovision display mode associated with the optical system, wherein activating the monovision display mode includes: outputting the left virtual image light at the first left eyepiece lens to cause the left optical stack to display the virtual content only at the first focal plane, and outputting the right virtual image light at the second right eyepiece lens to cause the right optical stack to display the virtual content only at the second focal plane; or or outputting the left virtual image light at the second left eyepiece lens to cause the left optical stack to display the virtual content only at the second focal plane, and outputting the right virtual image light at the first right eyepiece lens to cause the right optical stack to display the virtual content only at the first focal plane, and a processing module configured to perform operations including the above, An optical system comprising the above.

2. The optical system according to claim 1, wherein a refractive power associated with the first focal plane and a refractive power associated with the second focal plane differ by an offset amount.

3. The optical system according to claim 2, wherein the offset amount exceeds a threshold value.

4. The optical system according to claim 3, wherein the threshold value is one of 0.1 D, 0.2 D, 0.3 D, 0.4 D, 0.5 D, 0.6 D, 0.7 D, 0.8 D, 0.9 D, or 1.0 D.

5. Determining whether the activation condition is satisfied includes capturing eye-tracking data corresponding to one or more than one eye of a user of the optical system using an eye-tracking camera of the optical system, determining whether a convergence / divergence movement distance can be determined based on the eye-tracking data, determining that the convergence / divergence movement distance cannot be determined based on the eye-tracking data, The optical system according to claim 1, comprising.

6. Determining whether the activation condition is satisfied includes determining that eye-tracking data is unavailable, The optical system according to claim 1.

7. Determining whether the activation condition is satisfied includes determining that the virtual content to be displayed is represented on both the first focal plane and the second focal plane, The optical system according to claim 1, comprising.

8. The operation further includes deactivating the monovision display mode in response to determining that the activation condition is no longer satisfied, The optical system according to claim 1, comprising.

9. The operation further includes modifying the monovision display mode after a predetermined amount of time, The optical system according to claim 1, comprising.

10. A method comprising providing a wearable device including a left optical stack and a right optical stack, wherein the left optical stack has a first left eyepiece lens and a second left eyepiece lens respectively associated with a first focal plane and a second focal plane, and the right optical stack has a first right eyepiece lens and a second right eyepiece lens respectively associated with the first focal plane and the second focal plane, and the left optical stack and the right optical stack are each configured to switch between displaying virtual content on the first focal plane or the second focal plane, receiving virtual content to be displayed in the left optical stack and the right optical stack, determining whether an activation condition is satisfied, activating a monovision display mode associated with the wearable device in response to determining that the activation condition is satisfied, wherein activating the monovision display mode includes By outputting left virtual image light in the first left eyepiece lens, causing the left optical stack to display the virtual content only on the first focal plane, and by outputting right virtual image light in the second right eyepiece lens, causing the right optical stack to display the virtual content only on the second focal plane; or By outputting the left virtual image light in the second left eyepiece lens, causing the left optical stack to display the virtual content only on the second focal plane, and by outputting the right virtual image light in the first right eyepiece lens, causing the right optical stack to display the virtual content only on the first focal plane; including including

11. The refractive power associated with the first focal plane and the refractive power associated with the second focal plane differ by an offset amount, the method according to claim 10.

12. The offset amount exceeds a threshold value, the method according to claim 11.

13. The threshold value is one of 0.1 D, 0.2 D, 0.3 D, 0.4 D, 0.5 D, 0.6 D, 0.7 D, 0.8 D, 0.9 D, or 1.0 D, the method according to claim 12.

14. Determining whether the activation condition is satisfied using one or more eye-tracking cameras of the wearable device to capture eye-tracking data corresponding to one or both eyes of a user of the wearable device; determining whether a convergence / divergence movement distance can be determined based on the eye-tracking data; determining that the convergence / divergence movement distance cannot be determined based on the eye-tracking data including, the method according to claim 10.

15. Determining whether the activation condition is satisfied includes determining that eye-tracking data is unavailable the method according to claim 10.

16. Determining whether the activation condition is satisfied includes determining that the virtual content to be displayed is represented on both the first focal plane and the second focal plane including, the method according to claim 10.

17. The method according to claim 10, further comprising deactivating the monovision display mode in response to a determination that the activation condition is no longer satisfied.

18. The method according to claim 10, further comprising modifying the monovision display mode after a predetermined amount of time.

19. A non-transitory computer-readable medium, the non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to receive virtual content to be displayed on a left optical stack and a right optical stack of a wearable device, the left optical stack having a first left eyepiece lens and a second left eyepiece lens respectively associated with a first focal plane and a second focal plane, the right optical stack having a first right eyepiece lens and a second right eyepiece lens respectively associated with the first focal plane and the second focal plane, the left optical stack and the right optical stack each being configured to switch between displaying virtual content at the first focal plane or the second focal plane, determine whether an activation condition is satisfied; in response to a determination that the activation condition is satisfied, activate a monovision display mode associated with the wearable device, wherein activating the monovision display mode comprises causing the left optical stack to display the virtual content only on the first focal plane by outputting left virtual image light in the first left eyepiece lens, and causing the right optical stack to display the virtual content only on the second focal plane by outputting right virtual image light in the second right eyepiece lens; or or causing the left optical stack to display the virtual content only on the second focal plane by outputting the left virtual image light in the second left eyepiece lens, and causing the right optical stack to display the virtual content only on the first focal plane by outputting the right virtual image light in the first right eyepiece lens; including perform operations including. A non-transitory computer-readable medium.

20. The refractive power associated with the first focal plane and the refractive power associated with the second focal plane differ by an offset amount, the non-transitory computer-readable medium according to claim 19.

Images