Display system and method for clipping content and increasing viewing comfort - Patents.com

Abstract

To provide display systems and methods for clipping content to increase viewing comfort.SOLUTION: Augmented and virtual reality display systems increase viewer comfort by reducing viewer exposure to virtual content that causes undesirable accommodation-vergence mismatches (AVM), limit displaying content that exceeds an accommodation-vergence mismatch threshold, which may define a volume around the viewer. The volume may be subdivided into two or more zones, including an innermost loss-of-fusion zone (LoF) in which no content is displayed, and one or more outer AVM zones in which the displaying of content may be stopped, or clipped, under certain conditions. For example, content may be clipped if the viewer is verging within an AVM zone and if the content is displayed within the AVM zone for more than a threshold duration.SELECTED DRAWING: Figure 13A

Description

[Technical field]

[0001] (Priority Claim) This application claims the benefit of priority to U.S. Provisional Application No. 62 / 642,761, filed March 14, 2018, which is incorporated by reference in its entirety.

[0002] (Incorporated by reference) This application is incorporated by reference into the following patent applications: U.S. Patent Application No. 14 / 555,585, filed on November 27, 2014, and published on July 23, 2015 as U.S. Patent Publication No. 2015 / 0205126; U.S. Patent Application No. 14 / 555,585, filed on April 18, 2015, and published on October 22, 2015 as U.S. Patent Publication No. 2015 / 0302652; No. 690,401, filed March 14, 2014 (now U.S. Patent No. 9,417,452, issued August 16, 2016), and U.S. Patent Application No. 14 / 331,218, filed July 14, 2014 and published as U.S. Patent Publication No. 2015 / 0309263 on October 29, 2015, in their entireties.

[0003] This application also incorporates by reference the following patent applications and publications: U.S. Patent Application No. 15 / 469,369, filed March 24, 2017, and published September 20, 2017 as U.S. Patent Publication No. 2017 / 0276,948; U.S. Patent Application No. 62 / 618,559, entitled “DEPTH PLANE SELECTION AND RENDER CAMERA POSITIONING IN AUGMENTED REALITY DISPLAY SYSTEMS” filed January 17, 2018 (Attorney Docket No. MLEAP.175PR); and U.S. Patent Application No. 62 / 618,559, entitled “DEPTH PLANE SELECTION AND RENDER CAMERA POSITIONING IN AUGMENTED REALITY DISPLAY SYSTEMS” filed January 17, 2018 (Attorney Docket No. MLEAP.175PR). This application incorporates in its entirety U.S. Patent Application No. 16 / 250,931, filed January 17, 2019 (Attorney Docket No. MLEAP.175), entitled "METHOD FOR USE IN A HIGH-SPEED PHOTOCELL SYSTEM," and U.S. Patent Publication No. 2018 / 0039,083, published February 8, 2018.

[0004] The present disclosure relates to display systems, and more particularly to augmented reality display systems. [Background technology]

[0005] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or can be perceived as real. Virtual reality, or "VR", scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs, and augmented reality, or "AR", scenarios typically involve the presentation of digital or virtual image information as an extension to the visualization of the real world around the user. Mixed reality, or "MR", scenarios are a type of AR scenario that typically involve virtual objects that are integrated into and respond to the natural world. For example, in MR scenarios, AR image content may be perceived as being occluded by or otherwise interacting with objects in the real world.

[0006] 1, an augmented reality scene 10 is depicted in which a user of the AR technology sees a real-world park-like setting 20 featuring people, trees, and buildings in the background, and a concrete platform 30. In addition to these items, the user of the AR technology also perceives that they are "seeing" "virtual content," such as a robotic figure 40 standing on the real-world platform 30, and a flying cartoon-like avatar character 50 that appears to be an anthropomorphic bumblebee, although these elements 40, 50 are "virtual" in that they do not exist in the real world. The human visual perception system is complex, making it difficult to create AR technology that facilitates a comfortable, natural-feeling, and rich presentation of virtual image elements among other virtual or real-world image elements.

[0007] The systems and methods disclosed herein address various challenges associated with AR or VR technology. Summary of the Invention [Means for solving the problem]

[0008] In some embodiments, a display system is provided for displaying a virtual object. The display system includes one or more processors and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform various operations. The operations include determining whether a virtual object is located within a fusion loss zone, determining whether a user is converging on a point within the fusion loss zone, and refraining from displaying the virtual object within the fusion loss zone if the virtual object is determined to be located within the fusion loss zone and the user is converging on a point within the fusion loss zone.

[0009] In some embodiments, a method is provided for operating a display device configured to display a virtual object, the method including determining that the virtual object is located within a fusion loss zone of a user of the display system and refraining from displaying the virtual object within the fusion loss zone.

[0010] In another embodiment, a display system is provided for displaying a virtual object. The display system includes one or more processors and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform various operations. The operations include determining whether the virtual object is user-locked content or in-world content, setting an accommodation-vergence-divergence mismatch threshold based on whether the virtual object is user-locked content or in-world content, determining whether a location of the virtual object has an accommodation-vergence-divergence mismatch that exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object, and limiting the display of the virtual object to the user if the accommodation-vergence-divergence mismatch of the location of the virtual object exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object.

[0011] In yet another embodiment, a method is provided for operating a display device configured to display a virtual object, the method including determining whether the virtual object is user-locked content or in-world content, setting an accommodation-vergence-divergence mismatch threshold based on whether the virtual object is user-locked content or in-world content, determining whether a location of the virtual object has an accommodation-vergence-divergence mismatch that exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object, and limiting display of the virtual object to the user if the accommodation-vergence-divergence mismatch of the location of the virtual object exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object.

[0012] In another embodiment, a display system is provided for displaying a virtual object. The display system includes one or more processors and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform various operations. The operations include determining whether a virtual object crosses a boundary between different zones of a user's binocular field of view (FOV), and, in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode, in which the virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode, in which the virtual object is displayed to only one eye of the user.

[0013] In another embodiment, a method is provided for operating a head-mounted display device configured to display a virtual object. The method includes obtaining a first estimate of a location where a virtual object will be perceived by a user and a second estimate of a location after which the virtual object will be perceived by the user, determining that the virtual object crosses a boundary between different zones of a binocular field of view (FOV) of the user based on at least the first estimate of a location where the virtual object will be perceived by the user and the second estimate of a location where the virtual object will be perceived by the user, and switching between (i) a binocular presentation mode in which the virtual object is displayed to both eyes of the user and (ii) a monocular presentation mode in which the virtual object is displayed to only one eye of the user in response to determining that the virtual object crosses a boundary between different zones of the binocular FOV of the user.

[0014] In another embodiment, a display system is provided for displaying a virtual object. The display system includes one or more processors and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform various operations. The operations include determining whether a location in a user's binocular field of view (FOV) where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV, determining whether the user's convergence-divergence point crosses a distance threshold defined for the virtual object, and switching between two different modes of presenting the virtual object to the user in response to determining that a location in the user's binocular FOV where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV and determining that the user's convergence-divergence point crosses a distance threshold defined for the virtual object.

[0015] In another embodiment, a method is provided for operating a head mounted display device configured to display a virtual object, the method including: determining that a location in a user's binocular field of view (FOV) where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV; obtaining a first estimate of the user's vergence-divergence point and a second estimate of the user's vergence-divergence point; determining that the user's vergence-divergence point crosses a distance threshold defined for the virtual object based on at least the first estimate of the user's vergence-divergence point and the second estimate of the user's vergence-divergence point; and switching between two different modes of presenting the virtual object to the user in response to the determination that a location in the user's binocular FOV where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV and the determination that the user's vergence-divergence point crosses a distance threshold defined for the virtual object.

[0016] In another embodiment, a display system is provided for displaying a virtual object. The display system includes one or more processors and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform various operations. The operations include determining whether a location in a user's binocular field of view (FOV) where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV, determining whether the user's convergence-divergence point crosses a distance threshold defined for the virtual object, and switching between two different modes of presenting the virtual object to the user in response to determining that a location in the user's binocular FOV where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV and determining that the user's convergence-divergence point crosses a distance threshold defined for the virtual object.

[0017] In another embodiment, a method is provided for operating a head mounted display device configured to display a virtual object, the method including: determining that a location in a user's binocular field of view (FOV) where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV; obtaining a first estimate of the user's vergence-divergence point and a second estimate of the user's vergence-divergence point; determining that the user's vergence-divergence point crosses a distance threshold defined for the virtual object based on at least the first estimate of the user's vergence-divergence point and the second estimate of the user's vergence-divergence point; and switching between two different modes of presenting the virtual object to the user in response to the determination that a location in the user's binocular FOV where the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV and the determination that the user's vergence-divergence point crosses a distance threshold defined for the virtual object.

[0018] Additional examples of embodiments are provided below.

[0019] 1. A display system for displaying virtual objects, comprising: one or more processors; When executed by one or more processors, the one or more processors determining whether the virtual object is located within a fusion loss zone; determining whether the user is converging to a point within a fusion loss zone; refraining from displaying the virtual object in the fusion loss zone if the virtual object is determined to be located within the fusion loss zone and the user is converging on a point within the fusion loss zone; one or more computer storage media having instructions stored thereon for performing operations, the instructions including: A display system comprising:

[0020] 2. The display system of Example 1, wherein the step of determining whether the virtual object is located within a fusion loss zone includes the step of determining whether the virtual object is located less than a fusion loss threshold distance from a user of the display system.

[0021] 3. The display system of Example 1, wherein the step of determining whether the user is converging to a point within the fusion loss zone includes the step of determining whether the user is converging to a distance less than a fusion loss threshold distance.

[0022] 4. The display system of Example 1, wherein the step of determining whether the user is converging at a point within the fusion loss zone includes the step of determining whether the user is converging at a point corresponding to a vergence-divergence angle less than a fusion loss threshold vergence-divergence angle.

[0023] 5. The operation is further determining whether the virtual object is located within an accommodation-vergence-divergence mismatch discomfort zone; determining whether the user's convergence-divergence point is within an accommodation-vergence-divergence mismatch discomfort zone; the virtual object is located within the accommodation-vergence-divergence inconsistency discomfort zone, and the user's convergence-divergence point is within the accommodation-vergence-divergence inconsistency discomfort zone; limiting the display of the virtual object to the user if The display system of Example 1, comprising:

[0024] 6. The display system of Example 5, wherein the step of determining whether the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining whether the virtual object is located less than an accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone from the user.

[0025] 7. The display system of example 6, wherein the accommodation-vergence-divergence mismatch threshold distance is 15 cm.

[0026] 8. The display system of Example 5, wherein the step of determining whether the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining whether the virtual object is located farther from the user than an accommodation-vergence-divergence mismatch threshold distance of the accommodation-vergence-divergence mismatch discomfort zone.

[0027] 9. The display system of Example 8, wherein the step of determining whether the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone further includes a step of determining whether the virtual object is located less than an accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone from the user.

[0028] 10. A display system as described in Example 5, wherein the step of determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes a step of determining whether the user's convergence-divergence distance is less than the accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone.

[0029] 11. The display system of example 10, wherein the fusion loss threshold distance is 10 cm.

[0030] 12. A display system as described in Example 5, wherein the step of determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining whether the user's convergence-divergence distance exceeds the accommodation-vergence-divergence mismatch threshold distance of the far accommodation-vergence-divergence mismatch discomfort zone.

[0031] 13. A display system as described in Example 12, wherein the step of determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone further includes a step of determining whether the user's convergence-divergence distance is less than the accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone.

[0032] 14. The step of determining whether the user's convergence-divergence point is within an accommodation-vergence-divergence mismatch discomfort zone includes: A display system as described in Example 5, comprising a step of determining whether a user is congested on a virtual object.

[0033] 15. A display system as described in Example 5, wherein the step of determining whether the user is convergent to a point within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining whether the user is convergent to a point corresponding to a convergence-divergence angle within a predetermined range of the accommodation-vergence-divergence mismatch comfort angle.

[0034] 16. A display system as described in Example 15, wherein the step of determining whether the user is converging to a point corresponding to a convergence angle outside a predetermined range of accommodation-vergence-divergence mismatch comfort angles includes the step of determining whether the user is converging to a point corresponding to a convergence angle less than a minimum accommodation-vergence-divergence mismatch comfort angle.

[0035] 17. A display system as described in Example 15, wherein the step of determining whether the user is converging to a point corresponding to a convergence-divergence angle outside a predetermined range of accommodation-vergence-divergence mismatch comfort angles includes the step of determining whether the user is converging to a point corresponding to a convergence-divergence angle above a maximum accommodation-vergence-divergence mismatch comfort angle.

[0036] 18. A display system as described in Example 5, wherein the step of determining whether the user is convergent to a point within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining whether the user's convergence-divergence distance is within a predetermined range of the accommodation-vergence-divergence mismatch comfort distance.

[0037] 19. A display system as described in Example 18, wherein the step of determining whether the user is converging to a point within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining whether the user's convergence-divergence distance is less than a minimum accommodation-vergence-divergence mismatch comfort distance.

[0038] 20. A display system as described in Example 18, wherein the step of determining whether the user is converging to a point within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining whether the user's convergence-divergence distance exceeds a maximum accommodation-vergence-divergence mismatch comfort distance.

[0039] 21. The display system of Example 5, further comprising a step of redisplaying the virtual content after ceasing to display the virtual content for a cooling period, wherein the step of redisplaying the virtual content comprises a step of displaying the virtual content at a location outside the fusion loss zone.

[0040] 22. The operation further includes: determining whether the virtual object is user locked content or in-world content; setting a size of an accommodation-vergence-divergence motion mismatch discomfort zone based on whether the virtual object is user-locked content or in-world content, where a size of the associated accommodation-vergence-divergence motion mismatch discomfort zone for the in-world content exceeds a size of the associated accommodation-vergence-divergence motion mismatch discomfort zone for the user-locked content; The display system of Example 5, comprising:

[0041] 23. The step of limiting the display is as follows: determining a user's exposure time to the virtual object, the exposure time being a duration of the user's exposure to the virtual content during which the virtual content is persistently located within the accommodation-vergence-divergence mismatch discomfort zone; refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone after exceeding the exposure time threshold; The display system of Example 5, comprising:

[0042] 24. The display system of Example 23, wherein the step of refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone includes ceasing the display of the virtual object to one eye of the viewer while continuing to display the virtual object to the other eye of the viewer.

[0043] 25. The display system of Example 23, wherein the step of refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch threshold distance includes the step of intermittently ceasing display of the virtual object to each eye of the viewer by alternating the display of the virtual object between the viewer's eyes.

[0044] 26. The operation further includes: determining whether the virtual object is user locked content or in-world content; setting an exposure time threshold based on whether the virtual object is user-locked content or in-world content, where an associated exposure time threshold for in-world content exceeds an associated exposure time threshold for user-locked content; 24. The display system of Example 23, comprising:

[0045] 27. The display system of Example 1, further comprising an optical system configured to output light that forms a virtual object with varying amounts of wavefront divergence.

[0046] 28. The display system of example 27, wherein the optical system comprises a stack of waveguides, different ones of the waveguides outputting light with different amounts of wavefront divergence.

[0047] 29. The display system of example 27, further comprising a projection system for directing light containing image information into the optical system for output to a viewer.

[0048] 30. The display system of example 27, wherein the projection system comprises a light source and a spatial light modulator configured to modulate light from the light source.

[0049] 31. The display system of Example 1, further comprising a sensor for determining an orientation of the viewer's eyes, the display system being configured to determine a fixation point based on information from the sensor.

[0050] 32. The display system of example 31, wherein the sensor comprises an inward-facing camera configured to image the viewer's eye.

[0051] 33. The display system of Example 1, wherein the step of refraining from displaying the virtual object within the fusion loss zone includes a step of ceasing to display the virtual object to one eye of the viewer while continuing to display the virtual object to the other eye of the viewer.

[0052] 34. The display system of Example 1, wherein the step of refraining from displaying the virtual object within the fusion loss zone includes the step of intermittently ceasing display of the virtual object to each eye of the viewer by alternating the display of the virtual object between the viewer's eyes.

[0053] 35. A method for operating a display device configured to display a virtual object, comprising: determining that a virtual object is located within a fusion loss zone of a user of a display system; refraining from displaying the virtual object within the fusion loss zone; A method comprising:

[0054] 36. The method of example 35, wherein determining that the virtual object is located within the fusion loss zone includes determining that the virtual object is located less than a fusion loss threshold distance from the user.

[0055] 37. determining that the virtual object is located within a user's accommodation-vergence-divergence mismatch discomfort zone; determining that a user's convergence-divergence point is within an accommodation-vergence-divergence mismatch discomfort zone; limiting the display of the virtual object to the user; The method of example 35, further comprising:

[0056] 38. The method described in example 37, wherein the step of determining that the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining that the virtual object is located less than an accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone from the user.

[0057] 39. The method of example 37, wherein determining that the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes determining that the virtual object is located farther from the user than an accommodation-vergence-divergence mismatch threshold distance of the accommodation-vergence-divergence mismatch discomfort zone.

[0058] 40. The method described in example 39, wherein the step of determining that the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone further includes a step of determining that the virtual object is located less than an accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone from the user.

[0059] 41. The method described in Example 37, wherein the step of determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining that the user's convergence-divergence distance is less than the accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone.

[0060] 42. The method described in Example 37, wherein the step of determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes the step of determining that the user's convergence-divergence distance exceeds the accommodation-vergence-divergence mismatch threshold distance of the far accommodation-vergence-divergence mismatch discomfort zone.

[0061] 43. The method described in Example 42, wherein the step of determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone further includes a step of determining that the user's convergence-divergence distance is less than the accommodation-vergence-divergence mismatch threshold distance of the near accommodation-vergence-divergence mismatch discomfort zone.

[0062] 44. The step of determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes: 38. The method of example 37, comprising determining that a user is congested with a virtual object.

[0063] 45. The method of Example 37, further comprising the step of redisplaying the virtual content after ceasing to display the virtual content for a cooling period, wherein the step of redisplaying the virtual content comprises the step of displaying the virtual content at a location outside the fusion loss zone.

[0064] 46. ​​Determining whether the virtual object is user-locked content or in-world content; setting a size of an accommodation-vergence-divergence motion mismatch discomfort zone based on whether the virtual object is user-locked content or in-world content, where a size of the associated accommodation-vergence-divergence motion mismatch discomfort zone for the in-world content exceeds a size of the associated accommodation-vergence-divergence motion mismatch discomfort zone for the user-locked content; The method of Example 37, further comprising:

[0065] 47. The step of limiting the display is as follows: determining a user's exposure time to the virtual object, the exposure time being a duration of the user's exposure to the virtual content during which the virtual content is persistently located within the accommodation-vergence-divergence mismatch discomfort zone; refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone after exceeding the exposure time threshold; The method of Example 37, comprising:

[0066] 48. The action further includes: determining whether the virtual object is user locked content or in-world content; setting an exposure time threshold based on whether the virtual object is user-locked content or in-world content, where an associated exposure time threshold for in-world content exceeds an associated exposure time threshold for user-locked content; The method of Example 47, comprising:

[0067] 49. The method described in Example 47, wherein the step of refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone includes a step of ceasing to display the virtual object to one eye of the viewer while continuing to display the virtual object to the other eye of the viewer.

[0068] 50. The method described in Example 47, wherein the step of refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone includes a step of intermittently ceasing the display of the virtual object to each of the viewer's eyes by alternating the display of the virtual object between the viewer's eyes.

[0069] 51. The method of Example 35, wherein the step of refraining from displaying the virtual object within the fusion loss zone includes a step of ceasing to display the virtual object to one eye of the viewer while continuing to display the virtual object to the other eye of the viewer.

[0070] 52. The method of example 35, wherein the step of refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone includes the step of intermittently ceasing the display of the virtual object to each of the viewer's eyes by alternating the display of the virtual object between the viewer's eyes.

[0071] 53. A display system for displaying virtual objects, comprising: one or more processors; When executed by one or more processors, the one or more processors determining whether the virtual object is user locked content or in-world content; setting an accommodation-vergence-divergence movement mismatch threshold based on whether the virtual object is user-locked content or in-world content; determining whether the location of the virtual object has an accommodation-vergence-divergence mismatch that exceeds an accommodation-vergence-divergence mismatch threshold for the virtual object; limiting the display of the virtual object to the user if an accommodation-vergence-divergence mismatch of the location of the virtual object exceeds an accommodation-vergence-divergence mismatch threshold for the virtual object; one or more computer storage media having instructions stored thereon for performing operations, the instructions including: A display system comprising:

[0072] 54. A display system as described in Example 53, wherein the accommodation-vergence-divergence mismatch threshold is a distance from the user, the accommodation-vergence-divergence mismatch threshold for in-world content is greater than the accommodation-vergence-divergence mismatch threshold for content locked to the user, and the accommodation-vergence-divergence mismatch of the location of the virtual object exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object when the location is closer to the user of the display system than the accommodation-vergence-divergence mismatch threshold distance.

[0073] 55. The action further includes: setting an exposure time threshold for the virtual object based on whether the virtual object is user-locked content or in-world content, where an associated exposure time threshold for the in-world content exceeds an associated exposure time threshold for the user-locked content; The step of limiting the display of the virtual object includes: determining a user's exposure time to the virtual object, the exposure time being a duration of the user's exposure to the virtual content while the virtual content is persistently located within an accommodation-vergence-divergence mismatch zone having boundaries defined by an accommodation-vergence-divergence mismatch threshold distance; refraining from displaying the virtual object after an exposure time threshold is exceeded; The display system of Example 53, comprising:

[0074] 56. A method for operating a display device configured to display a virtual object, comprising: determining whether the virtual object is user locked content or in-world content; setting an accommodation-vergence-divergence movement mismatch threshold based on whether the virtual object is user-locked content or in-world content; determining that the location of the virtual object has an accommodation-vergence-divergence mismatch that exceeds an accommodation-vergence-divergence mismatch threshold for the virtual object; limiting the display of the virtual object to the user; A method comprising:

[0075] 57. The method of example 56, wherein the accommodation-vergence-divergence mismatch threshold is a distance from the user, the accommodation-vergence-divergence mismatch threshold for in-world content is greater than the accommodation-vergence-divergence mismatch threshold for content locked to the user, and the accommodation-vergence-divergence mismatch of the location of the virtual object exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object when the location is closer to the user of the display system than the accommodation-vergence-divergence mismatch threshold distance.

[0076] 58. The method further includes setting an exposure time threshold for the virtual object based on whether the virtual object is user-locked content or in-world content, where an associated exposure time threshold for the in-world content exceeds an associated exposure time threshold for the user-locked content, and limiting the display of the virtual object further includes: determining a user's exposure time to the virtual object, the exposure time being a duration of the user's exposure to the virtual content while the virtual content is persistently located within an accommodation-vergence-divergence mismatch zone having boundaries defined by an accommodation-vergence-divergence mismatch threshold distance; refraining from displaying the virtual object after an exposure time threshold is exceeded; The method of Example 56, comprising:

[0077] 59. A display system for displaying virtual objects, comprising: one or more processors; When executed by one or more processors, the one or more processors determining whether the virtual object crosses a boundary between different zones of a user's binocular field of view (FOV); in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode, in which the virtual object is displayed to both of the user's eyes, and (ii) a monocular presentation mode, in which the virtual object is displayed to only one of the user's eyes; one or more computer storage media having instructions stored thereon for performing operations, the instructions including: A display system comprising:

[0078] 60. The action further includes: Identifying a type of content corresponding to a virtual object; determining, at least in part, based on the identified type of content corresponding to the virtual object, a particular boundary between different zones of the user's binocular FOV, where determining whether the virtual object crosses a boundary between the different zones of the user's binocular FOV includes determining whether the virtual object crosses a particular boundary between the different zones of the user's binocular FOV; The display system of Example 59, comprising:

[0079] 61. The action further includes: A display system as described in Example 59, including a step of selecting a specific eye of the user from among both eyes of the user, wherein in a monocular presentation mode, the virtual object is displayed only to the specific eye of the user.

[0080] 62. The action further includes: A display system as described in Example 61, comprising a step of determining which of the user's eyes is the user's dominant eye, wherein the step of selecting a particular eye of the user from among the user's eyes comprises a step of selecting a particular eye of the user from among the user's eyes based, at least in part, on the step of determining which of the user's eyes is the user's dominant eye.

[0081] 63. The action further includes: The display system of Example 61, comprising a step of determining whether a virtual object is located within a left portion of the user's binocular FOV or a right portion of the user's binocular FOV, wherein the step of selecting a particular eye of the user from among the user's eyes comprises a step of selecting a particular eye of the user from among the user's eyes based, at least in part, on the step of determining whether a virtual object is located within a left portion of the user's binocular FOV or a right portion of the user's binocular FOV.

[0082] 64. The action further includes: The display system of Example 61, comprising a step of determining a distance between a virtual object and one or more other virtual objects within the same virtual scene, wherein the step of selecting a particular eye of the user from among the user's eyes comprises a step of selecting a particular eye of the user from among the user's eyes based, at least in part, on the step of determining a distance between a virtual object and one or more other virtual objects within the same virtual scene.

[0083] 65. The action further includes: The display system of Example 61, comprising a step of determining whether a duplicate version of a virtual object exists within the same virtual scene, wherein the step of selecting a particular eye of the user from among the user's eyes comprises a step of selecting a particular eye of the user from among the user's eyes based, at least in part, on the step of determining whether a duplicate version of a virtual object exists within the same virtual scene.

[0084] 66. The step of switching between (i) a binocular presentation mode, in which the virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode, in which the virtual object is displayed to only one eye of the user, comprises: The display system of Example 59 includes a step of switching between (i) a binocular presentation mode in which a virtual object is displayed to each eye of the user at a first level of brightness, and (ii) a monocular presentation mode in which a virtual object is displayed to only one eye of the user at a second level of brightness, the second level of brightness being greater than the first level of brightness.

[0085] 67. The action further includes: determining whether a distance between the virtual object and another virtual object in the same virtual scene is less than a threshold distance; in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV and determining that a distance between the virtual object and other virtual objects in the same virtual scene is less than a threshold distance, switching between (i) a binocular presentation mode, in which the other virtual objects are displayed to both of the user's eyes, and (ii) a monocular presentation mode, in which the other virtual objects are displayed to only one of the user's eyes; The display system of Example 59, comprising:

[0086] 68. The action further includes: determining whether another virtual object crosses a boundary between different zones of the user's binocular FOV; in response to determining that the other virtual object crosses a boundary between different zones of the user's binocular FOV, and independent of determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode, in which the other virtual object is displayed to both of the user's eyes, and (ii) a monocular presentation mode, in which the other virtual object is displayed to only one eye of the user; The display system of Example 59, comprising:

[0087] 69. A display system as described in Example 59, wherein the step of determining whether the virtual object crosses a boundary between different zones of the user's binocular FOV includes a step of determining whether the virtual object crosses a boundary between an accommodation-vergence-divergence mismatch discomfort zone and an accommodation-vergence-divergence mismatch comfort zone.

[0088] 70. A method for operating a head mounted display device configured to display a virtual object, comprising: obtaining a first estimate of a location where a virtual object will be perceived by a user and a second estimate after where the virtual object will be perceived by the user; determining that the virtual object crosses a boundary between different zones of a binocular field of view (FOV) of the user based on at least a first estimate of where the virtual object will be perceived by the user and a second estimate of where the virtual object will be perceived by the user; in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode, in which the virtual object is displayed to both of the user's eyes, and (ii) a monocular presentation mode, in which the virtual object is displayed to only one of the user's eyes; A method comprising:

[0089] 71. A display system for displaying virtual objects, comprising: one or more processors; When executed by one or more processors, the one or more processors determining whether a location within a user's binocular field of view (FOV) where the virtual object will be perceived is outside a comfort zone of the user's binocular FOV; determining whether a convergence-divergence motion point of the user crosses a distance threshold defined for the virtual object; in response to determining that a location within the user's binocular FOV at which the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV and determining that the user's vergence-divergence movement point crosses a distance threshold defined for the virtual object, switching between two different modes of presenting the virtual object to the user; one or more computer storage media having instructions stored thereon for performing operations, the instructions including: A display system comprising:

[0090] 72. A method for operating a head mounted display device configured to display a virtual object, comprising: determining that a location within a user's binocular field of view (FOV) at which a virtual object is to be perceived is outside a comfort zone of the user's binocular FOV; obtaining a first estimate of a convergence-divergence motion point of the user and a second estimate of a subsequent convergence-divergence motion point of the user; determining, based on at least a first estimate of the user's convergence-divergence motion point and a second estimate of the user's convergence-divergence motion point, that the user's convergence-divergence motion point crosses a distance threshold defined for the virtual object; in response to determining that a location within the user's binocular FOV at which the virtual object is to be perceived is outside a comfort zone of the user's binocular FOV and determining that the user's vergence-divergence movement point crosses a distance threshold defined for the virtual object, switching between two different modes of presenting the virtual object to the user; A method comprising:

[0091] 73. A display system for displaying virtual objects, comprising: one or more processors; When executed by one or more processors, the one or more processors Identifying a type of content corresponding to a virtual object; determining boundaries between different zones of a binocular field of view (FOV) of the user based at least in part on the identified type of content corresponding to the virtual object; determining whether a convergence-divergence motion point of the user crosses a distance threshold defined for the virtual object; in response to determining that the virtual object crosses the determined boundary between zones of the user's binocular FOV, toggling between starting and resetting a timer associated with the identified type of content; determining whether a timer associated with the identified type of content has expired; in response to determining that a timer associated with the identified type of content has expired, modifying the virtual object according to the identified type of content; one or more computer storage media having instructions stored thereon for performing operations, the instructions including: A display system comprising:

[0092] 74. A method for operating a head mounted display device configured to display a virtual object, comprising: Identifying a type of content corresponding to a virtual object; determining boundaries between different zones of a binocular field of view (FOV) of the user based at least in part on the identified type of content corresponding to the virtual object; obtaining a first estimate of a location where a virtual object will be perceived by a user and a second estimate after where the virtual object will be perceived by the user; determining that the virtual object crosses a determined boundary between different zones of the user's binocular FOV based on at least a first estimate of where the virtual object will be perceived by the user and a second estimate of where the virtual object will be perceived by the user; in response to determining that the virtual object crosses the determined boundary between zones of the user's binocular FOV, toggling between starting and resetting a timer associated with the identified type of content; A method comprising: The present invention provides, for example, the following: (Item 1) 1. A display system for displaying a virtual object, the display system comprising: one or more processors; One or more computer storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: determining whether the virtual object is located within a fusion loss zone; determining whether the user is convergent at a point within the fusion loss zone; refraining from displaying the virtual object within the fusion loss zone if the virtual object is determined to be located within the fusion loss zone and the user is converging on a point within the fusion loss zone; one or more computer storage media for performing operations including: A display system comprising: (Item 2) 2. The display system of claim 1, wherein determining whether the virtual object is located within a fusion loss zone includes determining whether the virtual object is located less than a fusion loss threshold distance from a user of the display system. (Item 3) 2. The display system of claim 1, wherein determining whether the user is converging to a point within the fusion loss zone includes determining whether the user is converging to a distance less than a fusion loss threshold distance. (Item 4) The display system of item 1, wherein determining whether the user is converging at a point within the fusion loss zone includes determining whether the user is converging at a point corresponding to a vergence-divergence angle less than a fusion loss threshold vergence-divergence angle. (Item 5) The operation further comprises: determining whether the virtual object is located within an accommodation-vergence-divergence mismatch discomfort zone; determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone; limiting display of the virtual object to the user when the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone and when the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone; 2. The display system of item 1, comprising: (Item 6) 6. The display system of claim 5, wherein determining whether the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes determining whether the virtual object is located less than a accommodation-vergence-divergence mismatch threshold distance of a near accommodation-vergence-divergence mismatch discomfort zone from the user. (Item 7) 7. The display system of claim 6, wherein the accommodation-vergence-divergence mismatch threshold distance is 15 cm. (Item 8) 6. The display system of claim 5, wherein determining whether the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes determining whether the virtual object is located beyond an accommodation-vergence-divergence mismatch threshold distance of an accommodation-vergence-divergence mismatch discomfort zone away from the user. (Item 9) 9. The display system of claim 8, wherein determining whether the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone further comprises determining whether the virtual object is located less than a accommodation-vergence-divergence mismatch threshold distance of a near accommodation-vergence-divergence mismatch discomfort zone from the user. (Item 10) 6. The display system of claim 5, wherein determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes determining whether the user's convergence-divergence distance is less than a near accommodation-vergence-divergence mismatch threshold distance of a near accommodation-vergence-divergence mismatch discomfort zone. (Item 11) Item 11. The display system of item 10, wherein the fusion loss threshold distance is 10 cm. (Item 12) 6. The display system of claim 5, wherein determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes determining whether the user's convergence-divergence distance exceeds a accommodation-vergence-divergence mismatch threshold distance of a far accommodation-vergence-divergence mismatch discomfort zone. (Item 13) Item 13. The display system of item 12, wherein determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone further includes determining whether the user's convergence-divergence distance is less than a near accommodation-vergence-divergence mismatch threshold distance of a near accommodation-vergence-divergence mismatch discomfort zone. (Item 14) Determining whether the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone Determining whether the user is congested with the virtual object. 6. The display system of item 5, comprising: (Item 15) 6. The display system of claim 5, wherein determining whether the user is converging to a point within the accommodation-vergence-divergence mismatch discomfort zone comprises determining whether the user is converging to a point corresponding to a vergence angle within a predetermined range of accommodation-vergence-divergence mismatch comfort angles. (Item 16) Item 16. The display system of item 15, wherein determining whether the user is converging to a point corresponding to a vergence angle outside a predetermined range of accommodation-vergence-divergence mismatch comfort angles includes determining whether the user is converging to a point corresponding to a vergence angle less than a minimum accommodation-vergence-divergence mismatch comfort angle. (Item 17) Item 16. The display system of item 15, wherein determining whether the user is converging to a point corresponding to a vergence angle outside a predetermined range of accommodation-vergence-divergence mismatch comfort angles includes determining whether the user is converging to a point corresponding to a vergence angle above a maximum accommodation-vergence-divergence mismatch comfort angle. (Item 18) 6. The display system of claim 5, wherein determining whether the user is converging to a point within the accommodation-vergence-divergence mismatch discomfort zone comprises determining whether the user's convergence-divergence distance is within a predetermined range of accommodation-vergence-divergence mismatch comfort distance. (Item 19) Item 19. The display system of item 18, wherein determining whether the user is convergent to a point within the accommodation-vergence-divergence mismatch discomfort zone comprises determining whether the user's convergence-divergence distance is less than a minimum accommodation-vergence-divergence mismatch comfort distance. (Item 20) Item 19. The display system of item 18, wherein determining whether the user is converging to a point within the accommodation-vergence-divergence mismatch discomfort zone comprises determining whether the user's convergence-divergence distance exceeds a maximum accommodation-vergence-divergence mismatch comfort distance. (Item 21) 6. The display system of claim 5, further comprising: redisplaying the virtual content after ceasing to display the virtual content for a cooling period, wherein redisplaying the virtual content comprises displaying the virtual content at a location outside the fusion loss zone. (Item 22) The operation further comprises: determining whether the virtual object is user-locked content or in-world content; setting a size of the accommodation-vergence-divergence motion mismatch discomfort zone based on whether the virtual object is user-locked content or in-world content, wherein a size of an associated accommodation-vergence-divergence motion mismatch discomfort zone for in-world content exceeds a size of an associated accommodation-vergence-divergence motion mismatch discomfort zone for user-locked content; 6. The display system of item 5, comprising: (Item 23) To limit the display, determining a time of exposure of the user to the virtual object, the exposure time being a duration of the user's exposure to the virtual content while the virtual content is persistently located within the accommodation-vergence-divergence mismatch discomfort zone; refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone after an exposure time threshold has been exceeded; 6. The display system of item 5, comprising: (Item 24) Item 24. The display system of item 23, wherein refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone comprises ceasing to display the virtual object to one eye of the viewer while continuing to display the virtual object to the other eye of the viewer. (Item 25) 24. The display system of claim 23, wherein refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch threshold distance comprises intermittently ceasing display of the virtual object to each eye of the viewer by alternating display of the virtual object between the viewer's eyes. (Item 26) The operation further comprises: determining whether the virtual object is user-locked content or in-world content; setting the exposure time threshold based on whether the virtual object is user-locked content or in-world content, an associated exposure time threshold for in-world content exceeding an associated exposure time threshold for user-locked content; 24. The display system of claim 23, comprising: (Item 27) Item 10. The display system of item 1, further comprising an optical system configured to output light that forms the virtual object with varying amounts of wavefront divergence. (Item 28) 28. The display system of claim 27, wherein the optical system comprises a stack of waveguides, different ones of the waveguides outputting light with different amounts of wavefront divergence. (Item 29) 28. The display system of claim 27, further comprising a projection system for directing light containing image information into the optical system for output to the viewer. (Item 30) 28. The display system of claim 27, wherein the projection system comprises a light source and a spatial light modulator configured to modulate light from the light source. (Item 31) 2. The display system of claim 1, further comprising a sensor for determining an orientation of the viewer's eyes, the display system being configured to determine a fixation point based on information from the sensor. (Item 32) Item 32. The display system of item 31, wherein the sensor comprises an inward-facing camera configured to image the viewer's eye. (Item 33) 2. The display system of claim 1, wherein refraining from displaying the virtual object within the fusion loss zone includes ceasing display of the virtual object to one eye of the viewer while continuing to display the virtual object to the other eye of the viewer. (Item 34) 2. The display system of claim 1, wherein refraining from displaying the virtual object within the fusion loss zone includes intermittently ceasing display of the virtual object to each eye of the viewer by alternating display of the virtual object between the viewer's eyes. (Item 35) 1. A method for operating a display device configured to display a virtual object, the method comprising: determining that the virtual object is located within a fusion loss zone of a user of the display system; refraining from displaying the virtual object within the fusion loss zone; A method comprising: (Item 36) 36. The method of claim 35, wherein determining that the virtual object is located within the fusion loss zone includes determining that the virtual object is located less than a fusion loss threshold distance from the user. (Item 37) determining that the virtual object is located within an accommodation-vergence-divergence mismatch discomfort zone of the user; determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone; limiting the display of the virtual object to a user; 36. The method of claim 35, further comprising: (Item 38) 38. The method of claim 37, wherein determining that the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes determining that the virtual object is located within a near accommodation-vergence-divergence mismatch discomfort zone threshold distance from the user. (Item 39) 38. The method of claim 37, wherein determining that the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone includes determining that the virtual object is located beyond an accommodation-vergence-divergence mismatch threshold distance of a far accommodation-vergence-divergence mismatch discomfort zone from the user. (Item 40) 40. The method of claim 39, wherein determining that the virtual object is located within the accommodation-vergence-divergence mismatch discomfort zone further comprises determining that the virtual object is located within a near accommodation-vergence-divergence mismatch discomfort zone less than a threshold distance of the accommodation-vergence-divergence mismatch threshold from the user. (Item 41) Item 38. The method of item 37, wherein determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes determining that the user's convergence-divergence distance is less than a near accommodation-vergence-divergence mismatch threshold distance of a near accommodation-vergence-divergence mismatch discomfort zone. (Item 42) Item 38. The method of item 37, wherein determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes determining that the user's convergence-divergence distance exceeds an accommodation-vergence-divergence mismatch threshold distance of a far accommodation-vergence-divergence mismatch discomfort zone. (Item 43) 43. The method of claim 42, wherein determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone further comprises determining that the user's convergence-divergence distance is less than a near accommodation-vergence-divergence mismatch threshold distance of a near accommodation-vergence-divergence mismatch discomfort zone. (Item 44) Determining that the user's convergence-divergence point is within the accommodation-vergence-divergence mismatch discomfort zone includes: Determining that the user is congested with the virtual object. 38. The method according to item 37, comprising: (Item 45) Item 38. The method of item 37, further comprising redisplaying the virtual content after ceasing to display the virtual content for a cooling period, wherein redisplaying the virtual content comprises displaying the virtual content at a location outside the fusion loss zone. (Item 46) determining whether the virtual object is user-locked content or in-world content; setting a size of the accommodation-vergence-divergence motion mismatch discomfort zone based on whether the virtual object is user-locked content or in-world content, wherein a size of an associated accommodation-vergence-divergence motion mismatch discomfort zone for in-world content exceeds a size of an associated accommodation-vergence-divergence motion mismatch discomfort zone for user-locked content; 38. The method of claim 37, further comprising: (Item 47) To limit the display, determining a time of exposure of the user to the virtual object, the exposure time being a duration of the user's exposure to the virtual content while the virtual content is persistently located within the accommodation-vergence-divergence mismatch discomfort zone; refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone after an exposure time threshold has been exceeded; 38. The method according to item 37, comprising: (Item 48) The operation further comprises: determining whether the virtual object is user-locked content or in-world content; setting the exposure time threshold based on whether the virtual object is user-locked content or in-world content, an associated exposure time threshold for in-world content exceeding an associated exposure time threshold for user-locked content; 48. The method according to item 47, comprising: (Item 49) Item 48. The method of item 47, wherein refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone comprises ceasing to display the virtual object to one eye of the viewer while continuing to display a virtual object to the other eye of the viewer. (Item 50) 48. The method of claim 47, wherein refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone comprises intermittently ceasing display of the virtual object to each eye of the viewer by alternating display of the virtual object between the viewer's eyes. (Item 51) Item 36. The method of item 35, wherein refraining from displaying the virtual object within the fusion loss zone includes ceasing to display the virtual object to one eye of the viewer while continuing to display the virtual object to the other eye of the viewer. (Item 52) 36. The method of claim 35, wherein refraining from displaying the virtual object within the accommodation-vergence-divergence mismatch discomfort zone comprises intermittently ceasing display of the virtual object to each eye of the viewer by alternating display of the virtual object between the viewer's eyes. (Item 53) 1. A display system for displaying a virtual object, the display system comprising: one or more processors; One or more computer storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: determining whether the virtual object is user-locked content or in-world content; setting an accommodation-vergence-divergence movement mismatch threshold based on whether the virtual object is user-locked content or in-world content; determining whether the location of the virtual object has an accommodation-vergence-divergence mismatch that exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object; limiting the display of the virtual object to the user if the accommodation-vergence-divergence mismatch at a location of the virtual object exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object. one or more computer storage media for performing operations including: A display system comprising: (Item 54) Item 54. The display system of item 53, wherein the accommodation-vergence-divergence mismatch threshold is a distance from the user, the accommodation-vergence-divergence mismatch threshold for in-world content is greater than the accommodation-vergence-divergence mismatch threshold for user-locked content, and the accommodation-vergence-divergence mismatch of the location of the virtual object exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object when the location is closer to the user of the display system than the accommodation-vergence-divergence mismatch threshold distance. (Item 55) The operation further comprises: setting an exposure time threshold for the virtual object based on whether the virtual object is user-locked content or in-world content, wherein an associated exposure time threshold for in-world content exceeds an associated exposure time threshold for user-locked content; Including, Limiting the display of the virtual object includes: determining a duration of exposure of the user to the virtual object, the exposure duration being a duration of exposure of the user to the virtual content while the virtual content is persistently located within an accommodation-vergence-divergence mismatch zone having a boundary defined by the accommodation-vergence-divergence mismatch threshold distance; refraining from displaying the virtual object after the exposure time threshold is exceeded; Item 54. The display system of item 53, comprising: (Item 56) 1. A method for operating a display device configured to display a virtual object, the method comprising: determining whether the virtual object is user-locked content or in-world content; setting an accommodation-vergence-divergence movement mismatch threshold based on whether the virtual object is user-locked content or in-world content; determining that a location of the virtual object has an accommodation-vergence-divergence mismatch that exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object; limiting the display of the virtual object to a user; A method comprising: (Item 57) 57. The method of claim 56, wherein the accommodation-vergence-divergence mismatch threshold is a distance from the user, wherein the accommodation-vergence-divergence mismatch threshold for in-world content is greater than the accommodation-vergence-divergence mismatch threshold for user-locked content, and wherein the accommodation-vergence-divergence mismatch of a location of the virtual object exceeds the accommodation-vergence-divergence mismatch threshold for the virtual object when the location is closer to a user of the display system than the accommodation-vergence-divergence mismatch threshold distance. (Item 58) setting an exposure time threshold for the virtual object based on whether the virtual object is user-locked content or in-world content, wherein an associated exposure time threshold for in-world content exceeds an associated exposure time threshold for user-locked content; Further comprising: Limiting the display of the virtual object includes: determining a duration of exposure of the user to the virtual object, the exposure duration being a duration of exposure of the user to the virtual content while the virtual content is persistently located within an accommodation-vergence-divergence mismatch zone having a boundary defined by the accommodation-vergence-divergence mismatch threshold distance; refraining from displaying the virtual object after the exposure time threshold is exceeded; 57. The method according to item 56, comprising: (Item 59) 1. A display system for displaying a virtual object, the display system comprising: one or more processors; One or more computer storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: determining whether the virtual object crosses a boundary between different zones of a user's binocular field of view (FOV); in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode in which the virtual object is displayed to both eyes of the user and (ii) a monocular presentation mode in which the virtual object is displayed to only one eye of the user; one or more computer storage media for performing operations including: A display system comprising: (Item 60) The operation further comprises: identifying a type of content corresponding to the virtual object; determining, at least in part, a particular boundary between different zones of the user's binocular FOV based on the identified type of content corresponding to the virtual object, wherein determining whether the virtual object crosses a particular boundary between different zones of the user's binocular FOV includes determining whether the virtual object crosses a particular boundary between different zones of the user's binocular FOV; Item 60. The display system of item 59, comprising: (Item 61) The operation further includes selecting a particular eye of the user out of both eyes of the user, and the virtual object is displayed only to the particular eye of the user in the monocular presentation mode. Item 60. The display system of item 59, comprising: (Item 62) The operation further comprises: determining an eye of the user which is a dominant eye of the user, wherein selecting a particular eye of the user among the eyes of the user includes selecting a particular eye of the user among the eyes of the user based at least in part on determining an eye of the user which is a dominant eye of the user. Item 62. The display system of item 61, comprising: (Item 63) The operation further comprises: determining whether the virtual object is located within a left portion of the user's binocular FOV or a right portion of the user's binocular FOV, and selecting the particular one of the user's eyes includes selecting the particular one of the user's eyes based at least in part on determining whether the virtual object is located within a left portion of the user's binocular FOV or a right portion of the user's binocular FOV. Item 62. The display system of item 61, comprising: (Item 64) The operation further comprises: determining a distance between the virtual object and one or more other virtual objects in the same virtual scene, wherein selecting a particular one of the user's eyes includes selecting a particular one of the user's eyes based at least in part on determining a distance between the virtual object and one or more other virtual objects in the same virtual scene; Item 62. The display system of item 61, comprising: (Item 65) The operation further comprises: determining whether a duplicate version of the virtual object exists within the same virtual scene, wherein selecting a particular one of the user's eyes includes selecting a particular one of the user's eyes based, at least in part, on determining whether a duplicate version of the virtual object exists within the same virtual scene; Item 62. The display system of item 61, comprising: (Item 66) (i) switching between the binocular presentation mode, in which the virtual object is displayed to both eyes of the user, and (ii) the monocular presentation mode, in which the virtual object is displayed to only one eye of the user, (i) switching between a binocular presentation mode in which the virtual object is displayed to each eye of the user at a first level of brightness, and (ii) a monocular presentation mode in which the virtual object is displayed to only one eye of the user at a second level of brightness, the second level of brightness being greater than the first level of brightness. Item 60. The display system of item 59, comprising: (Item 67) The operation further comprises: determining whether a distance between the virtual object and another virtual object in the same virtual scene is less than a threshold distance; in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV and determining that a distance between the virtual object and the other virtual object in the same virtual scene is less than the threshold distance, switching between (i) a binocular presentation mode, in which the other virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode, in which the other virtual object is displayed to only one eye of the user; Item 60. The display system of item 59, comprising: (Item 68) The operation further comprises: determining whether another virtual object crosses a boundary between different zones of the user's binocular FOV; in response to determining that the other virtual object crosses a boundary between different zones of the user's binocular FOV, and independent of determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode, in which the other virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode, in which the other virtual object is displayed to only one eye of the user; Item 60. The display system of item 59, comprising: (Item 69) Item 60. The display system of item 59, wherein determining whether the virtual object crosses a boundary between different zones of the user's binocular FOV includes determining whether the virtual object crosses a boundary between an accommodation-vergence-divergence mismatch discomfort zone and an accommodation-vergence-divergence mismatch comfort zone. (Item 70) 1. A method for operating a head mounted display device configured to display a virtual object, the method comprising: obtaining a first estimate of a location where the virtual object will be perceived by a user and a second estimate after where the virtual object will be perceived by the user; determining that the virtual object crosses a boundary between different zones of a binocular field of view (FOV) of the user based on at least a first estimate of where the virtual object will be perceived by a user and a second estimate of where the virtual object will be perceived by the user; in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode in which the virtual object is displayed to both eyes of the user and (ii) a monocular presentation mode in which the virtual object is displayed to only one eye of the user; A method comprising: (Item 71) 1. A display system for displaying a virtual object, the display system comprising: one or more processors; One or more computer storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: determining whether a location within a user's binocular field of view (FOV) where the virtual object will be perceived is outside a comfort zone of the user's binocular FOV; determining whether the user's convergence-divergence motion point crosses a distance threshold defined for the virtual object; in response to determining that a location within the user's binocular FOV at which the virtual object will be perceived is outside a comfort zone of the user's binocular FOV and determining that the user's vergence-divergence movement point crosses a distance threshold defined for the virtual object, switching between two different modes of presenting the virtual object to the user; one or more computer storage media for performing operations including: A display system comprising: (Item 72) 1. A method for operating a head mounted display device configured to display a virtual object, the method comprising: determining that a location within a user's binocular field of view (FOV) at which the virtual object will be perceived is outside a comfort zone of the user's binocular FOV; obtaining a first estimate of a convergence-divergence motion point of the user and a second estimate of a subsequent convergence-divergence motion point of the user; determining that the user's convergence-divergence motion point crosses a distance threshold defined for the virtual object based on at least a first estimate of the user's convergence-divergence motion point and a second estimate of the user's convergence-divergence motion point; in response to determining that a location within the user's binocular FOV at which the virtual object will be perceived is outside a comfort zone of the user's binocular FOV and determining that the user's vergence-divergence movement point crosses a distance threshold defined for the virtual object, switching between two different modes of presenting the virtual object to the user; A method comprising: (Item 73) 1. A display system for displaying a virtual object, the display system comprising: one or more processors; One or more computer storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: identifying a type of content corresponding to the virtual object; determining boundaries between different zones of a binocular field of view (FOV) of a user based, at least in part, on the identified type of content corresponding to the virtual object; determining whether the user's convergence-divergence motion point crosses a distance threshold defined for the virtual object; in response to determining that the virtual object crosses a determined boundary between zones of the user's binocular FOV, toggling between starting and resetting a timer associated with the identified type of content; determining whether a timer associated with the identified type of content has expired; modifying the virtual object in accordance with the identified type of content in response to determining that a timer associated with the identified type of content has expired; one or more computer storage media for performing operations including: A display system comprising: (Item 74) 1. A method for operating a head mounted display device configured to display a virtual object, the method comprising: identifying a type of content corresponding to the virtual object; determining boundaries between different zones of a binocular field of view (FOV) of a user based, at least in part, on the identified type of content corresponding to the virtual object; Obtaining a first estimate of a location where a virtual object will be perceived by a user and a second estimate after where the virtual object will be perceived by the user; determining that the virtual object crosses a determined boundary between different zones of a binocular FOV of the user based on at least a first estimate of where the virtual object will be perceived by a user and a second estimate of where the virtual object will be perceived by the user; toggle between starting and resetting a timer associated with the identified type of content in response to determining that the virtual object crosses a determined boundary between zones of the user's binocular FOV; A method comprising: (Item 75) 1. A display system for displaying a virtual object, the display system comprising: one or more processors; One or more computer storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: determining whether the virtual object crosses a boundary between different zones of a user's binocular field of view (FOV); in response to determining that the virtual object crosses a boundary between different zones of the user's binocular FOV, switching between (i) a binocular presentation mode in which the virtual object is displayed to both eyes of the user and (ii) a monocular presentation mode in which the virtual object is displayed to only one eye of the user; one or more computer storage media for performing operations including: A display system comprising: [Brief description of the drawings]

[0093] [Figure 1] FIG. 1 illustrates a user's view of an augmented reality (AR) device.

[0094] [Diagram 2] FIG. 2 illustrates a conventional display system for simulating a three-dimensional image for a user.

[0095] [Diagram 3] 3A-3C illustrate the relationship between the radius of curvature and the radius of focus.

[0096] [Figure 4A] Figure 4A illustrates a representation of the accommodation-vergence-divergence motor response of the human visual system.

[0097] [Figure 4B] FIG. 4B illustrates examples of different accommodation and convergence states of a pair of a user's eyes.

[0098] [Figure 4C] FIG. 4C illustrates an example of a top-down view representation of a user viewing content through a display system.

[0099] [Figure 4D] FIG. 4D illustrates another example of a top-down view representation of a user viewing content through a display system.

[0100] [Diagram 5] FIG. 5 illustrates aspects of an approach for simulating a three-dimensional image by modifying wavefront divergence.

[0101] [Figure 6] FIG. 6 illustrates an embodiment of a waveguide stack for outputting image information to a user.

[0102] [Figure 7] FIG. 7 illustrates an example of an output beam output by a waveguide.

[0103] [Figure 8] FIG. 8 illustrates an example of a stacked waveguide assembly where each depth plane contains an image formed using multiple different primary colors.

[0104] [Figure 9A] FIG. 9A illustrates a cross-sectional side view of an example of a set of stacked waveguides, each of which includes an internal coupling optical element.

[0105] [Figure 9B] FIG. 9B illustrates a perspective view of the multiple stacked waveguide embodiment of FIG. 9A.

[0106] [Figure 9C] FIG. 9C illustrates a top-down plan view of the multiple stacked waveguide embodiment of FIGS. 9A and 9B.

[0107] [Figure 9D] FIG. 9D illustrates an example of a wearable display system.

[0108] [Figure 10] FIG. 10 illustrates an example of an accommodation-vergence-divergence mismatch plot for a single depth plane display system.

[0109] [Figure 11A] FIG. 11A illustrates examples of accommodation-vergence-divergence mismatch plots for a two-depth flat display system and a one-depth display system.

[0110] [Figure 11B] FIG. 11B illustrates another example of an accommodation-vergence-divergence mismatch plot for a two-depth plane display system.

[0111] [Figure 11C] FIG. 11C illustrates another representation of the two-depth configuration of FIG. 11B.

[0112] [Figure 12A] FIG. 12A illustrates an example of a schematic diagram of superior and inferior accommodative-vergence-divergence mismatch (AVM) discomfort zones and fusion loss zones for a viewer.

[0113] [Figure 12B] FIG. 12B illustrates another example of a schematic diagram of superior and inferior accommodative-vergence-divergence mismatch (AVM) discomfort zones and fusion loss zones for a viewer.

[0114] [Figure 12C] FIG. 12C illustrates an example of a schematic top-bottom view of the AVM discomfort zone, taking into account the monocular field of view of a viewer.

[0115] [Figure 12D] FIG. 12D illustrates an example of a schematic top and bottom view of the AVM discomfort zone, encompassing optical infinity.

[0116] [Figure 12E] FIG. 12E illustrates an example of an upper and lower schematic diagram of AVM discomfort and comfort zones.

[0117] [Figure 13A]FIG. 13A illustrates a flowchart of an example process for determining whether to clip the display of a virtual object.

[0118] [Figure 13B] FIG. 13B illustrates a flowchart of another example process for determining whether to clip the display of a virtual object.

[0119] [Figure 13C] FIG. 13C illustrates a flowchart of yet another example process for determining whether to clip the display of a virtual object.

[0120] [Figure 14] FIG. 14 illustrates an example of a top-bottom schematic diagram of multiple accommodation-vergence-divergence mismatch discomfort zones and fusion loss zones for a viewer.

[0121] [Figure 15] FIG. 15 illustrates a flowchart of an example process for determining whether to clip the display of a virtual object based on whether the virtual object is user-locked or in-world content.

[0122] [Figure 16] FIG. 16 illustrates a flowchart of an example process for determining whether to clip the display of a virtual object in the case where a separate application defines its own clipping functionality. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0123] Virtual and augmented display systems can provide a variety of image content, the richness of which can increase with the user's ability to view the content presented by the system over time. For example, augmented reality display systems offer the potential to replace single devices with traditional displays (e.g., computer monitors, smartphone displays, etc.) and can also augment a user's perception of the world by providing content that would not otherwise be available.

[0124] As described herein, the display system may utilize both vergence and accommodation cues to present the displayed virtual content with a sense of depth. Vergence cues may be generated by presenting slightly different views of a virtual object to each eye of the user. Accommodation cues may be derived from the wavefront divergence of light that creates those slightly different views. Vergence cues may rotate the eyes and cause them to assume a particular vergence state, e.g., where the eyes converge on a virtual object. Accommodation cues may cause the eye's lens to assume a particular shape that provides a focused image of the virtual object on the eye's retina. Thus, certain vergence cues may cause the eyes to assume a particular vergence state (e.g., by rotating in a particular direction and converging on an object), and certain accommodation cues may cause the eyes to assume a particular accommodation state (e.g., by changing the shape of the eye's lens). The point in space where the eyes converge may also be referred to as a vergence point or fixation point.

[0125] It should be appreciated that real objects in space provide matched vergence and accommodation cues that vary based on their distance along the optical or z-axis from the viewer, such that a particular vergence cue can be correlated with a particular vergence distance and a particular accommodation cue can likewise be correlated with a particular accommodation distance away from the viewer. Because accommodation and vergence states can be caused by different stimuli, it is possible to have an accommodation-vergence-divergence mismatch (AVM), where a vergence cue may indicate that a particular object is at a particular distance from the viewer, while an accommodation cue may indicate that the object is located in a different position, i.e., at a different distance. The accommodation-vergence-divergence mismatch when displaying a virtual object can be defined as the difference in diopters between the vergence and accommodation distance for the virtual object.

[0126] It has been found that the human visual system can tolerate a certain level of accommodation-vergence mismatch. As a result, within the mismatch tolerance, in one embodiment, the accommodation cues can remain the same while the vergence cues can vary by a limited amount, thereby varying the perceived depth of the virtual object. In some embodiments, the vergence cues can vary continuously while the accommodation cues vary in discrete steps, such that the mismatch between accommodation and vergence is maintained below the mismatch tolerance level. Examples of accommodation-vergence mismatch tolerance include 0.75 diopters or less, 0.66 diopters or less, 0.5 diopters or less, 0.33 diopters or less, or 0.25 diopters or less. It should be understood that the mismatch tolerance may be programmed as a default value for the display system and / or may vary for different viewers (e.g., may be selectable by the viewer and / or may be selected for the viewer based on calibration).

[0127] With respect to accommodation cues, it should be understood that a particular amount of wavefront divergence is associated with a particular depth plane. That is, the wavefront divergence of light output by the display system corresponds to the wavefront divergence of light arising from a real object at a particular depth along the z-axis. As a result, varying the wavefront divergence, and therefore the accommodation cues, may be understood to involve switching the depth plane at which the display system presents the virtual object. In some embodiments, the depth planes may be switched to maintain accommodation-vergence-divergence mismatch below an acceptable tolerance level. Each depth plane may have a nominal focal depth with a corresponding wavefront divergence for light that appears to originate from that depth plane. However, due to accommodation-vergence-divergence mismatch tolerance, content may be displayed "above" that depth plane (i.e., with a wavefront divergence corresponding to that depth plane) even though vergence-divergence cues may be utilized to provide a perception of the virtual object being closer or farther from the viewer than the depth plane. The outer boundaries of the distance at which a particular depth plane may be utilized are determined by the accommodation-vergence-divergence mismatch, which may be measured in units of diopters, as disclosed herein. In some embodiments, the farthest depth plane may be within the accommodation-vergence-divergence mismatch tolerance of optical infinity, the next farthest depth plane may be within the accommodation-vergence-divergence mismatch tolerance of the volume created by the accommodation-vergence-divergence mismatch tolerance of the farthest depth plane, and so on. In some other embodiments, a reference depth plane (e.g., the nearest depth plane) may be a selected distance from the viewer, and the location of one or more other depth planes on the Z-axis may be selected to be within the accommodation-vergence-divergence mismatch tolerance of that reference depth plane. In some embodiments, a single depth plane (only the reference depth plane) may be utilized.

[0128] It should be appreciated that due to practical and / or technical constraints, a display system may be limited in its ability to vary the wavefront divergence and / or present an amount of wavefront divergence, particularly a relatively large amount of wavefront divergence associated with objects at distances close to the viewer, or a small amount of wavefront divergence associated with objects approaching or at optical infinity in a system whose furthest depth plane is close to the viewer and the system is therefore configured to provide a large amount of wavefront divergence. As a result, the display system's ability to present different accommodation cues may exceed its ability to present different vergence-divergence cues. At certain distances from the viewer, accommodation-vergence-divergence mismatch may exceed an acceptable tolerance, or threshold, level and thus cause viewer discomfort. It should be appreciated that these distances may be understood to define the boundaries of a volume or zone around the viewer. For example, the zone may be understood as an AVM discomfort zone where the accommodation-vergence-divergence mismatch of the object at any location within the zone exceeds the viewer's AVM threshold or accommodation-vergence-divergence mismatch tolerance. In some embodiments, the accommodation-vergence-divergence mismatch at any point outside the AVM discomfort zone remains within the accommodation-vergence-divergence mismatch tolerance. In some embodiments, the boundary of the AVM discomfort zone may be defined by a surface that has a constant distance from the viewer. In some other embodiments, the surface may have a variable distance from the viewer depending on the direction of the viewer's gaze.

[0129] In some embodiments, for scenarios where the display system does not provide a sufficiently high level of wavefront divergence for virtual objects at near distances, the near AVM discomfort zone may be presented as a volume proximate to the viewer. In certain other examples, for scenarios where the display system does not provide a sufficiently low level of wavefront divergence for virtual objects at far distances, the far AVM discomfort zone may be presented as a volume proximate to and encompassing optical infinity. In some embodiments, the display system may have both near and far AVM discomfort zones, one AVM discomfort zone proximate to the viewer and another AVM discomfort zone encompassing optical infinity.

[0130] Additionally, without being limited by theory, it is believed that the human visual system may be tolerant of accommodation-vergence-divergence mismatch above a threshold level for a limited amount of time, but it has been found that this tolerance may not extend to distances within the viewer's fusional loss zone. The fusional loss zone (LoF) should be understood to include distances close to the viewer where the human visual system will not combine the images seen by each eye (or the viewer must distort their eyes to allow the human visual system to combine the images seen by each eye) such that the viewer will see an overlap of objects at those distances (i.e., within the fusional loss zone) unless the viewer consciously distorts their eyes. In fact, viewer discomfort caused by relatively high accommodation-vergence-divergence mismatch may be amplified when combined with fusional loss (i.e., double vision). As described in more detail below, the fusion loss zone can also be defined in terms of vergence angles such that the LoF zone includes points in three-dimensional space associated with viewer vergence angles that are not conducive to image fusion. Given the trigonometric relationship between viewing distance and viewer vergence angles, relatively wide viewer vergence angles can be considered as impediments to binocular image fusion, as can relatively close viewing distances. In either case, each point in three-dimensional space along the outer boundary of the LoF zone can be considered, at least in some embodiments, to represent a "near point of convergence" (NPC) (i.e., the nearest point of binocular fused monocular vision) for a particular viewing position and orientation. For example, the human visual system may not combine images seen by each eye within the outer boundary, while outside each eye's outer boundary it does combine images seen by each eye. The outer boundary may be at a fixed distance from the viewer, or at a variable distance depending on the viewer's direction of gaze.

[0131] It should also be appreciated that the virtual content displayed to the viewer may include user-locked content and in-world content in some embodiments. User-locked content includes virtual content that is locked within the viewer's field of view such that the viewer cannot avoid the content by moving their head or looking away. For example, user-locked content may include notifications (e.g., advertisements, user interface objects, etc.) that follow the viewer and remain within the viewer's field of view. That is, they are not anchored in a fixed position relative to the external environment and cannot be avoided when utilizing the display system. In-world content, on the other hand, is virtual content that is anchored to the surrounding environment, an object within the surrounding environment, or other content that is not rigidly anchored to the viewer within the viewer's field of view. As a result, the viewer may change their view of the in-world content and / or avoid seeing it by moving, turning their head, etc. Without being limited by theory, it is believed that the viewer may be more sensitive to accommodation-vergence-divergence movement mismatch with respect to user-locked content than in-world content. It should be understood that virtual content may also be referred to as virtual objects, and both terms are used interchangeably herein.

[0132] In some embodiments, the display system may be configured to modify the display of the virtual object (i.e., to "clip" the virtual object) using the analysis, including determining whether the viewer is convergent to a point within the AVM discomfort zone (e.g., below the AVM threshold distance in instances where the AVM discomfort zone is proximate to the viewer, or above the AVM threshold distance when the AVM discomfort zone encompasses optical infinity) and / or whether the object is located within the AVM discomfort zone (e.g., below the accommodation-vergence-divergence mismatch (AVM) threshold distance when the AVM discomfort zone is proximate to the viewer, or above the AVM threshold distance when the AVM discomfort zone encompasses optical infinity). Modifying the display of the virtual object may include, for example, ceasing to display the virtual object (in one or both eyes), changing the position of the object (e.g., by moving the object away from the viewer), etc., as described herein. Additionally or alternatively, the display system may be configured to determine whether the virtual object is located within the LoF zone (e.g., below a loss of fusion (LoF) threshold distance from the viewer). If the virtual object is within the LoF zone, the system may be configured to clip the display of the object. If the virtual object is within the AVM discomfort zone but outside the LoF zone, the system may be configured to take a predetermined action related to limiting the display of the object. For example, the predetermined action may include limiting the amount of time the object is displayed within the AVM discomfort zone, then ceasing or fading out the display of the object, and / or moving the object outside the AVM discomfort zone. In some embodiments, after a predetermined cooling period, the display system may be configured to display the clipped virtual object again.It should be appreciated that the virtual object and its position may again be analyzed as disclosed herein to determine whether it should be clipped again.

[0133] In some embodiments, the amount of time that an object may be displayed within the AVM discomfort zone is limited to, for example, 1 second or less, 10 seconds or less, 30 seconds or less, 3 minutes or less, and / or a variable user or system selected amount of time. It should be understood that the amount of time that a viewer is exposed to content within the AVM discomfort zone may be referred to as an exposure time. In some embodiments, the exposure time may be utilized to determine a cooling period. For example, the duration of the cooling period may increase with the duration of the exposure time. In some embodiments, the cooling period may be 3 seconds or more, 5 seconds or more, or 10 seconds or more.

[0134] In some embodiments, the size of the AVM discomfort zone may vary depending on the type of content being displayed. For example, user-locked content may have a smaller AVM discomfort zone than in-world content. As a result, the display system may be configured to determine the type of content being displayed and therefore determine the appropriate AVM discomfort zone boundary before performing the analysis for clipping discussed above and further below. In some embodiments, in addition to, or as an alternative to, varying the AVM threshold distance depending on whether a virtual object constitutes user-locked content or in-world content, the time limit for exposure to the object may also be varied based on the content type. For example, user-locked content may have a shorter exposure time limit (e.g., 1 second or less) than in-world content (e.g., 10 seconds or less).

[0135] In some embodiments, the display system may be further configured to determine whether the viewer is actually converging on the virtual object (e.g., whether the convergence-divergence motion point corresponds to the location of the virtual object). If the system determines that the viewer is not converging on the virtual object, the object is not clipped. In some embodiments, the display system may be configured to determine whether the virtual object is within the user's field of view and not address displaying the object if it is outside the field of view. Advantageously, a more realistic experience and more functionality may be provided by limiting clipping of the object to situations where the viewer is actually converging within the AVM discomfort zone (e.g., below the AVM threshold distance for AVM discomfort zones close to the viewer, or above the AVM threshold distance for AVM discomfort zones encompassing optical infinity), rather than clipping all content within the AVM discomfort zone. For example, objects that are relatively close to the viewer may be presented without increasing viewer discomfort.

[0136] It should be appreciated that clipping analysis may be performed sequentially or in parallel for various virtual objects displayed to a user, e.g., virtual objects within a user's field of view.

[0137] Reference is now made to the drawings in which like reference numbers refer to like parts throughout, and, unless specifically indicated otherwise, the drawings are schematic and are not necessarily drawn to scale.

[0138] Exemplary Display Systems FIG. 2 illustrates a conventional display system for simulating a three-dimensional image for a user. It should be understood that when a user's eyes are spaced apart and viewing a real object in space, each eye may have a slightly different view of the object and form an image of the object at a different location on each eye's retina. This may be referred to as binocular disparity and may be exploited by the human visual system to provide a perception of depth. Conventional display systems simulate binocular disparity by presenting two distinctly different images 190, 200 with slightly different views of the same virtual object, one for each eye 210, 220, corresponding to the view of the virtual object as it would appear by each eye as if the virtual object were a real object at a desired depth. These images provide binocular cues that the user's visual system may interpret to derive a perception of depth.

[0139] Continuing with reference to FIG. 2, the images 190, 200 are spaced from the eyes 210, 220 by a distance 230 on the z-axis, which is parallel to the optical axis of the viewer when that eye is fixating on an object at optical infinity directly in front of the viewer. The images 190, 200 are flat and at a fixed distance from the eyes 210, 220. Based on slightly different views of the virtual object in the images presented to the eyes 210, 220, respectively, the eyes may necessarily rotate such that the image of the object falls on a corresponding point on the eye's respective retina to maintain single binocular vision. This rotation may cause the line of sight of each of the eyes 210, 220 to converge on a point in space where the virtual object is perceived to reside. As a result, providing three-dimensional images traditionally involves manipulating the convergence and divergence movements of the user's eyes 210, 220 to provide binocular cues that the human visual system interprets to provide the perception of depth.

[0140] However, creating a realistic and comfortable perception of depth is difficult. It should be understood that light from an object at different distances from the eye has a wavefront with different amounts of divergence. Figures 3A-3C illustrate the relationship between distance and divergence of light rays. The distance between the object and the eye 210 is represented in the order of decreasing distances R1, R2, and R3. As shown in Figures 3A-3C, the light rays become more divergent as the distance to the object decreases. Conversely, as the distance increases, the light rays become more collimated. In other words, the light field generated by a point (an object or part of an object) can be said to have a spherical wavefront curvature that is a function of the distance the point is away from the user's eye. The curvature increases with decreasing distance between the object and the eye 210. Although only a single eye 210 is illustrated in Figures 3A-3C and various other figures herein for clarity of illustration, the discussion regarding the eye 210 may apply to both eyes 210 and 220 of a viewer.

[0141] 3A-3C, light from an object that a viewer's eye is fixating on may have different wavefront divergences. Due to the different wavefront divergences, the light may be focused differently by the eye's lens, which in turn may require the lens to assume a different shape and form a focused image on the eye's retina. If a focused image is not formed on the retina, the resulting retinal blur acts as an accommodative cue, causing the shape of the eye's lens to change until a focused image is formed on the retina. For example, an accommodative cue may induce relaxation or contraction of the ciliary muscle surrounding the eye's lens, thereby modulating the force applied to the suspensory ligament that holds the lens, thus changing the shape of the eye's lens, thereby forming a focused image of the fixated object on the eye's retina (e.g., the fovea) until retinal blur of the fixated object is eliminated or minimized. The process by which the eye lens changes shape may be referred to as accommodation, and the shape of the eye lens required to form a focused image of a fixated object on the eye's retina (e.g., the fovea) may be referred to as the state of accommodation.

[0142] Referring now to FIG. 4A, a representation of the accommodation-vergence response of the human visual system is illustrated. A movement of the eye to fixate an object causes the eye to receive light from the object, which forms an image on each of the eye's retinas. The presence of retinal blur in the image formed on the retina may provide a cue for accommodation, and the relative location of the image on the retina may provide a cue for vergence. A cue for accommodation causes accommodation, resulting in the eye adopting a particular accommodation state in which the eye's lens forms a focused image of the object on the eye's retina (e.g., the fovea). On the other hand, a cue for vergence causes a vergence movement (eye rotation) such that the image formed on each retina of each eye is at a corresponding retinal point that maintains single binocular vision. In these positions, the eyes may be said to adopt a particular vergence state. Continuing with reference to FIG. 4A, accommodation can be understood to be the process by which the eye achieves a particular accommodation state, and convergence can be understood to be the process by which the eye achieves a particular convergence state. As shown in FIG. 4A, the accommodation and convergence state of the eye can change when the user fixates a different object. For example, the accommodated state can change when the user fixates a new object at a different depth on the z-axis.

[0143] Without being limited by theory, it is believed that a viewer of an object may perceive the object as being "three-dimensional" due to a combination of vergence-divergence and accommodation. As previously mentioned, vergence-divergence movements of the two eyes relative to one another (e.g., rotation of the eyes such that the pupils move toward or away from one another, converging the gaze of the eyes to fixate on an object) are closely linked to accommodation of the eye's lenses. Under normal conditions, changing the shape of the eye's lenses and shifting focus from one object to another at a different distance will automatically produce a matching change in vergence to the same distance, a relationship known as the "accommodation-vergence-divergence reflex." Similarly, a change in vergence will induce a matching change in lens shape under normal conditions.

[0144] 4B, an example of different accommodation and convergence-divergence states of the eyes is illustrated. Paired eye 222a fixates an object at optical infinity, while paired eye 222b fixates an object 221 at less than optical infinity. Notably, the convergence-divergence states of each pair of eyes are different, paired eye 222a points straight ahead, while paired eye 222 converges on object 221. The accommodation states of the eyes forming each pair of eyes 222a and 222b are also different, as represented by the different shapes of the lenses 210a, 220a.

[0145] Unfortunately, many users of conventional "3-D" display systems may find such conventional systems uncomfortable or may not perceive any depth sensation due to a mismatch between accommodation and vergence states in these displays. As previously mentioned, many stereoscopic or "3-D" display systems display a scene by providing a slightly different image to each eye. Such systems are uncomfortable for many viewers because, among other things, they simply provide a different presentation of the scene, causing changes in the vergence states of the eyes, but without a corresponding change in the accommodation states of those eyes. Rather, images are presented by the display at a fixed distance from the eyes, such that the eyes view all image information in a single accommodation state. Such an arrangement counters the "accommodation-vergence-divergence reflex" by causing changes in vergence states without a matching change in accommodation state. This mismatch is believed to cause viewer discomfort. By providing better matching between accommodation and convergence-divergence movements, display systems may form a more realistic and comfortable simulation of three-dimensional images.

[0146] Without being limited by theory, it is believed that the human eye is typically capable of interpreting a finite number of depth planes to provide depth perception. As a result, a highly realistic simulation of perceived depth may be achieved by providing the eye with different presentations of images corresponding to each of these limited number of depth planes. In some embodiments, the different presentations may provide both vergence cues and matching cues for accommodation, thereby providing physiologically acceptable accommodation-vergence divergence matching.

[0147] 4B, two depth planes 240 are illustrated, corresponding to different distances in space from the eyes 210, 220. For a given depth plane 240, vergence-divergence cues may be provided by displaying appropriately different perspective images for each eye 210, 220. Additionally, for a given depth plane 240, the light forming the image provided to each eye 210, 220 may have a wavefront divergence corresponding to the light field generated by a point at the distance of that depth plane 240.

[0148] In the illustrated embodiment, the distance along the z-axis of the depth plane 240 containing the point 221 is 1 m. As used herein, the distance or depth along the z-axis may be measured with the zero point located at the pupil of the user's eye. Thus, the depth plane 240 located at a depth of 1 m corresponds to a distance of 1 m away from the pupil of the user's eye on the optical axis of those eyes with the eyes pointed towards optical infinity. As an approximation, the depth or distance along the z-axis may be measured from the display (e.g., the surface of the waveguide) in front of the user's eye, and a value for the distance between the device and the pupil of the user's eye may be added. That value may be called the pupil distance and may correspond to the distance between the pupil of the user's eye and the display worn by the user in front of the eye. In practice, the value for the pupil distance may be a normalized value that is generally used for all viewers. For example, the pupil distance may be assumed to be 20 mm, and the depth plane at a depth of 1 m may be at a distance of 980 mm in front of the display.

[0149] 4C and 4D, examples of matched accommodation-vergence-divergence distances and mismatched accommodation-vergence-divergence distances are illustrated, respectively. As illustrated in FIG. 4C, the display system may provide an image of a virtual object to each eye 210, 220. The image may cause the eyes 210, 220 to assume a convergence-divergence state in which the eyes converge on point 15 on the depth plane 240. In addition, the image may be formed by light having a wavefront curvature corresponding to a real object in that depth plane 240. As a result, the eyes 210, 220 assume an accommodation state in which the image is focused on the retinas of those eyes. Thus, the user may perceive the virtual object as being at point 15 on the depth plane 240.

[0150] It should be appreciated that each of the accommodation and convergence states of the eyes 210, 220 is associated with a particular distance on the z-axis. For example, an object at a particular distance from the eyes 210, 220 will cause those eyes to assume a particular accommodation state based on the distance of the object. The distance associated with a particular accommodation state is referred to as the accommodation distance A. d Similarly, a particular convergence-divergence distance V associated with the eyes in a particular convergence-divergence state may be referred to as d or positions relative to each other. When the accommodation distance and the vergence distance match, the relationship between accommodation and vergence can be said to be physiologically acceptable. This is considered to be the most comfortable scenario for the viewer.

[0151] However, in a stereoscopic display, the accommodation distance and the convergence distance may not always match. For example, as illustrated in FIG. 4D, the images displayed to the eyes 210, 220 may be displayed with a wavefront divergence corresponding to the depth plane 240, and the eyes 210, 220 may be in a particular accommodation state in which the points 15a, 15b on that depth plane are in focus. However, the images displayed to the eyes 210, 220 may provide a convergence cue that causes the eyes 210, 220 to converge to a point 15 that is not located on the depth plane 240. As a result, the accommodation distance corresponds in some embodiments to the distance from the pupils of the eyes 210, 220 to the depth plane 240, while the convergence distance corresponds to a larger distance from the pupils of the eyes 210, 220 to the point 15. The accommodation distance is different from the convergence distance. As a result, an accommodation-vergence-divergence mismatch exists. Such a mismatch may be considered undesirable and may cause discomfort to the user. The mismatch may be related to distance (e.g., V d -A d ) and can be characterized in terms of diopters.

[0152] It should be understood that in some embodiments, reference points other than the pupils of the eyes 210, 220 may be used to determine the accommodation-vergence-divergence mismatch, so long as the same reference points are used for accommodation distance and vergence-divergence distance. For example, distance may be measured from the cornea to the depth plane, from the retina to the depth plane, from the eyepiece (e.g., a waveguide in a display device) to the depth plane, etc.

[0153] Without being limited by theory, it is believed that a user may still perceive an accommodation-vergence-divergence mismatch of up to about 0.25 diopters, up to about 0.33 diopters, up to about 0.5 diopters, up to about 0.66 diopters, and up to about 0.75 diopters as physiologically acceptable without the mismatch itself causing significant discomfort. In some embodiments, a display system disclosed herein (e.g., display system 250, FIG. 6) presents an image to a viewer having an accommodation-vergence-divergence mismatch of about 0.75 diopters or less. In some other embodiments, the accommodation-vergence-divergence mismatch of the image provided by the display system is about 0.66 diopters or less. In yet other embodiments, the accommodation-vergence-divergence mismatch of the image provided by the display system is about 0.50 diopters or less. In other embodiments, the accommodation-vergence-divergence mismatch of the images provided by the display system is about 0.33 diopters or less, hi yet other embodiments, the accommodation-vergence-divergence mismatch of the images provided by the display system is about 0.25 diopters or less, including about 0.1 diopters or less.

[0154] FIG. 5 illustrates aspects of an approach for simulating a three-dimensional image by modifying wavefront divergence. The display system includes a waveguide 270 configured to receive light 770 encoded with image information and output the light to a user's eye 210. The waveguide 270 may output light 650 with a defined amount of wavefront divergence that corresponds to the wavefront divergence of a light field generated by a point on a desired depth plane 240. In some embodiments, the same amount of wavefront divergence is provided for all objects presented on that depth plane. In addition, the user's other eye will be illustrated as being provided with image information from a similar waveguide.

[0155] In some embodiments, a single waveguide may be configured to output light with a set wavefront divergence corresponding to a single or limited number of depth planes, and / or the waveguide may be configured to output light of a limited range of wavelengths. As a result, in some embodiments, multiples or stacks of waveguides may be utilized to provide different wavefront divergence for different depth planes and / or output light of different ranges of wavelengths. It should be understood that as used herein, a depth plane may follow the contour of a flat or curved surface. In some embodiments, advantageously, for simplicity, a depth plane may follow the contour of a flat surface.

[0156] 6 illustrates an example of a waveguide stack for outputting image information to a user. The display system 250 includes a stack of waveguides or a stacked waveguide assembly 260 that can be utilized to provide a three-dimensional perception to the eye / brain using multiple waveguides 270, 280, 290, 300, 310. It should be understood that the display system 250 may be considered a light field display in some embodiments. Additionally, the waveguide assembly 260 may also be referred to as an eyepiece.

[0157] In some embodiments, the display system 250 may be configured to provide a substantially continuous cue for convergence and multiple discrete cues for accommodation. The cues for convergence may be provided by displaying different images to each of the user's eyes, and the cues for accommodation may be provided by outputting light forming images with selectable discrete amounts of wavefront divergence. In other words, the display system 250 may be configured to output light with variable levels of wavefront divergence. In some embodiments, each discrete level of wavefront divergence corresponds to a particular depth plane and may be provided by a particular one of the waveguides 270, 280, 290, 300, 310.

[0158] Continuing with reference to FIG. 6, the waveguide assembly 260 may also include a number of features 320, 330, 340, 350 between the waveguides. In some embodiments, the features 320, 330, 340, 350 may be one or more lenses. The waveguides 270, 280, 290, 300, 310 and / or the number of lenses 320, 330, 340, 350 may be configured to transmit image information to the eye with various levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane. The image input devices 360, 370, 380, 390, 400 may act as light sources for the waveguides and may be utilized to input image information into the waveguides 270, 280, 290, 300, 310, each of which may be configured to disperse incident light across each individual waveguide for output towards the eye 210, as described herein. Light exits output surfaces 410, 420, 430, 440, 450 of the image input devices 360, 370, 380, 390, 400 and is input into corresponding input surfaces 460, 470, 480, 490, 500 of the waveguides 270, 280, 290, 300, 310. In some embodiments, each of the input surfaces 460, 470, 480, 490, 500 may be an edge of the corresponding waveguide or a portion of the major surface of the corresponding waveguide (i.e., one of the waveguide surfaces that directly faces the world 510 or the viewer's eye 210). It should be understood that the major surface of the waveguide corresponds to the relatively large area surface of the waveguide between which the thickness of the waveguide extends. In some embodiments, a single beam of light (e.g., a collimated beam) may be launched into each waveguide, outputting a total field of cloned collimated beams that are directed towards the eye 210 at a particular angle (and divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, a single one of the image launch devices 360, 370, 380, 390, 400 may be associated with and launch light into multiple (e.g., three) waveguides 270, 280, 290, 300, 310.

[0159] In some embodiments, each of the image input devices 360, 370, 380, 390, 400 is a discrete display that generates image information for input into the corresponding waveguides 270, 280, 290, 300, 310, respectively. In some other embodiments, the image input devices 360, 370, 380, 390, 400 are the output of a single multiplexed display that may, for example, send image information via one or more optical conduits (such as fiber optic cables) to each of the image input devices 360, 370, 380, 390, 400. It should be understood that the image information provided by the image input devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (e.g., different primary colors as discussed herein).

[0160] In some embodiments, the light injected into the waveguides 270, 280, 290, 300, 310 is provided by a light projector system 520, which includes a light module 530, which may include a light emitter such as a light emitting diode (LED). Light from the light module 530 may be directed through a beam splitter 550 to and modified by a light modulator 540, e.g., a spatial light modulator. The light modulator 540 may be configured to change the perceived intensity of the light injected into the waveguides 270, 280, 290, 300, 310 and encode the light with image information. Examples of spatial light modulators include liquid crystal displays (LCDs), including liquid crystal on silicon (LCOS) displays. It should be understood that image injection devices 360, 370, 380, 390, 400 are illustrated diagrammatically and in some embodiments these image injection devices may represent different light paths and locations within a common projection system configured to output light into associated ones of waveguides 270, 280, 290, 300, 310. In some embodiments, the waveguides of the waveguide assembly 260 may act as ideal lenses, relaying light injected into the waveguides to the user's eye. In this concept, the object may be a spatial light modulator 540 and the image may be an image on a depth plane.

[0161] In some embodiments, the display system 250 may be a scanning fiber display comprising one or more scanning fibers configured to project light in various patterns (e.g., raster scan, spiral scan, Lissajous pattern, etc.) into one or more waveguides 270, 280, 290, 300, 310 and ultimately to the viewer's eye 210. In some embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may diagrammatically represent a single scanning fiber or a bundle of scanning fibers configured to inject light into one or more waveguides 270, 280, 290, 300, 310. In some other embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may diagrammatically represent multiple scanning fibers or multiple bundles of scanning fibers, each configured to inject light into an associated one of the waveguides 270, 280, 290, 300, 310. It should be understood that one or more optical fibers may be configured to transmit light from the optical module 530 to one or more of the waveguides 270, 280, 290, 300, 310. It should be understood that one or more intervening optical structures may be provided between the scanning fiber or fibers and one or more of the waveguides 270, 280, 290, 300, 310, for example, to redirect light exiting the scanning fiber into one or more of the waveguides 270, 280, 290, 300, 310.

[0162] The controller 560 controls the operation of one or more of the stacked waveguide assemblies 260, including the operation of the image input devices 360, 370, 380, 390, 400, the light source 530, and the light module 540. In some embodiments, the controller 560 is part of the local data processing module 140. The controller 560 includes programming (e.g., instructions in a non-transient medium) that coordinates the timing and provision of image information to the waveguides 270, 280, 290, 300, 310, for example, according to any of the various schemes disclosed herein. In some embodiments, the controller may be a single integrated device or may be a distributed system connected by wired or wireless communication channels. The controller 560 may be part of the processing module 140 or 150 (FIG. 9D) in some embodiments.

[0163] Continuing with reference to FIG. 6, the waveguides 270, 280, 290, 300, 310 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). Each of the waveguides 270, 280, 290, 300, 310 may be planar or have another shape (e.g., curved), with major top and bottom surfaces and edges extending between the major top and bottom surfaces. In the illustrated configuration, the waveguides 270, 280, 290, 300, 310 may each include an outcoupling optical element 570, 580, 590, 600, 610 configured to extract light from the waveguide by redirecting light propagating within each individual waveguide out of the waveguide and outputting image information to the eye 210. The extracted light may also be referred to as outcoupling light, and the outcoupling optical element of light may also be referred to as a light extraction optical element. The extracted beam of light may be output by the waveguide at a location where the light propagating within the waveguide strikes the light extraction optical element. The outcoupling optical element 570, 580, 590, 600, 610 may be a grating, for example, including diffractive optical features as further discussed herein. Although shown disposed on the bottom major surface of the waveguide 270, 280, 290, 300, 310 for ease of explanation and clarity of drawing, in some embodiments the outcoupling optical element 570, 580, 590, 600, 610 may be disposed on the top major surface and / or bottom major surface and / or directly within the volume of the waveguide 270, 280, 290, 300, 310, as further discussed herein. In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 may be formed in a layer of material that is attached to a transparent substrate and forms the waveguide 270, 280, 290, 300, 310. In some other embodiments, the waveguide 270, 280, 290, 300, 310 may be a monolithic piece of material and the outcoupling optical elements 570, 580, 590, 600, 610 may be formed on and / or within that piece of material.

[0164] With continued reference to FIG. 6, as discussed herein, each waveguide 270, 280, 290, 300, 310 is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye may be configured to deliver collimated light (injected into such waveguide 270) to the eye 210. The collimated light may represent an optical infinity focal plane. The next upper waveguide 280 may be configured to send collimated light that passes through a first lens 350 (e.g., a negative lens) before it can reach the eye 210. Such a first lens 350 may be configured to generate a slight convex wavefront curvature such that the eye / brain interprets the light originating from the next upper waveguide 280 as originating from a first focal plane closer inward from optical infinity toward the eye 210. Similarly, the third upper waveguide 290 passes its output light through both the first lens 350 and the second lens 340 before reaching the eye 210. The combined refractive powers of the first lens 350 and the second lens 340 may be configured to produce another incremental amount of wavefront curvature such that the eye / brain interprets the light emerging from the third waveguide 290 as originating from a second focal plane that is closer inward from optical infinity towards the person than was the light from the next upper waveguide 280.

[0165] The other waveguide layers 300, 310 and lenses 330, 320 are similarly configured, with the highest waveguide 310 in the stack sending its output through all of the lenses between it and the eye for an aggregate focal power that represents the focal plane closest to the person. To compensate the stack of lenses 320, 330, 340, 350 when viewing / interpreting light originating from the world 510 on the other side of the stacked waveguide assembly 260, a compensating lens layer 620 may be placed on top of the stack to compensate for the aggregate power of the lens stacks 320, 330, 340, 350 below. Such a configuration provides as many perceived focal planes as there are waveguide / lens pairs available. Both the outcoupling optical elements of the waveguides and the focusing sides of the lenses may be static (i.e., not dynamic or electroactive). In some alternative embodiments, one or both may be dynamic using electroactive features.

[0166] In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, multiple waveguides 270, 280, 290, 300, 310 may be configured to output images set at the same depth plane, or multiple subsets of the waveguides 270, 280, 290, 300, 310 may be configured to output images set at the same depth planes, with one set per depth plane. This may provide the advantage of forming tiled images to provide an extended field of view at those depth planes.

[0167] Continuing with reference to FIG. 6, the outcoupling optical elements 570, 580, 590, 600, 610 may be configured to redirect light from its respective waveguide and output the light with an appropriate amount of divergence or collimation for a particular depth plane associated with the waveguide. As a result, waveguides with different associated depth planes may have different configurations of outcoupling optical elements 570, 580, 590, 600, 610 that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light extraction optical elements 570, 580, 590, 600, 610 may be volume or surface features that may be configured to output light at specific angles. For example, the light extraction optical elements 570, 580, 590, 600, 610 may be volume holograms, surface holograms, and / or diffraction gratings. In some embodiments, the features 320, 330, 340, 350 may not be lenses. Rather, they may simply be spacers (eg, cladding layers and / or structures for forming air gaps).

[0168] In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 are diffractive features that form a diffraction pattern or "diffractive optical elements" (also referred to herein as "DOEs"). Preferably, the DOEs have a low enough diffraction efficiency so that only a portion of the light in the beam is deflected toward the eye 210 at each intersection of the DOE, while the remainder continues traveling through the waveguide via TIR. The light carrying the image information is thus split into several related exit beams that exit the waveguide at various locations, resulting in a very uniform pattern of exit emission toward the eye 210 for this particular collimated beam bouncing within the waveguide.

[0169] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer dispersed liquid crystal in which microdroplets comprise a diffractive pattern in a host medium, and the refractive index of the microdroplets may be switched to substantially match the refractive index of the host material (in which case the pattern does not significantly diffract incident light), or the microdroplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0170] In some embodiments, a camera assembly 630 (e.g., a digital camera, including visible and infrared light cameras) may be provided to capture images of the eye 210 and / or tissue surrounding the eye 210, for example, to detect user input and / or monitor a physiological condition of the user. As used herein, a camera may be any image capture device. In some embodiments, the camera assembly 630 may include an image capture device and a light source that projects light (e.g., infrared light) onto the eye, which may then be reflected by the eye and detected by the image capture device. In some embodiments, the camera assembly 630 may be mounted to the frame 80 (FIG. 9D) and may be in electrical communication with the processing modules 140 and / or 150, which may process image information from the camera assembly 630. In some embodiments, one camera assembly 630 may be utilized per eye, monitoring each eye separately.

[0171] 7, an example of an output beam output by a waveguide is shown. Although one waveguide is shown, it should be understood that other waveguides in the waveguide assembly 260 (FIG. 6) may function similarly and that the waveguide assembly 260 includes multiple waveguides. Light 640 is launched into the waveguide 270 at the input surface 460 of the waveguide 270 and propagates within the waveguide 270 by TIR. At the point where the light 640 impinges on the DOE 570, a portion of the light exits the waveguide as an output beam 650. The output beam 650 is shown as approximately parallel, but may be redirected to propagate to the eye 210 at an angle (e.g., forming a diverging output beam) as discussed herein and depending on the depth plane associated with the waveguide 270. It should be understood that a nearly collimated exit beam may refer to a waveguide with outcoupling optics that outcouples light to form an image that appears to be set at a depth plane at a large distance (e.g., optical infinity) from the eye 210. Other waveguides or other sets of outcoupling optics may output an exit beam pattern that is more divergent, which would require the eye 210 to accommodate to a closer distance and focus on the retina, and would be interpreted by the brain as light from a distance closer to the eye 210 than optical infinity.

[0172] In some embodiments, a full color image may be formed at each depth plane by overlaying an image in each of the primary colors, for example, three or more primary colors. FIG. 8 illustrates an example of a stacked waveguide assembly in which each depth plane includes an image formed using multiple different primary colors. The illustrated embodiment shows depth planes 240a-240f, but more or less depths are also contemplated. Each depth plane may have three or more primary color images associated with it, including a first image of a first color G, a second image of a second color R, and a third image of a third color B. The different depth planes are indicated in the diagram by different numbers for diopters (dpt) following the letters G, R, and B. Merely by way of example, the numbers following each of these letters indicate the diopters (1 / m), i.e., the inverse distance of the depth plane from the viewer, and each box in the diagram represents an individual primary color image. In some embodiments, the exact locations of the depth planes for the different primary colors may be varied to account for differences in the eye's focusing of light of different wavelengths. For example, different primary color images for a given depth plane may be placed on depth planes that correspond to different distances from the user. Such an arrangement may increase visual acuity and user comfort, and / or reduce chromatic aberration.

[0173] In some embodiments, the light of each primary color may be output by a single dedicated waveguide, such that each depth plane may have multiple waveguides associated with it. In such embodiments, each box in the figure containing the letter G, R, or B may be understood to represent an individual waveguide, and three waveguides may be provided per depth plane, with three primary color images provided per depth plane. The waveguides associated with each depth plane are shown adjacent to each other in this drawing for ease of illustration, but it should be understood that in a physical device, the waveguides may all be arranged in a stack with one waveguide per level. In some other embodiments, multiple primary colors may be output by the same waveguide, such that, for example, only a single waveguide may be provided per depth plane.

[0174] 8, in some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light, including magenta and cyan, may be used in addition to or in place of one or more of red, green, or blue.

[0175] It should be understood that references to a given color of light throughout this disclosure will be understood to encompass one or more wavelengths of light within the range of wavelengths of light that are perceived by a viewer as that given color. For example, red light may include one or more wavelengths of light within a range of about 620-780 nm, green light may include one or more wavelengths of light within a range of about 492-577 nm, and blue light may include one or more wavelengths of light within a range of about 435-493 nm.

[0176] In some embodiments, the light source 530 (FIG. 6) may be configured to emit light at one or more wavelengths outside the range of visual perception of a viewer, e.g., infrared and / or ultraviolet wavelengths. Additionally, the waveguide in-coupling, out-coupling, and other light redirecting structures of the display 250 may be configured to direct and emit this light from the display toward the user's eye 210, e.g., for imaging and / or user stimulation applications.

[0177] 9A, in some embodiments, light impinging on a waveguide may need to be redirected to in-couple the light into the waveguide. An in-coupling optical element may be used to redirect and in-couple the light into its corresponding waveguide. FIG. 9A illustrates a cross-sectional side view of an example of a plurality or set 660 of stacked waveguides, each including an in-coupling optical element. Each of the waveguides may be configured to output light of one or more different wavelengths or one or more different wavelength ranges. The stack 660 may correspond to the stack 260 (FIG. 6), and the illustrated waveguides of the stack 660 may correspond to a portion of the plurality of waveguides 270, 280, 290, 300, 310, but it should be understood that light from one or more of the image input devices 360, 370, 380, 390, 400 is launched into the waveguide from a position that requires the light to be redirected for in-coupling.

[0178] The illustrated set 660 of stacked waveguides includes waveguides 670, 680, and 690. Each waveguide includes an associated incoupling optical element (which may also be referred to as the light input area on the waveguide), for example, incoupling optical element 700 is disposed on a major surface (e.g., the top major surface) of waveguide 670, incoupling optical element 710 is disposed on a major surface (e.g., the top major surface) of waveguide 680, and incoupling optical element 720 is disposed on a major surface (e.g., the top major surface) of waveguide 690. In some embodiments, one or more of the incoupling optical elements 700, 710, 720 may be disposed on the bottom major surface of the respective waveguides 670, 680, 690 (particularly, one or more of the incoupling optical elements are reflective polarizing optical elements). As shown, the incoupling optical elements 700, 710, 720 may be disposed on the upper major surfaces of their respective waveguides 670, 680, 690 (or on top of the next lower waveguide), and in particular, the incoupling optical elements are transmissive polarizing optical elements. In some embodiments, the incoupling optical elements 700, 710, 720 may be disposed within the body of the respective waveguides 670, 680, 690. In some embodiments, as discussed herein, the incoupling optical elements 700, 710, 720 are wavelength selective, such that they selectively redirect one or more wavelengths of light while transmitting other wavelengths of light. Although illustrated on one side or corner of the respective waveguides 670, 680, 690, it should be understood that the incoupling optical elements 700, 710, 720 may be disposed within other areas of the respective waveguides 670, 680, 690 in some embodiments.

[0179] As shown, the in-coupling optical elements 700, 710, 720 may be laterally offset from one another. In some embodiments, each in-coupling optical element may be offset to receive light without the light passing through another in-coupling optical element. For example, each in-coupling optical element 700, 710, 720 may be configured to receive light from different image input devices 360, 370, 380, 390, and 400 as shown in FIG. 6 and may be separated (e.g., laterally spaced) from the other in-coupling optical elements 700, 710, 720 so as to not receive substantially any light from others of the in-coupling optical elements 700, 710, 720.

[0180] Each waveguide also includes an associated optically dispersive element, for example, optically dispersive element 730 is disposed on a major surface (e.g., a top major surface) of waveguide 670, optically dispersive element 740 is disposed on a major surface (e.g., a top major surface) of waveguide 680, and optically dispersive element 750 is disposed on a major surface (e.g., a top major surface) of waveguide 690. In some other embodiments, optically dispersive elements 730, 740, 750 may be disposed on the bottom major surfaces of associated waveguides 670, 680, 690, respectively. In some other embodiments, optically dispersive elements 730, 740, 750 may be disposed on both the top and bottom major surfaces of associated waveguides 670, 680, 690, respectively, or optically dispersive elements 730, 740, 750 may be disposed on different ones of the top and bottom major surfaces in different associated waveguides 670, 680, 690, respectively.

[0181] The waveguides 670, 680, 690 may be spaced apart and separated, for example, by gas, liquid, and / or solid layers of material. For example, as shown, layer 760a may separate the waveguides 670 and 680, and layer 760b may separate the waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed from a low index material (i.e., a material having a lower index of refraction than the material forming the immediately adjacent ones of the waveguides 670, 680, 690). Preferably, the index of refraction of the material forming layers 760a, 760b is 0.05 or greater, or 0.10 or less, relative to the index of refraction of the material forming the waveguides 670, 680, 690. Advantageously, the lower refractive index layers 760a, 760b may act as cladding layers to promote total internal reflection (TIR) ​​of light through the waveguides 670, 680, 690 (e.g., TIR between the top and bottom major surfaces of each waveguide). In some embodiments, the layers 760a, 760b are formed from air. Although not shown, it should be understood that the top and bottom of the illustrated set of waveguides 660 may include immediate cladding layers.

[0182] Preferably, for ease of manufacturing and other considerations, the materials forming the waveguides 670, 680, 690 are similar or the same, and the materials forming the layers 760a, 760b are similar or the same. In some embodiments, the materials forming the waveguides 670, 680, 690 may vary between one or more of the waveguides, and / or the materials forming the layers 760a, 760b may differ while still maintaining the various refractive index relationships described above.

[0183] 9A, light rays 770, 780, 790 are incident on the set of waveguides 660. It should be understood that light rays 770, 780, 790 may be injected into the waveguides 670, 680, 690 by one or more image injection devices 360, 370, 380, 390, 400 (FIG. 6).

[0184] In some embodiments, the light beams 770, 780, 790 have different properties, e.g., different wavelengths or different wavelength ranges, that may correspond to different colors. Each of the incoupling optical elements 700, 710, 720 deflects the incident light such that the light propagates through a respective one of the waveguides 670, 680, 690 by TIR. In some embodiments, each of the incoupling optical elements 700, 710, 720 selectively deflects one or more particular wavelengths of light while transmitting other wavelengths to the underlying waveguide and associated incoupling optical element.

[0185] For example, incoupling optical element 700 may be configured to deflect light beam 770 having a first wavelength or wavelength range while transmitting light beams 780 and 790 having different second and third wavelengths or wavelength ranges, respectively. The transmitted light beam 780 impinges on and is deflected by incoupling optical element 710, which is configured to deflect light of the second wavelength or wavelength range. Light beam 790 is deflected by incoupling optical element 720, which is configured to selectively deflect light of the third wavelength or wavelength range.

[0186] 9A, the deflected light rays 770, 780, 790 are deflected to propagate through the corresponding waveguides 670, 680, 690. That is, the incoupling optical element 700, 710, 720 of each waveguide deflects the light into its corresponding waveguide 670, 680, 690 and incouplings the light into its corresponding waveguide. The light rays 770, 780, 790 are deflected at an angle that causes the light to propagate through the respective waveguides 670, 680, 690 by TIR. The light rays 770, 780, 790 propagate through the respective waveguides 670, 680, 690 by TIR until they strike the corresponding optical dispersive element 730, 740, 750 of the waveguide.

[0187]

[0046] Referring now to Figure 9B, a perspective view of the multiple stacked waveguide embodiment of Figure 9A is illustrated. As described above, the in-coupled light rays 770, 780, 790 are deflected by the in-coupling optical elements 700, 710, 720, respectively, and then propagate by TIR within the waveguides 670, 680, 690, respectively. The light rays 770, 780, 790 then impinge on the optically dispersive elements 730, 740, 750, respectively. The optically dispersive elements 730, 740, 750 deflect the light rays 770, 780, 790 to propagate towards the out-coupling optical elements 800, 810, 820, respectively.

[0188] In some embodiments, the optically dispersive elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPEs deflect or disperse the light to the out-coupling optical elements 800, 810, 820, and in some embodiments may also increase the beam or spot size of this light as it propagates to the out-coupling optical elements. In some embodiments, the optically dispersive elements 730, 740, 750 may be omitted and the in-coupling optical elements 700, 710, 720 may be configured to deflect the light directly to the out-coupling optical elements 800, 810, 820. For example, referring to FIG. 9A, the optically dispersive elements 730, 740, 750 may be replaced with the out-coupling optical elements 800, 810, 820, respectively. In some embodiments, the outcoupling optical elements 800, 810, 820 are exit pupils (EPs) or exit pupil expanders (EPEs) that direct light to the viewer's eye 210 (FIG. 7). It should be understood that the OPEs may be configured to increase the size of the eyebox in at least one axis, and the EPEs may increase the eyebox in an axis that intersects, e.g., is perpendicular to, the axis of the OPE. For example, each OPE may be configured to redirect a portion of the light striking the OPE to the EPE of the same waveguide, while allowing the remaining portion of the light to continue propagating down the waveguide. In response to striking the OPE again, another portion of the remaining light is redirected to the EPE, and the remaining portion of that portion continues to propagate further down the waveguide, and so on. Similarly, in response to striking an EPE, a portion of the impinging light is directed out of the waveguide towards the user, and the remaining portion of that light continues to propagate through the waveguide until it again strikes an EP, at which point another portion of the impinging light is directed out of the waveguide, etc. As a result, a single beam of internally coupled light may be "replicated" each time a portion of that light is redirected by an OPE or EPE, thereby forming a cloned beam field of light, as shown in Figure 6. In some embodiments, the OPE and / or EPE may be configured to modify the size of the beam of light.

[0189] Thus, referring to Figures 9A and 9B, in some embodiments, a set of waveguides 660 includes, for each primary color, a waveguide 670, 680, 690, an in-coupling optical element 700, 710, 720, an optical dispersion element (e.g., OPE) 730, 740, 750, and an out-coupling optical element (e.g., EP) 800, 810, 820. The waveguides 670, 680, 690 may be stacked with an air gap / cladding layer between each one. The in-coupling optical element 700, 710, 720 redirects or deflects the incoming light into its waveguide (with different in-coupling optical elements receiving different wavelengths of light). The light then propagates at an angle that will result in TIR within the individual waveguides 670, 680, 690. In the illustrated embodiment, light ray 770 (e.g., blue light) is deflected by the first in-coupling optical element 700 in the manner described above, then continues to bounce back down the waveguide, interacting with the optically dispersive element (e.g., OPE) 730 and then the out-coupling optical element (e.g., EP) 800. Light rays 780 and 790 (e.g., green and red light, respectively) pass through the waveguide 670, where light ray 780 impinges on and is deflected by the in-coupling optical element 710. Light ray 780 will then proceed via TIR back down the waveguide 680, to its optically dispersive element (e.g., OPE) 740 and then the out-coupling optical element (e.g., EP) 810. Finally, light ray 790 (e.g., red light) passes through the waveguide 690 and impinges on the optically in-coupling optical element 720 of the waveguide 690. The light in-coupling optical element 720 deflects the light beam 790 such that it propagates by TIR to a light dispersive element (e.g., OPE) 750 and then by TIR to an out-coupling optical element (e.g., EP) 820. The out-coupling optical element 820 then finally out-couples the light beam 790 to a viewer, who also receives the out-coupled light from the other waveguides 670, 680.

[0190] FIG. 9C illustrates a top-down plan view of an example of the multiple stacked waveguides of FIGS. 9A and 9B. As shown, the waveguides 670, 680, 690 may be vertically aligned with each waveguide's associated light dispersive element 730, 740, 750 and associated out-coupling optical elements 800, 810, 820. However, as discussed herein, the in-coupling optical elements 700, 710, 720 are not vertically aligned. Rather, the in-coupling optical elements are preferably non-overlapping (e.g., laterally spaced apart, as seen in the top-down view). As discussed further herein, this non-overlapping spatial arrangement facilitates the injection of light from different sources into different waveguides on a one-to-one basis, thereby allowing a specific light source to be uniquely coupled to a specific waveguide. In some embodiments, an arrangement including non-overlapping spatially separated in-coupling optical elements may be referred to as a shifted pupil system, and the in-coupling optical elements in these arrangements may correspond to sub-pupils.

[0191] 9D illustrates an example of a wearable display system 60 into which the various waveguide and associated systems disclosed herein may be integrated. In some embodiments, the display system 60 is the system 250 of FIG. 6, which diagrammatically illustrates some portions of the system 60 in greater detail. For example, the waveguide assembly 260 of FIG. 6 may be part of the display 70.

[0192] 9D , the display system 60 includes a display 70 and various mechanical and electronic modules and systems to support the functionality of the display 70. The display 70 may be coupled to a frame 80, which is wearable by a display system user or viewer 90 and configured to position the display 70 in front of the eye of the user 90. The display 70 may be considered an eyepiece in some embodiments. In some embodiments, a speaker 100 is coupled to the frame 80 and configured to be positioned adjacent to the ear canal of the user 90 (in some embodiments, another speaker, not shown, may also be optionally positioned adjacent the user's other ear canal to provide stereo / shapeable sound control). The display system 60 may also include one or more microphones 110 or other devices to detect sound. In some embodiments, the microphones may be configured to allow the user to provide input or commands to the system 60 (e.g., voice menu command selections, natural language queries, etc.) and / or enable audio communication with other persons (e.g., other users of a similar display system). The microphone may further be configured as an ambient sensor and collect audio data (e.g., sounds from the user and / or the environment). In some embodiments, the display system 60 may further include one or more outwardly directed environmental sensors 112 configured to detect objects, stimuli, people, animals, places, or other aspects of the world around the user. For example, the environmental sensors 112 may include one or more cameras, which may be positioned, for example, facing outward, to capture images similar to at least a portion of the user 90's normal field of view. In some embodiments, the display system may also include an ambient sensor 120a, which may be separate from the frame 80 and mounted on the body of the user 90 (e.g., the head, torso, limbs, etc. of the user 90). The ambient sensor 120a may, in some embodiments, be configured to obtain data characterizing a physiological state of the user 90. For example, the sensor 120a may be an electrode.

[0193] 9D, the display 70 is operably coupled to the local data processing module 140 by a communication link 130, such as wired or wireless connectivity, which may be mounted in a variety of configurations, such as fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, embedded within headphones, or otherwise removably attached to the user 90 (e.g., in a backpack configuration, in a belt-coupled configuration). Similarly, the sensor 120a may be operably coupled to the local data processing module 140 by a communication link 120b, such as wired or wireless connectivity. The local processing and data module 140 may comprise a hardware processor and digital memory, such as non-volatile memory (e.g., flash memory or hard disk drive), both of which may be utilized to aid in processing, caching, and storing data. Optionally, the local processing and data module 140 may include one or more central processing units (CPUs), graphics processing units (GPUs), dedicated processing hardware, and the like. The data may include a) data captured from sensors (such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, gyroscopes, and / or other sensors disclosed herein (e.g., which may be operatively coupled to the frame 80 or otherwise attached to the user 90)) and / or b) data acquired and / or processed using the remote processing module 150 and / or remote data repository 160 (including data related to virtual content), possibly for passing to the display 70 after processing or retrieval. The local processing and data module 140 may be operatively coupled to the remote processing module 150 and the remote data repository 160 by communication links 170, 180, such as via wired or wireless communication links, such that these remote modules 150, 160 are operatively coupled to each other and available as resources to the local processing and data module 140.In some embodiments, local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyroscope. In some other embodiments, one or more of these sensors may be mounted on frame 80 or may be a stand-alone structure that communicates with local processing and data module 140 by a wired or wireless communication path.

[0194] 9D , in some embodiments, remote processing module 150 may comprise one or more processors configured to analyze and process data and / or image information, including, for example, one or more central processing units (CPUs), graphics processing units (GPUs), dedicated processing hardware, etc. In some embodiments, remote data repository 160 may comprise digital data storage facilities that may be available through the Internet or other networking configurations in a "cloud" resource configuration. In some embodiments, remote data repository 160 may include one or more remote servers, which provide information, for example, information for generating augmented reality content to local processing and data module 140 and / or remote processing module 150. In some embodiments, all data is stored and all computations are performed within the local processing and data module, allowing for fully autonomous use from the remote modules. Optionally, an external system (e.g., one or more processors, one or more computer systems), including a CPU, GPU, etc., may perform at least a portion of the processing (e.g., generating image information, processing data) and provide information to and receive information from modules 140, 150, 160, e.g., via a wireless or wired connection.

[0195] Clipping Virtual Objects As discussed herein, in some embodiments, a display system may be considered capable of displaying a virtual object with accommodation and vergence cues that are inconsistent and exceed a comfortable level. Although the embodiments herein are not limited to any particular cause of inconsistency, FIGS. 10-11B are provided as examples of how some inconsistencies may occur. Additionally, it should be understood that in some embodiments, the distance at which an unacceptable accommodation-vergence mismatch is understood to occur may vary depending on the viewer's gaze direction. FIGS. 10-11B provide an example of a misalignment that results from a viewer directing their gaze straight ahead.

[0196] In some cases, accommodation-vergence-divergence mismatch may occur because the range of cues a display system may provide for one of accommodation and vergence exceeds the range of cues for the other of vergence and accommodation. For example, a display system may have a relatively small number of possible accommodation cues and a larger number of possible vergence cues. FIG. 10 illustrates an example of an accommodation-vergence-divergence mismatch plot for a single depth plane display system. Such a display system outputs light with a wavefront divergence corresponding to a single depth plane, and thus may be understood to provide a single accommodation cue. The dotted lines show the accommodation-vergence-divergence mismatch at different (virtual) distances from the viewer. The horizontal axis corresponds to the distance from the viewer, and the vertical axis corresponds to the accommodation-vergence-divergence mismatch, also referred to herein as AVM. In the illustrated embodiment, the single depth plane is located at 2m, and virtual objects at that distance have zero AVM. At distances greater than 2m (e.g., z=2m to z=optical infinity), the accommodation-vergence-divergence mismatch remains below 0.5 diopters. At closer distances, the mismatch increases, and at distances less than 1m from the viewer, the mismatch may exceed what is considered physiologically tolerable without discomfort. As a result, at distances less than 1m, viewer discomfort may be expected simply from viewing image content at that distance. Other examples of single depth plane display systems are disclosed in US2018 / 0039083 and US2017 / 0276948, the entire disclosures of which are incorporated herein by reference.

[0197] It should be appreciated that utilizing a system capable of providing content on a greater number of depth planes can increase the number of possible accommodation cues, thereby reducing the range of distances at which viewer discomfort may be expected. FIG. 11A illustrates an example of a plot of accommodation-vergence-divergence mismatch for a two-depth plane display system and a one-depth display system. The solid line shows the two-depth plane system, and the dotted line shows the single-depth plane system. It should be appreciated that the accommodation-vergence-divergence mismatch at large distances from the viewer is less for the two-depth plane system, allowing the two-depth plane system to display content at closer distances to the viewer while still maintaining an acceptable mismatch value. In some embodiments, the farthest depth plane may be within the acceptable mismatch value of optical infinity. As discussed herein, the acceptable misalignment may be about 0.75 diopters or less, about 0.66 diopters or less, about 0.5 diopters or less, about 0.33 diopters or less, or about 0.25 diopters or less. As illustrated, the acceptable misalignment may be 0.33 diopters. In some embodiments, the farthest depth plane may be set to 0.33 diopters (corresponding to 3 m from the user) and the closer second depth plane may be set inward from the farthest depth plane by a value equal to twice the acceptable misalignment, for example, 0.33 diopters x 2, i.e., 0.66 diopters. As a result, the closer, second depth plane may be set to 1 diopter in some embodiments.

[0198] Continuing with reference to FIG. 11A, proceeding along the solid plot from right to left (with decreasing distance from the user's eyes), the accommodation-vergence-divergence mismatch increases until a mismatch value of 0.33 diopters is observed. As a result, given the illustrated placement of the farthest depth plane at 0.33 diopters, image content for virtual objects from a distance of 1.2 m to infinity may be displayed on the same depth plane (set at 0.33 diopters or 3 m from the user's eyes) without significant discomfort, as all image content within this range is within the acceptable mismatch. For virtual objects at distances closer than 1.2 m, a second depth plane may be provided, as shown. As previously mentioned, the second depth plane may be at 1 diopter.

[0199] It should be appreciated that the distance represented by the 0.33 diopter acceptable misalignment is smaller the closer the depth plane is placed to the user. In some other embodiments, it may be desirable to display the virtual object closer than the minimum distance provided by the 0.33 diopter separation from the 1 diopter depth plane. As shown, at distances of 0.75 m or less, the acceptable misalignment value increases above 0.33 diopters. As a result, one or more additional depth planes may be provided to display image content at closer distances. For example, a third depth plane (not shown) may be formed to display image content at a distance closer than 0.75 m. It should be appreciated that each depth plane is separated from the nearest depth plane by twice the acceptable misalignment or less. It is therefore possible that undesirably large accommodation-vergence-divergence motion misalignment (AVM) may be caused by setting the depth planes at different distances, changing the range of distances. For example, in some embodiments, an AVM that exceeds a threshold may be close to the viewer when the nearest depth plane (or single depth plane) is sufficiently far away from the viewer, while in certain other embodiments, an AVM that exceeds a threshold may occur at a distance far from the viewer, e.g., approaching optical infinity, when the farthest depth plane (or single depth plane) is sufficiently close to the viewer.

[0200] 11B, another example of an accommodation-vergence-divergence mismatch plot for a two depth plane display system is illustrated. In this example, the innermost or closest depth plane to the viewer is set at 0.5m (2 diopters), the outermost or farthest depth plane from the viewer is set at 1.5 meters (0.66 diopters), and the AVM threshold is 0.66 diopters. In some embodiments, the AVM threshold at distances closer than the innermost depth plane is set to a higher value than the AVM threshold at distances farther than the innermost depth plane. For example, the AVM threshold at distances closer than the innermost depth plane may be 0.75 diopters such that up to a distance of 0.37m or closer from the viewer, the observed AVM does not exceed the threshold for discomfort.

[0201] 11C illustrates another representation of the two-depth plane configuration of FIG. 11B, with the innermost depth plane set at 0.5 m from the viewer and the outermost depth plane set at 1.5 m from the viewer. As illustrated, assuming an AVM resistance of 0.66 diopters, the outermost depth plane is 2×0.66 diopters (1.32 diopters) from optical infinity and the innermost depth plane is 2×0.66 diopters (1.32 diopters) from the outermost depth plane. At a distance closer to the viewer than the innermost depth plane, the AVM resistance may be 0.75 diopters, which corresponds to a distance of 0.37 m from the viewer. Thus, in some embodiments, the AVM discomfort zone may be understood to encompass a volume formed by a boundary that intersects with a point at 0.37 m from the viewer. In addition, the viewer may have an LoF zone with a boundary that intersects with a point at 0.1 m from the viewer. More generally, it should be appreciated that in some embodiments, the boundary of the AVM discomfort zone may be set based on the AVM tolerance from the innermost depth plane.

[0202] Referring again to FIG. 11B, going from right to left, the accommodation-vergence-divergence mismatch increases until a mismatch value of 0.75 diopters is observed. As a result, given the illustrated placement of the farthest depth plane at 0.33 diopters, image content for virtual objects at a distance of 1.2 m to infinity may be displayed on the same depth plane (set at 0.33 diopters or 3 m from the user's eyes) without significant discomfort, since all image content within this range is within the acceptable mismatch. For virtual objects at distances closer than 1.2 m, a second depth plane may be provided, as shown. As noted above, the second depth plane may be at 1 diopter.

[0203] As is evident from FIGS. 10-11B, the potential for viewer discomfort may increase with decreasing distance to the viewer or increasing distance from the viewer depending on where the nearest or furthest depth plane provided by the display system is placed. In some embodiments, the display system may have a finite number of depth planes, with the nearest depth plane to the viewer set at a particular distance from the viewer. The display system may provide specific accommodation cues for the virtual content at this depth plane. Because vergence-divergence cues may be provided by varying the image provided to each eye, it may be possible for the display system to provide multiple vergence-divergence cues to the viewer at distances closer than the nearest depth plane. As a result, the display system may provide a single accommodation cue at a near distance, but multiple vergence-divergence cues toward or away from the viewer that increasingly diverge vergence and accommodation distances with increasing distance from the depth plane. That is, AVM may increase as content is closer to the viewer and may increase as content is further away from the viewer (from the depth plane). At some distance closer to the viewer, the difference between convergence-divergence and accommodation distance exceeds a threshold for viewer comfort, i.e., AVM threshold. In addition, at some distance away from the viewer, depending on where the farthest depth plane is, the difference between convergence-divergence and accommodation distance may also exceed the AVM threshold for viewer comfort. Thus, it should be understood that the AVM threshold may be expressed as a distance from the viewer, e.g., as a distance from the viewer's eyes. As the viewer rotates their head and / or eyes, these distances may be mapped to define a surface, which defines a volume around the viewer. For example, the distance within a continuous volume where the AVM exceeds a threshold for viewing comfort may define an AVM discomfort zone. In addition, as discussed herein, a viewer may also be unable to fuse images of objects viewed by their left and right eyes at close distances, or at least be unable to fuse images of objects viewed by their left and right eyes at close distances without experiencing undue eye strain.The volume defined by these close distances to the viewer may be referred to as the loss of fusion zone (LoF zone).

[0204] An example of a representation of AVM discomfort zones and LoF zones is shown in FIG. 12A. FIG. 12A illustrates a top-bottom schematic of accommodation-vergence-divergence mismatch (AVM) discomfort zones and loss of fusion (LoF) zones relative to a viewer. It should be understood that content within these zones close to the viewer may cause discomfort due to AVM exceeding a comfortable level, as described above with respect to FIGS. 10-11C, but such content may nevertheless be desirable for presentation to the viewer for other reasons. For example, such content may be present in a scene to preserve the naturalness of the scene. Simply clipping the content may reduce the viewer's sense of immersion and realism. For example, content may include virtual objects moving near the viewer's head (e.g., a virtual butterfly flying near the viewer's head, a leaf falling near the viewer, etc.), virtual "mud" or "dirt" on the virtual goggles, virtual "cracks" in the eyepieces or virtual goggles, a virtual firearm or other object associated with the user's arm / hand and that the user may bring close to his / her face, virtual skis on the user's shoulder, etc. However, all or part of this virtual content may be presented at close distances to the viewer, including distances at which the AVM creates undesirable viewer discomfort. In some embodiments, the display of content determined to create unacceptable accommodation-vergence-divergence movement mismatch (i.e., content within the AVM discomfort zone being converged by the viewer's eyes) is modified to protect against such viewer discomfort.

[0205] It should be understood that the determination of the AVM, as discussed herein, relies on a determination of a mismatch between accommodation and vergence-divergence distance. The mismatch occurs where the viewer is verging and accommodating to the virtual object. As a result, the mismatch is not expected to occur with respect to virtual objects that the viewer is not verging, since there may typically be no vergence or accommodation distance associated with such objects. In other words, preferably, any modification of the display of the virtual object occurs only with respect to those objects that are both within the AVM discomfort zone and are verged by the viewer's eyes. Without being limited by theory, it is believed that even if the content is within the AVM discomfort zone, the viewer will not experience discomfort unless the viewer is verging to content within the AVM discomfort zone. In some embodiments, it is assumed that if the viewer is not verging to a point within the AVM discomfort zone, the AVM threshold for discomfort is not exceeded and no modification of the content due to accommodation-vergence mismatch is required. Additionally or alternatively, in some embodiments, if a viewer is not engrossed by particular content, even if it is within the AVM discomfort zone, it is assumed that the AVM threshold for discomfort is not applicable and modification of that content is unnecessary.

[0206] In addition, there may be certain types of content for which modification is not implemented even if it is within the AVM discomfort zone and the viewer is congested to it. For example, "occlusion content" is preferably not modified even if it otherwise meets the criteria for modification. It should be understood that occlusion content includes content that occludes or blocks other virtual content or real objects. Examples of such content include virtual walls or other obstacles. Preventing the display of such content may undesirably reduce the viewer's immersion in the scene and / or undesirably allow the viewer to see other content or real objects, which may provide an unfair advantage in the game. In some embodiments, such content is not modified even if it may cause viewing discomfort. For example, content is not modified even if it is analyzed to determine that it causes viewer discomfort. More preferably, content is determined to be "occlusion content" and is simply not analyzed for modification as disclosed herein.

[0207] Continuing with reference to FIG. 12A, a viewer 90 is shown wearing a display system frame 80 that supports a display 70. The eyes of the viewer 90 are convergent at a point 15 (fixation point) on the optical z-axis. Point 15 is located at the outer boundary of an AVM discomfort zone 1000. As discussed herein, a virtual object displayed within the AVM discomfort zone 1000 has an AVM that exceeds a threshold for viewing comfort. In some embodiments, the AVM within the AVM discomfort zone 1000 is greater than 0.25 diopters. In certain other embodiments, the AVM within the AVM discomfort zone 1000 is greater than 0.33 diopters. In still other embodiments, the AVM within the AVM discomfort zone 1000 is greater than 0.5 diopters, 0.66 diopters, or 0.75 diopters.

[0208] With continued reference to FIG. 12A , the LoF zone 1010 is located within the AVM discomfort zone 1000. In some embodiments, the AVM discomfort zone 1000 has a circular or spherical outer boundary with a radius of about 15 cm from the midpoint between the viewer's eyes 90. The distance from the midpoint to the outer boundary of the AVM discomfort zone 1000 may be referred to as the accommodation-vergence-divergence movement mismatch threshold distance (AVM threshold distance). In some embodiments, the LoF zone 1010 has a circular or spherical outer boundary with a radius of about 10 cm from the same midpoint. The distance from the midpoint to the outer boundary of the LoF zone 1010 may be referred to as the loss of fusion threshold distance (LoF threshold distance). It should be understood that the illustrated shapes of the boundaries of the zones 1000, 1010 are examples and the zones may have outer boundaries that take other shapes. For example, such a boundary may extend outward in a cone shape (e.g., in a manner similar to the visual field or oculomotor field) or may take the shape of at least one theoretical or empirical monocular locus. In such an embodiment, the outer boundary of the AVM discomfort zone 1000 may vary, in which case the AVM threshold distance may also vary depending on the direction in which the viewer's eyes are pointed. Similarly, in such an embodiment, the outer boundary of the LoF zone 1010 may vary, in which case the LoF threshold distance may also vary depending on the direction in which the viewer's eyes are pointed. Furthermore, it should be appreciated that in examples in which the LoF zone 1010 is defined in terms of vergence-divergence angles (i.e., such that the LoF zone 1010 includes points in three-dimensional space associated with relatively wide viewer vergence-divergence angles), the geometry of the outer boundary may vary according to a trigonometric relationship between the viewing distance and the viewer vergence-divergence angles. 12A , the vergence angle associated with point 15 may correspond to the angle formed between (i) a line depicted as extending from point 15 to the right eye of viewer 90 and (ii) a line depicted as extending from point 15 to the left eye of viewer 90. The vergence angle associated with any other point in three-dimensional space may similarly correspond to the angle formed at the intersection of rays extending from the respective point to the left and right eyes of viewer 90.

[0209] 12B illustrates another example of a schematic top-bottom diagram of an accommodation-vergence-divergence mismatch (AVM) discomfort zone and fusion loss zone for a viewer. As discussed herein, the boundaries of one or both of the AVM discomfort zone 1000 and the LoF zone 1010 may generally follow the contour of a theoretical or empirical monocular locus. For example, the boundary 1012 of the LoF zone 1010 may include multiple points 16A, 16B, 16C, etc., projected onto corresponding locations on the retina of each eye 210, 220. In this embodiment, the angle formed by the intersection of rays extending outward from the eyes 210 and 220 at point 16A may be substantially equal to the angle formed by the intersection of rays extending outward from the eyes 210 and 220 at point 16B, which in turn may be substantially equal to the angle formed by the intersection of rays extending outward from the eyes 210 and 220 at point 16C. That is, all points located along the boundary 1012 of the LoF zone 1010 may be associated with substantially the same viewer vergence-divergence angles. Points that are not projected onto corresponding locations on the retinas of each eye 210, 220 may not be fused and may not be considered a single point by the viewer 90. For example, points closer to the viewer 90 than the boundary 1012 (i.e., points associated with vergence-divergence angles greater than those associated with points 16A, 16B, 16C) may not be considered a single point by the viewer 90. However, it should be understood that the human visual system has limited tolerance for points that are not projected onto the exact corresponding location of each retina. That is, there is still a range of points in front of and behind the monocular locus that are properly fused by the human visual system even if they are not projected onto the exact corresponding location of the retinas of the eyes 210, 220. Such a range of points is sometimes also referred to as "Panum's fusional area" or "Panum's area of ​​fusion". As a result, in some circumstances, the boundary 1012 may be closer to the viewer than the monocular locus. For example, the theoretical or empirical monocular locus may be between the boundaries 1012 and 1002.

[0210] As discussed herein, the AVM discomfort zone 1000 may be a volume where the accommodation-vergence-divergence mismatch level of the virtual content exceeds the viewer's accommodation-vergence-divergence mismatch tolerance. Thus, it should be understood that there are portions of the user's visual field where accommodation-vergence-divergence mismatch may not occur due to, for example, the occurrence of monocular vision. FIG. 12C illustrates an example of a top-bottom schematic diagram of an AVM discomfort zone, taking into account the viewer's monocular vision. The general shape and size of the AVM discomfort zone 1000 shown in FIG. 12C may be similar to the AVM discomfort zone illustrated in FIG. 12B, however, the AVM discomfort zone 1000 may be larger than the monocular vision region M. L and M. R does not extend into the monocular vision region M L and M. R corresponds to the peripheral portion of the viewer's field of vision where content is seen by only one eye. For example, the monocular vision region M L may be a portion of a field of view or volume where only the left eye 210 of the viewer 90 is able to see content, while the right eye 220 of the viewer is unable to see content within this portion or volume. R may be a portion of a field of view or volume in which only the right eye 220 of the viewer 90 is able to see the content, while the viewer's left eye 210 is unable to see any content within this portion or volume. Thus, in some embodiments, a display system configured to clip virtual content may include a monocular viewing region M L and M. R For example, the AVM discomfort zone 1000 may be configured not to clip content displayed within the monocular vision region M, which may be considered to be outside the AVM discomfort zone 1000. Similarly, it should be appreciated that content displayed to only one eye will not induce unacceptable accommodation-vergence-divergence mismatch and therefore will not be clipped by the display system. In some embodiments, the AVM discomfort zone 1000 and the monocular vision region M may be configured not to clip content displayed within the AVM discomfort zone 1000, which may be considered to be outside the AVM discomfort zone 1000. L The boundary between the AVM discomfort zone 1000 and the monocular vision region M may be oriented 50-60 degrees left of the center of the viewer's 90 field of view (FOV),R The boundary between may be oriented 50-60 degrees to the right of the center of the viewer's 90 FOV.

[0211] With continued reference to FIG. 12C , the AVM discomfort zone 1000 is a volume M N This volume M N It should be understood that the content in may be seen by both eyes 210, 220, but is too close to those eyes to be physically able to rotate sufficiently to converge on that content, even with conscious effort by the viewer 90. Thus, because there is no vergence-divergence movement on that content, the display system will experience unacceptable accommodation-vergence-divergence mismatch in the volume M N The method may be configured to assume that a given threshold will not occur within a certain time period.

[0212] As discussed herein, the location of the AVM discomfort zone depends on the AVM threshold distance (which may vary for different viewers) and the location of the depth plane provided by the display system. As a result, in some embodiments, if the farthest depth plane is sufficiently close to the viewer, the display system may provide a wavefront divergence corresponding to an object close to the viewer, but may not provide an amount of wavefront divergence corresponding to an object far from the viewer, e.g., an object at or close to optical infinity. Thus, in some embodiments, the AVM discomfort zone may encompass optical infinity. Additionally, in some embodiments, if the nearest depth plane is sufficiently far from the viewer, another AVM discomfort zone may be presented close to the viewer.

[0213] 12D illustrates an example of an above-and-below schematic of an AVM discomfort zone that encompasses optical infinity. In this example, the farthest depth plane provided by the display system is at a distance less than the boundary 1022 of the far AVM discomfort zone 1020. However, this depth plane is close enough to the viewer 90 that it exceeds the AVM threshold at the distance of the boundary 1022. As a result, content provided at a distance above the boundary 1022 is within the far AVM discomfort zone 1020. In some embodiments, if the nearest depth plane is close enough to the viewer 90 that the AVM threshold is not exceeded for the distance between the depth plane and the boundary 1012 of the LoF zone 1010, the display system may include only the far AVM discomfort zone 1020 and the LoF zone 1010.

[0214] In certain other embodiments, the display system may include both a far AVM discomfort zone 1020 and a near AVM discomfort zone 1000. This may occur, for example, when the nearest depth plane provided by the display system is at a distance far enough from the viewer 90 that the virtual content exceeds the AVM threshold before reaching the LoF zone 1010. The distance exceeding the AVM threshold near the viewer 90 defines a boundary 1002 of the near AVM discomfort zone 1000. In some embodiments, the volume between the boundaries 1002 and 1022 may be understood as an AVM comfort zone 1030 that does not exceed the AVM threshold. In certain other embodiments, if the AVM discomfort zone 1000 does not exist, the AVM comfort zone 1030 may be understood to extend between the boundary 1012 of the LoF zone 1010 and the boundary 1022 of the AVM discomfort zone 1020. In some implementations, the boundary 1022 may correspond to an AVM threshold that is different from the AVM threshold to which the boundary 1002 corresponds. In at least some such implementations, the AVM threshold to which boundary 1022 corresponds may also differ from the AVM value to which boundary 1012 corresponds.

[0215] It should be appreciated that in some embodiments, a display system may be configured to provide virtual content on a single depth plane. Depending on the distance of the single depth plane from the viewer, such a display system may include one or both of a near AVM discomfort zone 1000 and a far AVM discomfort zone 1020.

[0216] Continuing to refer to FIG. 12D, monocular zone M L and M. R It should be understood that the volume of the surrounding world visible to the viewer 90, excluding the AVM threshold, may be considered the viewer's binocular field of view (FOV) (i.e., visible / perceptible to each of the viewer's eyes). The binocular field of view may be understood to be divisible into an AVM comfort zone 1030 and an AVM discomfort zone 1000 and / or 1020. Although the viewer may not be able to combine the images seen by each eye at a distance corresponding to the LoF zone 1010 without experiencing any discomfort, the binocular FOV may also be understood to encompass the LoF zone 1010, since objects located within the LoF zone 1010 are still readily visible / perceptible to each of the viewer's eyes. As is evident from FIGS. 12A-12D, the volume around the viewer may be broadly divided into a zone above the AVM threshold and a zone below the AVM threshold. That is, the volume around the viewer may be divided into an AVM discomfort zone and an AVM comfort zone (also simply referred to as a comfort zone). FIG. 12E illustrates an example of a top-down schematic of AVM discomfort and comfort zones. As shown, the AVM discomfort zone is indicated by a diagonal line extending upward from left to right. The AVM discomfort zone may be considered to include zones 1000, 1010, and 1020. In addition, the comfort zone, indicated by a diagonal line extending upward from right to left as shown, may be considered to include the volume outside of zones 1000, 1010, and 1020. In some embodiments, the comfort zone may be considered to include the volume outside of zones 1000 and 1020 and monocular vision region M. L and M. Rand the volume between the comfort zone and the AVM discomfort zone. In other words, a virtual object within the comfort zone does not exceed the AVM threshold, while a virtual object within the AVM discomfort zone does exceed the AVM threshold. Thus, in some embodiments, the display system may be configured to determine whether the virtual object is within the comfort zone and avoid clipping if the virtual object is within the comfort zone. In some embodiments, determining whether the virtual object exceeds the AVM threshold or is within the AVM discomfort zone may include determining whether the virtual object is within the comfort zone. If the virtual object is within the zone comfort, the display system may be configured to interpret the result as meaning that the AVM threshold is not exceeded or the virtual object is not within the AVM discomfort zone.

[0217] In some embodiments, the display system may be configured to clip the virtual object based on the location of the virtual object relative to the AVM discomfort zones 1000, 1020 and / or the LoF zone 1010. FIG. 13A illustrates a flowchart of an example process for determining whether to clip the display of a virtual object. Any of the processes of FIG. 13A or other processes disclosed herein may be performed by a display system, which may have a computer storage medium that stores software containing instructions for performing various actions of the process. For example, the display system may be display system 60 (FIG. 9D), which may include processing hardware and software stored on a computer storage medium, and may optionally provide information for processing to one or more computer remote systems, e.g., offload processing to a local portion of the system and receive information from the remote systems. In some embodiments, the display system may be a virtual or augmented reality display system having one or more processors (e.g., within the local processing and data module 140 and / or the remote processing module 150) with computer storage media (e.g., within the local processing and data module 140, the remote processing module 150, and / or the remote data repository 160) for storing software with instructions for performing processes.

[0218] Continuing with reference to FIG. 13A, the display system may display or be prepared to display one or more virtual objects at one or more locations known to the display system. In block 2030, the display system may determine whether the user is fixating on a point that is within the AVM discomfort zone. If the fixation point is not within the AVM discomfort zone, it may be assumed that the AVM tolerance is not exceeded and no action is taken to clip the virtual object. However, if the fixation point is within the AVM discomfort zone, the display system may analyze the particular virtual object or virtual content in block 2020 to determine whether the virtual content is within the AVM discomfort zone. If the virtual content is not within the AVM discomfort zone, no clipping of the virtual content is performed. If the virtual content is within the AVM discomfort zone, in some embodiments, the display system may perform a predetermined action (e.g., a clipping action) in block 2022. The predetermined action is designed to address possible viewer discomfort caused by virtual content that is within the AVM discomfort zone. It should be understood that FIG. 13A provides a general overview of some embodiments. With respect to this figure and other figures discussed herein, additional details regarding the various blocks of this figure may be found in the descriptions of similarly numbered blocks in the other figures described herein.

[0219] 13B, the display system may also consider whether the user is fixating within the LoF zone or whether there is virtual content within the LoF zone. In block 230, the display system determines whether the user is fixating a point within the AVM discomfort zone. If not, no further action (e.g., clipping action) may be taken. If so, the process proceeds to block 2070. In block 2070, the display system determines whether the user is fixating within the LoF zone. If not, the process proceeds to block 2020, where the display system determines whether the virtual content is within the AVM discomfort zone. If not, no clipping action is taken. If so, a predetermined action is taken in block 2050 to address the virtual content that is within the AVM discomfort zone.

[0220] Returning to block 2070, if the user is fixating within the LoF zone, the process proceeds to block 2020. In block 2020, the display system determines whether the virtual content displayed or to be displayed is within the LoF zone. If so, the process proceeds to block 2080, where the display system refrains from displaying the virtual object. The process may then begin again. As discussed herein, it should be understood that the various blocks disclosed herein may be implemented in a different sequence than that shown and discussed in connection with the illustrations. For example, the order of blocks 2020 and 2030 (in various figures herein, including FIGS. 13A and 13B) may be reversed. As another example, the order of blocks 2070 and 2040 may also be reversed.

[0221] 13C, additional details regarding the process are provided, according to some embodiments. In some embodiments, each one of the virtual objects is analyzed to determine whether to clip the virtual object or modify its display. In block 2010, the displayed (or to be displayed) virtual objects in the scene are selected for analysis. In some embodiments, the selection may be based on certain criteria. For example, only virtual objects within the user's field of view may be selected in some embodiments. In some other embodiments, the selected virtual objects may be virtual objects that are proximate to the convergence-divergence movement point of the viewer's eyes.

[0222] In block 2020, the display system determines whether the virtual object is located within an AVM discomfort zone (e.g., within a distance less than an AVM threshold distance from the viewer, where the AVM threshold distance is determined based on the direction in which the viewer's eyes are fixating). The AVM discomfort zone size and shape may be determined based on the display system's capabilities to provide different convergence-divergence and accommodation cues. In some embodiments, the size and shape of the AVM discomfort zone may be preset values ​​based on the above capabilities of the display system and the expected AVM tolerance of the viewer, e.g., as discussed with respect to FIGS. 10-11C. In some other embodiments, the size and shape of the AVM discomfort zone may be set for an individual user based on the viewer's AVM tolerance in addition to the display system's capabilities.

[0223] If the virtual object is not located within the AVM discomfort zone (e.g., less than or greater than the relevant AVM threshold distance for the user depending on whether the AVM discomfort zone is a near AVM discomfort zone or a far AVM discomfort zone, respectively), the process proceeds to block 2090 and the virtual object is displayed or continues to be displayed. If the virtual object is located within the AVM discomfort zone (e.g., less than or greater than the relevant AVM threshold distance), the process proceeds to block 2030. In block 2030, the display system determines whether the user is convergent to a fixation point within the AVM discomfort zone (e.g., the vergence-divergence distance is less than or greater than the relevant AVM threshold distance). For example, the display system determines the user's vergence-divergence point of the eye and the distance of this vergence-divergence point from the user. This distance may be referred to as the vergence-divergence distance. If the vergence-divergence movement distance is not less than the AVM threshold distance (if a near AVM discomfort zone 1000, FIGS. 12A-12D exists) or if the vergence-divergence movement distance is not greater than the AVM threshold distance (if a far AVM discomfort zone 1020, FIG. 12D exists), the process proceeds to block 2090 and the virtual object is displayed or continues to be displayed. On the other hand, if the vergence-divergence movement distance is less than the AVM threshold distance (if a near AVM discomfort zone 1000, FIGS. 12A-12D exists) or if the vergence-divergence movement distance is greater than the AVM threshold distance (if a far AVM discomfort zone 1020, FIG. 12D exists), the process continues to block 2040. In some embodiments, if the convergence-divergence movement distance is less than the AVM threshold distance (if a near AVM discomfort zone 1000, FIG. 12A-12D exists) or if the convergence-divergence movement distance is greater than the AVM threshold distance (if a far AVM discomfort zone 1020, FIG. 12D exists), the process proceeds directly from block 2030 to block 2050.Examples of methods for determining vergence distance may be found in U.S. Patent Application No. 62 / 618,559, entitled "DEPTH PLANE SELECTION AND RENDER CAMERA POSITIONING IN AUGMENTED REALITY DISPLAY SYSTEMS," filed January 17, 2018 (Attorney Docket No. MLEAP.175PR), the entire disclosure of which is incorporated herein by reference. Additional examples of methods for determining vergence distance may be found in U.S. Patent Application No. 15 / 469,369, filed March 24, 2017, and published on September 20, 2017 as U.S. Patent Publication No. 2017 / 0276948, the entire disclosure of which is incorporated herein by reference.

[0224] In some embodiments, the AVM discomfort zone may be defined in terms of vergence angles. In some embodiments, different vergence angles may be associated with different distances from the viewer such that boundaries of the AVM discomfort zone may be associated with particular vergence angles. For example, an inner boundary of the AVM discomfort zone (closest to the viewer) may be associated with a first discomfort vergence angle, and an outer boundary of the AVM discomfort zone (farthest from the viewer) may be associated with a second discomfort vergence angle. Thus, the display system may be configured to determine whether the viewer is merging into the AVM discomfort zone by determining whether the vergence angle of the viewer's eyes is within a range defined by the first and second discomfort vergence angles. It will be appreciated that these convergence-divergence angles are also associated with distance from the viewer, and as discussed herein, determining whether the viewer is convergent within an AVM discomfort zone may include determining whether the viewer's convergence-divergence distance is within a distance defined by the distance of the boundary of the AVM discomfort zone.

[0225] In addition, outside the AVM discomfort zone may be an AVM comfort zone, where the accommodation-vergence-divergence mismatch is within a comfortable tolerance. This AVM comfort zone may also be defined by a vergence-divergence angle, with a maximum AVM comfort angle defining an inner boundary (closest to the viewer) and a minimum AVM comfort angle defining an outer boundary (farthest from the viewer), between which the AVM comfort zone lies. Thus, in some embodiments, the display system may be configured to determine whether a viewer is verging within the AVM comfort zone (and therefore not verging within the AVM discomfort zone) by determining whether the viewer's ocular vergence-divergence angle is within a predetermined range of the AVM comfort angle. It will be appreciated that, like the AVM discomfort zone, these comfortable convergence-divergence angles are associated with a distance from the viewer, and in some embodiments, determining whether the viewer is convergent within the AVM comfort zone may include determining whether the viewer's convergence-divergence distance is within a distance defined by the distance of the boundary of the AVM comfort zone.

[0226] Continuing with reference to FIG. 13, in block 2040, the display system determines whether the virtual object is within a LoF zone (e.g., a distance that is less than the LoF threshold distance from the user). In some embodiments, the size and shape of the LoF zone may be a set value that has been determined to be generally applicable to the viewer. In some embodiments, the size and shape of the LoF zone may be calibrated for a particular viewer, for example, by displaying test images and determining the virtual distance at which fusion loss occurs, for example, by imaging the user's eyes or by receiving input from the user that overlap of objects is observed. Thus, in these embodiments, the LoF threshold distance for a given gaze direction of a viewer may be considered as the near point of convergence (NPC) (i.e., the nearest point of binocular fused single vision) determined for the particular viewer. It follows that for at least some embodiments in which the LoF threshold distance is a set value, the LoF threshold distance may represent an average NPC value or an NPC value determined for one or more populations. Although the LoF zone (e.g., LoF zone 1010) is described herein primarily as encompassing portions of three-dimensional space that are less than a threshold distance from the user (LoF threshold distance), it should be understood that the LoF zone may also represent portions of three-dimensional space where the user convergence angle (i.e., convergence angle) is above a threshold LoF convergence angle. Thus, in some examples, the display system (at block 2040) determines whether the virtual object is at a location in three-dimensional space relative to the user such that if the user were to fixate / converge on the virtual object, the user convergence angle would be higher than the threshold convergence angle. Further, in these examples, the threshold convergence angle may be a set value or may be calibrated for a particular viewer, similar to that described above such that the LoF threshold distance is either a set value or calibrated for a particular viewer.

[0227] If the virtual object is within the LoF zone at a distance that is less than the LoF threshold distance (or at a location that meets the threshold vergence-divergence angle in the manner described above), the process proceeds to block 2070. On the other hand, if the virtual object is outside the LoF zone (e.g., at a distance that is not less than the LoF threshold distance), the process proceeds to block 2050. In block 2050, a predetermined action is taken to address the virtual object at a location where the AVM exceeds a comfortable level for the viewer. As noted above, in some embodiments, the process may proceed directly from block 2030 to block 2050 in response to a determination that the vergence-divergence distance is within an AVM discomfort zone (e.g., less than or greater than the applicable AVM threshold distance from the near and far AVM discomfort zones, respectively). Possible predefined actions for block 2050 include, for example, immediately ceasing to display the virtual object to the viewer, fading out the display of the virtual object to the viewer, moving the virtual object out of the AVM discomfort zone, or otherwise limiting the viewer's exposure to the virtual object within the AVM discomfort zone. As described in further detail below with reference to FIG. 16 (specifically with reference to block 4060 of FIG. 16), in some embodiments, the possible predefined actions for block 2050 may also include sending a message to an application to invoke execution of one or more additional predefined actions. In some implementations, the possible predefined actions for block 2050 may include performing one or more of the operations associated with setting, resetting, evaluating, and / or responding to a timer, as described below with reference to FIGS. 15 and 16. For situations in which the virtual object has not yet been presented, the possible predefined actions for block 2050 may further include, for example, refraining from proceeding with the display of the virtual object. In some embodiments, the exposure is limited by modifying the content displayed to one eye without modifying the content displayed to the other eye.For example, an object may be displayed to one eye but not the other eye, or the display may alternate the display of virtual objects to each eye. Preferably, the display of objects to each eye is alternated at a rate sufficient to allow the viewer to "recover" from the discomfort caused by the display of virtual objects to both eyes. In some embodiments, the alterations occur at a rate perceptible to the viewer. For example, the display and cessation of display of objects may occur at a rate below the flicker fusion threshold of the human visual system. In some embodiments, the rate of alterations in each eye is less than 60 times / second, including less than about 30 times / second.

[0228] In certain other embodiments, the content is displayed to only one eye. For example, rather than alternating the display of the content between the viewer's two eyes, the display of the content to one of the eyes is simply stopped for a "cooling" period, e.g., as discussed further herein. The cooling period may be long enough to allow the viewer to "recover" from any discomfort that may be caused by the content.

[0229] In some embodiments, the predetermined action may include timing the duration over which the viewer is sustainedly exposed to the virtual object in the AVM discomfort zone. In some embodiments, the permitted exposure duration may be, for example, 1 second or less, 10 seconds or less, 30 seconds or less, 3 minutes or less, and / or a variable user or system selected amount of time. After the permitted exposure duration has elapsed, exposure to the virtual object in the AVM discomfort zone may be stopped or limited. As noted above, stopping or limiting this exposure may include simply ceasing to display the virtual object, fading the virtual object, moving the virtual object out of the AVM discomfort zone, modifying the display of the virtual object to one or both eyes, etc.

[0230] In some embodiments, a predefined action may be implemented consistent with an identification of the virtual object and / or the application causing the virtual object to be displayed. For example, in a scene in which the virtual object is a flying entity, moving the virtual object out of the AVM discomfort zone may include showing the virtual object flying away from or towards the user to a distance outside the near or far AVM discomfort zone, respectively. As a result, the predefined action may vary depending on the application causing the virtual object to be displayed and may be determined by the application itself, with the application also dictating the particular action to be taken. In some embodiments, as further described herein, if the predefined action provided by the application is not determined to be sufficient to address the viewer's exposure to the virtual object within the AVM discomfort zone, the display system may override the application and take another predefined action to sufficiently limit the viewer's exposure.

[0231] In some embodiments, after ceasing display of a virtual object or after a virtual object has been moved out of the AVM discomfort zone and after a "cooling off" period has elapsed, the virtual object may be redisplayed, allowed to be moved back into the AVM discomfort zone, or simply deemed eligible for display. In some embodiments, the cooling period may be 3 seconds or more, 5 seconds or more, or 10 seconds or more. In some embodiments, the cooling period may be variable and may be a function of exposure duration. For example, for content with a longer allowed exposure duration, the cooling period may be longer than content with a correspondingly shorter exposure duration. For example, in some embodiments, the cooling period for in-world content may be shorter than the cooling period for user-locked content.

[0232] After the predetermined action is performed at block 2050, the process may proceed to block 2060. At block 2060, a next virtual object is selected for clipping analysis, and the process then proceeds again to block 2020. It should be understood that the next virtual object to be analyzed may be a different virtual object than the one most recently analyzed. However, in some cases, the next virtual object may be the same virtual object. For example, the next virtual object may still be filled by the previously analyzed virtual object, may be selected based on criteria, or it is possible that there may be only one virtual object being displayed.

[0233] Referring again to block 2040 of FIG. 13C, if it is determined that the virtual object is within the LoF zone (e.g., at a distance that is less than the LoF threshold distance from the user), the process proceeds to block 2070. In block 2070, the display system determines whether the user is fixating on a point within the LoF zone (e.g., whether the user has a vergence-divergence distance that is less than the LoF threshold distance). It should be understood that the user vergence-divergence distance value utilized for block 2070 may be the same user vergence-divergence distance value utilized for block 2030, or the display system may make a new determination of the user vergence-divergence distance. For an embodiment in which the LoF zone represents a portion of three-dimensional space where the user vergence angle (i.e., the convergence angle) exceeds a threshold vergence-divergence angle (at block 2070), it follows that the display system determines whether the user vergence-divergence angle exceeds a threshold LoF vergence-divergence angle.

[0234] If the user is fixating on a point outside the LoF zone (e.g., if the vergence-divergence distance is not less than the LoF threshold distance or if the user vergence-divergence angle does not exceed the threshold LoF vergence-divergence angle), the process proceeds to block 2090 and the virtual object is displayed or continues to be displayed. If the user's fixation point is within the LoF zone (e.g., if the vergence-divergence distance is less than the LoF threshold distance) (or if the user vergence-divergence angle exceeds the threshold LoF vergence-divergence angle), the process proceeds to block 2080 and the display system refrains from displaying the virtual object. The determination at block 2070 that the user vergence-divergence distance is, in fact, less than the LoF threshold distance effectively indicates that both the virtual object and the user vergence-divergence correspond to a location that is located within the LoF zone (e.g., LoF zone 1010 of FIG. 12A described above). Refraining from displaying the virtual object may include ceasing or preventing the display of the virtual object to both eyes entirely, or intermittently (e.g., one eye at a time). For example, the display system may stop displaying the virtual object in one eye but continue to display the object to the other eye, or the display may alternate between stopping and displaying the virtual object to each eye. In some embodiments, the stopping and starting of the display of the object (e.g., flickering) to each eye occurs at a rate below a flicker fusion threshold (e.g., the rate of stopping and starting the display of the object may be 60 times / second or less). In some implementations, block 2080 may include performing one or more of the operations associated with setting, resetting, evaluating, and / or responding to a timer, as described below with reference to FIGS. 15 and 16. In such implementations, the system may perform the operation of refraining from displaying the virtual object only in response to one or more timers expiring. After block 2080 or block 2090, the process then proceeds to block 2060, where a further selection of virtual objects is analyzed for clipping, as discussed herein.In some embodiments, in response to a determination at block 2040 that the virtual object is not less than the LoF threshold distance from the user, the process may proceed to block 2070 or another block similar thereto before proceeding directly to block 2050.

[0235] With continued reference to FIG. 13C, it should be understood that the order of the various blocks 2020, 2030, 2040, 2070 may be varied from that shown. For example, the order in which blocks 2020, 2030 are performed may be reversed. Additionally, the order in which blocks 2040, 2070 are performed may also be reversed. Other sequences of steps may also be performed according to alternative embodiments. Furthermore, each block illustrated in FIG. 13 may include multiple sub-steps, which may be performed in various sequences depending on the needs of the individual steps. Furthermore, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives. For example, in some embodiments, a process may be divided into two or more independently executable processes. In some implementations, the system may perform a first process including blocks 2010, 2020, 2030, 2050, 2060, and 2090 (e.g., the process may proceed directly from block 2030 to either block 2050 or block 2090) and a second process including blocks 2010, 2040, 2050, 2060, 2070, 2080, and 2090 (e.g., the process may proceed directly from block 2010 to block 2040 and may proceed directly from block 2040 to either block 2070 or block 2050) independently of each other. In this manner, the system may perform AVM and LoF-based evaluation in parallel. In some embodiments, the system may perform only one of the first and second processes. Further, as will be described in further detail below with reference to FIG. 16, splitting and / or extending one or more of the processes described herein (e.g., the processes of FIGS. 13A-13C, 15, 16, etc.) in this manner may enable a process to be performed at a different layer within a software architecture.

[0236] Also, in some embodiments, as an alternative to or in addition to determining whether the user's fixation point (e.g., the user's convergence-divergence distance) is less than the AVM threshold distance, block 2030 may include a determination as to whether the user is convergent to the virtual object. If the user is not convergent to the virtual object, the process proceeds to block 2090, and the virtual object is displayed. If the user is convergent to the virtual object, the process proceeds to block 2040. In some examples, the process may include a determination as to whether the user is convergent to a point in three-dimensional space that is located less than a threshold distance away from the virtual object. In these examples, the system may determine a Euclidean distance from the user's convergence-divergence point to the virtual object (or a portion thereof), compare the determined Euclidean distance to a threshold, and determine whether to proceed to block 2040 or block 2090. Additionally, if one of the user's eyes is occluded, the display system may be programmed to not register the vergence-divergence distance such that the display system provides a result in which the vergence-divergence distance is not less than the AVM threshold distance or the vergence-divergence distance is not less than the LoF threshold distance in blocks 2030 and 2070. In some implementations, the system may actively determine whether one (or both) of the user's eyes are occluded and may transition to block 2090 in response to a determination that one (or both) of the user's eyes are, in fact, occluded.

[0237] For ease of explanation, the display system is discussed herein as performing various actions or making various decisions. It should be understood that performing these various actions or making these various decisions may include performing calculations using one or more processors of the display system and / or utilizing external computing resources in performing the various actions or making the various decisions.

[0238] Without being limited by theory, it is believed that viewers may be more sensitive to accommodation-vergence-divergence mismatch for user-locked content than for in-world content. Furthermore, after an extended exposure period, some users may find certain portions of user-locked content disorienting and / or uncomfortable. As a result, the AVM threshold distance for in-world content may be greater than the corresponding AVM threshold distance for user-locked content. These differences may be expressed as a difference in the size of the AVM discomfort zone for the user-locked content and the in-world content. Additionally, as disclosed herein, if the content is occlusion content, the content may be treated as not having an AVM discomfort zone in some embodiments.

[0239] Referring now to FIG. 14, an example of a top-bottom schematic of multiple accommodation-vergence-divergence mismatch discomfort zones and loss of fusion zones for a viewer is illustrated. FIG. 14 is similar to FIG. 12A, but two near AVM discomfort zones are illustrated. AVM discomfort zone 1010a is the AVM discomfort zone for user-locked content, and AVM discomfort zone 1010b is the AVM discomfort zone for in-world content. AVM discomfort zone 1010b completely overlaps AVM discomfort zone 1010a, which extends beyond AVM discomfort zone 1010b due to the viewer's expected greater tolerance to accommodation-vergence-divergence mismatch for in-world content. In some embodiments, different loss of fusion (LoF) zones may also be provided for different content types (e.g., user-locked content, in-world content, etc.). 14 and 15 (e.g., relating to AVM discomfort zones 1010a and 1010b, etc.) may be applied not only in the context of enforcing AVM-related rules, but also in the context of enforcing LoF-related rules. It should be understood that the far AVM discomfort zone 1020 (FIG. 12D) may similarly be bifurcated into two AVM discomfort zones, with the far in-world AVM discomfort zone for in-world content encompassing the entirety of the far AVM discomfort zone 1020 and the far user-locked AVM discomfort zone for user-locked content encompassing less than the entirety of the far AVM discomfort zone 1020.

[0240] In some embodiments, a viewer's greater tolerance to accommodation-vergence-divergence mismatch for in-world content compared to user-locked content may manifest as a longer allowed exposure duration to in-world content located within the AVM discomfort zone. As a result, in some embodiments, a longer allowed exposure duration may be added to the larger AVM discomfort zone for in-world content, or the AVM discomfort zones for both types of content may be assumed to be similar, with only the allowed exposure duration varying between the two types of content. As discussed herein, examples of allowed exposure durations include 1 second or less, 10 seconds or less, 30 seconds or less, 3 minutes or less, and / or a variable user or system selected amount of time. These values ​​may apply to both in-world and user-locked content, although the allowed exposure duration for in-world content is preferably longer than the allowed exposure duration for user-locked content.

[0241] Due to differences in viewer sensitivity to different types of content, clipping analysis may also take into account content type. FIG. 15 illustrates a flowchart of an example process for determining whether to clip the display of a virtual object based on whether the virtual object is user-locked or in-world content. The process may be performed by a display system, which may have a computer storage medium storing software containing instructions for performing various actions of the process. For example, the display system may be display system 60 (FIG. 9D), which may include processing hardware and software stored on a computer storage medium, and may optionally provide information for the process to one or more computer external systems, e.g., offload processing to the external system and receive information from the external system. In some embodiments, the display system may be a virtual reality display system having one or more processors with a computer storage medium for storing software with instructions for performing the process of FIG. 15.

[0242] At block 2010, a virtual object from the scene is selected for clipping analysis, as discussed herein. At block 3010, the display system determines whether the virtual object is user-locked content. This determination may be made as a function of whether the virtual object meets criteria for user-locked content. Examples of such criteria include the virtual object being locked within the viewer's field of view such that the viewer is unable to avoid the content by moving their head or looking away. Other examples of criteria that may be used in addition to or as an alternative to the above criteria include determining whether the virtual object is an advertisement, a user interface object, or other type of object that is presented exclusively as user-locked content. In some implementations, the virtual object may already be defined as either "in-world" or "user-locked" content. In such implementations, at block 3010, the display system may simply access and analyze information maintained in association with the selected virtual object regarding the content type.

[0243] If it is determined that the virtual object is user-locked content, the process proceeds to block 3020. If the virtual object is not user-locked content, the process proceeds to block 3080, which will be discussed after the discussion of block 3020 and subsequent blocks.

[0244] 15, in block 3020, the display system determines whether the virtual object is within an AVM discomfort zone (e.g., whether the distance between the virtual object and the user is less than or greater than the AVM threshold distance for user-locked content for near and far AVM discomfort zones, respectively). Block 3020 is similar to block 2020, and the discussion regarding block 2020 applies fully to block 3020, except that the AVM discomfort zone shape and size (e.g., threshold distance) used for the determination is the AVM discomfort zone shape and size (e.g., threshold distance) for user-locked content (e.g., as used in the determination regarding whether the virtual object is located within AVM discomfort zone 1010a for user-locked content of FIG. 14 described above). If the distance between the virtual object and the user is not less than the AVM threshold distance for the user-locked content (if a near AVM discomfort zone 1000, FIG. 12A-12D exists) or if the distance between the virtual object and the user is not greater than the AVM threshold distance for the user-locked content (if a far AVM discomfort zone 1020, FIG. 12D exists), no action is taken to clip the virtual object, the process proceeds to block 2060, a next virtual object is selected for clipping analysis, and the process then proceeds again to block 3010. Although not explicitly shown in FIG. 15, in some implementations, the process proceeds from block 3020 to a block where a timer reset is performed before finally proceeding to block 2060 and / or block 2010. Exemplary timers of interest are described in further detail below with reference to blocks 3040-3060. If the distance between the virtual object and the user is less than the AVM threshold distance for the user-locked content, the process proceeds to block 3030.

[0245] In block 3030, the display system determines whether the user's fixation point is within an AVM discomfort zone for the user-locked content (e.g., whether the user's convergence-divergence movement distance is less than the AVM threshold distance for the user-locked content if a near AVM discomfort zone 1000, FIG. 12A-12D, exists, or whether the distance between the virtual object and the user exceeds the AVM threshold distance for the user-locked content if a far AVM discomfort zone 1020, FIG. 12D, exists). Block 3030 is similar to block 2030, and the discussion regarding block 2030 (FIGS. 13A-13C) applies fully to block 3030, except that the AVM discomfort zone shape and size (e.g., AVM threshold distance) used for the determination is the AVM discomfort zone shape and size (e.g., AVM threshold distance) for the user-locked content. If the fixation point is outside the AVM discomfort zone for the user-locked content (e.g., if the vergence-divergence distance is not less than the AVM threshold distance in the case of a near AVM discomfort zone, or if the vergence-divergence distance is not greater than the AVM threshold distance in the case of a far AVM discomfort zone), no action is taken to clip the virtual object, and the process proceeds to block 2060 to repeat the clipping analysis for the next virtual object. Although not explicitly shown in FIG. 15, in some implementations, the process proceeds from block 3030 to a block where a timer reset is performed before finally proceeding to block 2060 and / or block 2010. If the fixation point is inside the AVM discomfort zone for the user-locked content (e.g., if the vergence-divergence distance is less than the AVM threshold distance in the case of a near AVM discomfort zone, or if the vergence-divergence distance is greater than the AVM threshold distance in the case of a far AVM discomfort zone), the process proceeds to block 3040.

[0246] In block 3040, the display system determines whether a timer has been started to track the user's exposure time to the virtual object. The timer may be set to expire after the allowed exposure duration to the user-locked content has elapsed. It should be understood that this exposure time is the amount of time the user is sustainedly exposed to the virtual object while the vergence-divergence distance is within the AVM discomfort zone (e.g., below the AVM threshold distance for a near AVM discomfort zone or above the AVM threshold distance for a far AVM discomfort zone) and while the virtual object is within the AVM discomfort zone (e.g., within a distance from the user that is below the AVM threshold distance for the user-locked content for a near AVM discomfort zone or above the AVM threshold distance for a far AVM discomfort zone). If the vergence-divergence distance change moves out of the AVM discomfort zone (e.g., goes above the AVM threshold distance for a near AVM discomfort zone or below the AVM threshold distance for a far AVM discomfort zone) or the object moves out of the AVM discomfort zone, the timer will be reset.

[0247] At block 3040, if the timer is not running, the process proceeds to block 3050. At block 3050, a timer is started and set to run for a duration equal to the allowed exposure duration to the user-locked content. As noted above, user-locked content may have a shorter exposure time limit (e.g., 1 second or less, 2 seconds or less, etc.) than in-world content (e.g., 10 seconds or less). Because a given allowed exposure duration may be about 1 second or more, it is possible that within this time frame, virtual objects may move or the user's convergence-divergence motion may change. As a result, while the timer is running, the process may return to block 3020 and loop from block 3020 to block 3060 indefinitely until the timer expires. If the timer is running at block 3040, the process proceeds to block 3060.

[0248] In block 3060, the display system may be configured to determine whether a timer has expired, at which point the allowed exposure duration has elapsed. If the timer has not expired, the process returns to block 3020. If the timer has expired, the process proceeds to block 3070.

[0249] At block 3070, the display system performs a predefined action for the user-locked content represented by the virtual object. The predefined action may be similar to that discussed with respect to block 2050 (FIG. 13). For example, the predefined action may include ceasing display of the virtual object, moving the virtual object out of the AVM discomfort zone for the user-locked content, flickering or alternating display of the virtual object to one or both eyes, or redisplaying the content after a cooling period has passed. After the predefined action is performed, the process proceeds to block 2060. Although not explicitly shown in FIG. 15, in some implementations, the process proceeds from block 3070 to a block where a timer reset is performed before finally proceeding to block 2060 and / or block 2010. At block 2060, the next virtual object from the scene is selected for clipping analysis, and the process repeats.

[0250] Returning now to block 3010, if the display system determines that the virtual object is not user-locked content, the process proceeds to block 3080. In some embodiments, the display system proceeds to block 3080 in response to determining that the virtual object is in-world content. Such a determination as to whether the virtual object is in-world content may be made instead of or in addition to a determination as to whether the virtual object is user-locked content (e.g., block 3010). In block 3080, the display system may determine whether the virtual object is within an AVM discomfort zone for the in-world content (e.g., whether the distance between the virtual object and the user is less than the AVM threshold distance in the case of a near AVM discomfort zone, or whether the distance between the virtual object and the user is greater than the AVM threshold distance in the case of a far AVM discomfort zone). Block 3080 is similar to block 2020, and the discussion regarding block 2020 applies fully to block 3080, except that the size and shape of the AVM discomfort zone (e.g., AVM threshold distance) used for the determination is the previously described shape (e.g., AVM threshold distance) of the AVM discomfort zone for the in-world content (e.g., the determination regarding whether the virtual object is located within AVM discomfort zone 1010b for the in-world content of FIG. 14 described above). If the virtual object is outside the AVM discomfort zone for the in-world content (e.g., if the distance between the virtual object and the user is not less than the AVM threshold distance if a near AVM discomfort zone exists, or if the distance between the virtual object and the user is not more than the AVM threshold distance if a far AVM discomfort zone exists), no action is taken to clip the virtual object, and the process proceeds to block 2060. As discussed herein, in block 2060, the next virtual object is selected for clipping analysis and the process then proceeds again to block 3010.15, in some embodiments, the process proceeds from block 3080 to a block where a timer reset is performed before eventually proceeding to block 2060 and / or block 2010. Relevant example timers are described in further detail below with reference to blocks 3100-3120. If the virtual object to be displayed is within an AVM discomfort zone (e.g., if the distance between the virtual object and the user is less than the AVM threshold distance for the in-world content if a near AVM discomfort zone exists, or if the distance between the virtual object and the user is greater than the AVM threshold distance if a far AVM discomfort zone exists), the process proceeds to block 3090.

[0251] In some embodiments, before determining whether the virtual object is within an AVM discomfort zone for in-world content in block 3080 (e.g., in the case of a near AVM discomfort zone, whether the distance between the virtual object and the user is less than the AVM threshold distance, or in the case of a far AVM discomfort zone, whether the distance between the virtual object and the user is greater than the AVM threshold distance), the display system may check whether the virtual object is in-world content. For example, the system may determine whether the virtual object meets criteria for in-world content. Examples of criteria include determining whether the virtual object is anchored to the surrounding environment or to the user. For example, the movement of the object may be tracked relative to the user to determine whether it is anchored to the user (and thus does not move relative to the user) or not anchored to the user. If the virtual object is not anchored to the user, it may be determined that it is not user-locked content and is therefore in-world content. In some embodiments, the virtual object may have an identifier assigned to it to identify its content type.

[0252] 15, the illustrated process may next proceed through one or more of blocks 3090, 3100, 3110, 3120, 3130, and 2060. It should be understood that blocks 3090, 3100, 3110, 3120, 3130, and 2060 are similar to blocks 3030, 2040, 3050, 3060, 3070, and 2060, respectively. The discussion herein for blocks 3030, 2040, 3050, 3060, 3070, and 2060 applies in full to blocks 3090, 3100, 3110, 3120, 3130, and 2060, respectively, with the following exceptions: the size and shape of the AVM discomfort zone for block 3090 is the AVM discomfort zone size or shape for in-world content (e.g., AVM threshold distance block 3090 is the AVM threshold distance for in-world content), the predetermined time period (allowed exposure duration) of block 3110 is that predetermined time period or allowed exposure duration for in-world content, and the predetermined action of block 3130 involves including actions defined for in-world content. In some embodiments, the possible predetermined actions of blocks 3070 and 3130 are similar.

[0253] However, for clarity, blocks 3090, 3100, 3110, 3120, 3130, and 2060 are briefly discussed below. In block 3090, the display system determines whether the user's fixation point is within an AVM discomfort zone for in-world content (e.g., whether the vergence-divergence movement distance is less than the AVM threshold distance for the in-world content if a near AVM discomfort zone exists, or whether the vergence-divergence movement distance is greater than the AVM threshold distance for the in-world content if a far AVM discomfort zone exists). If the fixation point is outside the AVM discomfort zone for the in-world content (e.g., if a near AVM discomfort zone exists, the vergence-divergence movement distance is not less than the AVM threshold distance for the in-world content, or if a far AVM discomfort zone exists, the vergence-divergence movement distance is not greater than the AVM threshold distance for the in-world content), no action is taken to clip the virtual object, and the process proceeds to block 2060 (to repeat the clipping analysis for the next virtual object). Although not explicitly shown in FIG. 15, in some embodiments, the process proceeds from block 3090 to a block where a timer reset is performed before ultimately proceeding to block 2060 and / or block 2010. If the vergence-divergence movement distance is within the AVM discomfort zone for the in-world content (e.g., if a near AVM discomfort zone exists, the vergence-divergence movement distance is less than the AVM threshold distance for the in-world content, or if a far AVM discomfort zone exists, the vergence-divergence movement distance is greater than the AVM threshold distance for the in-world content), the process proceeds to block 3100.

[0254] In block 3100, the display system determines whether a timer is running to track user exposure time to the virtual object. The timer may be set to expire after the allowed exposure duration to the in-world content. If the timer is not running, the process proceeds to block 3110. In block 3110, a timer is started and set to run for a duration equal to the allowed exposure duration to the in-world content. While the timer is running, the process may return to block 3080 and continually loop from block 3080 to block 3120 until the timer expires. If the timer is running in block 3100, the process proceeds to block 3120.

[0255] In block 3120, the display system may be configured to determine whether a timer has expired, i.e., whether the allowed exposure duration has elapsed. If the timer has not expired, the process returns to block 3080. If the timer has expired, the process proceeds to block 3130.

[0256] In block 3130, the display system takes the prescribed predefined action with respect to the in-world content represented by the virtual object. The predefined action may be similar to that discussed with respect to block 2050 (FIG. 13). Examples of such predefined actions include ceasing display of the virtual object, fading the virtual object, moving the virtual object out of the AVM discomfort zone, modifying or alternating the display of the virtual object to one or both eyes, etc.

[0257] After the predetermined action is performed, the process proceeds to block 2060. Although not explicitly shown in FIG. 15, in some examples, the process proceeds from block 3130 to a block where a timer reset is performed before finally proceeding to block 2060 and / or block 2010. In block 2060, the next virtual object from the scene is selected for clipping analysis, and the process repeats. It should be understood that the specific blocks illustrated in FIG. 15 provide a particular method for determining whether and how to modify the presentation of a virtual object based on a content type, according to some embodiments of the present invention. Other sequences of steps may also be performed according to alternative embodiments. Furthermore, each block illustrated in FIG. 15 may include multiple sub-steps that may be performed in various sequences depending on the needs of the individual steps. Furthermore, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0258] Referring now to FIG. 16, a flowchart of an exemplary process 4000 for determining whether to clip the display of a virtual object when a separate application (e.g., a video game, a streaming media player, or other application running on the display system) defines its own clipping function is illustrated. In such a case, the display system may contain software that sends instructions to the application to perform the clipping function. This clipping function may vary depending on the application and may serve to limit user exposure to content provided by the application for output through the display system. However, independent of the particular application and clipping function, the display system may be configured to perform the process 4000. That is, the process 4000 and such clipping function may be performed in different layers of a software architecture associated with the display system or another computing device that communicates with the display system. Further, in some implementations, independent of the particular application and clipping functionality, the display system may be configured to perform one or more LoF-based evaluations, such as one or more of those described above with reference to blocks 2010, 2040, 2060, 2070, 2080, and 2090 of Figure 13. In this manner, LoF-based content restriction rules may be enforced globally (e.g., the display system may perform one or more of the operations described above with reference to block 2060 of Figure 13 to modify the presentation of application-generated content), while AVM-based content restriction rules may be enforced, at least in part, locally at the application (e.g., application-defined functionality may include one or more of the operations described above with reference to block 2050 of Figure 13 to modify the content that the application generates and provides for output through the display system).However, in some embodiments, AVM-based content restriction rules may be enforced, at least in part, globally if the display system determines that an application is unable to properly enforce such rules locally.

[0259] In block 2030, the display system determines whether the user's fixation point is within the AVM discomfort zone (e.g., whether the user's vergence-divergence distance is less than the AVM threshold distance if a near AVM discomfort zone exists, or whether the user's vergence-divergence distance is greater than the AVM threshold distance if a far AVM discomfort zone exists). If the fixation point is outside the AVM discomfort zone (e.g., if the vergence-divergence distance is not less than the AVM threshold distance for a near AVM discomfort zone, or if the vergence-divergence distance is not greater than the AVM threshold distance for a far AVM discomfort zone), no further action is required to clip the virtual object, and the process proceeds to block 4020. It should be appreciated that the display system may include a timer to determine whether the allowed exposure duration has elapsed. Because the allowed exposure duration relates to the time period during which the user is engrossed in a point within the AVM discomfort zone (e.g., engrossed below the AVM threshold distance if a near AVM discomfort zone exists, or above the AVM threshold distance if a far AVM discomfort zone exists), in block 4020, this timer may be reset because the user is not engrossed in content within the AVM threshold distance. The process may then return to block 2030.

[0260] Referring again to block 2030, if the user's fixation point is within the AVM discomfort zone (e.g., if the user's convergence-divergence distance is less than the AVM threshold distance if a near AVM discomfort zone exists, or greater than the AVM threshold distance if a far AVM discomfort zone exists), the process proceeds to block 4030. Optionally, in some embodiments, before proceeding to block 4030, the process may proceed from block 2030 to block 4025, where the display system determines whether the content to be displayed is within the AVM discomfort zone. If not, the process proceeds to block 4020. If the content to be displayed is within the AVM discomfort zone, the process then proceeds to block 4030.

[0261] In block 4030, the display system determines whether a timer is running, which tracks the user's exposure time to the virtual object when the virtual object is within the AVM discomfort zone (e.g., when the virtual object is less than the AVM threshold distance if a near AVM discomfort zone exists, or when the virtual object is greater than the AVM threshold distance if a far AVM discomfort zone exists). If the timer is not running, the process proceeds to block 4040. In block 4040, the timer is started and set to run for the allowed exposure duration for the virtual object. While the timer is running, the process may return to block 2030 and loop from block 2030 to block 4050 indefinitely until the timer expires.

[0262] In block 4050, the display system may be configured to determine whether a timer has expired, i.e., whether the allowed exposure duration has elapsed. If the timer has not expired, the process returns to block 2030. If the timer has expired, the process proceeds to block 4060. In some embodiments, blocks 4020, 4030, 4040, and 4050 may serve to enable the display system to effectively filter slight movements of the user's eyes, for example, during fast movements (e.g., saccades, microsaccades), and update the fixation point in response to determining that the eyes are fixating on a location in three-dimensional space. In this manner, the display system may be configured to fixate a point for less than a threshold duration, ignore eye movements, and / or ignore involuntary eye movements (e.g., blinks), and thus proceed to block 4060 only in response to a determination that the user is fixating a point in three-dimensional space that is located within an AVM discomfort zone (e.g., less than an AVM threshold distance away from the user if a near AVM discomfort zone is present, or greater than an AVM threshold distance if a far AVM discomfort zone is present). In some implementations, some or all of blocks 4020, 4030, 4040, and 4050 may be excluded from process 4000.

[0263] At block 4060, the display system is configured to send a message to an application currently launched on the display system to invoke execution of a clipping function. As discussed herein, examples of possible clipping functions include ceasing to display the virtual object, fading the virtual object, moving the virtual object out of the AVM discomfort zone, and the like. Indeed, as noted above, at least some of the operations associated with block 4060 may represent examples of predefined actions, such as those described above with reference to blocks 2050, 3070, and / or 3130. In some embodiments, the messages described above are communicated to the application through an application programmable interface (API). In some examples, such messages communicated through an API may be of function calls. The functions invoked through the API and subsequently executed by the application may correspond to one or more of the operations described above with reference to FIGS. 13 and 15. In some embodiments, the functions executed by the application may include one or more operations for determining whether any virtual object is within the user's field of view. In this manner, the display system may refrain from displaying only those virtual objects determined to be located within the user's field of view and within an AVM discomfort zone (e.g., less than an AVM threshold distance away from the user if a near AVM discomfort zone exists, or greater than an AVM threshold distance away from the user if a far AVM discomfort zone exists). In certain other embodiments, the functionality performed by the application may include one or more operations to indiscriminately refrain from displaying all virtual objects determined to be located within an AVM discomfort zone (e.g., less than an AVM threshold distance away from the user if a near AVM discomfort zone exists, or greater than an AVM threshold distance away from the user if a far AVM discomfort zone exists). In some implementations, the message may include time-stamped information indicating the user's convergence-divergence distance.In at least some of these implementations, the application may further leverage such information to identify virtual objects that are to be displayed and virtual objects that are to not be displayed. As noted above, in some embodiments, the display system may monitor the activity of the application to determine whether the application has responded satisfactorily to the issued message to invoke execution of the clipping function, and may take additional action if the application's response to the issued message is determined to be insufficient. That is, one or more operations for clipping or otherwise modifying the content may be performed globally, or, following block 4060, at a hierarchically higher layer within the software architecture of the display system (i.e., to bypass the application) if the application-defined functionality fails to properly clip or otherwise modify the content.

[0264] It should be understood that the specific blocks illustrated in FIG. 16 provide a particular method for determining whether and how to modify the presentation of a virtual object when a separate application defines its own clipping function according to some embodiments of the present invention. Other sequences of steps may also be performed according to alternative embodiments. Moreover, the individual blocks illustrated in FIG. 16 may include multiple sub-steps that may be performed in various sequences depending on the needs of the individual steps. Furthermore, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0265] In addition, and again with reference to FIGS. 12A-18, it should be understood that the various values ​​utilized to make the comparison or otherwise make the determination may vary. For example, values ​​for the AVM threshold distance and the allowed exposure duration may vary depending on the application or program providing the content or establishing the rules for content modification within the scene. In some embodiments, the display system may be configured to identify different types of user content, each with an associated distance that establishes the size of the AVM discomfort zone and / or the allowed exposure duration, e.g., due to a program or application with its own categorization of the content or its own content parameters. For example, the display system may recognize multiple in-world content types, each with a different identifier and a separate AVM discomfort zone size and / or a separate allowed exposure duration. Additionally or alternatively, the display system may recognize multiple user-locked content types, each with a different identifier and a separate AVM discomfort zone size and / or a separate allowed exposure duration. In some embodiments, there may be only one type each of in-world content and user-locked content, but the AVM discomfort zone size and / or allowed exposure duration may nevertheless vary and have different values ​​set by different applications. Thus, in the various processes and flow charts disclosed herein, the AVM discomfort zone size and shape, AVM threshold distance, and / or time period for exposure to content may vary depending on the virtual object being analyzed and the program or application that defines the content type, associated AVM zone size and shape, and exposure duration. For example, depending on the application, the display system may have different AVM threshold distances and allowed exposure durations stored in memory for different types of content.As a result, values ​​for AVM size shape, AVM threshold distance, and allowed exposure duration for different content types can be dynamically varied and updated based on the virtual objects selected for analysis within the scene.

[0266] In addition, as discussed herein, if the AVM threshold is exceeded, the display system may be configured to present content to only one eye of the viewer (e.g., switch from presenting content to both eyes to presenting content to only one eye, i.e., from binocular presentation of virtual content to monocular presentation of virtual content). For example, again referring to FIGS. 13A-13C and 15-16, performing a predetermined action at blocks 2050, 3070, 3130, 4060 or refraining from displaying a virtual object at block 2080 may include presenting virtual content to only one eye. In some embodiments, as discussed herein, the evaluation of whether to present a virtual object to one eye is performed on a virtual object-by-virtual object basis. For example, when multiple virtual objects are presented to the viewer, each virtual object is evaluated as to whether it would be perceived as a location that exceeds the AVM threshold, and therefore whether to present the virtual object to only one eye. In a situation where one or more virtual objects of a first set in a scene are determined to be perceived to be at a location that exceeds the AVM threshold (e.g., outside the AVM comfort zone), while one or more virtual objects of a second set in the scene are determined to be perceived to be at a location that does not exceed the AVM threshold (e.g., within the AVM comfort zone), it follows that the display system may, in some embodiments, be configured to present one or more objects of the first set to the user monocularly and present one or more objects of the second set to the user binocularly. In the following discussion, monocular content may be used to refer to virtual content or objects that are nominally presented to both eyes and then presented to one eye in response to exceeding the AVM threshold.

[0267] In some embodiments, the perceived location of the virtual object may be tracked or estimated to determine whether the virtual object is within an AVM discomfort zone, and the display system may be configured to determine whether the virtual object crosses a boundary between different zones of the viewer's monocular field of view. For example, the display system may estimate the perceived location of the virtual object at a first moment in time and later at a second moment in time. If the virtual object is initially assumed to be within a comfort zone, and then crosses a boundary between the zones, the display system may interpret this to mean that the virtual object crosses into an AVM discomfort zone.

[0268] In certain other embodiments, to determine whether a virtual object is within the AVM discomfort zone, the display system may be configured to determine whether the virtual object is at a location outside the comfort zone and whether the viewer's vergence-divergence point crosses a distance threshold for the virtual object (e.g., whether the user's vergence-divergence point is close enough to the virtual object such that the viewer may be said to be "seeing" the virtual object). If the virtual object is at a location outside the comfort zone and the viewer's vergence-divergence point crosses a threshold distance for the virtual object, the display system may interpret this to mean that the virtual object is currently located within the AVM discomfort zone and the viewer is viewing the virtual object such that corrective action should be taken by the display system.

[0269] As discussed herein, different content types (e.g., user-locked content or in-world content) may have different associated AVM discomfort zones. As a result, different content types may have different boundaries for their associated AVM discomfort zones. In some embodiments, the display system may be configured to determine the content type of a virtual object and determine the boundaries of the associated AVM discomfort zone. Examples of such different boundaries include AVM discomfort zone boundaries for in-world and user-locked content, as described above with reference to FIGS. 14 and 15. Assuming that the virtual object is initially within a comfort zone, if the virtual object crosses a boundary between the zones, the display system may interpret this to mean that the virtual object crosses into an AVM discomfort zone. In addition, the display system may be configured to determine whether the user's convergence-divergence movement point crosses a distance threshold for the virtual object. If applicable, the display system may interpret this to mean that the viewer is looking at a virtual object within the AVM discomfort zone.

[0270] As a result of the above various schemes for determining whether a virtual object is within an AVM discomfort zone and requires an action, the display system may conclude that a predetermined action should be performed to mitigate the presence of the virtual object within the AVM discomfort zone. It should be understood that the predetermined action may be any of the predetermined actions discussed herein. For example, the display system may be configured to cease presenting the virtual object within one of the viewer's eyes, i.e., the display system may be configured to transition from a binocular presentation mode (wherein the virtual object is presented to both eyes of the viewer) to a monocular presentation mode (wherein the virtual object is presented to one eye of the viewer).

[0271] If a virtual object is presented to both eyes, then ceasing the presentation of the virtual object to an eye so that the virtual object is presented only to the opposing eye may cause a reduction in the perceived brightness of the presented image, and therefore a reduction in the perceived brightness of the virtual object. Without being limited by theory, it is believed that this is due to binocular summation of the image, i.e., the perceived brightness of the image is determined by the human visual system by the summation of the images in the left and right eyes. Ceasing the presentation of the image to an eye effectively removes the image from that eye so that the summation is less, thereby causing a reduction in the perceived brightness.

[0272] In some embodiments, the display system may be configured to mitigate the reduction in perceived brightness by increasing the brightness of an image of a monocular presentation of a virtual object to a viewer. In some embodiments, when presenting a virtual object to one eye, the display system may be configured to increase the brightness of the presented image of the virtual object to a level sufficient such that the viewer perceives the image of the virtual object presented to one eye as having substantially the same brightness as the virtual object presented to both eyes. In some embodiments, the perceived brightness of the virtual object presented to one eye is within 30%, within 20%, or within 10% of the perceived brightness of the virtual object when presented to both eyes.

[0273] In some embodiments, the display system may be configured to default or biased to favor the presentation of monocular content to a particular eye. For example, as a default, monocular content may be presented to the user's dominant eye. In such embodiments, the display system may store or be able to access in memory an indication of the viewer's eye that is dominant. It should be appreciated that this parameter may be a user-selectable parameter input by the user. In certain other embodiments, the display system may determine the user's eye that is dominant based on one or more tests / assessments performed by the system, for example, during a calibration routine and / or performed by the system in real-time during operation of the system. Without being limited by theory, it is believed that the human visual system may in some scenarios prioritize the processing of information from the dominant eye, and that visual information from the non-dominant eye may be suppressed in some instances. Advantageously, it is believed that the presentation of monocular content to the dominant eye may provide the viewer with visual information that is perceived at a higher visual acuity, reducing the suppression of that information. Additional details regarding visual processing, including ocular dominance, inhibition, and binocular rivalry, are described in US2017 / 0365101, the entire disclosure of which is incorporated herein by reference.

[0274] In some embodiments, the display system may be configured to move or otherwise change the visual properties of static monocular content. Without being limited by theory, it is believed that a static virtual object may disappear so as not to be perceived by the viewer, or may appear to disappear and reappear intermittently, even if the display system persistently presents its monocular content. This may be due to binocular rivalry, or other phenomena such as motion-induced blindness (MIB) and motion-induced interocular inhibition, in which the visual system ignores static content in favor of dynamic content presented to the other eye. In some embodiments, the display system may be configured to protect against such disappearance of displayed monocular content by determining whether the monocular content is static. If the monocular content is determined to be static, the display system may be configured to cause the otherwise static virtual content to move (e.g., cause the monocular content to appear to oscillate in a fixed position, adjust the position of one or more virtual light sources in the virtual scene to cause movement in the virtual shadows cast by the monocular content, etc.), change other properties (e.g., cause the monocular content to oscillate in perceived brightness and / or size), or otherwise present monocular stimuli (e.g., time-varying stimuli) to the user. In some embodiments, the display system may be configured to invoke one or more predefined animation sequences within the monocular content to prevent involuntary perceptual suppression of such monocular content by the user. In some embodiments, the display system may be configured to treat user-locked content as static content. In at least some of such implementations, the display system may be configured to "unlock" user-locked content from the user when presenting such content monocularly to the user, such that the position of the content relative to the user's FOV may change as the user moves his or her head.In some examples, the display system may be configured to adjust the behavior of content locked to a user such that such content is only loosely tethered to the user when presenting such content monocularly to the user. Additional details regarding binocular rivalry, motion-induced blindness (MIB), and motion-induced interocular inhibition are described in US 2017 / 0365,101, the entirety of which is incorporated herein by reference.

[0275] When providing monocular content, the display system may make a determination regarding which eye will be presented with the monocular content. In some embodiments, the display system may be configured to present the monocular content to the eye on which the side of the field of view in which the virtual object is represented by the monocular content is located. For example, a virtual object on the right side of the user's field of view may be presented as monocular content to the right eye, while virtual content on the left side of the field of view may be presented as monocular content to the left eye. In some embodiments, the display system may be configured to determine whether the virtual object is on the right or left side of the field of view, and if the virtual object is to be presented as monocular content, present the monocular content to the eye corresponding to the side of the field of view on which the virtual object is located.

[0276] In some embodiments, the display system may be configured to determine the direction of a user's gaze and to determine the eye that will be presented with the monocular content. For example, if the user's gaze is directed to the left side of the field of view, the monocular content may be presented to the left eye, and if the user's gaze is directed to the right side of the field of view, the monocular content may be presented to the right eye.

[0277] In some embodiments, the display system may have different virtual objects for presentation to different sides of the user's field of view, e.g., one virtual object on the left side of the field of view and a second virtual object on the right side. In such an instance, the display system may be configured to determine whether the virtual objects are within an AVM discomfort zone. If they are within the AVM discomfort zone, the display system may be further configured to make a decision as to whether to present both virtual objects monocularly to the same eye. In some embodiments, the display system may present both virtual objects monocularly to the same eye as a default. In certain other embodiments, with respect to the first and second virtual objects to be presented on different sides of the field of view (FOV), the display system may be configured to determine whether one or more of the following conditions are met. For example, the display system may be configured to evaluate the presence of any one or all of the following conditions. If any of these conditions are met, the display system may be configured to present the first and second virtual objects monocularly to the same eye. · The first and second virtual objects are closer to each other in virtual space than a predefined threshold separation value. · The first and second virtual objects are duplicates or different instances of the same virtual object or asset from a library of virtual objects / assets (e.g., two identical or nearly identical cardboard box assets on the floor of the virtual scene), or are at least similar in appearance (e.g., similar in size, shape, and color). Without being limited by theory, it is believed that the human visual system will attempt to blend similar looking virtual objects together as if they were the same object presented to different eyes. Presenting both the first and second virtual objects to the same eye avoids such attempts to blend the objects.

[0278] In some embodiments, the first and second virtual objects will be similar in appearance but presented at different depths. For example, the first virtual object may be within the AVM discomfort zone while the second virtual object may be outside the AVM discomfort zone, or vice versa. Note that the display system may be configured to present the first and second virtual objects monocularly to the same eye to protect against attempts by the human visual system to blend the first and second virtual objects.

[0279] In certain other embodiments, under some circumstances, the display system may be configured to present a first virtual object to its corresponding eye (the eye on the same side of the field of view as the first virtual object) and a second virtual object to its corresponding eye (the eye on the same side of the field of view as the second virtual object). Thus, for two virtual objects to be presented on different sides of the field of view (FOV), the display system may be configured to determine whether one or more of the following conditions are met. The display system may be configured to determine the presence of only one of the following conditions or the presence of more than one of the following conditions. If any of these conditions are met, the display system may be configured to present the first virtual object monocularly to the corresponding one of the user's eyes and the second virtual object monocularly to the corresponding other one of the user's eyes. · Two virtual objects are farther apart than the threshold separation value. One or both of the virtual objects are outside the user's binocular FOV (e.g., at least one of the virtual objects is outside the M of FIG. 12D). L or M R For example, if the first virtual object is within the monocular FOV (M L ) and the second virtual object is positioned within the monocular FOV (M R), the display system may be configured to simultaneously (i) present a first virtual object monocularly to the user's left eye and (ii) present a second virtual object monocularly to the user's right eye.

[0280] It should be appreciated that after switching to monocular presentation of a virtual object, a viewer may accidentally close the eye receiving the monocular content while the monocular content is presented to that eye. Typically, when virtual content is provided to both eyes, closing one eye will not result in loss of the content. However, monocular content provided to one eye may be susceptible to loss if that eye is closed. To avoid loss of this monocular content, the display system may be configured to shift to presenting monocular content to the viewer's other eye in response to detecting that the eye has been or is being closed. It should be appreciated that one or more cameras, such as camera assembly 630 (FIG. 6), may be utilized to image the eye and determine whether the eye is closed, occluded, or open.

[0281] For example, if the system determines that the user has closed or is closing the left eye while presenting content monocularly to the user's left eye (and is not presenting content to the user's right eye), the system may begin presenting content to the user's right eye. In some embodiments, the display system may continue to present content to the left eye, thus presenting content to both eyes simultaneously. However, the presentation of content effectively only reaches the open eye, such that the AVM threshold is not exceeded even when the content is directed to both eyes. In such an embodiment, the display system may eventually stop presenting content to the user's closed eye once it is determined that that eye has been closed for at least the threshold amount of time, after which the system presents content only to the user's other open eye.

[0282] In some embodiments, the display system may present content to one eye and may cease presenting content to that eye (e.g., the user's left eye) once it is determined that the user has closed or is closing that eye. Alternatively, the display system may be configured to transition to presenting monocular content only to the user's other eye (e.g., the right eye).

[0283] It should be understood that in any of the embodiments in which the presentation of monocular content is shifted from a first eye to the other eye in response to closure of the first eye, the display system may immediately revert to presenting content monocularly to the first eye (and stop presenting monocular content to the other eye) in response to the system determining that the user has opened or is opening the first eye.

[0284] The switch from binocular to monocular presentation of the virtual content may be visible to the viewer. To mask this switch, in some embodiments, the display system may be configured to determine whether a switch to monocular presentation is appropriate, as discussed above, and then wait for a mask event to occur before making the switch to monocular presentation. Thus, the switch to monocular presentation may occur during a mask event. Examples of mask events include an eye blink, a saccade, or other eye event that reduces the viewer's ability to perceive the switch to monocular presentation. For example, the display system may determine that a virtual object crosses into an AVM discomfort zone, and then wait for a mask event to occur before making the switch to monocular presentation. Similarly, the display system may determine that a virtual object crosses outside an AVM discomfort zone, and may switch from monocular to binocular presentation of the virtual object in response to the occurrence of a mask event. Additional details regarding mask events are disclosed in US2017 / 0276948, the entire disclosure of which is incorporated herein by reference.

[0285] It should be understood that each of the processes, methods, and algorithms described herein and / or depicted in the figures may be embodied in code modules executed by one or more physical computing systems, hardware computer processors, application-specific circuits, and / or electronic hardware configured to execute specific and particular computer instructions, and thereby be fully or partially automated. For example, a computing system may include a general-purpose computer (e.g., a server) or a special-purpose computer programmed with specific computer instructions, special-purpose circuits, etc. The code modules may be written in a programming language that may be compiled and linked into an executable program, installed in a dynamic link library, or interpreted. In some implementations, certain operations and methods may be performed by circuitry specific to a given function.

[0286] Furthermore, certain implementations of the functionality of the present disclosure may be sufficiently mathematically, computationally, or technically complex that special purpose hardware (utilizing appropriate specialized executable instructions) or one or more physical computing devices may be required to implement the functionality, e.g., due to the amount or complexity of the calculations involved, or to provide results in substantially real-time. For example, a video may contain many frames, each frame may have millions of pixels, and specifically programmed computer hardware may be required to process the video data to provide the desired image processing task or application in a commercially reasonable amount of time.

[0287] The code modules or any type of data may be stored on any type of non-transitory computer readable medium, such as physical computer storage devices, including hard drives, solid state memory, random access memory (RAM), read only memory (ROM), optical disks, volatile or non-volatile storage devices, combinations of the same, and / or the like. In some embodiments, the non-transitory computer readable medium may be part of one or more of the local processing and data module (140), the remote processing module (150), and the remote data repository (160). The methods and modules (or data) may also be transmitted as a data signal generated (e.g., as part of a carrier wave or other analog or digital propagating signal) over a variety of computer readable transmission media, including wireless-based and wired / cable-based media, and may take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The results of the disclosed processes or process steps may be stored persistently or otherwise in any type of non-transitory tangible computer storage device, or communicated via a computer readable transmission medium.

[0288] Any process, block, state, step, or functionality in the flow diagrams described herein and / or depicted in the accompanying figures should be understood as potentially representing a code module, segment, or portion of code that includes one or more executable instructions for implementing a specific function (e.g., logical or arithmetic) or step in the process. Various processes, blocks, states, steps, or functionality may be combined, rearranged, added, deleted, modified, or otherwise altered from the illustrative examples provided herein. In some embodiments, additional or different computing systems or code modules may perform some or all of the functionality described herein. The methods and processes described herein are also not limited to any particular sequence, and the blocks, steps, or states associated therewith may be performed in other sequences as appropriate, e.g., serially, in parallel, or in some other manner. Tasks or events may be added to or removed from the disclosed exemplary embodiments. Furthermore, the separation of various system components in the embodiments described herein is for illustrative purposes and should not be understood as requiring such separation in all embodiments. It should be understood that the program components, methods, and systems described may generally be integrated together in a single computer product or packaged into multiple computer products.

[0289] In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be apparent that various modifications and changes can be made therein without departing from the broader spirit and scope of the invention. The specification and drawings are, therefore, to be regarded in an illustrative rather than a restrictive sense.

[0290] Indeed, it should be understood that the systems and methods of the present disclosure each have several innovative aspects, none of which is solely responsible for or required for the desirable attributes disclosed herein. The various features and processes described above may be used independently of one another or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure.

[0291] Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a combination and may even be initially claimed as such, one or more features from the claimed combination may in some cases be deleted from the combination, and the claimed combination may be directed to a subcombination or variation of the subcombination. No single feature or group of features is necessary or essential to every embodiment.

[0292] In particular, it should be understood that conditional terms used herein, such as "can," "could," "might," "may," "eg," and the like, are generally intended to convey that certain embodiments include certain features, elements, and / or steps while other embodiments do not, unless specifically stated otherwise or understood otherwise within the context as used. Thus, such conditional terms are generally not intended to agree that features, elements, and / or steps are in any way required for one or more embodiments, or that one or more embodiments necessarily include logic for determining whether those features, elements, and / or steps should be included or performed in any particular embodiment, with or without authorial input or prompting. The terms "comprising," "including," "having," and the like, are synonymous and are used inclusively in a non-limiting manner and do not exclude additional elements, features, acts, operations, etc. Also, the term "or" is used in its inclusive sense (and not its exclusive sense), thus, for example, when used to connect a list of elements, the term "or" means one, some, or all of the elements in the list. In addition, the articles "a," "an," and "the," as used in this application and the appended claims, should be interpreted to mean "one or more" or "at least one," unless otherwise specified. Similarly, although operations may be depicted in the figures in a particular order, it should be recognized that such operations need not be performed in the particular order shown, or in sequential order, or that all of the illustrated operations need not be performed to achieve desirable results. Additionally, the figures may diagrammatically depict one or more exemplary processes in the form of a flow chart. However, other operations not depicted may be incorporated within the diagrammatically illustrated exemplary methods and processes.For example, one or more additional operations can be performed before, after, simultaneously with, or between any of the illustrated operations. Additionally, operations may be rearranged or reordered in other implementations. In some situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated together in a single software product or packaged in multiple software products. Additionally, other implementations are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results.

[0293] Thus, the claims are not intended to be limited to the embodiments shown herein but are to be accorded the widest scope consistent with the present disclosure, the principles and novel features disclosed herein.

Claims

1. 1. A head mounted display system for displaying a virtual object, the head mounted display system comprising: one or more processors; one or more computer storage media; Equipped with The one or more computer storage media have instructions stored thereon that, when executed by the one or more processors, determining whether a virtual object is displayed at a location having an accommodation-vergence mismatch above a threshold; determining whether an eye of a user of the head mounted display system is closed; refraining from displaying the virtual object after determining that the user's eyes are not occluded and that the virtual object will be displayed at a location that has the accommodation-vergence mismatch exceeding the threshold; displaying the virtual object after determining that the eye of the user is closed and that the virtual object is to be displayed at a location having the accommodation-vergence mismatch exceeding the threshold; A head mounted display system, causing the one or more processors to perform operations including:

2. 2. The head mounted display system of claim 1, wherein displaying the virtual object after determining that the eye of the user is closed includes displaying the virtual object to the open eye of the user after determining that the other eye of the user is closed.

3. 2. The head mounted display system of claim 1, wherein displaying the virtual object after determining that the eye of the user is closed comprises displaying the virtual object within a fusion loss zone of the user.

4. The head mounted display system of claim 1 comprising an eyepiece configured to output light with a variable amount of wavefront divergence.

5. 5. The head mounted display system of claim 4, wherein the eyepiece comprises a stack of waveguides, at least one of the waveguides being configured to output light with a different amount of wavefront divergence than other waveguides of the waveguide stack.

6. 10. The head mounted display system of claim 1, wherein the operations further include identifying a type of content corresponding to the virtual object and setting the threshold based at least in part on the identified type of content.

7. The head mounted display system of claim 6 , wherein the types of content include user locked content or in-world content.

8. 1. A method for displaying a virtual object by a head mounted display system, the method comprising: determining whether a virtual object is displayed at a location having an accommodation-vergence mismatch above a threshold; determining whether an eye of a user of the head mounted display system is closed; refraining from displaying the virtual object after determining that the user's eyes are not occluded and that the virtual object will be displayed at a location that has the accommodation-vergence mismatch exceeding the threshold; displaying the virtual object after determining that the eye of the user is closed and that the virtual object is to be displayed at a location having the accommodation-vergence mismatch exceeding the threshold; A method comprising:

9. 10. The method of claim 8, wherein displaying the virtual object after determining that the eye of the user is closed comprises displaying the virtual object to the open eye of the user after determining that the other eye of the user is closed.

10. 10. The method of claim 8, wherein after determining that the eye of the user is closed, displaying the virtual object comprises displaying the virtual object within a fusion loss zone of the user.

11. The method of claim 8 , wherein the head mounted display system comprises an eyepiece configured to output light with a variable amount of wavefront divergence.

12. 12. The method of claim 11, wherein the eyepiece comprises a stack of waveguides, at least one waveguide of the stack of waveguides configured to output light with a different amount of wavefront divergence than other waveguides of the stack of waveguides.

13. The method comprises: identifying a type of content corresponding to the virtual object; setting the threshold based at least in part on the identified type of content; The method of claim 11 further comprising:

14. The method of claim 13 , wherein the type of content includes user-locked content or in-world content.

15. 1. A display system for displaying a virtual object within a binocular field of view (FOV) of a user, the display system comprising: one or more processors; one or more computer storage media; Equipped with The one or more computer storage media have instructions stored thereon that, when executed by the one or more processors, determining whether the virtual object crosses a boundary between different zones of a plurality of zones corresponding to different volumes of space within the binocular field of view (FOV), wherein determining whether the virtual object crosses the boundary between the different zones includes determining whether the virtual object crosses the boundary between an accommodation-convergence mismatch discomfort zone and an accommodation-convergence comfort zone; in response to determining that the virtual object crosses the boundary between the different zones, switching between (i) a binocular presentation mode, in which the virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode, in which the virtual object is displayed to only one eye of the user. The display system further comprising:

16. The operation includes: selecting a particular eye of the user from among both eyes of the user; The display system of claim 15 , wherein the virtual object is displayed only to the particular eye of the user in the monocular presentation mode.

17. The operation includes: determining which eye of the user is the dominant eye of the user; The display system of claim 16 , wherein selecting the particular eye of the user includes the dominant eye for displaying the virtual object.

18. The operation includes: determining whether the virtual object is located within a left portion of the user's binocular field of view (FOV) or a right portion of the user's binocular field of view (FOV); 17. The display system of claim 16, wherein selecting the particular eye of the user from among the user's eyes comprises selecting the particular eye of the user from among the user's eyes based at least in part on determining whether the virtual object is located within the left portion of the user's binocular field of view (FOV) or within the right portion of the user's binocular field of view (FOV).

19. (i) switching between the binocular presentation mode, in which the virtual object is displayed to both eyes of the user, and (ii) the monocular presentation mode, in which the virtual object is displayed to only one eye of the user, (i) switching between a binocular presentation mode in which the virtual object is displayed to each eye of the user at a first level of brightness, and (ii) a monocular presentation mode in which the virtual object is displayed to only one eye of the user at a second level of brightness, the second level of brightness being greater than the first level of brightness in which another virtual object is displayed to only one eye of the user.

16. The display system of claim 15, comprising:

20. The operation includes: determining whether another virtual object crosses the boundary between different zones of the user's binocular field of view (FOV); in response to determining that the another virtual object crosses the boundary between different zones of the binocular field of view (FOV) of the user, and independent of determining that the virtual object crosses the boundary between different zones of the binocular field of view (FOV) of the user, switching between (i) a binocular presentation mode in which the another virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode in which the another virtual object is displayed to only one eye of the user.

16. The display system of claim 15 further comprising:

Images