Projection apparatus

Abstract

An optical system for near-eye projection of an image into an eye of a user, the optical system comprising: a projection portion configured to project light encoding the image, and a central concave portion comprising at least one switchable optical element, the at least one switchable optical element being controllable, the central concave portion is configured to project light from the projection portion onto a central region of the retina of the eye and to switch between a central concave mode in which the central concave portion is configured to project light from the projection portion onto a central region of the retina of the eye and a peripheral mode in which the central concave portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region is larger than and includes the central region. A corresponding method and projection apparatus are also provided.

Description

Technical Field

[0001] The exemplary aspects of this paper relate to near-eye projection of an image into a user's eye, and specifically to optical systems, projection devices, and methods. background

[0002] Optical systems used to project light into a user's eyes (or both eyes) are used in extended reality (XR) settings, such as augmented reality, virtual reality, or mixed reality. These optical systems are used to form virtual images that can be combined with light incident from the environment the user is viewing (in the case of augmented or mixed reality), or they can recreate the virtual environment the user is viewing (in the case of virtual reality).

[0003] Typically, the components of an optical system are placed near the user's eyes (i.e., these components are optical systems used for near-eye projection of images), allowing for image projection with a more compact optical system, which helps improve portability and user convenience. For example, the components of these optical systems can be part of a device worn or held by the user, such as a head-mounted display.

[0004] To provide an immersive user experience, the virtual image projected onto the user's eyes should be adapted to the user's current field of vision. In particular, the light projected onto the user's eyes reaches a specific part of the retina, which in turn determines how the user will see the virtual image, so the orientation of the projected light, especially the angle at which the light enters the user's eyes, should be adapted to the user.

[0005] Therefore, there is a need for an optical system and method that enhances the user experience by improving the adaptability of the virtual image projected onto the user's eyes, and in particular, provides a more immersive user experience.

[0006] Specifically, a virtual image can be adapted to the user's field of vision by projecting light so that it converges at a point in the eye that coincides with the eye's center of rotation, thus allowing the image to always be centered on the retina. However, because the angle of incidence of light is limited by the pupil, this results in only a small portion of the retina being projected. This, in turn, limits the size of the virtual image seen by the user.

[0007] Therefore, there is a need for an optical system and method that is adaptable to the user's field of view while allowing light to be projected onto a wider portion of the retina.

[0008] Furthermore, optical systems have limited resources in terms of size, battery life, and computing resources. Therefore, there is a need for an optical system and method that improves the user experience while improving compactness and suppressing any increase in required resources. Overview

[0009] According to a first example aspect, an optical system for projecting an image near the eye into a user's eye is provided. The optical system includes a projection portion and a focusing portion, wherein the projection portion is configured to project light that encodes the image, and the focusing portion is configured to converge the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina. The at least one convergence point is located on or near the pupil of the eye on a convergence arc, wherein the corresponding location of each convergence point on the convergence arc corresponds to a rotation angle of the eye about an axis of rotation of the eye, and the convergence arc has a center of curvature that substantially coincides with the axis of rotation of the eye.

[0010] By converging the projected light to a point on the convergence arc, the projected light can enter at a larger angle of incidence than if the light were to converge to a point near the center of the eye's rotation. This allows for a larger image to be displayed to the user, and the projected light can enter the eye across an angular range that remains centered on the retina, thus avoiding the so-called window effect.

[0011] Therefore, the virtual image projected onto the user's eyes better adapts to the user's field of vision, thereby improving the user experience.

[0012] Preferably, the distance between the center of curvature and the center of eye rotation is 1.5 cm or less, more preferably 1 cm or less, and most preferably 0.5 cm or less.

[0013] Reducing the distance between the center of curvature of the converging arc and the center of eye rotation allows for an increase in the field of view provided by the optical system.

[0014] Preferably, the distance between the arc and the pupil remains essentially constant regardless of the eye's rotation angle.

[0015] Therefore, regardless of the eye's rotation angle (i.e., the pupil's orientation), the range of incident angles of the projected light entering the eye can remain essentially constant, and the image can be projected onto the same area of ​​the retina, allowing the user to see a virtual image of essentially constant size.

[0016] Preferably, the optical system is configured to move at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably substantially coincides with the macula of the retina, and more preferably substantially coincides with one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea minor, and umbo of the retina.

[0017] By moving the convergence point along the convergence arc, light can continue to enter as the eye rotates, while avoiding an increase in the energy required to project light into the eye (relative to the case where light converges to multiple different convergence points corresponding to different rotation angles of the eye).

[0018] Preferably, the optical system further includes a steering portion configured to move at least one convergence point around a convergence arc in response to rotation of the eye about the eye's rotation axis, the arc having a center of curvature substantially coinciding with the eye's rotation axis.

[0019] Preferably, the steering portion is configured to move at least one element of the projection portion and the focusing portion.

[0020] Preferably, the deflection portion includes at least one optical element for deflecting light from the projection portion toward the focusing portion.

[0021] Preferably, at least one optical element includes a steerable reflector.

[0022] Preferably, at least one optical element includes a plurality of switchable lenses, each switchable lens having a corresponding orientation and configured to direct light from the projection portion to different portions of the focusing portion, such that the focusing portion converges the light to different convergence points on the convergence arc.

[0023] Therefore, the position of the converging light can be changed by controlling the state of each switchable lens.

[0024] Preferably, at least one optical element includes an element with controllable phase modulation.

[0025] Preferably, the projection portion is configured to project multiple lights, each light encoding a corresponding portion of the image, and the focusing portion is configured to converge each light to a corresponding convergence point among multiple convergence points, the positioning of each convergence point on the convergence arc corresponding to a corresponding rotation angle of the eye around the eye rotation axis, such that the light converging at adjacent convergence points encodes adjacent portions of the image.

[0026] Therefore, depending on the rotation angle of the eye, light focused on different points of convergence will overlap with the pupil of the eye and enter the eye to form a (virtual) image on the retina.

[0027] By projecting light that converges at multiple points simultaneously (which can be called a static window), optical systems can be simplified, thus becoming more compact and requiring fewer resources, for example by omitting components for moving the convergence points or for tracking the eye's position.

[0028] Preferably, at least one convergence point is located on a convergence surface that is substantially parallel to the pupil of the eye and includes a convergence arc.

[0029] The converging surface can have a center of curvature that is substantially coincident with the center of rotation of the eye. The converging surface can be substantially coincident with the surface of a sphere that approximates the eye, or the converging surface can be a surface that is substantially parallel to the surface of the sphere, with a larger or smaller radius of curvature, a center of curvature that is displaced relative to the center of the sphere, or both, as explained above in the case of the converging arc.

[0030] An optical system can be configured to move at least one convergence point on a convergence surface along two dimensions of the surface, or an optical system can be configured to project light onto a plurality of convergence points on a convergence surface (not necessarily simultaneously), wherein the convergence points are spaced apart (or distributed) from each other along two dimensions of the convergence surface.

[0031] Therefore, even when the eye rotates around the two eye rotation axes (or around the eye rotation center), the projected light can continue to enter the eye.

[0032] Preferably, the projection portion is configured to project a first light encoding a first image onto the user's first eye and a second light encoding a second image onto the user's second eye, and the focusing portion is configured to: converge the first light to at least one first convergence point so that the projected light enters the first eye and forms an image on the retina of the first eye, and converge the second light to at least one second convergence point so that the projected light enters the second eye and forms an image on the retina of the second eye.

[0033] Therefore, optical systems can project stereoscopic images, thus giving users a sense of depth in virtual images.

[0034] Preferably, the projection portion includes a first projection component configured to project a first light and a second projection component configured to project a second light, and / or the focusing portion includes a first focusing component and a second focusing component, the first focusing component being configured to converge the first light to at least one first convergence point, and the second focusing component being configured to converge the second light to at least one second convergence point.

[0035] Therefore, each component of the projection section and / or focusing section can be controlled individually to project an image into the user's eyes.

[0036] Preferably, the projection portion includes a display configured to emit an image to be displayed, and a focusing element for converging light toward the focusing portion, wherein, optionally, the projection portion includes a spatial filter for receiving the converged light.

[0037] Preferably, the projection section includes a coherent light source, preferably a laser, configured to emit substantially coherent light, wherein the projection section optionally includes at least one of the following: a collimating element configured to collimate substantially coherent light, a phase modulator configured to encode an image by modulating substantially coherent light, and a laser beam scanning system including at least one steerable mirror.

[0038] Preferably, the optical system further includes a foveal portion configured to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the foveal portion is configured to project light from the projection portion onto a central region of the retina of the eye, and in the peripheral mode, the foveal portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and including the central region.

[0039] Because the central region of the retina has higher sensitivity, images projected in foveal mode can have higher resolution than those projected in peripheral mode without causing a significant change in image quality perceived by the user.

[0040] Therefore, the resource requirements of the optical system (e.g., for resolution, bandwidth, optical power, and computation) can be reduced, while avoiding a significant impact on the quality of the projected image.

[0041] Preferably, the projection portion of the optical system, including the central concave portion, is configured to project light as substantially collimated light.

[0042] Preferably, the central concave portion is configured to increase the angle of light incident on the eye in the peripheral mode.

[0043] In other words, the angle of light projected onto the eye is larger in peripheral mode than in foveal mode (i.e., in peripheral mode, light enters the pupil from a wider range of angles).

[0044] Preferably, the concave portion is configured to increase the focal length of light incident on the eye in the concave mode.

[0045] Preferably, the central concave portion includes at least one focusing element with a controllable focal length.

[0046] Preferably, the central recess includes at least one switchable optical element, and optionally, each of the at least one switchable element is configured to allow light to pass through when switched to the first state without substantially changing the angular size of the light.

[0047] Preferably, at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length different from the first focal length.

[0048] Preferably, the first switchable element is arranged at a first distance from the display, and the second switchable element is arranged at a second distance from the display.

[0049] Preferably, the difference between the first distance and the second distance is such that the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

[0050] Preferably, in the central recess mode, the first switchable element is configured to switch to the second state and the second switchable element is configured to switch to the first state, while in the peripheral mode, the first switchable element is configured to switch to the first state and the second switchable element is configured to switch to the second state.

[0051] Preferably, at least one switchable optical element includes a first switchable diverging element, a first switchable converging element, and a second switchable converging element, wherein the first switchable diverging element is arranged at a first distance from the projection portion, and the first switchable converging element is arranged at a second distance from the projection portion, the second distance being greater than the first distance.

[0052] Preferably, in the foveal mode, the first switchable diverging element is configured to switch to a first state, the first switchable converging element is configured to switch to a first state, and the second switchable converging element is configured to switch to a second state, such that light from the projection portion is converged by the second switchable converging element to form an image on the central region of the retina; and in the peripheral mode, the first switchable diverging element is configured to switch to a second state, the first switchable converging element is configured to switch to a second state, and the second switchable converging element is configured to switch to a first state, such that light from the projection portion is diverged by the first switchable diverging element onto the first switchable converging element and converged by the first switchable converging element to form an image on the peripheral region of the retina.

[0053] Preferably, the second switchable converging element is arranged at a third distance from the projection portion, such that the convergence point of the first switchable converging element and the convergence point of the second switchable converging element substantially coincide with each other.

[0054] Preferably, at least one switchable optical element further includes a second switchable diverging element, wherein the second switchable diverging element is configured to switch to a second state in the concave mode and increase the angle of light converged by the first switchable converging element, wherein the second switchable converging element is arranged at a third distance from the projection portion, and the second switchable diverging element is arranged at a fourth distance from the projection portion, wherein the first distance, the second distance, the third distance, and the fourth distance are configured such that the convergence point of light in the concave mode and the convergence point of light in the peripheral mode substantially coincide with each other.

[0055] Preferably, the projection portion includes a light field display configured to project light encoding multiple element images into a three-dimensional light field, each element image forming part of an image.

[0056] Preferably, the projection portion includes a phase modulator configured to project light that encodes the holographic image.

[0057] According to a second example aspect, an optical system for projecting an image near the eye into a user's eye is provided. The optical system includes: a projection portion configured to project light encoding the image; and a foveal portion including at least one switchable optical element, the at least one switchable optical element being controllable to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the foveal portion is configured to project light from the projection portion onto a central region of the retina of the eye, and in the peripheral mode, the foveal portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and encompassing the central region.

[0058] Using at least one switchable optical element allows for the differentiation between images projected onto the central region of the retina (e.g., the fovea) with higher sensitivity and images projected onto the peripheral region of the retina, without the need for additional elements.

[0059] This allows higher-resolution images to be projected onto the central area of ​​the retina, ensuring a higher quality image perceived by the user. Conversely, lower-resolution images can be projected onto the peripheral area of ​​the retina without affecting the quality of the image perceived by the user.

[0060] Therefore, the optical system improves the user experience while also enhancing compactness and suppressing any increase in required resources.

[0061] Preferably, the projection portion is configured to project light as substantially collimated light.

[0062] Therefore, the focal length of the light projected into the eye can be determined by the focal length of the switchable optical element.

[0063] Preferably, each of at least one switchable element is configured to allow light to pass through when switched to the first state without substantially changing the angular size of the light.

[0064] Preferably, the central concave portion is configured to increase the angle of light incident on the eye in the peripheral mode.

[0065] Preferably, the concave portion is configured to increase the focal length of light incident on the eye in the concave mode.

[0066] Preferably, the central concave portion includes at least one focusing element with a controllable focal length.

[0067] Preferably, at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length different from the first focal length.

[0068] Preferably, the first switchable element is arranged at a first distance from the display, and the second switchable element is arranged at a second distance from the display.

[0069] Preferably, the difference between the first distance and the second distance is such that the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

[0070] For example, the distance that separates the second switchable element from the first switchable element (i.e., the difference between the second distance and the first distance) can be set based on the difference between the first focal length and the second focal length.

[0071] Therefore, regardless of whether the focal length of the light is changed by the first switchable element or the second switchable element, the light will essentially converge at the same point.

[0072] Preferably, in the central recess mode, the first switchable element is configured to switch to the second state and the second switchable element is configured to switch to the first state, while in the peripheral mode, the first switchable element is configured to switch to the first state and the second switchable element is configured to switch to the second state.

[0073] Preferably, at least one switchable optical element includes a first switchable diverging element, a first switchable converging element, and a second switchable converging element, wherein the first switchable diverging element is arranged at a first distance from the projection portion, and the first switchable converging element is arranged at a second distance from the projection portion, the second distance being greater than the first distance.

[0074] Preferably, in the foveal mode, the first switchable diverging element is configured to switch to a first state, the first switchable converging element is configured to switch to a first state, and the second switchable converging element is configured to switch to a second state, such that light from the projection portion is converged by the second switchable converging element to form an image on the central region of the retina; and in the peripheral mode, the first switchable diverging element is configured to switch to a second state, the first switchable converging element is configured to switch to a second state, and the second switchable converging element is configured to switch to a first state, such that light from the projection portion is diverged by the first switchable diverging element onto the first switchable converging element and converged by the first switchable diverging element to form an image on the peripheral region of the retina.

[0075] Preferably, the second switchable converging element is arranged at a third distance from the projection portion, such that the convergence point of the first switchable converging element and the convergence point of the second switchable converging element substantially coincide with each other.

[0076] For example, the distance that separates the second switchable convergent element from the first switchable convergent element (i.e., the difference between the third distance and the first distance) can be set based on the difference between the focal length of the second switchable convergent element and the focal length of the first switchable convergent element.

[0077] Preferably, at least one switchable optical element further includes a second switchable diverging element, wherein the second switchable diverging element is configured to switch to a second state in the concave mode and increase the angle of light converged by the first switchable converging element, wherein the second switchable converging element is arranged at a third distance from the projection portion, and the second switchable diverging element is arranged at a fourth distance from the projection portion, wherein the first distance, the second distance, the third distance, and the fourth distance are configured such that the convergence point of light in the concave mode and the convergence point of light in the peripheral mode substantially coincide with each other.

[0078] Preferably, the concave portion is configured to converge light to at least one convergence point so that the projected light enters the eye and forms an image on the retina, wherein at least one convergence point is located on or near the pupil of the eye on a convergence arc, wherein the corresponding location of each convergence point on the convergence arc corresponds to the rotation angle of the eye about the eye's rotation axis, and the convergence arc has a center of curvature that substantially coincides with the eye's rotation axis.

[0079] Preferably, the optical system is configured to move at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably substantially coincides with the macula of the retina, and more preferably substantially coincides with one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea minor, and macula of the retina.

[0080] Preferably, the optical system further includes a steering portion configured to move at least one convergence point around a convergence arc in response to rotation of the eye about the eye's rotation axis, the arc having a center of curvature substantially coinciding with the eye's rotation axis.

[0081] Preferably, the steering portion is configured to move at least one element of the projection portion and the central recess portion.

[0082] Preferably, the deflecting portion includes at least one optical element for deflecting light from the projection portion toward the central concave portion.

[0083] Preferably, at least one optical element includes a steerable reflector.

[0084] Preferably, at least one optical element includes a plurality of switchable lenses, each switchable lens having a corresponding orientation and configured to direct light from the projection portion to different portions of the central concave portion, such that the central concave portion converges the light to different convergence points on the convergence arc.

[0085] Preferably, at least one optical element includes a phase modulator with controllable phase modulation.

[0086] Preferably, the projection portion is configured to project multiple lights, each light encoding a corresponding portion of the image, and the central concave portion is configured to converge each light to a corresponding convergence point among multiple convergence points, the positioning of each convergence point on the convergence arc corresponding to a corresponding rotation angle of the eye around the eye rotation axis, such that the light converging at adjacent convergence points encodes adjacent portions of the image.

[0087] Preferably, at least one convergence point is located on a convergence surface that is substantially parallel to the pupil of the eye and includes a convergence arc.

[0088] Preferably, multiple convergence points are distributed along two dimensions of the convergence surface.

[0089] Preferably, the optical system is configured to move at least one convergence point along two dimensions of the converging surface.

[0090] Preferably, the projection portion is configured to project a first light encoding a first image onto the user's first eye and a second light encoding a second image onto the user's second eye, and the concave portion includes two concave components, each corresponding to a corresponding eye of the user, each concave component being configured to switch between a concave pattern that projects light onto the central region of the retina of the corresponding eye and a peripheral pattern that projects light onto the peripheral region of the retina of the corresponding eye, the peripheral region being larger than and including the central region.

[0091] Preferably, the projection portion includes a first projection component configured to project a first light and a second projection component configured to project a second light.

[0092] Preferably, the projection portion includes a display configured to emit an image to be displayed with substantially collimated light, and a focusing element for converging the light toward a central recess, wherein, optionally, the projection portion includes a spatial filter for receiving the converged light.

[0093] Preferably, the projection section includes a coherent light source, preferably a laser, configured to emit substantially coherent light, wherein the projection section optionally includes at least one of the following: a collimating element configured to collimate substantially coherent light, a phase modulator configured to encode an image by modulating substantially coherent light, and a laser beam scanning system including at least one steerable mirror.

[0094] Preferably, the projection portion includes a light field display configured to project light encoding multiple element images into a three-dimensional light field, each element image forming part of an image.

[0095] Preferably, in the concave mode, the concave portion is configured to project light encoded from the light field display for a first plurality of elements onto the central region of the retina of the eye, and in the peripheral mode, the concave portion is configured to project light encoded from the light field display for a second plurality of elements onto the peripheral region of the retina, wherein the second plurality of elements is different from the first plurality of elements.

[0096] Preferably, the projection portion includes a phase modulator configured to project light that encodes the holographic image.

[0097] Preferably, in the concave mode, the concave portion is configured to project light encoding a first portion of the holographic image onto the central region of the retina of the eye, and in the peripheral mode, the concave portion is configured to project light encoding a second portion of the holographic image onto the peripheral region of the retina, wherein the first and second portions of the holographic image are different from each other.

[0098] According to a third example aspect, a projection device is provided, which includes an optical system according to a first example aspect or a second example aspect of this document, and at least one of the following: one or more eye trackers for determining the rotation angle of the pupil of an eye, and a computing unit.

[0099] Optionally, each eye tracker includes at least one camera for capturing images of the eye, and the eye tracker is configured to determine the rotation angle of the pupil of the eye based on the image, the rotation angle of the pupil being used to determine the location of at least one convergence point on the convergence arc.

[0100] By tracking the rotation angle of the pupil, the optical system can determine the position on the converging arc (or converging surface) where light will be allowed to enter the eye. This can be used to determine the position on the converging arc where the convergence point should be moved (e.g., in the case of a dynamic view), and to determine which convergence points the light should be converged to.

[0101] In the case of a static view, the optical system can determine a subset of one or more convergence points to which the light encoding the image will converge (because the light will enter the eye) based on the pupil's rotation angle, and can selectively project only the light that will converge at the convergence points within the subset, thereby reducing the energy required to project the image. In other words, the optical system can interrupt the projection of light that converges at convergence points outside the subset.

[0102] Optionally, the eye tracker is configured to determine the focal length of the eye's lens.

[0103] For example, a tracker can determine the instantaneous focal length at recurring moments (e.g., periodically, such as every millisecond) or when a predetermined trigger (such as eye movement) is detected.

[0104] Therefore, the image projected onto the eye can be adapted to the eye's focal length. This allows, for example, the introduction of artificial (or digital) blur into the image to improve the perceived depth of elements in a virtual image.

[0105] Optionally, the eye tracker includes at least one light source for illuminating the eye.

[0106] Preferably, the light source cannot be detected by the eye to avoid affecting the image projected onto the eye.

[0107] Preferably, the projection device is held or worn by the user.

[0108] Preferably, the projection device includes a head-mounted device to be mounted on the user's head.

[0109] Preferably, at least one component of the optical system is located within the housing of the head-mounted device.

[0110] Preferably, the computing unit is configured to obtain a value indicating the rotation angle of the eye and generate one or more first control signals for causing the projection portion to project light of the rendered image based on the indicated rotation angle of the eye.

[0111] Preferably, the computing unit is configured to obtain the value from one or more eye trackers.

[0112] Preferably, the computing unit is configured to determine, based on the obtained value, a position on the converging arc corresponding to the indicated eye rotation angle, and to generate one or more second control signals for controlling at least one of the projection portion and the focusing portion, the one or more second control signals causing the light to converge at one or more convergence points corresponding to the determined position.

[0113] Preferably, the computing unit is configured to determine the direction within the scene to which the pupil is oriented based on the obtained value, and render the portion of the scene corresponding to that direction.

[0114] Preferably, the optical system includes a computing unit configured to: generate one or more first control signals to cause the projection portion to project light of a first portion of the rendered image onto the central region of the retina and to switch the foveal portion to a foveal mode; and generate one or more second control signals to cause the projection portion to project light of a second portion of the rendered image onto the peripheral region of the retina and to switch the foveal portion to a peripheral mode.

[0115] Preferably, the computing unit is configured to switch the projection portion and the central concave portion substantially simultaneously by generating one or more first control signals and one or more second control signals, wherein the projection portion switches between projecting light from the first part and projecting light from the second part, and the central concave portion switches between a central concave mode and a peripheral mode.

[0116] According to a fourth example aspect, a method for projecting an image near the eye onto a user's eye is provided, the method comprising: projecting light encoding the image, and converging the light to at least one convergence point such that the projected light enters the eye and forms an image on the retina, wherein the at least one convergence point is located on or near a convergence arc on the pupil of the eye, wherein the corresponding location of each convergence point on the convergence arc corresponds to a rotation angle of the eye about an axis of rotation of the eye, the convergence arc having a center of curvature substantially coinciding with the axis of rotation of the eye.

[0117] According to a fifth example aspect, a method for projecting an image near the eye into a user's eye is provided, the optical system comprising: projecting light encoding the image, and controlling at least one switchable optical element to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the at least one switchable optical element projects the image-encoded light onto a central region of the retina of the eye, and in the peripheral mode, the at least one switchable optical element projects the image-encoded light onto a peripheral region of the retina, the peripheral region being larger than and including the central region.

[0118] Preferably, the optical system according to the first or second example aspect, the projection device according to the third example aspect, and the method according to the fourth or fifth example aspect are used to project extended reality XR images, and more preferably, one of augmented reality (AR) images, virtual reality (VR) images, or mixed reality (MR) images is projected.

[0119] Preferably, the XR image includes at least one virtual element representing digital information.

[0120] Preferably, each virtual element is either overlaid on the user's physical environment, captured by the imaging device or seen by the user, or forms part of the virtual environment.

[0121] For simplicity, when the eye is approximated as a sphere, the center of the sphere can be defined as the center of eye rotation, and therefore, eye rotation can be defined as rotation about the center of the sphere (i.e., the center of eye rotation). Rotation about the center of eye rotation can also be defined as rotation about two distinct axes of eye rotation that intersect at the center of eye rotation and lie in a plane parallel to the pupil. These axes of eye rotation can be two axes defined according to Listing's law (e.g., the vertical and horizontal axes of the eye), i.e., two axes located in the Listing plane of the eye.

[0122] In some aspects of this article, light projected into the eye converges to at least one point of convergence, which may be a point in front of the eye's rotation center and substantially along the eye's pupillary axis or central concave axis (i.e., a point that may coincide with the eye's pupil, such that light converging to the point of convergence enters the eye and is projected onto the retina).

[0123] In some aspects of this paper, the convergence point is located on the convergence arc. The center of curvature of the convergence arc can be defined as an axis coinciding with one of the rotation axes of the eye, and the radius of curvature corresponds to the radius of a sphere approximating the eye. However, it should be understood that there may be a difference between the center of curvature of the convergence arc and the center of rotation of the eye (i.e., the center of the sphere), a difference between the radius of curvature of the convergence arc and the radius of the sphere (preferably, a difference of 1 cm or less), or both, which may be caused by variations in the size of the human eye.

[0124] If the center of the converging arc does not coincide with the center of the sphere, the center of curvature of the converging arc can be between the pupil and the center of rotation of the eye (e.g., in the volume around the pupillary axis or the central fovea axis of the eye, such as in a cone with the iris of the eye as its base and the center of rotation of the eye as its vertex), or between the center of rotation of the eye and the retina of the eye (e.g., in the volume around the pupillary axis or the central fovea axis of the eye, such as in a cone with the central region of the retina as its base and the center of rotation of the eye as its vertex). If the converging radius of the converging arc is greater than the radius of the sphere, the converging arc will be outside the eye and in front of the pupil.

[0125] The term "eye rotation angle" can also be defined as the orientation of the eye (or pupil), the rotation angle of the pupil, or the position of the pupil. The rotation angle can be defined relative to a default axis, which corresponds to the orientation of the pupil when the eye is considered to be stationary.

[0126] In some aspects of this paper, the optical system moves at least one convergence point based on the eye's rotation angle. For example, this can be done to converge light to a point that substantially coincides with the center of the eye's pupil. This can be referred to as a dynamic window because the convergence point moves dynamically as the eye rotates.

[0127] In certain aspects of this document, one or more steering portions may be provided to move at least one element of the projection portion and the focusing portion. The steering portion (or each steering portion) may include an electromechanical device, such as an actuator, motor, etc., configured to move an element of the projection portion, an element of the focusing portion, or an element of both the projection portion and the focusing portion.

[0128] In some aspects of this document, the steering portion (or each steering portion) may include at least one optical element. The optical element of the steering portion may be positioned along the optical path of light between the projection portion and the focusing portion. The optical element of the steering portion may be movable to change the optical path of the light toward the focusing portion, thereby causing a movement of the convergence point.

[0129] In some aspects of this paper, at least one optical element may have controllable phase modulation, which may be, for example, a phase or amplitude spatial light modulator (SLM), a lens with controllable phase modulation, etc.

[0130] In some aspects of this paper, at least one convergence point is located on the convergence surface. In other words, the optical system can move the convergence point along two dimensions of the convergence surface, or the optical system can be configured to converge light to a convergence point distributed along two dimensions of the convergence surface. These two dimensions will be understood as different (e.g., orthogonal) dimensions.

[0131] In some aspects of this paper, multiple beams of light, each encoding a corresponding portion of an image, converge to a single convergence point among a plurality of convergence points. These convergence points may be spaced (or distributed) at regular intervals along a convergence arc or a convergence surface, or the distance between adjacent convergence points may vary along an arc or surface, for example by providing greater density at a portion of an arc corresponding to the angle of rotation when the eye is stationary. In some cases, adjacent convergence points may be close enough to each other that light converging at two or more convergence points enters the eye simultaneously, thereby projecting different portions of the image onto different portions of the retina (which may overlap).

[0132] In certain aspects of this paper, one or more eye trackers may be used to track one or both of a user's eyes. Each eye tracker may be a camera-based tracker, a tracker using physiological sensors (e.g., ECG-based signals), or other sensing methods such as LiDAR, and each eye tracker may track the pupil or a protrusion formed by the cornea. Each tracker may be a camera-based tracker, in which case the camera may be a camera that captures images in the visible wavelength range (e.g., a red-green-blue RGB camera) or a camera that captures images in the infrared (IR) range (e.g., an IR camera, a near-IR camera). Each tracker may include at least one light source, which may be IR light, or visible (e.g., white) light that is off-center and not detected by the eye, or has a brightness below a threshold to avoid or reduce any influence on the image projected onto the eye.

[0133] In some aspects of this paper, an optical system can project an image into a user’s two eyes, for example by including separate parts for each eye (e.g., separate projection parts, focusing parts, steering parts, and / or eye trackers), wherein the parts(one or more) for each eye can be controlled individually.

[0134] In some aspects of this document, a display and a focusing element may be used. For example, this could be a self-emitting display that emits light. The display (e.g., a self-emitting display) may emit collimated light or may include collimating optics to collimate the light, or the light emitted by the display may be divergent, for example, if the focusing element is configured, due to its position and / or focal length, to receive divergent light and converge it toward the focusing portion. The spatial filter may be, for example, a pinhole (or Fourier) filter.

[0135] In some aspects of this document, a coherent light source may be used. A coherent light source can be any light source that emits light having at least a predetermined level of coherence, such as having coherence sufficient to be modulated by a phase modulator (e.g., an SLM, such as a device including a steerable micromirror array, such as a digital light processing (DLP) or other types of digital micromirror devices (DMDs)). A coherent light source may include a laser.

[0136] When the coherent light source is a laser, the laser may include one or more laser diodes, each of which can be configured to emit light of a given wavelength (e.g., diodes emitting red, green, and blue wavelengths, or infrared wavelengths). Preferably, the laser may include multiple diodes (e.g., at least red, green, and blue diodes) to emit light substantially across the visible spectrum.

[0137] In some aspects of this paper, a light field display can be used to project light that encodes an element image. Each element image of the light field can have an associated perceptual depth (e.g., corresponding to the distance between the retina and the focal point where the light rays associated with the element image converge), and the light for each element image can be projected onto different portions of the retina. The portion formed by each element image can overlap with at least one portion formed by adjacent element images.

[0138] In some aspects of this paper, the optical system may include a concave portion and a projection portion having a light field display. In this case, the light field display may be configured to project light that encodes different element images for the concave mode and for the peripheral mode.

[0139] In some aspects of this document, the optical system may include a concave portion and a projection portion, the projection portion including a phase modulator to project light encoding a holographic image. In this case, the phase modulator may project light encoding different holographic images for a concave mode and for a peripheral mode.

[0140] In some aspects of this paper, at least one switchable optical element may be used. In this case, each switchable optical element may be, for example, a converging or diverging element configured to change the angular magnitude of the transmitted light. For example, each switchable optical element may be a converging / diverging lens, a mirror, a holographic polymer-dispersed liquid crystal (HPDLC) layer, an Alvarez lens, or a polarization-switching lens such as a polarization volume grating.

[0141] In some aspects of this article, a switchable optical element (hereinafter also referred to as SOE) can be defined as switching to a first state or a second state. In the second state, the switchable optical element can have a different focal length than in the first state.

[0142] For example, the first state could correspond to a state in which the SOE allows light to pass through without substantially changing the angular size of the light, while in the second state, the SOE can change the angular size of the light (e.g., by converging or diverging the incident light, or by collimating converging / diverging the incident light). In other words, in one of the states (e.g., the first state), the SOE could correspond to a neutral lens.

[0143] In another example, in both the first and second states, the SOE can change the angle of the incident light.

[0144] In some aspects of this paper, the SOE can be switched to a first state or a second state by controlling the electrical signal received by the SOE (e.g., control voltage). The SOE can switch to the second state when it receives an electrical signal and switch to the first state (or vice versa) when it no longer receives an electrical signal. Therefore, the first state and the second state can be defined as "off" and "on" states, or "default" and "active" states.

[0145] As used herein, the term "central region of the retina" may refer to, for example, the macula of the retina, and preferably one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea fovea minor, and fovea fovea.

[0146] As used herein, the term "viewing window" may refer to the area in front of the projection device in which the user's eyes can be placed to view the image displayed by the projection device without distortion or loss of part of the displayed image.

[0147] As used in this article, the term "field of view" can refer to the angular range of the image displayed to the user's eyes. Brief description of the attached diagram

[0148] Figure 1A and Figure 1B A schematic diagram of a conventional projection device is shown; Figure 2A and Figure 2B A schematic diagram of another conventional projection device is shown; Figure 3A and Figure 3B A schematic diagram of the projection device in an example embodiment is shown; Figure 4 A schematic diagram of the projection device in an example embodiment is shown; Figure 5 A computing unit for controlling an optical system is shown in an example embodiment; Figure 6A and Figure 6B A schematic diagram of the optical system in an example embodiment is shown; Figure 7A and Figure 7B A schematic diagram of the optical system in an example embodiment is shown; Figure 8A and Figure 8B A schematic diagram of the optical system in an example embodiment is shown; Figure 9A and Figure 9B A schematic diagram of the optical system in an example embodiment is shown; Figure 10A and Figure 10B A schematic diagram of the optical system in an example embodiment is shown; Figure 11A and Figure 11B A schematic diagram of the optical system in an example embodiment is shown; Figure 12A and Figure 12B A schematic diagram of the optical system in an example embodiment is shown; Figure 13A and Figure 13B A schematic diagram of an optical system using waveguides is shown in an example embodiment; Figure 14A A schematic diagram of a peripheral image projected by an optical system in an example embodiment is shown; Figure 14B A schematic diagram of a concave image projected by an optical system in an example embodiment is shown; Figures 14C to 14D A schematic diagram of a central concave image and a peripheral image projected by an optical system in an example embodiment is shown; Figure 15A and Figure 15B A schematic diagram of an optical system using a concave portion in peripheral mode and concave mode, respectively, is shown in an example embodiment; Figure 16A and Figure 16B A schematic diagram of an optical system using a central concave portion in both central and peripheral modes is shown in an example embodiment. Figure 17A and Figure 17B A schematic diagram of an optical system using a central concave portion in both central and peripheral modes is shown in an example embodiment. Figure 18A and Figure 18B A schematic diagram of an optical system using a central concave portion in both central and peripheral modes is shown in an example embodiment. Figure 19A and Figure 19B A schematic diagram of an optical system using a concave portion in peripheral mode and concave mode, respectively, is shown in an example embodiment; Figure 20A and Figure 20B A schematic diagram of an optical system using a concave portion in peripheral mode and concave mode, respectively, is shown in an example embodiment; Figure 21A and Figure 21B A schematic diagram of an optical system using a concave portion in peripheral mode and concave mode, respectively, is shown in an example embodiment; Figure 22A and Figure 22BA schematic diagram of an optical system using a central concave portion in both central and peripheral modes is shown in an example embodiment. Figure 23A and Figure 23B A schematic diagram of an optical system using a concave portion in peripheral mode and concave mode, respectively, is shown in an example embodiment; Figure 24A A schematic diagram of an optical system using a light field display in an example embodiment is shown; Figures 25A to 25E A schematic diagram of an elemental image projected by an optical system using a light field display is shown in an example embodiment. Detailed description

[0149] For simplicity, the following description will refer to an eye rotating about an axis of rotation in two-dimensional space, which will be understood as an axis perpendicular to the plane shown in the accompanying drawings and near the center of the sphere representing the eye. Thus, the optical system described herein can adapt the image projected onto the eye to the movement of the eye along one of the eye's axes of rotation (e.g., rotation about a vertical axis when the eye moves laterally / left and right, or rotation about a horizontal axis when the eye moves up and down).

[0150] However, it should be understood that the optical system described herein can substantially adapt the image to the movement of the eye along both axes of rotation of the eye (e.g., both the vertical and horizontal axes described above), allowing the image to adapt to any movement of the eye. In this case, the rotation angle of the eye can be considered as the angle along the two axes of rotation of the eye. Therefore, the optical system described herein can thus be configured to converge light to at least one convergence point located on an arcuate surface comprising converging arcs. In other words, the “converging surface” can be defined by two converging arcs in two different dimensions, each arc having a center of curvature substantially coinciding with the rotation axis of the eye, and each arc being on or near the pupil of the eye. Thus, the defined surface can be substantially parallel to the outer surface of the cornea, iris, or lens of the eye.

[0151] As explained below, the components of an optical system (e.g., eye tracker, computing unit, projection section, etc.) can be communicatively connected to each other. The connection can be the establishment of one or more suitable communication links. Each communication link can be, for example, a wireless communication link (e.g., Wi-Fi, Bluetooth, or Bluetooth Low Energy (BLE), Near Field Communication (NFC)) or a wired communication link, such as a serial communication link (e.g., I...). 2 Communication links may include C, SPI, RS232, RS422, RS432, etc. Each communication link may not be permanent.

[0152] Figure 1A and Figure 1B A schematic diagram of a conventional projection device 1 for projecting an image onto an eye is shown. Projection device 1 may also be referred to as a near-eye display. Projection device 1 includes a projection section (not shown) configured to generate and modulate light such that the projected light encodes an image in an optical system 2. Optical system 2 includes one or more optical elements configured to converge the projected light onto an eye 6, such that an image 5 is formed on the retina of the eye 6. Figure 1A In the diagram, device 1 is shown to converge light to a first convergence point 3a, which is located on a convergence plane 4 just outside the eye (although in some configurations the convergence plane 4 is located on or inside the surface of the eye). Projection device 1 also includes an eye tracker (not shown) that tracks the movement of the eye 6 about a rotation axis 7 of the eye 6.

[0153] Figure 1B Device 1 in a second configuration is shown. Eye 6 rotates about axis 7 to a new position. An eye tracker measures the position of eye 6, and optical system 2 dynamically changes to converge light to a second convergence point 3b, which is displaced along convergence plane 4 from first convergence point 3a. The position of the convergence point is selected to correspond to the position of the pupil of eye 6, such that light enters the eye and image 5 is formed on the retina. The viewing window E of device 1 is defined by the maximum displacement position of convergence point 3 that device 1 can produce.

[0154] The system described above provides a dynamic viewing window in which only the portion of the window that enters the pupil is projected into the eye. This saves energy for the projection device 1 because light is not projected onto the point on the converging plane that does not enter the pupil of the eye 6. However, from Figure 1B As shown in the configuration, when the eye rotates away from the center of the converging plane 4, the image 5 shifts laterally on the retina, causing the center C' of the image 5 to no longer align with the center C of the retina. Therefore, the viewer of the projection device 1 perceives the edge of the window (or the edge of the field of view) and is able to move their eye 6 so that the retina focuses on the edge of the window. The resulting effect is that the user perceives the projected virtual scene as being displayed through a window with edges. This is in… Figure 1C As shown in the figure, Figure 1C An image from projection device 1 as perceived by a user is shown. When the user's eye 6 is in the center of the window, the user's central vision, indicated by F1, is contained within the edges of the image, making the user unaware of any edges. When the user's eye 6 moves away to the periphery of the window, the user's central vision, indicated by F1, makes the edges of the image perceptible, causing the user to generally perceive that they are viewing an image through a fixed window (also referred to herein as the window effect).

[0155] To overcome this problem, solutions such as... Figure 2Aand Figure 2B The projection device 1 shown is a projection section (not shown) configured to project light that encodes an image in the optical system 2. The optical system 2 includes one or more optical elements configured to converge the projected light onto the eye 6, such that an image 5 is formed on the retina of the eye 6. Figure 1A Unlike the devices shown, device 1 focuses light onto a convergence point on or near the rotation axis 7 of the eye 6. Projection device 1 also includes an eye tracker (not shown) that tracks the movement of the eye 6 around the rotation axis 7 of the eye 6. Figure 2B Device 1 in the second configuration is shown. Eye 6 rotates around axis 7. Figure 2B The new position is shown. The eye tracker measures the position of the eye 6, and the optical system 2 dynamically changes to rotate the projected light to match the rotation of the eye 6 around axis 7, so that light enters the eye and image 5 is formed on the retina. The viewing window E of device 1 is defined by the maximum rotational position of the projected light that device 1 can produce.

[0156] and Figure 1A and Figure 1B Unlike the system shown, image 5 remains centered on the retina, thus eliminating the windowing effect. However, image 5 formed on the retina is limited by the optical aperture of the pupil, therefore... Figure 1A and Figure 1B The device shown is smaller in size, thus limiting the field of view that a user can see at any given time.

[0157] Figure 3A and Figure 3B A schematic diagram of a projection device 10 according to one or more embodiments is shown. The projection device 10 includes a projection portion (not shown) configured to project light that encodes an image in an optical system 20. The optical system 20 includes one or more optical elements configured to converge the projected light onto an eye 6, such that an image 5 is formed on the retina of the eye 6. Figure 1A , Figure 1B , Figure 2A and Figure 2BUnlike the device shown, device 10 converges light to a point on a converging arc 41, which is in front of the rotation axis 7 of the eye 6 (i.e., the arc lies between the rotation axis 7 and the optical system 20), and more specifically at or near the pupil of the eye 6 (inside or outside the eye), while the image still rotates around a rotation axis 40 that is close to or coincides with the rotation axis 7 of the eye. In other words, the distance between the point of convergence of light and the rotation axis 7 of the eye (and the image) is chosen to be approximately equal to the distance between the pupil or lens of the eye 6 and the rotation axis 7. For example, the distance between the point of convergence and the rotation axis 7 (which may correspond to the radius of curvature of the converging arc) may be between 1 cm and 2 cm.

[0158] The projection device 10 may also include an eye tracker (not shown) that tracks the movement of the eye 6 about the rotation axis 7 of the eye 6. Figure 3B The device 10 is shown in the second configuration. The eye 6 rotates around the axis 7. Figure 3B The new position is shown. The eye tracker measures the position of the eye 6, and the optical system 20 dynamically changes to rotate the image 5 to match the rotation of the eye 6 around the axis 7, so that light enters the eye and the image 5 is formed centered on the retina. The window E of the device 10 can correspond to the area defined by the maximum rotational position of the projected light that the device 10 is capable of producing.

[0159] and Figure 2A and Figure 2B Unlike device 1 shown, device 10 allows for a wider field of view because the convergence point of the projected light is near the pupil, while still keeping the image centered on the retina of the eye 6 (i.e., focused on or near the central region of the retina) to avoid the windowing effect. Preferably, the image 5 may be focused on or near the macula of the retina. More preferably, the image 5 is focused on or near the area forming the macula, such as the area around the fovea, the perifovea, the fovea itself, the avascular area of ​​the fovea, the fovea minor, or the fovea fovea.

[0160] In three-dimensional space, the rotation axis 40 can be, for example, a vertical axis that allows the image 5 to remain focused on the central region of the retina during lateral eye movements, or a horizontal axis that allows the image 5 to remain focused on the central region of the retina when the eye moves up or down.

[0161] Figure 4A projection device 10 according to one or more embodiments is shown. The projection device 10 includes an eye tracker 60, a computing unit 50, and an optical system 20. The projection device may also include any suitable power source, such as a battery or other AC or DC power supply, and / or may include a power connector for connecting to an external AC or DC power supply. The eye tracker 60 may be any suitable eye tracker configured to track the positioning of a user's pupil. The eye tracker 60 is communicatively connected to the computing unit 50 (e.g., via a communication line or via a wireless communication link) and is configured to output a value to the computing unit 50 indicating the positioning of the pupil 6a of the user's eye 6. The computing unit 50 is configured to render a scene (i.e., a 2D or 3D image or video, which may include one or more virtual elements) and is configured to render the scene based on the position of the eye 6. More specifically, the computing unit 50 is configured to calculate which direction the pupil 6a is pointing within the scene and render that portion of the scene.

[0162] Eye-tracking element 60 may include, for example, an IR (infrared) and / or RGB camera and an IR source for the eye. The IR source illuminates the eye to enable tracking in low-light conditions without being seen by the user (because the frequency of the light is outside the human visual spectrum). Eye-tracking element 60 may be mounted separately from optical system 20 to device 100, or one or more optical elements within optical system 20 may be used to direct illumination light to eye 6 and reflect reflected light back to eye-tracking element 60. In some cases where both eyes are to be tracked, the same eye-tracking element 60 may be configured to track each eye, or separate eye-tracking elements may each track their respective eyes.

[0163] The projection device also includes an optical system 20. The optical system 20 includes a projection section 21 configured to project light encoded onto an image of the rendered scene generated by the computing unit 50. The optical system also includes a steering section 22 configured to deflect light emitted from the optical system 20 around a rotation axis 40 based on the positioning of the pupil 6a indicated by the computing unit 50 (or, alternatively, based on direct communication with the eye tracker 60). Figure 3A and Figure 3BFinally, the optical system includes a focusing section 23 configured to converge the light rays encoding the image to a point on the converging arc 41. The projection section 21, the steering section 22, and the focusing section 23 can be formed as separate optical elements, or two or more of these elements can be included in a single optical element that performs two or more of the functions of elements 21, 22, and 23. Separate projection sections 21, steering elements 22, and focusing elements 23 can be provided for each eye, or the same element 21, the same element 22, and / or the same element 23 can be used to project an image into both eyes (e.g., sequentially in time or space). In this case, the images projected into each eye may be different from each other. For example, offset can be used to provide a stereoscopic image, or a portion (e.g., half) of a larger image can be projected into each eye.

[0164] The computing unit 50 can control all the components, or it can provide a separate computing unit to control different optical components for each eye (i.e., the projection device includes a first eye tracker, a first computing unit and / or a first optical system for the left eye, and a second eye tracker, a second computing unit and / or a second optical system for the right eye).

[0165] In some embodiments, the projection portion 21 may include a light generating element and a light modulation element, wherein the light modulation element modulates the light generated by the light generating element to encode an image.

[0166] The focusing element 23 can be an optical combiner that redirects and converges light so that an image is projected onto the eye while allowing ambient light to pass through, thus superimposing the virtual image formed by the optical system 20 onto the real-world environment (augmented reality). In embodiments where the projection device is a pair of smart glasses, the optical combiner can be formed as part of one or more lenses of the smart glasses. In other embodiments, the focusing element 23 can be combined with an opaque lens so that light from the environment is not combined with the image, thereby providing a completely virtual view (virtual reality).

[0167] In some embodiments, the focusing element 23 or optical combiner may be a curved transparent reflector or an off-axis concave (or parabolic) mirror. The projection portion 21 and the steering element 22 may be mounted in or on the arm of the eyeglasses, and light may be projected through the air toward the eyeglass lens, which is reflected by the concave reflector on the lens and converged to the eye, thus acting as a converging lens.

[0168] In some embodiments, the focusing element 23 or optical combiner may include one or more flat or curved holographic optical elements (HOEs), each of which may be a photopolymer formed on a glass substrate. The HOEs may be recorded to act as off-axis concave mirrors, such that they behave similarly to curved reflectors. The HOEs may be manufactured to have the same optical properties as curved reflectors, but are much thinner.

[0169] In some embodiments, the focusing element 23 or optical combiner may include one or more holographic polymer dispersed liquid crystals (HPDLCs) or switchable Bragg gratings, each of which acts as a HOE that can be turned on and off (when off, the HPDLC or Bragg grating is transparent and allows light to pass through without substantially changing the angular size of the light).

[0170] When using HOE or a single HPDLC, there may be low tolerance for incident light with an angle of incidence different from the recording angle. This can limit the viewport of the projection device. Therefore, in some embodiments, the optical combiner may include multiple HPDLC layers recorded for different steering angles, and the computing unit 50 may be configured to turn on the HPDLC corresponding to the desired steering angle, so that light is correctly focused onto the pupil.

[0171] In some embodiments, the focusing element 23 or optical combiner may include a waveguide through which light is projected and converged to the eye by means of total internal reflection (TIR). It should be understood that many suitable waveguide configurations can be used. In some embodiments, a holographic waveguide including an input coupler and an output coupler is used.

[0172] In some embodiments, at least one of the projection portion 21 and the focusing portion 23 may be movable by the steering portion 22 to direct the light about the rotation axis 40. In some embodiments, the projection portion 21 may be movable to move the convergence point along one dimension of the converging surface, and the focusing portion may be movable to move the convergence point along another dimension of the converging surface. Thus, by moving both the projection portion 21 and the focusing portion 23, the convergence point can be moved according to any movement of the eye.

[0173] In some embodiments, the eye tracker 60 may be omitted. Instead, the projection portion 21 may be configured to project light that encodes multiple portions of the image 5, and the focusing portion 23 may be configured to converge each light to a corresponding convergence point among multiple convergence points located on a convergence arc (or convergence surface). The location of each convergence point corresponds to a corresponding rotation angle of the eye 6 about the eye rotation axis 4. Thus, when the eye rotates at an angle corresponding to one of the convergence points, the light converged at that convergence point enters the eye and forms a centered image on the retina. Light converged to adjacent convergence points can encode adjacent portions of the image, allowing the eye to rotate from an angle corresponding to one convergence point to an angle corresponding to an adjacent convergence point without causing a visible transition in the image formed on the retina.

[0174] In some embodiments, the eye tracker 60 may include a light source. The light source may be configured to be undetectable by the eye. Thus, the light source can be prevented from affecting the image to be projected onto the eye. As a non-limiting example, the light source may be configured such that light emitted by the light source does not enter the eye due to the position and / or orientation of the light source, or light from the light source is not projected onto the retina or the light is not visible (i.e., detectable by the cells of the retina).

[0175] In some embodiments, the projection device will be held or worn by a user. As a non-limiting example, the projection device may include a head-mounted device to be mounted on a user's head, wherein elements of the optical system are positioned near and in front of the user's eyes (or both eyes).

[0176] In some embodiments, the projection device includes a housing that stores components of the optical system. The housing may, for example, be located on the head-mounted portion of the projection device. Thus, the components stored in the housing can be protected from environmental influences, for example, by preventing unwanted light from the environment from affecting the light encoding the image to be projected.

[0177] Figure 5 A computing unit 50 for controlling a projection device according to one or more embodiments is illustrated. The computing unit 50 includes one or more communication interfaces 510 for communicating with components of the eye tracker 60 and the optical system 20. The communication interfaces may allow the computing unit 50 to communicate with other computing units external to the projection device 10, for example, to exchange data related to the image to be projected. Each communication interface may use any suitable communication link as described herein.

[0178] The computing unit 50 also includes a processor 515. The processor 515 may include one or more processing units, such as a microprocessor, GPU, CPU, multi-core processor, or similar computer processing unit.

[0179] The computing unit 50 also includes a memory 520. The memory 520 can be any suitable storage medium, including but not limited to optical discs, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archiving / warehouse devices, and / or any other type of device suitable for storing instructions and / or data. The memory 520 can store two-dimensional or three-dimensional image data and any other data to be displayed to the user, and can store data received from components of the eye tracker 60 and / or the optical system 20.

[0180] The computing unit 50 may further include an eye-tracking module 525, a rendering module 530, a display module 535, and a steering module 540. All modules of the computing unit 50 may be implemented in hardware, software, or a combination of both. For example, modules may be stored as software in memory 520. Modules may include any suitable sequence of instructions stored in the computing unit 50, which is configured to perform the methods disclosed herein.

[0181] In operation, eye tracker 60 tracks the positioning of the user's pupils (or both pupils) and outputs a value indicating the rotation angle of the pupils, which is received by interface 510. Eye tracking module 525 can convert this data into coordinates in a predetermined coordinate system describing the direction the pupils are pointing. Then, rendering module 530 can receive the coordinates describing the direction the pupils are pointing, receive image data from memory 520, and render a 2D or 3D image to be displayed to the pupils. Rendering module 530 can render different images for each pupil, taking parallax into account, so that the user perceives a 3D environment. The rendered image data is sent to display module 535, which generates instructions to projection section 21, causing projection section 21 to project light encoded with the rendered image. Steering module 540 calculates the steering angle of steering element 22 corresponding to the direction of the pupils and instructs steering element 22 to steering the image to the correct steering angle. The rendered image is then converged by focusing element 23 and formed on the retina of the user's eye. The value output by eye tracker 60 can specify the rotation angle of the pupil (e.g., the output of eye tracker 60 can include an angle value in degrees or radians), or the output of eye tracker can be an image of the eye (e.g., from a camera) or a sensor value (e.g., from an ECG sensor), and the eye tracking module can determine the rotation angle of the pupil based on the received image or sensor value.

[0182] Figure 6AA schematic diagram of an optical system 20 for a projection device according to one or more embodiments is shown. The projection device includes a self-emitting display 201 (e.g., a micro-LED display) that emits an image to be displayed with collimated light (or a standard display may be used with collimating optics placed in front of the display). The device also includes a focusing element 203 that converges the light rays. The device also includes a steerable mirror 205, which may be, for example, a MEMS mirror (i.e., a mirror including a microelectromechanical system MEMS), and is positioned after the convergence point F of the focused light rays. The steerable mirror 205 reflects light toward the focusing element 207, which converges the light to the convergence point on arc 41. An image is formed on the retina of the eye 6. Orienting the mirror 205 as indicated by arrow R rotates the virtual image V of the convergence point F, which in turn translates the convergence point of the light rays along arc 41. Thus, the mirror 205 can be steered to follow the position of the eye 6, such that the image projected onto the retina remains centered on the retina.

[0183] Note that although focusing elements 203 and 207 are shown as transmissive lenses, they can alternatively be concave mirrors or HOEs with the same focal length. For example, Figure 6B Alternative embodiments are shown in which a curved mirror or HOE is used instead of a lens for focusing element 207. This curved mirror or HOE can also be an optical combiner configured to allow light from the environment to pass through the mirror / HOE. If coherence of the light is required, for example, to use the HOE for components 203 or 207, a pinhole filter 204 (which may be referred to as a spatial or Fourier filter) can be added to the optical system.

[0184] for Figure 6A and Figure 6B The optical system 20, display 201, and focusing element 203 shown can be considered as projection part 21, the steerable mirror 205 can be considered as steering element 22, and the focusing element 207 can be considered as... Figure 4 The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a smart glasses, while element 23 can be integrated or coupled to the smart glasses lens).

[0185] Figure 7AA schematic diagram of an optical system 20 for a projection device according to one or more embodiments is shown. The optical system includes a laser 210 comprising one or more laser diodes (e.g., individual RGB laser diodes or a single RGB diode) configured to emit light along red, green, and blue light. It should be understood that the laser may also emit IR light or light of other wavelengths in the visible spectrum. The light emitted by the laser 210 is collimated by a collimating element 211 (which is shown as a lens, but may alternatively be a concave mirror or HOE, such as...). Figure 7B (As shown) Collimation. The collimated laser is incident on a phase or amplitude spatial light modulator (SLM) 212, which modulates the light such that the image to be projected is encoded in the light transmitted by the SLM 212. Any suitable SLM can be used, such as a transmissive or reflective amplitude and / or phase modulator based on liquid crystal (LC) or liquid crystal on silicon (LCoS) or digital micromirror device (DMD). It should be noted that although Figure 7A SLM 212 is shown as a reflective SLM, but alternatively, a transmissive SLM can be used. The modulated light is then focused to a convergence point F by a focusing element 213 (shown as a lens, but could be a concave mirror or HOE) before incident on the steerable mirror 205. The steerable mirror 205 can be any suitable steerable mirror, such as a MEMS mirror. The light is reflected by the steerable mirror 205 toward the focusing element 207 (a curved reflector or HOE), which focuses the light to a point on the convergence arc 41. In other embodiments, the focusing element 207 is a focusing lens, similar to... Figure 6A The embodiment shown. An image is projected onto the retina of eye 6. When the steerable mirror 205 is turned, as indicated by arrow R, the virtual image V of the convergence point F rotates around the arc, and the convergence point at the pupil of eye 6 shifts along arc 41. Therefore, the steerable mirror 205 is controllable to rotate the image projected onto the retina so that the image remains centered on the retina. Collimating lenses 211 and 213 can be replaced by a single focusing element 215 comprising a concave mirror (or one or more lenses), such as Figure 7B As shown. Focusing element 215 is configured to guide the laser light from laser 210 to SLM 212, such that the light is incident on SLM 212 in a decollimated manner and converges to a convergence point F in front of the steerable mirror 205. SLM 212 modulates the decollimated light, such that an image is encoded into the light. Once the image has been encoded into the light, focusing element 215 can also be placed after SLM 212.

[0186] for Figure 7AIn the optical system 20 shown, elements 210, 211, 212, and 213 can be considered as the projection section 21, the steerable mirror 205 can be considered as the steering element 22, and the focusing element 207 can be considered as... Figure 4 The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a pair of smart glasses). For Figure 7B In the optical system 20 shown, elements 210, 215, and 212 can be considered as the projection section 21, the steerable mirror 205 can be considered as the steering element 22, and the focusing element 207 can be considered as... Figure 4 The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a pair of smart glasses).

[0187] Figure 8A A schematic diagram of an optical system 20 for a projection device according to one or more embodiments is shown. The optical system includes a laser 210, such as an RGB laser diode configured to emit red, green, and blue light. The laser 210 is configured to emit a laser beam to a steerable mirror 205' (such as a MEMS mirror). The steerable mirror 205' is steerable (indicated by arrow R'), and the laser 210 and the steerable mirror 205' together emit light that encodes an image to be viewed by the user (selective emission of the laser from the laser 210 and selective positioning of the steerable mirror 205' together provide the necessary light generation and modulation to generate an image). Light is guided from the steerable mirror 205' to a steerable mirror 205 (any suitable mirror, such as a MEMS mirror), which is configured to rotate (indicated by arrow R). The light reflected by the steerable mirror 205 corresponds to a virtual image V and is guided toward a focusing element 207. Figure 8A In this embodiment, the focusing element 207 is a lens, while in other embodiments (such as...) Figure 8B As shown, the focusing element is a concave mirror, HOE, HPDLC, or stacked HPDLC. The focusing element 207 converges light to a convergence point on the convergence arc 41, and an image is formed on the retina of the eye 6. Rotation of the steerable mirror 205 shifts the convergence point along the convergence arc 41 and rotates the projected image about an axis 40 that coincides with or is near the rotation axis of the eye, such that the image projected onto the retina remains centered on the retina as the user's eye rotates.

[0188] for Figure 8A In the optical system 20 shown, elements 210 and 205' can be considered as projection part 21, the steerable mirror 205 can be considered as steering element 22, and the focusing element 207 can be considered as... Figure 4The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a pair of smart glasses).

[0189] Figure 9A A schematic diagram of an optical system 20 for a projection device according to one or more embodiments is shown. The optical system 20 includes a self-emitting display 217 (e.g., a micro-LED display) that emits an image to be displayed with collimated light (or a standard display may be used with collimating optics placed in front of the display). The optical system also includes a diverging element 218 configured to form a virtual image V of a converging arc 41. While the diverging element 218 is illustrated as a diverging lens, in other embodiments, a convex mirror, HOE, or HPDLC having the same optical properties as a diverging lens may be used. The self-emitting display 217 and the diverging element 218 are mounted to a steerable element (not shown), such as a MEMS, such that these components can be steered by rotation, the direction of rotation indicated by arrow R. Light emitted by the diverging element is directed to a focusing element 207, shown as a focusing lens, which converges the light to a point of convergence on the arc 41, and the image is projected onto the retina of an eye 6. Display 217 and diverging element 218 are rotatable, allowing the convergence point to move to a point on the convergence arc corresponding to the position of eye 6. Therefore, as eye 6 moves, the image remains centered on the retina of eye 6. Display 217 and diverging element 218 can be integrated into a single self-emitting display that emits diverging light.

[0190] In an alternative embodiment, the self-emitting display 217 may include a laser configured to illuminate an optical modulation element (such as a spatial light modulator (SLM) or a digital light circuit (DLP)) with divergent light. The divergent light is transmitted or reflected to a focusing element 207. The display 217, including the laser and the optical modulator, is mounted to a steerable element such as a MEMS, such that the projected image follows the position of the eye and remains centered on the retina. In embodiments using a laser configured to emit divergent light, the divergent element 218 is not required because the laser already emits a divergent beam.

[0191] In some embodiments, the focusing lens of the focusing element 207 may alternatively be a concave mirror, HOE, HPDLC, or HPDLC stack having the same optical properties as the focusing lens, such as Figure 9B As shown.

[0192] Reference Figure 9A and Figure 9B In the described embodiment, the display 217 and the diverging element 218 (or SLM) mounted on the steerable element can be considered as both the projection portion 21 and the steering element 22, and the focusing element 207 can be considered as Figure 4The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a pair of smart glasses).

[0193] Figure 10A A schematic diagram of an optical system 20 for a projection device according to one or more embodiments is shown. The optical system 20 includes a display 219. The display 219 may be a self-emitting display configured to emit coherent light, or a reflective or transmissive light modulation element, such as a spatial light modulator (SLM) or DLP illuminated with substantially coherent light (e.g., coherent point light or collimated light). The coherent light emitted by the display 219 is incident on an HPDLC stack 220 comprising multiple HPDLC layers. Each layer of the HPDLC is configured to act as a diverging lens, with each stack recorded at a different angle. Therefore, the angle at which the diverging beam is emitted depends on which HPDLC layer (or combination thereof) is switched on. The diverging beam is directed to a focusing element 207, which, in the illustrated embodiment, is a focusing lens. The light converges to a convergence point on a convergence arc 41, and an image is projected onto the retina of the eye. The layers of the HPDLC stack 220 may be selectively switched on in different combinations to emit the diverging light to the focusing element 207 at different selectable angles. Depending on the selected angle, light converges to different points on the converging arc 41. Therefore, the angle at which the image is projected onto the retina can be rotated about axis 40, keeping the image projected onto the retina centered on it. Advantageously, this embodiment does not include moving parts, thus simplifying the structure of the projection device.

[0194] In some embodiments, the light emitted by the display 219 may be incoherent, or alternatively divergent. In this case, the HPDLC stack 220 may modify the divergent light from the display 219 substantially as described above.

[0195] The focusing element 207 may alternatively include a concave mirror, a HOE or HPDLC, or an HPDLC stack having the same optical properties as the focusing lens, such as... Figure 10B As shown.

[0196] exist Figure 10A and Figure 10B In the illustrated embodiment, the display 219 can be considered as the projection portion 21, the HPDLC stack 220 can be considered as the steering element 22, and the focusing element 207 can be considered as... Figure 4 The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a pair of smart glasses).

[0197] Figure 11AA schematic diagram of an optical system 20 for a projection device according to one or more embodiments is shown. The optical system 20 includes a laser 210, such as an RGB laser, configured to emit diverging light toward a phase modulator 212. The phase modulator may be, for example, a reflective or transmissive SLM or DLP. The phase modulator 212 is configured to spatially modulate the phase of the light (and optionally also modulate the amplitude of the light) to encode an image. The phase modulator 212 is also configured to modulate the light to direct the beam in a given direction. The light is guided by the phase modulator 212 to a focusing element 207, which, in the illustrated embodiment, is a focusing lens. The light is converged by the focusing element 207 to a convergence point on a convergence arc 41, and an image is projected onto the retina of the eye. The phase modulation of the light by the phase modulator 212 can be selected to project an image in different directions, such that the light converges to different points along the convergence arc 41. Therefore, phase modulation can be controlled so that the image is projected onto the point on the converging arc 41 corresponding to the position of the eye 6, so that the projected image remains centered on the retina of the eye 6.

[0198] In some embodiments, the focusing element 207 may be a concave mirror, a HOE, an HPDLC, or an HPDLC stack, such as Figure 11B As shown. The phase modulator 212 can also be reflective, such as... Figure 11B As shown.

[0199] exist Figure 11A and Figure 11B In the illustrated embodiment, the laser 210 and phase modulator 212 can be considered together as the projection section 21 and the steering element 22, and the focusing element 207 can be considered as... Figure 4 The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a pair of smart glasses).

[0200] Figure 12AA schematic diagram of an optical system 20 for a projection device according to one or more embodiments is shown. The optical system 20 includes a self-emitting steerable display 221 configured to generate and modulate light. The display 221 is configured to emit light that encodes an image to be projected onto the retina of an eye 6. The light is transmitted to a focusing element 207, which in the illustrated embodiment is a focusing lens that converges the light to a point on a convergence arc 41, and the image is projected onto the retina of the eye 6. The display 221 is also configured to spatially modulate the phase of the light such that the image is projected in a specific direction and converges to a point along the convergence arc 41. Therefore, the display 221 is controllable to steer the projected light so that the convergence point moves to a point on the convergence arc 41 corresponding to the position of the eye, thereby keeping the projected image centered on the retina of the eye 6. In some embodiments, the focusing element 207 may be a concave mirror, a HOE, HPDLC, or a stack of HPDLCs, such as... Figure 12B As shown.

[0201] exist Figure 12A and Figure 12B In the illustrated embodiment, the self-emitting steerable display 221 can be considered as both the projection portion 21 and the steering element 22, and the focusing element 207 can be considered as... Figure 4 The focusing element 23 (and elements 21 and 22 can both be mounted in or on the arm of a pair of smart glasses).

[0202] With respect to any of the embodiments disclosed herein, it should be noted that the projected light may be transmitted between optical components via waveguides rather than via air. Figures 13A to 13C A schematic diagram of an optical system 20 using waveguides is shown.

[0203] Although the following exemplary embodiments are described using a single waveguide, this is not limiting, as any number of waveguides can be used. For example, in Figure 7B In the illustrated embodiment, a first waveguide may be provided between the laser 210 and the focusing element 215, and a second waveguide may be provided after the rotatable mirror 205. Each waveguide may have a corresponding input coupler and output coupler, and the output coupler of the second waveguide may be used instead. Figure 7B The concave reflector 207 shown.

[0204] In the following exemplary embodiments, the waveguide's output coupler is located on a surface of the waveguide away from the eye, and a TIR is used to reflect light from the waveguide's output coupler and converge that light to the eye. However, these are non-limiting examples, as the output coupler may alternatively be located on a surface of the waveguide near the eye, and the light may be converged to the eye by focusing the light.

[0205] exist Figure 13A In the illustrated embodiment, a self-emitting directional display 221 generates and modulates light. The light is transmitted to an input coupler 250 of waveguide 251. The light is guided through waveguide 251 to an output coupler 252 of waveguide 252, which converges the light through waveguide 251 to the eye via TIR.

[0206] exist Figure 13B In the illustrated embodiment, a self-emitting display 201 (emitting collimated light or including collimating optics) emits an image to be displayed using collimated light. The collimated light passes through a focusing element 203, which converges the light through a waveguide 251. The light is then reflected by a steerable mirror 205 (e.g., a MEMS mirror). The light reflected from the steerable mirror 205 is transmitted to an input coupler 250 of the waveguide 251. Figure 13A As in the embodiment shown, light from the input coupler 250 is guided through the waveguide 251 and converged to the eye by the output coupler 252 via TIR.

[0207] exist Figure 13C In the illustrated embodiment, a coherent light source 210 (e.g., a laser, an LED emitting sufficiently coherent light, etc.) (e.g., at red, blue, and green wavelengths) emits light along the visible spectrum, which is collimated by a collimating element 211 (e.g., such as...). Figure 13C The lens shown, or alternatively, a concave mirror or HOE, converges the light. The converged light passes through waveguide 251 and is reflected by SLM 212. The light reflected by SLM 212 is transmitted to the input coupler 250 of waveguide 251. Then, as shown... Figure 13A and Figure 13B As shown, light is guided through waveguide 251 to output coupler 252 and then converged into the eye. The coherent light source can be replaced by a self-emitting display that emits amplitude-coded images or by a reflective display (e.g., a laser / coherent light source that emits light reflected by an amplitude modulator).

[0208] In the previously disclosed embodiments, an increased field of view (such as when the projected light is focused at a point in front of the eye's rotation center) is obtained. Figure 3A and Figure 3B Image 5 formed in the middle and by Figure 2A and Figure 2B(The image formed by the system shown is compared with that shown). The projection system according to the invention allows a monocular field of view of 150° or greater. The natural resolution of the human eye is about 60 pixels per degree, which means that such a field of view requires a display with a resolution higher than 9000*9000 pixels. While this is possible, to render such a resolution, the final display device would be expensive, bulky, and computationally demanding. However, the retina has the highest sensitivity in the central foveal region and lower sensitivity in the peripheral region. Therefore, some projection devices can combine a foveal portion that is used to combine a low-resolution image for the periphery of the user's field of view with a high-resolution image for the foveal region. This reduces the requirements for resolution, bandwidth, optical power, and computation. Typical display devices use two displays (one for the high-resolution image and one for the low-resolution image), which also makes the display device more expensive and bulky. According to one aspect of this disclosure, a foveal portion 24 of an optical system 20 for a display device 10 is provided, which uses a single projection portion 21 for both images, which reduces the cost and size of the system. The foveal portion 24 can be actuated to switch between a foveal mode and a peripheral mode, in which a high-resolution image is projected onto the retina with a small field of view, and in the peripheral mode, a lower-resolution image is projected onto the periphery of the retina. The foveal portion 24 can be controlled by the computing unit 50 to switch between the foveal and peripheral modes. The computing unit 50 is configured to also cause the projection portion 21 to switch between the foveal and peripheral modes in a consistent manner with the foveal portion 24, such that the projection portion 21 projects an image in a high-resolution, low-field-of-view foveal mode or a lower-resolution, high-field-of-view peripheral mode. In such an embodiment, the computing unit 50 includes a foveal module 545 that can be implemented in hardware, software, or a combination of both (e.g., stored as software in memory 520). The foveal module includes a sequence of instructions stored in the computing unit 50 to consistently control the projection portion 21 and the foveal portion 24 as described herein. The foveal portion 24 can be placed at any suitable point along the optical path. For example, the central recess 24 can be placed after the projection portion 21 and before the steering element 22, or it can be placed after the steering element 22 and before the focusing element 23.

[0209] Figure 14A A schematic diagram of the optical system 20 of the projection device operating in peripheral mode is shown. The optical system (when it includes the central recess 24) can be, for example... Figure 4The optical system 20 of the projection device projects a peripheral image 5a onto the retina of the eye in peripheral mode. The peripheral image is a lower-resolution image projected onto the periphery of the retina when the eye is centered as shown. In embodiments where the steering element 22 is included in the projection device, the center of the peripheral image 5a remains on the center of the retina as the eye moves, such that the peripheral image remains on the periphery of the retina. The center C of the peripheral image 5a remains blank (i.e., no light or image is projected onto the retina in the central portion C in peripheral mode). Figure 14A It also shows the peripheral image I that the user will observe when using a projection device. p .

[0210] Figure 14B A schematic diagram of an optical system operating in concave mode is shown. The optical system (when it includes the concave portion 24) can be, for example... Figure 4 The optical system 20 projects a foveal image 5b onto the retina of the eye in foveal mode. A foveal image is a higher resolution image projected onto the foveal portion of the retina with a lower field of view when the image is focused on the central portion of the retina (as shown). In embodiments where a steering element 22 is included in the projection device, the center of the foveal image 5b remains on the center of the retina as the eye moves, such that the foveal image remains on the fovea of ​​the retina. Figure 14B It also shows the peripheral image I that the user will observe when using a projection device. f .

[0211] The optical system 20 is controlled by the computing unit 50, such as... Figure 5 The computing unit 50 is shown. The computing unit 50 controls the projection section 21 and the optical system 20 to switch between a central concave mode and a peripheral mode. The switching is performed at a sufficiently high rate so that the user does not perceive the switching, but rather perceives the complete image 5, for example... Figure 14C The image shown is a combination of a central concave image 5b and a peripheral image 5a. For example, the projection section 21 is configured to switch between a peripheral mode and a central concave mode at a rate of 60 Hz or higher (i.e., each of the central concave image 5b and the peripheral image 5a is shown at a rate of 30 frames per second or higher).

[0212] Note that the resolution and field of view of the peripheral and central concave images can vary depending on the requirements of the projection device. For example, as Figure 14D As shown, the central concave image can include a larger... Figure 14C This results in a wider field of view and lower resolution. Furthermore, the projection device can include three or more concave modes. For example, such as... Figure 14EAs shown, the image may include: an inner region image with a first resolution; an inner periphery image surrounding the inner region image, having a lower resolution than the inner region image; and an outer periphery image surrounding the inner periphery image, having a lower resolution than the inner periphery image. In such an embodiment, the foveal portion 24 may switch between three or more modes, each mode projecting an image onto a retina with a different field of view.

[0213] Figure 15A A schematic diagram of an optical system 20 incorporating a concave portion 24 according to one or more embodiments is shown. The optical system 20 includes a projection portion 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection portion 21 can be any suitable projection portion for projecting an image onto the retina of the eye (such as...). Figure 6A , Figure 6B , Figure 7A , Figure 7B , Figure 8A , Figure 8B , Figure 9A , Figure 10A , Figure 10B , Figure 11A , Figure 11B or Figure 12A (Any projection portion shown above). In embodiments where the steering element 22 is incorporated into the optical system 20 and includes mechanically steerable elements (such as mirror 205), the projection portion 21 may be, for example, as referenced Figure 6A and Figure 6B The described display 201 (and optional focusing element 203); as referenced Figure 7A The laser 210, collimating element 211, and SLM 212 (and optional focusing element 213) are described; see reference. Figure 7B The laser 210, focusing element 215, and SLM 212 described herein; or as referenced Figure 8A and Figure 8B The laser 210 and the steerable mirror 205' described.

[0214] In such an embodiment, the central recess 24 includes one or more switchable focusing and / or diverging elements (one or more switchable lenses or HPDLC layers) positioned between the projection portion 21 and the steering element 23 (i.e., the steerable reflector 205). Figure 15A In the peripheral mode shown, the switchable element and projection section 21 are switched to the peripheral mode, in which the projection section 21 projects the peripheral image 5a onto the periphery of the retina with a lower resolution and a higher field of view. Figure 15B It shows when Figure 15AThis is a schematic diagram of the optical system in foveal mode. In foveal mode, the switchable element and projection section 21 are switched to foveal mode, in which projection section 21 projects a foveal image onto the fovea of ​​the retina with higher resolution and a lower field of view. Note that by switching between these two modes, the resolution of the combined image viewed by the user is twice that of the resolution output by projection section 21, because the combined image is a combination of the two images projected by projection section 21 (and considering the non-uniform distribution of photoreceptors on the human retina, the perceived resolution can be much higher). The steerable mirror 205 steers the image to follow the position of the retina, as described in the previous embodiments. Note that in the illustrated embodiment, the light from both foveal and peripheral modes converges to the same convergence point F, such that the light from the peripheral image 5a and the foveal image 5b converges to a point on the convergence arc 41 and rotates with the position of the eye. This can be achieved through many different combinations of the switchable element. In the illustrated embodiment, in the peripheral mode, only the first focusing element 24c is activated, which converges the collimated light from the projection portion 21 at point F. In the concave mode, the second focusing element 24a and the collimating element 24b are also activated, causing a narrower collimated beam to be guided by the first focusing element 24c, which again converges the light to the convergence point F. It should be understood that any number of elements (e.g., two elements, such as a first diverging lens and a subsequent converging lens, or any other suitable combination of optical elements) can be activated in the peripheral mode.

[0215] Figure 16A An optical system 20 for a projection device, including a concave portion 24, is shown according to one or more embodiments. The optical system 20 includes a projection portion 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection portion 21 can be any suitable projection portion for projecting an image onto the retina of the eye (such as...). Figure 6A , Figure 6B , Figure 7A , Figure 7B , Figure 8A , Figure 8B , Figure 9A , Figure 10A , Figure 10B , Figure 11A , Figure 11B or Figure 12A (Any projection portion shown above). The illustrated embodiment shows an optical system 20 that includes only a projection portion 21 and a central concave portion 24 that directly converges the projected light to a convergence point F near the pupil of the eye (i.e., a projection device excluding the steering element 22). In such an embodiment, the central concave portion 24 can also be considered as a focusing element 23. It should be understood that, referring to... Figure 16AThe described central concave portion 24 can be compared with the reference. Figure 15A and Figure 15B The central concave portion 24 is described as interchangeable (and vice versa). In this case, as regarding Figure 15A As discussed, the central concave portion 24 converges the projected light to a convergence point F in front of the steerable reflector 205, such as... Figure 15A and Figure 15B As shown. The central concave portion 24 includes a first switchable focusing element 230 and a second switchable focusing element 231 (such as a switchable lens, a mirror, and / or an HPDLC layer), which are configured to receive collimated light from the projection portion 21. The first lens 230 and the second lens 231 have different focal lengths and are positioned away from the projection portion 21 and separated from each other, such that the convergence point F of the two lenses 230, 231 coincides. Figure 16A In the diagram, the optical system 20 is shown in a foveal mode, in which the projection portion 21 is controlled to project light that encodes the foveal image 5b, the first switchable focusing element 230 is turned on and the second switchable focusing element 231 is turned off. The light converges to the convergence point F, and an image is formed on the retina.

[0216] Figure 16B It shows the peripheral mode. Figure 16A In the optical system 20, in this peripheral mode, the projection section 21 is controlled to project light encoding the peripheral image 5a, the second switchable focusing element 231 is turned on and the first switchable focusing element 230 is turned off. The peripheral image is projected onto the convergence point F, and the image is formed on the retina. Because the second switchable focusing element 231 has a shorter focal length, the field of view of the peripheral image is wider. Note that by switching between these two modes, the resolution of the combined image viewed by the user is twice the resolution output by the projection section 21, because the combined image is a combination of the two images projected by the projection section 21. It should be understood that the field of view of the foveal image and the peripheral image depends on the size of the collimated light projected from the projection section, the focal lengths of the focusing elements 230 and 231, and the distance from the convergence point F to the retina.

[0217] Figure 17AAn optical system 20 for a projection device, including a fovea portion 24, is illustrated according to one or more embodiments. The optical system 20 includes a projection portion 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection portion 21 can be any suitable projection portion for projecting an image onto the retina. The illustrated embodiment shows an optical system 20 that includes only the projection portion 21 and a fovea portion 24 that converges the projected light directly to a convergence point F near the pupil of the eye (i.e., a projection device excluding the steering element 22). In such an embodiment, the fovea portion 24 can also be considered as a focusing element 23. It should be understood that reference... Figure 17A The described central concave portion 24 can be compared with the reference. Figure 15A and Figure 15B The central concave portion 24 is described as interchangeable (and vice versa). In this case, as regarding Figure 15A As discussed, the central concave portion 24 converges the projected light to a convergence point F in front of the steerable reflector 205, such as... Figure 15A and Figure 15B As shown. The central concave portion 24 includes a switchable diverging element 232 (e.g., a switchable lens, a convex mirror, or an HPDLC layer), and a first switchable focusing element 233 and a second switchable focusing element 234 (e.g., a switchable lens, a convex mirror, or an HPDLC layer, or any combination thereof). The switchable diverging element 232 is positioned away from and in front of the first switchable focusing element 233. Figure 17A In the diagram, the optical system 20 is shown in a foveal mode, in which the projection section 21 is controlled to project light that encodes the foveal image 5b, the second switchable focusing element 234 is turned on, and the first switchable focusing element 233 and the diverging element 232 are turned off. The collimated light encoding the image is guided from the projection section 21 to the second focusing element 234, the light converges to the convergence point F, and the image is formed on the retina.

[0218] Figure 17B It shows the peripheral mode. Figure 17AIn the optical system 20, in this peripheral mode, the projection section 21 is controlled to project light encoding the peripheral image 5a. In the peripheral mode, the second switchable focusing element 234 is turned off, and the first switchable focusing element 233 and the diverging element 232 are turned on. The projection section 21 projects collimated light encoding the peripheral image onto the switchable diverging element 232. The diverging light is directed to the first switchable focusing element 233, which converges the light to a convergence point F. The peripheral image 5a is then projected onto the retina. Because the first switchable focusing element 233 has a shorter focal length than the second switchable focusing element 231, the field of view of the peripheral image is wider. It should be understood that the exact positions of elements 232, 233, and 234 depend on the focal length. It should be understood that the field of view of the foveal image and the peripheral image depends on the size of the collimated light projected from the projection section, the focal lengths of elements 232, 233, and 234, and the distance from the convergence point F to the retina. Note that by switching between these two modes, the resolution of the combined image viewed by the user is twice that of the output resolution of the projection section 21, because the combined image is a combination of the two images projected by the projection section 21. By using the diverging element 232, the minimum space required for the optical system can be advantageously reduced. Specifically, the optical system requires a certain amount of space, which is determined in part by the focal length of each optical element used. Particularly, in Figure 16A and Figure 16B In the system shown, switchable focusing elements 230 and 231 should be located at distances to the eye corresponding to their respective focal lengths. Therefore, these switchable focusing elements need to be spaced apart from each other, which may be a major factor in determining how compact the optical system 20 can be. By using a diverging element 232 and a separate switchable focusing element 234, the two switchable focusing elements 233 and 234 do not need to be spaced apart from each other.

[0219] Figure 18A An optical system 20 for a projection device, including a fovea portion 24, is illustrated according to one or more embodiments. The optical system 20 includes a projection portion 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection portion 21 can be any suitable projection portion for projecting an image onto the retina. The illustrated embodiment shows an optical system 20 that includes only the projection portion 21 and a fovea portion 24 that converges the projected light directly to a convergence point F near the pupil of the eye (i.e., a projection device excluding the steering element 22). In such an embodiment, the fovea portion 24 can also be considered as a focusing element 23. It should be understood that reference... Figure 18A The described central concave portion 24 can be compared with the reference. Figure 15A and Figure 15B The central concave portion 24 is described as interchangeable (and vice versa). In this case, as regarding Figure 15A As discussed, the central concave portion 24 converges the projected light to a convergence point F in front of the steerable reflector 205, such as... Figure 15A and Figure 15B As shown. The central recessed portion 24 includes a first switchable diverging element 235a and a second switchable diverging element 235b, as well as a first switchable focusing element 236a and a second switchable focusing element 236b. The diverging and focusing elements can be, for example, a switchable lens, a switchable mirror, or an HPDLC layer, or any combination thereof. The first focusing element 236a is positioned away from and in front of the second diverging element 235b. Figure 18A An optical system 20 in foveal mode is shown. In foveal mode, a first focusing element 236a and a second diverging element 236b are turned on, and a first diverging element 235a and a second focusing element 236a are turned off. The projection section 21 is controlled to project collimated light encoding the foveal image 5b onto the first focusing element 236a. The first focusing element 236a converges the light toward the second diverging element 235b, and the second diverging element 235b diverges the light toward the convergence point F. The foveal image 5B is formed on the retina.

[0220] Figure 18B It shows the peripheral mode. Figure 18A The optical system 20. In peripheral mode, the first diverging element 235a and the second focusing element 236b are turned on, and the first focusing element 236a and the second diverging element 235b are turned off. The projection section is controlled to project collimated light encoding the peripheral image 5a onto the first diverging element 235a. The first diverging element 235a diverges the light and directs it to the second focusing element 235b, which converges the light to the same convergence point F as in foveal mode. The peripheral image 5a is formed on the retina.

[0221] Reference Figures 14A to 18B In all the embodiments described, the convergence point F of the projected light in the central concave mode and the peripheral mode may not coincide. Figure 19A An optical system 20 in peripheral mode is shown. In peripheral mode, the projection portion 21 projects collimated light onto the concave portion 24, which converges the light to a convergence point F1 in front of the steerable mirror 205. The light is guided to the focusing element 207, and a peripheral image 5a is formed on the retina. Note that the concave portion 24 includes a first switchable focusing element 237 and a second switchable focusing element 238, which are coupled to... Figure 16B and Figure 16BThe central recess 24 shown operates in a similar manner, except that in the current case the focusing elements are positioned such that the convergence points do not coincide. In peripheral mode, the first focusing element 237 is turned on and the second focusing element 237 is turned off. Note that the central recess 24 shown can be used... Figure 17A and Figure 17B or Figure 18A and Figure 18B The central concave portion is replaced, wherein the lens has optical properties and is positioned such that the convergence points in the central concave mode and the peripheral mode are offset from each other.

[0222] Figure 19B The central concave mode is shown. Figure 19A The optical system 20. The projection section 21 projects collimated light onto the concave section 24, which converges the light to a convergence point F2, which may, for example, coincide with the surface of the steerable mirror 205. The steerable mirror 205 guides the light to the focusing element 207, which converges the light so that the concave image 5b is projected onto the retina.

[0223] Figure 20A A schematic diagram of an optical system 20 incorporating a concave portion 24 according to one or more embodiments is shown. The optical system 20 includes a display 217 configured to emit collimated light that encodes an image to be projected onto the eye. The optical system also includes a concave portion 24, which can be any previously disclosed concave portion. The display 217 and the concave portion 24 are mounted to a steerable element such as a MEMS. Figure 20A In this embodiment, the optical system 20 is shown operating in peripheral mode. In this mode, the display 217 projects light that encodes the peripheral image. The concave portion 24 switches to peripheral mode and diffuses the light to a wide angle, while the focusing element 207 converges the light to a convergence point F, and the peripheral image 5a is projected onto the retina. A steerable element allows the peripheral image 5a to remain centered on the retina, as described in the previous embodiment.

[0224] Figure 20B It shows Figure 20A The optical system 20 operates in foveal mode. The display 217 projects light that encodes the foveal image. The foveal portion 24 is switched to foveal mode and diverges the light to a narrower angle than in peripheral mode, and the focusing element 207 converges the light to a convergence point F (coinciding with the convergence point in peripheral mode), and the foveal image 5b is projected onto the retina. A steerable element allows the foveal image to remain centered on the retina, as described in the previous embodiments.

[0225] As previously mentioned, the light projected by the focusing element 207 in the peripheral mode and the central concave mode does not need to have a coincident convergence point. Figure 21A and Figure 21B It shows something similar to Figure 20A and Figure 20B The optical system 20 differs in that the optical elements for each mode of the concave portion 24 are selected such that in the peripheral mode, the projected light converges to a convergence point F1, while in the concave mode, the projected light converges to a convergence point F2 offset from F1. The offset can be defined substantially around the optical axis of the eye, the pupillary axis of the eye, or the axis formed by the pupil and the center of rotation of the eye.

[0226] In the exemplary embodiments described above, the optical system 20 includes a central recess 24 that uses elements separate from the projection portion 21 and the focusing portion. However, this is not limiting, as a central recess display can be provided without the need for separate optical elements for the central recess (which can therefore be omitted).

[0227] Figure 22A and Figure 22B A schematic diagram of an optical system 20 configured to provide a concave display according to one or more embodiments is shown.

[0228] Specifically, as mentioned above... Figure 11A As described, laser 210 is configured to emit divergent light to phase modulator 212 (e.g., reflective or transmissive SLM or DLP), and phase modulator 212 is configured to spatially modulate the light to steer the beam in a given direction.

[0229] Figure 22A An optical system 20 in foveal mode is shown. A laser 210 and a phase modulator 212 transmit light to the central portion of a focusing element 207 (shown as a lens by way of a non-limiting example). The focusing element 207 converges the light from its central portion to a convergence point located near the pupil of the eye. The converged light enters the eye and projects a foveal image 5a onto the central portion 5a of the retina (e.g., the fovea).

[0230] Figure 22B An optical system 20 in peripheral mode is shown. A laser 210 and a phase modulator 212 transmit light to a wider portion of a focusing element 207 (wider than the central portion used in foveal mode). The focusing element 207 converges the light from the wider portion into the eye, and this light projects the peripheral image 5b onto the retina.

[0231] It should be understood that Figure 22A and Figure 22BThe implementation shown is a non-limiting example. Alternatively, the focusing element 207 can converge light in foveal mode to a point closer to the eye's rotation center (i.e., the center of the sphere representing the eye), and in foveal mode to a point closer to the pupil (i.e., the position of the convergence point in foveal and peripheral modes can be different). Figure 22A and Figure 22B (The example shown is the opposite). In other examples, one of the convergence points (in either the foveal or peripheral pattern) may be located in front of the eye, and the other convergence point (in the other of the foveal or peripheral pattern) may be located between the pupil and the center of eye rotation.

[0232] Advantageously, this optical system eliminates the need for movable components and additional optical elements to provide the central concave display. Therefore, the optical system 20 can be made more compact and requires fewer resources to operate.

[0233] and Figure 10A and Figure 10B As in the exemplary embodiment shown, the optical system 20 without moving parts can project different images onto the central portion and the peripheral portion of the retina.

[0234] Figure 23A and Figure 23B A schematic diagram of an optical system 20 configured to provide a concave display according to one or more embodiments is shown.

[0235] The optical system 20 includes a display 219, a central recess 24, a steering portion (shown by way of a non-limiting example as a stack 220 of HPDLCs), and a focusing element 207 (shown by way of a non-limiting example as a lens). The central recess includes one or more switchable optical elements (e.g., converging or diverging elements).

[0236] Figure 23A An optical system 20 in peripheral mode is shown. A display 219 emits collimated light onto a central recess 24. In peripheral mode, the switchable element of the central recess 24 is turned off, and thus the collimated light from the display 219 is substantially unchanged in angular magnitude (i.e., the light remains collimated).

[0237] The collimated light rays are transmitted to the deflecting portion 220, which is described above in conjunction with... Figure 10A and Figure 10B As explained, it causes the light to turn; for the sake of brevity, the explanation will be omitted here.

[0238] Figure 23BAn optical system 20 in concave mode is shown. In concave mode, a switchable element of the concave portion 24 is activated. Therefore, focusing element 24a focuses collimated light from display 219. Collimating element 24b collimates the focused light, which is then transmitted to deflecting portion 220. In concave mode, the light collimated by collimating element 24b reaches a narrower portion of deflecting portion 220 than in peripheral mode (e.g., the collimated light is projected onto the central portion of the HPDLC stack). Therefore, the light reaches focusing element 207 and thus the eye in concave mode with a narrower angular range than in peripheral mode.

[0239] Despite Figure 23A and Figure 23B In the example shown, the central recess 24 includes a focusing element 24a and a collimating element 24b, but this is not limiting, as it should be understood that any number of switchable optical elements can be provided. More generally, the central recess may alternatively correspond to the central recess described in any exemplary embodiment herein, for example... Figures 15A-15B , Figures 16A-16B , Figures 17A-17B , Figures 18A-18B , Figures 19A-19B , Figures 20A-20B , Figures 21A-21B The central concave portion shown in any of the above.

[0240] The above embodiments use examples of displays that emit light that encodes an image. In some embodiments, the display may be a light field display configured to project light that encodes an image of multiple elements into a three-dimensional light field.

[0241] Figure 24A A schematic diagram of an optical system 20 configured to provide a light field display according to one or more embodiments is shown.

[0242] The optical system 20 includes a light field display 260 and a focusing element 203 (as an example of a projection portion 21), a rotatable mirror 205 (as an example of a steering portion 22), and a focusing element 207 (as an example of a focusing portion 23).

[0243] Focusing element 203, rotatable mirror 205, and focusing element 207 are combined as described above. Figure 6A As explained, these components will be operated as described above, so for the sake of brevity, some explanations of these components will be omitted here.

[0244] The light field display 260 emits rays that encode multiple element images within a three-dimensional light field. Each element image is rendered so that it is perceived by the user as arriving from a virtual point in the environment. Figure 24ATwo exemplary virtual points, V1 and V2, are shown, although it should be understood that any other number of virtual points may be provided.

[0245] Light rays corresponding to virtual point V1 are reflected away from rotatable mirror 205 and pass through focusing element 207 to converge at point V1' on the retina. Because these rays converge at the point on the retina, the image of the elements encoded in these rays appears to be in focus to the user.

[0246] Similarly, the light rays corresponding to the virtual point V2 are reflected off the rotatable mirror 205 and pass through the focusing element 207 to be projected into the eye. However, the light rays corresponding to the virtual point V2 converge at point V2' in front of the retina. Therefore, these light rays are projected onto the retinal area indicated by V2'. Because these light rays are not focused at a single point when they reach the retina, the image of the elements encoded by these light rays is blurred (i.e., they appear out of focus to the user).

[0247] Therefore, the user will perceive that the element image encoded by the light rays corresponding to virtual point V2 is at a different depth than the element image encoded by the light rays corresponding to virtual point V1. Thus, the image displayed by the light field display acquires a third dimension with the depth perceived by the user.

[0248] It should be understood that, in the example shown in Figure 24C, the light focused by the focusing element 207 is focused at multiple convergence points on the converging arc near the pupil. As a non-limiting example, Figure 24A Three convergence points C1, C2, and C3 are shown.

[0249] Figures 25A-25E Schematic diagrams are shown, illustrating different combinations of element images that can be encoded in light from a light field display.

[0250] like Figure 25A As shown, nine element images with different perceptual depths can be combined to form a three-dimensional light field. Specifically, the images in the middle column can correspond to a first perceptual depth (as shown by solid lines), the images in the left column can correspond to a second perceptual depth (as shown by dotted lines), and the images in the right column can correspond to a third perceptual depth (as shown by dashed lines), where the first, second, and third perceptual depths are different from each other. The nine element images have partial overlap with adjacent images, allowing the user to perceive the image, where each element has a different depth.

[0251] The number of element images, the configuration of each image, or the perceptual depth are not limited to Figure 25A Examples. Specifically, the number of element images can be different, for example, using... Figure 25BThe four element images shown are different configurations (i.e., the element images do not need to be configured in columns with an equal number of element images).

[0252] Furthermore, some images do not need to overlap. For example, such as Figure 25D As shown, the corresponding parts of each element image that will overlap with other element images during combination can be omitted. Figure 25D (The shaded portion above). In this case, a single element image among multiple element images (e.g., the element image in the upper right corner) can include the corresponding shaded portion, so the combined image is complete.

[0253] like Figure 25E As shown, light field displays can also be used to display different three-dimensional light fields to the central and peripheral portions of the retina. For example, a five-element image (with...) can be used... Figure 25C The light field of the configuration shown is projected onto the central part, and can be used to project a four-element image (with...) Figure 25D The light field of the configuration shown is projected onto the peripheral portion. It should be understood that the number of element images described herein is merely exemplary and any number of element images can be used alternatively.

[0254] Various examples of three-dimensional light fields that can be provided have been described, and for the sake of brevity, further details of light field displays that will now be apparent to those skilled in the art will be omitted here.

[0255] Modifications and changes Many modifications and variations can be made to the above example embodiments.

[0256] although Figure 4 An optical system 20 with a steering portion 22 and a central recess portion 24 is shown, but in some embodiments, either or both of the steering portion 22 and the central recess portion 24 may be omitted.

[0257] For example, such as Figures 6A to 13C As shown in either of them, the optical system 20 does not need to project different images onto the central region and the peripheral region of the retina.

[0258] As another example, such as Figure 10A or Figure 10B As shown, the projection section 21 can be configured to emit light in different directions, thereby eliminating the need for the steering section 22.

[0259] Furthermore, when the optical system 20 includes a central concave portion, the optical system 20 does not need to move the convergence point along the convergence arc (in the case of a dynamic view).

[0260] Although the above example embodiments use specific optical elements, it should be understood that each of these optical elements can be replaced by other interchangeable optical elements. For example, each diverging lens can be replaced by a convex mirror, each converging lens can be replaced by a concave mirror, or each lens or mirror can be replaced by HPDLC, HOE, stacked HPDLC, etc.

[0261] Although in the example embodiments described above, the mirror or other rotatable element is shown to have one axis of rotation, these elements may have two different axes of rotation (or the optical system 20 may include additional rotatable elements configured to rotate along a second, different axis of rotation), thereby allowing the optical system to control the positioning of the convergence point of light along two dimensions.

[0262] In the example embodiments described above, the image projected onto the eye can be a portion of a scene (such as a still image or video showing one or more virtual elements). The projected portion of the scene can correspond to the rotation angle of the eye (i.e., the orientation of the pupil), such that the projected portion changes as the eye moves.

[0263] In other cases, such as when using a dynamic view (and the optical system moves the convergence point along a convergence arc or surface), the displayed image can instead remain fixed in the user's field of view. In other words, the image seen by the user can be static and not move in the user's vision.

[0264] In any of the example embodiments described above that include a phase modulator, the optical system can be configured to project light that encodes the holographic image. For example, in Figure 7A In this configuration, phase modulator 212 can be configured to project light encoding the holographic image by combining wavefronts. For the sake of brevity, details of holographic image generation and projection, which will now be apparent to those skilled in the art, will be omitted.

[0265] References above Figure 17A and Figure 17B In the described example embodiment, two switchable focusing elements 233 and 234 are included, each of which is used to focus light in one of a foveal mode and a peripheral mode. Alternatively, a single switchable focusing element can be used and controlled to switch between a first state and a second state (e.g., in the first state when the foveal portion is in the foveal mode and in the second state when the foveal portion is in the peripheral mode, or vice versa), wherein the single switchable focusing element may have different focal lengths in the first state and the second state.

[0266] References above Figure 18A and Figure 18BIn the described example embodiment, the first switchable diverging element 235a and the first focusing element 236a may be replaced by a single switchable optical element (wherein, in concave mode, the single switchable optical element has a focal length corresponding to the first switchable diverging element 235a, and in peripheral mode, the single switchable optical element has a focal length corresponding to the first focusing element 236a). Additionally or alternatively, elements 235b and 236b may be replaced by a single switchable optical element.

[0267] In the foregoing description, exemplary aspects have been described with reference to several exemplary embodiments. Therefore, the specification should be considered illustrative rather than restrictive. Similarly, the accompanying drawings, which highlight the functionality and advantages of exemplary embodiments, are presented merely for illustrative purposes. The architecture of the exemplary embodiments is flexible and configurable enough that it can be utilized in ways other than those shown in the drawings.

[0268] Some embodiments can also be implemented by preparing application-specific integrated circuits, field-programmable gate arrays, or by interconnecting appropriate networks of conventional component circuits.

[0269] The apparatus described herein may be implemented in other specific forms without departing from its characteristics. The foregoing embodiments are illustrative and not intended to limit the systems and methods described. Therefore, the scope of the apparatus described herein is indicated by the appended claims rather than the foregoing description, and thus variations falling within the meaning and scope of equivalents of the claims are included therein.

Claims

1. An optical system for projecting an image near the eye onto a user's eye, the optical system comprising: - A projection section configured to project light that encodes the image; as well as - A central recessed portion, the central recessed portion including at least one switchable optical element, the at least one switchable optical element being controllable to switch between a central recessed mode and a peripheral mode, wherein -- In the foveal pattern, the foveal portion is configured to project light from the projection portion onto the central region of the retina of the eye, and -- In the peripheral mode, the concave portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and including the central region.

2. The optical system according to claim 1, wherein, The projection portion is configured to project the light as substantially collimated light.

3. The optical system according to claim 1 or 2, wherein, Each of the at least one switchable element is configured to allow light to pass through when switched to a first state without substantially changing the angular size of the light.

4. The optical system according to any one of claims 1 to 3, wherein, The central concave portion is configured to increase the angle of light incident on the eye in the peripheral mode.

5. The optical system according to any one of claims 1 to 4, wherein, The concave portion is configured to increase the focal length of light incident on the eye in the concave pattern.

6. The optical system according to claim 5, wherein, The central recess includes at least one focusing element with a controllable focal length.

7. The optical system according to any one of claims 1 to 6, wherein, The at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length different from the first focal length.

8. The optical system according to claim 7, wherein, The first switchable element is arranged at a first distance from the display, and the second switchable element is arranged at a second distance from the display.

9. The optical system according to claim 8, wherein, The difference between the first distance and the second distance makes the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

10. The optical system according to any one of claims 7 to 9, wherein, In the central recessed mode, the first switchable element is configured to switch to the second state, and the second switchable element is configured to switch to the first state. In the peripheral mode, the first switchable element is configured to switch to a first state, and the second switchable element is configured to switch to a second state.

11. The optical system according to any one of claims 1 to 6, wherein, The at least one switchable optical element includes a first switchable diverging element, a first switchable converging element, and a second switchable converging element. The first switchable diverging element is arranged at a first distance from the projection portion, and the first switchable converging element is arranged at a second distance from the projection portion, the second distance being greater than the first distance.

12. The optical system according to claim 11, wherein, In the foveal mode, the first switchable diverging element is configured to switch to a first state, the first switchable converging element is configured to switch to a first state, and the second switchable converging element is configured to switch to a second state, such that light from the projection portion is converged by the second switchable converging element to form an image on the central region of the retina. In the peripheral mode, the first switchable diverging element is configured to switch to a second state, the first switchable converging element is configured to switch to a second state, and the second switchable converging element is configured to switch to a first state, such that light from the projection portion is diverged by the first switchable diverging element onto the first switchable converging element and converged by the first switchable diverging element to form an image on the peripheral region of the retina.

13. The optical system according to claim 11 or claim 12, wherein, The second switchable converging element is arranged at a third distance from the projection portion such that the convergence point of the first switchable converging element and the convergence point of the second switchable converging element substantially coincide with each other.

14. The optical system according to claim 11 or claim 12, wherein, The at least one switchable optical element further includes a second switchable divergence element. The second switchable diverging element is configured to switch to a second state in the concave mode and increase the angle of light focused by the first switchable converging element. The second switchable converging element is arranged at a third distance from the projection portion, and the second switchable diverging element is arranged at a fourth distance from the projection portion. The first distance, the second distance, the third distance, and the fourth distance are configured such that the convergence point of light in the concave mode and the convergence point of light in the peripheral mode substantially coincide with each other.

15. The optical system according to any one of claims 1 to 14, wherein, The concave portion is configured to converge the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina. The at least one convergence point is located on or near the pupil of the eye on a convergence arc, wherein the corresponding location of each convergence point on the convergence arc corresponds to the rotation angle of the eye about the eye's rotation axis, and the convergence arc has a center of curvature that substantially coincides with the eye's rotation axis.

16. The optical system according to claim 15, wherein, The optical system is configured to move the at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably substantially coincides with the macula of the retina, and more preferably substantially coincides with one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea fovea minor, and fovea fovea of ​​the retina.

17. The optical system of claim 16, further comprising a steering portion configured to move the at least one convergence point about the convergence arc in response to rotation of the eye about an eye rotation axis, the arc having a center of curvature substantially coinciding with the eye rotation axis.

18. The optical system according to claim 17, wherein, The steering portion is configured to move at least one element of the projection portion and the central recessed portion.

19. The optical system according to claim 17 or claim 18, wherein, The deflecting portion includes at least one optical element for deflecting light from the projection portion toward the central concave portion.

20. The optical system according to claim 19, wherein, The at least one optical element includes a steerable reflector.

21. The optical system according to claim 19 or claim 20, wherein, The at least one optical element includes a plurality of switchable lenses, each switchable lens having a corresponding orientation and configured to direct light from the projection portion onto different portions of the central concave portion, such that the central concave portion converges the light to different convergence points on the convergence arc.

22. The optical system according to any one of claims 19 to 21, wherein, The at least one optical element includes a phase modulator with controllable phase modulation.

23. The optical system according to any one of claims 1 to 15, wherein, The projection portion is configured to project multiple lights, each light encoding a corresponding portion of the image, and the central concave portion is configured to converge each light to a corresponding convergence point among multiple convergence points, the positioning of each convergence point on the convergence arc corresponding to a corresponding rotation angle of the eye about the eye's rotation axis, such that light converging at adjacent convergence points encodes adjacent portions of the image.

24. The optical system according to any one of claims 15 to 23, wherein, The at least one convergence point is located on a convergence surface that is substantially parallel to the pupil of the eye and includes the convergence arc.

25. The optical system according to any one of claims 1 to 24, wherein, The projection section is configured to project a first light encoding a first image onto the user's first eye, and a second light encoding a second image onto the user's second eye. The central recess portion includes two central recess components, each corresponding to one of the user's eyes. Each foveal component is configured to switch between a foveal pattern that projects light onto a central region of the retina of the corresponding eye and a peripheral pattern that projects light onto a peripheral region of the retina of the corresponding eye, the peripheral region being larger than and encompassing the central region.

26. The optical system according to any one of claims 1 to 25, wherein, The projection portion includes a display and a focusing element. The display is configured to emit an image to be displayed using substantially collimated light, and the focusing element is used to converge the light towards the central concave portion. Optionally, the projection section includes a spatial filter for receiving the converged light rays.

27. The optical system according to any one of claims 1 to 26, wherein, The projection section includes a coherent light source, preferably a laser, which is configured to emit substantially coherent light. The projection portion may optionally include at least one of the following: A collimating element configured to collimate the substantially coherent light. A phase modulator, configured to encode the image by modulating the substantially coherent light, and A laser beam scanning system, the laser beam scanning system including at least one steerable reflector.

28. The optical system according to any one of claims 1 to 27, wherein, The projection portion includes a light field display configured to project light encoding multiple element images into a three-dimensional light field, each element image forming a portion of the image.

29. The optical system according to any one of claims 1 to 27, wherein, The projection section includes a phase modulator configured to project light that encodes the holographic image.

30. A projection device comprising an optical system according to any one of claims 1 to 29 and an eye tracker for determining the rotation angle of the pupil of an eye. in, Optionally, the eye tracker includes at least one camera for capturing images of the eye, the eye tracker being configured to determine a rotation angle of the pupil of the eye based on the images, the pupil rotation angle being used to determine the positioning of the at least one convergence point on the convergence arc. Optionally, the eye tracker is configured to determine the focal length of the lens of the eye, and Optionally, the eye tracker includes at least one light source for illuminating the eye.

31. A method for projecting an image near the eye onto a user's eye, the optical system comprising: Projecting light that encodes the image, and Control at least one switchable optical element to switch between a central concave mode and a peripheral mode, wherein In the foveal mode, light encoding the image is projected onto the central region of the retina of the eye by the at least one switchable optical element, and In the peripheral mode, light encoding the image is projected onto a peripheral region of the retina by the at least one switchable optical element, the peripheral region being larger than and including the central region.

Images