Projection device

Abstract

An optical system for near-eye projection of an image into an eye of a user, the optical system comprising: a projection portion configured to project light encoding the image, and a central concave portion comprising at least one switchable optical element, the at least one switchable optical element being controllable, the central concave portion is configured to project light from the projection portion onto a central region of the retina of the eye and to switch between a central concave mode in which the central concave portion is configured to project light from the projection portion onto a central region of the retina of the eye and a peripheral mode in which the central concave portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region is larger than and includes the central region. A corresponding method and projection apparatus are also provided.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202380097815.3 (22) Application Date 2023.05.12 (85) PCT International Application Entering National Phase Date 2025.11.03 (86) PCT International Application Application Data PCT / IB2023 / 000246 2023.05.12 (87) PCT International Application Publication Data WO2024 / 236334 EN 2024.11.21 (71) Applicant Inmesia Computer Co., Ltd. Address Barcelona, ​​Spain (72) Inventors Gilliam de la Torrero Vera Otman Mayo Amarqui D Treptow (74) Patent Agency Beijing Anxin Fangda Intellectual Property Agency Co., Ltd. 11262 Patent Attorneys Zhang Shaobo Yang Mingzhao (51) Int.Cl. G02B 27 / 01(2006.01) G02B 27 / 00(2006.01) (54) Title of Invention: Projection Device (57) Abstract: An optical system for projecting an image near the eye onto a user's eye, the optical system comprising: a projection portion configured to project light encoding the image, and a foveal portion including at least one switchable optical element controlled to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the foveal portion is configured to project light from the projection portion onto a central region of the retina of the eye, and in the peripheral mode, the foveal portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and encompassing the central region. Corresponding methods and projection devices are also provided. Claims 4 pages, Description 29 pages, Drawings 25 pages, CN 121219620 A 2025.12.26 CN 1 21 21 96 20 A 1. An optical system for projecting an image near the eye onto a user's eye, the optical system comprising: - a projection portion configured to project light encoding the image; and - a foveal portion including at least one switchable optical element controllable to switch between a foveal mode and a peripheral mode, wherein - in the foveal mode, the foveal portion is configured to project light from the projection portion onto a central region of the retina of the eye, and - in the peripheral mode, the foveal portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and encompassing the central region. 2. The optical system of claim 1, wherein the projection portion is configured to project the light as a basic...3. The optical system of claim 1 or 2, wherein each of the at least one switchable element is configured to allow the light to pass through without substantially changing the angular magnitude of the light when switched to a first state. 4. The optical system of any one of claims 1 to 3, wherein the concave portion is configured to increase the angular magnitude of the light incident on the eye in the peripheral mode. 5. The optical system of any one of claims 1 to 4, wherein the concave portion is configured to increase the focal length of the light incident on the eye in the concave mode. 6. The optical system of claim 5, wherein the concave portion includes at least one focusing element having a controllable focal length. 7. The optical system of any one of claims 1 to 6, wherein the at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length, the second focal length being different from the first focal length. 8. The optical system of claim 7, wherein the first switchable element is arranged at a first distance from the display, and the second switchable element is arranged at a second distance from the display. 9. The optical system of claim 8, wherein the difference between the first distance and the second distance causes the convergence point of the first switchable element and the convergence point of the second switchable element to substantially coincide with each other. 10. The optical system of any one of claims 7 to 9, wherein in the concave mode, the first switchable element is configured to switch to a second state, and the second switchable element is configured to switch to a first state; and in the peripheral mode, the first switchable element is configured to switch to the first state, and the second switchable element is configured to switch to the second state. 11. The optical system of any one of claims 1 to 6, wherein the at least one switchable optical element comprises a first switchable diverging element, a first switchable converging element, and a second switchable converging element, wherein the first switchable diverging element is arranged at a first distance from the projection portion, and the first switchable converging element is arranged at a second distance from the projection portion, the second distance being greater than the first distance. 12. The optical system of claim 11, wherein, in the foveal mode, the first switchable diverging element is configured to switch to a first state, the first switchable converging element is configured to switch to a first state, and the second switchable converging element is configured to switch to a second state, such that light from the projection portion is converged by the second switchable converging element to form an image on the central region of the retina, and claim 1 / 4 page 2 CN 121219620 AIn the peripheral mode, the first switchable diverging element is configured to switch to a second state, the first switchable converging element is configured to switch to a second state, and the second switchable converging element is configured to switch to a first state, such that light from the projection portion is diverged by the first switchable diverging element onto the first switchable converging element and converged by the first switchable diverging element to form an image on the peripheral region of the retina. 13. The optical system of claim 11 or claim 12, wherein the second switchable converging element is arranged at a third distance from the projection portion such that the convergence point of the first switchable converging element and the convergence point of the second switchable converging element substantially coincide with each other. 14. The optical system of claim 11 or claim 12, wherein the at least one switchable optical element further comprises a second switchable diverging element, wherein the second switchable diverging element is configured to switch to a second state in the concave mode and increase the angular magnitude of the light converged by the first switchable converging element, wherein the second switchable converging element is arranged at a third distance from the projection portion, and the second switchable diverging element is arranged at a fourth distance from the projection portion, wherein the first distance, the second distance, the third distance, and the fourth distance are configured such that the convergence point of the light in the concave mode and the convergence point of the light in the peripheral mode substantially coincide with each other. 15. The optical system of any one of claims 1 to 14, wherein the foveal portion is configured to converge the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina, wherein the at least one convergence point is located on or near the pupil of the eye on a convergence arc, wherein the corresponding positioning of each convergence point on the convergence arc corresponds to a rotation angle of the eye about an axis of rotation of the eye, the convergence arc having a center of curvature substantially coinciding with the axis of rotation of the eye. 16. The optical system of claim 15, wherein the optical system is configured to move the at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably substantially coincides with the macula of the retina, and more preferably substantially coincides with one of the perifoveal region, parafoveal region, fovea, avascular region of the fovea, fovea minor, and macula of the retina. 17. The optical system of claim 16, further comprising a steering portion configured to move the at least one convergence point about the convergence arc in response to rotation of the eye about an eye rotation axis, the arc having a center of curvature substantially coinciding with the eye rotation axis. 18. The optical system of claim 17, wherein the steering portion is configured to move the projection portion.19. The optical system of claim 17 or claim 18, wherein the steering portion includes at least one optical element for steering light from the projection portion toward the central concave portion. 20. The optical system of claim 19, wherein the at least one optical element includes a steerable mirror. 21. The optical system of claim 19 or claim 20, wherein the at least one optical element includes a plurality of switchable lenses, each switchable lens having a corresponding orientation and configured to direct light from the projection portion onto different portions of the central concave portion, such that the central concave portion converges the light to different convergence points on the convergence arc. 22. The optical system of any one of claims 19 to 21, wherein the at least one optical element includes a phase modulator with controllable phase modulation as described in claim 2 / 4 (CN 121219620 A). 23. The optical system of any one of claims 1 to 15, wherein the projection portion is configured to project a plurality of light beams, each light beam encoding a corresponding portion of an image, and the concave portion is configured to converge each light beam to a corresponding convergence point among a plurality of convergence points, the positioning of each convergence point on the convergence arc corresponding to a corresponding rotation angle of the eye about the eye's rotation axis, such that light converged at adjacent convergence points encodes adjacent portions of the image. 24. The optical system of any one of claims 15 to 23, wherein the at least one convergence point is located on a convergence surface substantially parallel to the pupil of the eye and includes the convergence arc. 25. The optical system of any one of claims 1 to 24, wherein the projection portion is configured to project a first light encoding a first image onto a first eye of the user and a second light encoding a second image onto a second eye of the user, and the foveal portion includes two foveal members, each corresponding to a respective eye of the user, each foveal member being configured to switch between a foveal pattern projecting light onto a central region of the retina of the corresponding eye and a peripheral pattern projecting light onto a peripheral region of the retina of the corresponding eye, the peripheral region being larger than and encompassing the central region. 26. The optical system of any one of claims 1 to 25, wherein the projection portion includes a display and a focusing element, the display being configured to emit an image to be displayed with substantially collimated light, the focusing element being used to converge the light toward the foveal portion, wherein, optionally, the projection portion includes a spatial filter for receiving the converged light.27. The optical system of any one of claims 1 to 26, wherein the projection portion comprises a coherent light source, preferably a laser, the coherent light source being configured to emit substantially coherent light, wherein the projection portion optionally comprises at least one of: a collimating element configured to collimate the substantially coherent light, a phase modulator configured to encode the image by modulating the substantially coherent light, and a laser beam scanning system comprising at least one steerable mirror. 28. The optical system of any one of claims 1 to 27, wherein the projection portion comprises a light field display configured to project light encoding a plurality of element images into a three-dimensional light field, each element image forming a portion of the image. 29. The optical system of any one of claims 1 to 27, wherein the projection portion comprises a phase modulator configured to project light encoding a holographic image. 30. A projection device comprising an optical system according to any one of claims 1 to 29 and an eye tracker for determining a rotation angle of the pupil of an eye, wherein, optionally, the eye tracker includes at least one camera for capturing an image of the eye, the eye tracker being configured to determine a rotation angle of the pupil of the eye based on the image, the rotation angle of the pupil being used to determine the positioning of the at least one convergence point on the convergence arc, optionally, the eye tracker being configured to determine the focal length of the lens of the eye, and optionally, the eye tracker including at least one light source for illuminating the eye. Claims 3 / 4 pages 4 CN 121219620 A 31. A method for projecting an image near the eye into a user's eye, the optical system comprising: projecting light encoding the image, and controlling at least one switchable optical element to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the at least one switchable optical element projects light encoding the image onto a central region of the retina of the eye, and in the peripheral mode, the at least one switchable optical element projects light encoding the image onto a peripheral region of the retina, the peripheral region being larger than and encompassing the central region. Claims 4 / 4 pages 5 CN 121219620 A Projection Device Technical Field

[0001] Exemplary aspects of this document relate to near-eye projection of an image into a user's eye, and more particularly to optical systems, projection devices, and methods. Background

[0002] Optical systems for projecting light into a user's eyes (or both eyes) are used in extended reality (XR) settings,Such as augmented reality, virtual reality, or mixed reality. These optical systems are used to form virtual images that can be combined with light incident from the environment viewed by the user (in the case of augmented reality or mixed reality), or can recreate the virtual environment viewed by the user (in the case of virtual reality).

[0003] Typically, the elements of the optical system are placed near the user's eyes (i.e., these elements are optical systems for near-eye projection of images), so that images can be projected with more compact optical systems, which helps to improve portability and user convenience. For example, the elements of these optical systems can be part of a device worn or held by the user (such as a head-mounted display).

[0004] In order to provide an immersive user experience, the virtual image projected onto the user's eyes should be adapted to the user's current field of view. In particular, the light projected onto the user's eyes reaches a specific part of the retina, which in turn determines how the user will see the virtual image, such that the orientation of the projected light, in particular the angle at which the light enters the user's eyes, should be adapted to the user.

[0005] Therefore, there is a need for an optical system and method that enhances the user experience by improving the adaptability of the virtual image projected onto the user's eyes, and in particular, provides a more immersive user experience.

[0006] In particular, a virtual image can be adapted to the user's field of view by projecting light such that it converges at a point in the eye that coincides with the eye's center of rotation, thus allowing the image to always be centered on the retina. However, since the angle of incidence of light is limited by the pupil of the eye, this results in light being projected onto only a small portion of the retina. This, in turn, limits the size of the virtual image seen by the user.

[0007] Therefore, there is a need for an optical system and method that has improved adaptability to the user's field of view while allowing light to be projected onto a wider portion of the retina.

[0008] Furthermore, optical systems have limited resources in terms of size, battery life, and computational resources. Therefore, there is a need for an optical system and method that improves the user experience while improving compactness and suppressing any increase in required resources.

[0009] According to a first example aspect, an optical system is provided for projecting an image near the eye into a user's eye. The optical system includes a projection portion and a focusing portion, wherein the projection portion is configured to project light that encodes the image, and the focusing portion is configured to converge the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina. The at least one convergence point is located on or near the pupil of the eye on a convergence arc, wherein the corresponding location of each convergence point on the convergence arc corresponds to an angle of rotation of the eye about an axis of rotation of the eye, and the convergence arc has a center of curvature substantially coinciding with the axis of rotation of the eye.

[0010] By converging the projected light to a point located on the convergence arc, compared to converging the light near the center of rotation of the eye…In the case of a point, the projected light can enter at a larger angle of incidence, thus allowing a larger image to be displayed to the user, and the projected light can enter the eye across an angular range that can remain centered on the retina, thus avoiding the so-called window effect.

[0011] Therefore, the virtual image projected onto the user's eye better adapts to the user's field of vision, thereby improving the user experience.

[0012] Preferably, the distance between the center of curvature and the center of eye rotation is 1.5 cm or less, more preferably 1 cm or less, and most preferably 0.5 cm or less.

[0013] Reducing the distance between the center of curvature of the converging arc and the center of eye rotation allows for an increase in the field of vision provided by the optical system.

[0014] Preferably, the distance between the arc and the pupil remains substantially constant regardless of the eye's rotation angle.

[0015] Therefore, regardless of the eye's rotation angle (i.e., the pupil's orientation), the range of incident angles of the projected light entering the eye can remain substantially constant, and the image can be projected onto the same area of ​​the retina, allowing the user to see a virtual image of substantially constant size.

[0016] Preferably, the optical system is configured to move at least one convergence point such that the central portion of the image substantially coincides with the central region of the eye's retina, preferably substantially coincides with the macula of the retina, more preferably substantially coincides with one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea minor, and umbo of the retina.

[0017] By moving the convergence point along the convergence arc, light can continue to enter as the eye rotates, while avoiding an increase in the energy required to project light into the eye (relative to the case where light converges to multiple different convergence points corresponding to different rotation angles of the eye).

[0018] Preferably, the optical system further includes a steering portion configured to move at least one convergence point around a convergence arc in response to rotation of the eye about its rotation axis, the arc having a center of curvature substantially coinciding with the eye's rotation axis.

[0019] Preferably, the steering portion is configured to move at least one element of the projection portion and the focusing portion.

[0020] Preferably, the steering portion includes at least one optical element for steering light from the projection portion toward the focusing portion.

[0021] Preferably, at least one optical element includes a steerable mirror.

[0022] Preferably, at least one optical element includes a plurality of switchable lenses, each switchable lens having a corresponding orientation and configured to direct light from the projection portion onto different portions of the focusing portion, such that the focusing portion converges light to different convergence points on the convergence arc.

[0023] Therefore, the positioning of the convergence point light can be changed by controlling the state of each switchable lens.

[0024] Preferably, at least one optical element includes an element with controllable phase modulation.

[0025] Preferably, the projection portion is configured to project a plurality of lights, each light encoding a corresponding portion of the image, and the focusing portion is configured to converge each light to a corresponding convergence point among a plurality of convergence points, the positioning of each convergence point on the convergence arc corresponding to a corresponding rotation angle of the eye about the eye's rotation axis, such that light converging at adjacent convergence points encodes adjacent portions of the image.

[0026] Thus, depending on the rotation angle of the eye, light focused at different convergence points will overlap with the pupil of the eye and enter the eye to form a (virtual) image on the retina.

[0027] By projecting light that converges simultaneously at multiple convergence points (which can be referred to as a static window), the optical system can be simplified, thus becoming more compact and requiring fewer resources, for example by omitting elements for moving the convergence points or for tracking the eye's position. Specification 2 / 29 pages 7 CN 121219620 A

[0028] Preferably, at least one convergence point is located on a convergence surface that is substantially parallel to the pupil of the eye and includes a convergence arc.

[0029] The convergence surface may have a center of curvature that substantially coincides with the center of rotation of the eye. The convergence surface may substantially coincide with the surface of a sphere approximating the eye, or the convergence surface may be a surface substantially parallel to the surface of the sphere, having a larger or smaller radius of curvature, having a center of curvature displaced relative to the center of the sphere, or both, as explained above in the case of a convergence arc.

[0030] The optical system may be configured to move at least one convergence point on the convergence surface along two dimensions on the surface, or the optical system may be configured to project light onto a plurality of convergence points on the convergence surface (not necessarily simultaneously), wherein the convergence points are spaced apart (or distributed) from each other along two dimensions on the convergence surface.

[0031] Thus, even when the eye rotates about two axes of rotation of the eye (or about the center of rotation of the eye), the projected light can continue to enter the eye.

[0032] Preferably, the projection portion is configured to project a first light encoding a first image onto a user's first eye and a second light encoding a second image onto a user's second eye, and the focusing portion is configured to: converge the first light to at least one first convergence point so that the projected light enters the first eye and forms an image on the retina of the first eye, and converge the second light to at least one second convergence point so that the projected light enters the second eye and forms an image on the retina of the second eye.

[0033] Therefore, the optical system can project a stereoscopic image, thereby giving the user an impression of depth in a virtual image.

[0034] Preferably, the projection portion includes a first projection element configured to project a first light and a second projection element configured to project a second light, and / or the focusing portion includes a first focusing element and a second focusing element, the first focusing element being configured to converge the first light to at least one first convergence point, and the second focusing element being configured to converge the second light to at least one second convergence point.

[0035] Thus, each component of the projection portion and / or the focusing portion can be controlled individually to project an image into the user's eye.

[0036] Preferably, the projection portion includes a display configured to emit an image to be displayed, and a focusing element for converging light toward the focusing portion, wherein, optionally, the projection portion includes a spatial filter for receiving the converged light.

[0037] Preferably, the projection portion includes a coherent light source, preferably a laser, configured to emit substantially coherent light, wherein the projection portion optionally includes at least one of the following: a collimating element configured to collimate substantially coherent light, a phase modulator configured to encode an image by modulating substantially coherent light, and a laser beam scanning system including at least one steerable mirror.

[0038] Preferably, the optical system further includes a foveal portion configured to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the foveal portion is configured to project light from the projection portion onto a central region of the retina of the eye, and in the peripheral mode, the foveal portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and encompassing the central region.

[0039] Since the central region of the retina has higher sensitivity, the image projected in the foveal mode can have a higher resolution than the image projected in the peripheral mode without causing a significant change in image quality perceived by the user.

[0040] Therefore, the resource requirements of the optical system (e.g., for resolution, bandwidth, optical power, and computation) can be reduced while avoiding a significant impact on the quality of the projected image. Specification 3 / 29 pages 8 CN 121219620 A

[0041] Preferably, the projection portion in the optical system including the foveal portion is configured to project light as substantially collimated light.

[0042] Preferably, the concave portion is configured to increase the angular magnitude of light incident on the eye in the peripheral mode.

[0043] In other words, the angular magnitude of light projected onto the eye is larger in the peripheral mode than in the concave mode (i.e., in the peripheral mode, light enters the pupil from a wider angular range).

[0044] Preferably, the concave portion is configured to increase the focal length of light incident on the eye in the concave mode.

[0045] Preferably, the concave portion includes at least one focusing element having a controllable focal length.

[0046] Preferably, the concave portion includes at least one switchable optical element, and optionally, each of the at least one switchable element is configured to allow light to pass through when switched to a first state without substantially changing the angular size of the light.

[0047] Preferably, the at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length different from the first focal length.

[0048] Preferably, the first switchable element is arranged at a first distance from the display, and the second switchable element is arranged at a second distance from the display.

[0049] Preferably, the difference between the first distance and the second distance is such that the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

[0050] Preferably, in the concave mode, the first switchable element is configured to switch to the second state and the second switchable element is configured to switch to the first state, while in the peripheral mode, the first switchable element is configured to switch to the first state and the second switchable element is configured to switch to the second state.

[0051] Preferably, at least one switchable optical element includes a first switchable diverging element, a first switchable converging element, and a second switchable converging element, wherein the first switchable diverging element is arranged at a first distance from the projection portion, and the first switchable converging element is arranged at a second distance from the projection portion, the second distance being greater than the first distance.

[0052] Preferably, in the foveal mode, the first switchable diverging element is configured to switch to a first state, the first switchable converging element is configured to switch to a first state, and the second switchable converging element is configured to switch to a second state, such that light from the projection portion is converged by the second switchable converging element to form an image on the central region of the retina; and in the peripheral mode, the first switchable diverging element is configured to switch to a second state, the first switchable converging element is configured to switch to a second state, and the second switchable converging element is configured to switch to a first state, such that light from the projection portion is diverged by the first switchable diverging element onto the first switchable converging element and converged by the first switchable converging element to form an image on the peripheral region of the retina.

[0053] Preferably, the second switchable converging element is arranged at a third distance from the projection portion such that the convergence point of the first switchable converging element and the convergence point of the second switchable converging element substantially coincide with each other.

[0054] Preferably, at least one switchable optical element further includes a second switchable diverging element, wherein the second switchable diverging element is configured to switch to a second state in the concave mode and increase the angular size of the light converged by the first switchable converging element, wherein the second switchable converging element is arranged at a third distance from the projection portion, and the second switchable diverging element is arranged at a fourth distance from the projection portion, wherein the first distance, the second distance, the third distance, and the fourth distance are...The projection portion is configured such that the convergence point of light in the concave mode and the convergence point of light in the peripheral mode substantially coincide with each other.

[0055] Preferably, the projection portion includes a light field display configured to project light encoding multiple element images into a three-dimensional light field, each element image forming a part of an image.

[0056] Preferably, the projection portion includes a phase modulator configured to project light encoding the holographic image. Specification 4 / 29 pages 9 CN 121219620 A

[0057] According to a second example aspect, an optical system for projecting an image near the eye into a user's eye is provided. The optical system includes: a projection portion configured to project light encoding an image, and a foveal portion including at least one switchable optical element, the at least one switchable optical element being controllable to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the foveal portion is configured to project light from the projection portion onto a central region of the retina of the eye, and in the peripheral mode, the foveal portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and encompassing the central region.

[0058] Using at least one switchable optical element allows for the differentiation of an image projected onto a central region of the retina (e.g., the fovea) with higher sensitivity and an image projected onto a peripheral region of the retina without the need for additional elements.

[0059] This allows an image with higher resolution to be projected onto the central region of the retina, which ensures a higher quality image perceived by the user. On the other hand, images with lower resolution can be projected onto the peripheral area of ​​the retina without affecting the quality of the image perceived by the user.

[0060] Thus, the optical system improves the user experience while improving compactness and suppressing any increase in required resources.

[0061] Preferably, the projection portion is configured to project light as substantially collimated light.

[0062] Thus, the focal length of the light projected into the eye can be determined by the focal length of the switchable optical elements.

[0063] Preferably, each of at least one switchable element is configured to allow light to pass through when switched to a first state without substantially changing the angular size of the light.

[0064] Preferably, the foveal portion is configured to increase the angular size of the light incident on the eye in the peripheral mode.

[0065] Preferably, the foveal portion is configured to increase the focal length of the light incident on the eye in the foveal mode.

[0066] Preferably, the foveal portion includes at least one focusing element with a controllable focal length.

[0067] Preferably, at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length different from the first focal length.

[0068] Preferably, the first switchable element is arranged at a first distance from the display, and the second switchable element is arranged at a second distance from the display.

[0069] Preferably, the difference between the first distance and the second distance is such that the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

[0070] For example, the distance separating the second switchable element and the first switchable element (i.e., the difference between the second distance and the first distance) can be set based on the difference between the first focal length and the second focal length.

[0071] Therefore, regardless of whether the focal length of the light is changed by the first switchable element or by the second switchable element, the light substantially converges at the same point.

[0072] Preferably, in the concave mode, the first switchable element is configured to switch to the second state and the second switchable element is configured to switch to the first state, while in the peripheral mode, the first switchable element is configured to switch to the first state and the second switchable element is configured to switch to the second state.

[0073] Preferably, at least one switchable optical element includes a first switchable diverging element, a first switchable converging element and a second switchable converging element, wherein the first switchable diverging element is arranged at a first distance from the projection portion and the first switchable converging element is arranged at a second distance from the projection portion, the second distance being greater than the first distance.

[0074] Preferably, in the foveal mode, the first switchable diverging element is configured to switch to a first state, the first switchable converging element is configured to switch to a first state, and the second switchable converging element is configured to switch to a second state, such that light from the projection portion is converged by the second switchable converging element to form an image on the central region of the retina (see page 5 / 29 of the specification, 10 CN 121219620 A). In the peripheral mode, the first switchable diverging element is configured to switch to a second state, the first switchable converging element is configured to switch to a second state, and the second switchable converging element is configured to switch to a first state, such that light from the projection portion is diverged by the first switchable diverging element onto the first switchable converging element and converged by the first switchable diverging element to form an image on the peripheral region of the retina.

[0075] Preferably, the second switchable converging element is arranged at a third distance from the projection portion, such that the convergence point of the first switchable converging element and the convergence point of the second switchable converging element substantially coincide with each other.

[0076] For example, the distance separating the second switchable converging element and the first switchable converging element (i.e., the difference between the third distance and the first distance) can be set based on the difference between the focal length of the second switchable converging element and the focal length of the first switchable converging element.

[0077] Preferably, at least one switchable optical element further includes a second switchable diverging element, wherein the second switchable diverging element is configured to switch to a second state in the concave mode, and to increase the divergence caused by the first switchable converging element.The angle of the focused light, wherein the second switchable converging element is arranged at a third distance from the projection portion, and the second switchable diverging element is arranged at a fourth distance from the projection portion, wherein the first distance, the second distance, the third distance, and the fourth distance are configured such that the convergence point of the light in the foveal mode and the convergence point of the light in the peripheral mode substantially coincide with each other.

[0078] Preferably, the foveal portion is configured to converge light to at least one convergence point so that the projected light enters the eye and forms an image on the retina, wherein at least one convergence point is located on or near the pupil of the eye on a convergence arc, wherein the corresponding positioning of each convergence point on the convergence arc corresponds to the rotation angle of the eye about the eye rotation axis, the convergence arc having a center of curvature substantially coincident with the eye rotation axis.

[0079] Preferably, the optical system is configured to move at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably substantially coincides with the macula of the retina, more preferably substantially coincides with one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea minor, and fovea fovea.

[0080] Preferably, the optical system further includes a steering portion configured to move at least one convergence point about a convergence arc having a center of curvature substantially coincident with the eye's rotation axis in response to rotation of the eye about the eye's rotation axis.

[0081] Preferably, the steering portion is configured to move at least one element of the projection portion and the foveal portion.

[0082] Preferably, the steering portion includes at least one optical element for steering light from the projection portion toward the foveal portion.

[0083] Preferably, at least one optical element includes a steerable mirror.

[0084] Preferably, at least one optical element includes a plurality of switchable lenses, each switchable lens having a corresponding orientation and configured to direct light from the projection portion onto different portions of the concave portion, such that the concave portion converges the light to different convergence points on a convergence arc.

[0085] Preferably, at least one optical element includes a phase modulator with controllable phase modulation.

[0086] Preferably, the projection portion is configured to project a plurality of lights, each light encoding a corresponding portion of an image, and the concave portion is configured to converge each light to a corresponding convergence point among a plurality of convergence points, the positioning of each convergence point on the convergence arc corresponding to a corresponding rotation angle of the eye about the eye's rotation axis, such that light converging at adjacent convergence points encodes adjacent portions of the image.

[0087] Preferably, at least one convergence point is located on a convergence surface that is substantially parallel to the pupil of the eye and includes a convergence arc.

[0088] Preferably, the plurality of convergence points are distributed along two dimensions of the convergence surface. Instruction manual, page 6 / 29, 11 CN 121219620 A

[0089] Preferably, the optical system is configured to move at least one convergence point along two dimensions of the converging surface.

[0090] Preferably, the projection portion is configured to project a first light encoding a first image onto a user's first eye and a second light encoding a second image onto a user's second eye, and the concave portion includes two concave members, each corresponding to a user's respective eye, each concave member being configured to switch between a concave pattern projecting light onto a central region of the retina of the corresponding eye and a peripheral pattern projecting light onto a peripheral region of the retina of the corresponding eye, the peripheral region being larger than and encompassing the central region.

[0091] Preferably, the projection portion includes a first projection member configured to project the first light and a second projection member configured to project the second light.

[0092] Preferably, the projection portion includes a display configured to emit an image to be displayed with substantially collimated light, and a focusing element for converging the light toward the concave portion, wherein, optionally, the projection portion includes a spatial filter for receiving the converged light.

[0093] Preferably, the projection portion includes a coherent light source, preferably a laser, configured to emit substantially coherent light, wherein the projection portion optionally includes at least one of: a collimating element configured to collimate substantially coherent light, a phase modulator configured to encode an image by modulating substantially coherent light, and a laser beam scanning system including at least one steerable mirror.

[0094] Preferably, the projection portion includes a light field display configured to project light encoding a plurality of element images into a three-dimensional light field, each element image forming a portion of an image.

[0095] Preferably, in a foveal mode, the foveal portion is configured to project light encoding a first plurality of element images from the light field display onto a central region of the retina of the eye, and in a peripheral mode, the foveal portion is configured to project light encoding a second plurality of element images from the light field display onto a peripheral region of the retina, wherein the second plurality of element images are different from the first plurality of element images.

[0096] Preferably, the projection portion includes a phase modulator configured to project light encoding the holographic image.

[0097] Preferably, in a foveal mode, the foveal portion is configured to project light encoding a first portion of the holographic image onto a central region of the retina of the eye, and in a peripheral mode, the foveal portion is configured to project light encoding a second portion of the holographic image onto a peripheral region of the retina, wherein the first and second portions of the holographic image are different from each other.

[0098] According to a third exemplary aspect, a projection device is provided, comprising the components according to the first exemplary aspect herein or according to...The optical system of the second example aspect of this document, and at least one of the following: one or more eye trackers for determining the rotation angle of the pupil of the eye, and a computing unit.

[0099] Optionally, each eye tracker includes at least one camera for capturing an image of the eye, the eye tracker being configured to determine the rotation angle of the pupil of the eye based on the image, the rotation angle of the pupil being used to determine the location of at least one convergence point on a convergence arc.

[0100] By tracking the rotation angle of the pupil, the optical system can determine the location on the convergence arc (or convergence surface) where light will be allowed to enter the eye. This can be used to determine the location on the convergence arc to which the convergence point should be moved (e.g., in the case of a dynamic view), and can be used to determine which convergence points the light should converge to.

[0101] In the case of a static view, the optical system can determine, based on the rotation angle of the pupil, a subset of one or more convergence points to which the light encoding the image will converge (because the light will enter the eye), and can selectively project only the light that will converge to the convergence points in the subset, thereby reducing the energy required to project the image. In other words, the optical system specification 7 / 29 page 12 CN 121219620 A can interrupt the projection of light that converges at a convergence point outside the subset.

[0102] Optionally, the eye tracker is configured to determine the focal length of the eye's lens.

[0103] For example, the tracker can determine the instantaneous focal length at recurring moments (e.g., periodically, such as every millisecond) or when a predetermined trigger (such as eye movement) is detected.

[0104] Thus, the image projected onto the eye can be adapted to the eye's focal length. This allows, for example, the introduction of artificial (or digital) blur into the image to improve the perceived depth of elements in a virtual image.

[0105] Optionally, the eye tracker includes at least one light source for illuminating the eye.

[0106] Preferably, the light source cannot be detected by the eye to avoid affecting the image projected onto the eye.

[0107] Preferably, the projection device is held or worn by the user.

[0108] Preferably, the projection device includes a head-mounted device to be mounted on the user's head.

[0109] Preferably, at least one element of the optical system is located within a housing on the head-mounted device.

[0110] Preferably, the computing unit is configured to obtain a value indicating the rotation angle of the eye and generate one or more first control signals for causing the projection portion to project light for rendering the image based on the indicated eye rotation angle.

[0111] Preferably, the computing unit is configured to obtain the value from one or more eye trackers.

[0112] Preferably, the computing unit is configured to determine a position on the convergence arc corresponding to the indicated eye rotation angle based on the obtained value and generate one or more signals for controlling at least one of the projection portion and the focusing portion.A plurality of second control signals, the one or more second control signals causing light to converge at one or more convergence points corresponding to the determined position.

[0113] Preferably, the computing unit is configured to determine the direction in which the pupil is oriented within the scene based on the obtained value, and render a portion of the scene corresponding to that direction.

[0114] Preferably, the optical system includes a computing unit configured to: generate one or more first control signals to cause the projection portion to project light of a first portion of the rendered image onto the central region of the retina, and to switch the foveal portion to a foveal mode; and generate one or more second control signals to cause the projection portion to project light of a second portion of the rendered image onto the peripheral region of the retina, and to switch the foveal portion to a peripheral mode.

[0115] Preferably, the computing unit is configured to cause the projection portion and the foveal portion to switch substantially simultaneously by generating one or more first control signals and one or more second control signals, the projection portion switching between projecting light to render the first portion and projecting light to render the second portion, and the foveal portion switching between a foveal mode and a peripheral mode.

[0116] According to a fourth example aspect, a method for projecting an image near the eye into a user's eye is provided, the method comprising: projecting light encoding the image, and converging the light to at least one convergence point such that the projected light enters the eye and forms an image on the retina, wherein the at least one convergence point is located on or near a convergence arc of the eye, wherein the corresponding positioning of each convergence point on the convergence arc corresponds to a rotation angle of the eye about an axis of rotation of the eye, the convergence arc having a center of curvature substantially coinciding with the axis of rotation of the eye.

[0117] According to a fifth example aspect, a method for projecting an image near the eye into a user's eye is provided, the optical system comprising: projecting light encoding the image, and controlling at least one switchable optical element to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the at least one switchable optical element projects the light encoding the image onto a central region of the retina of the eye, and in the peripheral mode, the at least one switchable optical element projects the light encoding the image onto a peripheral region of the retina, the peripheral region being larger than and including the central region. Specification page 8 / 29, CN 121219620 A

[0118] Preferably, the optical system according to the first example aspect or the second example aspect, the projection device according to the third example aspect, and the method according to the fourth example aspect or the fifth example aspect are used to project extended reality XR images, and more preferably, one of augmented reality (AR) images, virtual reality (VR) images, or mixed reality (MR) images is projected.

[0119] Preferably, the XR image includes at least one virtual element representing digital information.

[0120] Preferably, each virtual element is either superimposed on the user's physical environment, captured by the imaging device or seen by the user, or forms part of the virtual environment.

[0121] For simplicity, when the eye is approximated as a sphere, the center of the sphere can be defined as the center of eye rotation, and thus the rotation of the eye can be defined as rotation about the center of the sphere (i.e., the eye rotation center). Rotation about the eye rotation center can also be defined as rotation about two different eye rotation axes that intersect at the eye rotation center and are in a plane parallel to the pupil of the eye. The eye rotation axes can be two axes defined according to Listing's law (e.g., the vertical and horizontal axes of the eye), i.e., two axes located in the Listing plane of the eye.

[0122] In some aspects herein, light projected into the eye converges to at least one convergence point, which in this case can be a point in front of the eye rotation center and substantially along the pupil axis or concave axis of the eye (i.e., a point that may coincide with the pupil of the eye, such that light converging to the convergence point enters the eye and is projected onto the retina).

[0123] In some aspects of this document, the convergence point is located on the convergence arc. The center of curvature of the convergence arc can be defined as an axis coinciding with one of the rotation axes of the eye, and the radius of curvature corresponds to the radius of a sphere approximating the eye. However, it should be understood that there may be a difference between the center of curvature of the convergence arc and the center of rotation of the eye (i.e., the center of the sphere), a difference between the radius of curvature of the convergence arc and the radius of the sphere (preferably, a difference of 1 cm or less), or both of these differences may exist, which may be caused by variations in the size of the human eye.

[0124] If it does not coincide with the center of the sphere, the center of curvature of the convergence arc may be between the pupil and the center of rotation of the eye (e.g., in the volume around the pupil axis or the central fovea axis of the eye, such as in a cone with the iris of the eye as its base and the center of rotation of the eye as its apex), or between the center of rotation of the eye and the retina of the eye (e.g., in the volume around the pupil axis or the central fovea axis of the eye, such as in a cone with the central region of the retina as its base and the center of rotation of the eye as its apex). If the converging radius of the converging arc is greater than the radius of the sphere, then the converging arc will be outside the eye and in front of the pupil.

[0125] The term "eye rotation angle" can also be defined as the orientation of the eye (or pupil), the rotation angle of the pupil, or the position of the pupil. The rotation angle can be defined relative to a default axis that corresponds to the orientation of the pupil when the eye is considered to be stationary.

[0126] In some aspects herein, the optical system moves at least one convergence point based on the eye rotation angle. For example, this can be done so that light converges to a point substantially coinciding with the center of the pupil of the eye. This can be referred to as a dynamic window because the convergence point moves dynamically as the eye rotates.

[0127] In some aspects of this document, one or more steering portions may be provided to move at least one element of the projection portion and the focusing portion. The steering portion (or each steering portion) may include an electromechanical device, such as an actuator, motor, etc., configured to move an element of the projection portion, an element of the focusing portion, or an element of both the projection portion and the focusing portion.

[0128] In some aspects of this document, the steering portion (or each steering portion) may include at least one optical element. The optical element of the steering portion may be positioned along the optical path of light between the projection portion and the focusing portion. The optical element of the steering portion may be movable to change the optical path of light toward the focusing portion, thereby causing a movement of the convergence point. Specification 9 / 29 pages 14 CN 121219620 A

[0129] In some aspects of this document, at least one optical element may have controllable phase modulation, which may be, for example, a phase or amplitude spatial light modulator (SLM), a lens with controllable phase modulation, etc.

[0130] In some aspects of this document, at least one convergence point is located on a convergence surface. In other words, the optical system can move the convergence point along two dimensions of the convergence surface, or the optical system can be configured to converge light to convergence points distributed along two dimensions of the convergence surface. These two dimensions will be understood as different (e.g., orthogonal) dimensions.

[0131] In some aspects of this document, multiple beams of light, each encoding a corresponding portion of an image, converge to a corresponding convergence point among multiple convergence points. These convergence points may be separated (or distributed) at regular intervals along a convergence arc or along a convergence surface, or the distance between adjacent convergence points may vary along an arc or surface, for example by providing greater density at a portion of an arc corresponding to the angle of rotation when the eye is stationary. In some cases, adjacent convergence points may be close enough to each other that light converging at two or more convergence points enters the eye simultaneously, thereby projecting different portions of the image onto different portions of the retina (which may overlap).

[0132] In some aspects of this document, one or more eye trackers may be used to track one or both of a user's eyes. Each eye tracker can be a camera-based tracker, a tracker using physiological sensors (e.g., ECG-based signals), or other sensing methods such as LiDAR. Each eye tracker can track the pupil or a protrusion formed by the cornea. Each tracker can be a camera-based tracker, in which case the camera can be a camera that captures images in the visible wavelength range (e.g., a red-green-blue RGB camera) or a camera that captures images in the infrared (IR) range (e.g., an IR camera, a near-IR camera). Each tracker can include at least one light source, which can be IR light, or visible (e.g., white) light that is off-center and not detected by the eye, or light with a brightness below a threshold to avoid or reduce interference with the projection onto the eye.Any effect of the image in the eye.

[0133] In some aspects of this document, the optical system may project an image into the user's two eyes, for example by including separate portions for each eye (e.g., separate projection portions, focusing portions, steering portions, and / or eye trackers), wherein the portions(one or more) for each eye may be individually controlled.

[0134] In some aspects of this document, a display and focusing element may be used. For example, this may be a self-emitting display that emits light. The display (e.g., a self-emitting display) may emit collimated light or may include collimating optics to collimate the light, or the light emitted by the display may be divergent, for example, if the focusing element is configured, due to its position and / or focal length, to receive divergent light and converge it toward the focusing portion. The spatial filter may be, for example, a pinhole (or Fourier) filter.

[0135] In some aspects of this document, a coherent light source may be used. A coherent light source can be any light source that emits light having at least a predetermined level of coherence, such as having coherence sufficient to be modulated by a phase modulator (e.g., an SLM, such as a device including a steerable micromirror array, such as a digital light processing (DLP) or other types of digital micromirror devices (DMDs). A coherent light source can include a laser.

[0136] In the case where the coherent light source is a laser, the laser can include one or more laser diodes, wherein each laser diode can be configured to emit light of a given wavelength (e.g., a diode emitting red, green, and blue wavelengths, or an infrared wavelength). Preferably, the laser can include multiple diodes (e.g., at least red, green, and blue diodes) to emit light substantially across the visible spectrum.

[0137] In some aspects herein, a light field display can be used to project light that encodes an element image. Each element image of the light field can have an associated perceptual depth (e.g., corresponding to the distance between the retina and the focal point where the light associated with the element image converges), and light for each element image can be projected onto different portions of the retina. The portion formed by each element image can overlap with at least one portion formed by adjacent element images. Specification 10 / 29 pages 15 CN 121219620 A

[0138] In some aspects of this document, the optical system may include a concave portion and a projection portion having a light field display. In this case, the light field display may be configured to project light encoding different element images for a concave mode and for a peripheral mode.

[0139] In some aspects of this document, the optical system may include a concave portion and a projection portion including a phase modulator to project light encoding a holographic image. In this case, the phase modulator may project light encoding different holographic images for a concave mode and for a peripheral mode.

[0140] In some aspects of this document, at least one switchable optical element may be used. In this case, each switchable optical element may be, for example, a converging or diverging element configured to change the angular size of the light passing through. For example, each switchable optical element may be a converging / diverging lens, a mirror, a holographic polymer-dispersed liquid crystal HPDLC layer, an Alvarez lens, or a polarization-switching lens such as a polarization volume grating.

[0141] In some aspects of this document, a switchable optical element (hereinafter also referred to as SOE) may be defined as switching to a first state or a second state. The switchable optical element in the second state may have a different focal length than in the first state.

[0142] For example, the first state may correspond to a state in which the SOE allows light to pass through without substantially changing the angular size of the light, while in the second state, the SOE may (e.g., by converging or diverging the incident light, or by collimating the converging / diverging incident light) change the angular size of the light. In other words, in one of the states (e.g., the first state), the SOE may correspond to a neutral lens.

[0143] In another example, the SOE can change the angle of the incident light in both the first and second states.

[0144] In some aspects herein, the SOE can be switched to the first or second state by controlling the electrical signal received by the SOE (e.g., control voltage). The SOE can switch to the second state when it receives an electrical signal and switch to the first state (or vice versa) when it no longer receives an electrical signal. Thus, the first and second states can be defined as “off” and “on” states, or “default” and “active” states.

[0145] As used herein, the term “central region of the retina” can mean, for example, the macula of the retina, and preferably one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea minor, and fovea fovea.

[0146] As used herein, the term “viewing window” can mean the area in front of the projection device in which the user’s eye can be placed to view the image displayed by the projection device without distortion or loss of a portion of the displayed image.

[0147] As used herein, the term "field of view" may refer to the angular range of the image displayed to the user's eyes. Brief Description of the Drawings

[0148] Figures 1A and 1B show schematic diagrams of a conventional projection device; Figures 2A and 2B show schematic diagrams of another conventional projection device; Figures 3A and 3B show schematic diagrams of a projection device in an example embodiment; Figure 4 shows a schematic diagram of a projection device in an example embodiment; Figure 5 shows a computing unit for controlling an optical system in an example embodiment; Figures 6A and 6B show schematic diagrams of an optical system in an example embodiment; Figures 7A and 7B show schematic diagrams of an optical system in an example embodiment; Figures 8A and 8B show schematic diagrams of an optical system in an example embodiment;Figures 9A and 9B show schematic diagrams of the optical system in the example embodiment; Figures 10A and 10B show schematic diagrams of the optical system in the example embodiment; Figures 11A and 11B show schematic diagrams of the optical system in the example embodiment; Figures 12A and 12B show schematic diagrams of the optical system in the example embodiment; Figures 13A and 13B show schematic diagrams of the optical system using waveguides in the example embodiment; Figure 14A shows a schematic diagram of the peripheral image projected by the optical system in the example embodiment; Figure 14B shows a schematic diagram of the concave image projected by the optical system in the example embodiment; Figures 14C to 14D show schematic diagrams of the concave and peripheral images projected by the optical system in the example embodiment; Figures 15A and 15B show schematic diagrams of the optical system using the concave portion in the peripheral mode and the concave mode, respectively, in the example embodiment; Figures 16A and 16B show schematic diagrams of the optical system using the concave portion in the concave mode and the peripheral mode, respectively, in the example embodiment; Figures 17A and 17B show schematic diagrams of optical systems using concave portions in central and peripheral modes, respectively, in an example embodiment; Figures 18A and 18B show schematic diagrams of optical systems using concave portions in central and peripheral modes, respectively, in an example embodiment; Figures 19A and 19B show schematic diagrams of optical systems using concave portions in peripheral and central modes, respectively, in an example embodiment; Figures 20A and 20B show schematic diagrams of optical systems using concave portions in peripheral and central modes, respectively, in an example embodiment; Figures 21A and 21B show schematic diagrams of optical systems using concave portions in peripheral and central modes, respectively, in an example embodiment; Figures 22A and 22B show schematic diagrams of optical systems using concave portions in central and peripheral modes, respectively, in an example embodiment; Figures 23A and 23B show schematic diagrams of optical systems using concave portions in peripheral and central modes, respectively, in an example embodiment; Figure 24A shows a schematic diagram of an optical system using a light field display in an example embodiment; Figures 25A to 25E illustrate schematic diagrams of elemental images projected by an optical system using a light field display in an example embodiment. Detailed Description

[0149] For simplicity, the following description will refer to an eye rotating about an axis of rotation in two-dimensional space, which will be understood as an axis perpendicular to the plane shown in the figures and near the center of the sphere representing the eye. Therefore, the optical system described herein can adapt the image projected onto the eye to the movement of the eye along one of the eye's rotational axes (e.g.,For example, rotation about a vertical axis when the eye moves laterally / left and right, or rotation about a horizontal axis when the eye moves up and down.

[0150] However, it should be understood that the optical system described herein can make the image substantially adaptable to the movement of the eye along both axes of rotation of the eye (e.g., both the vertical and horizontal axes described above), such that the image can adapt to any movement of the eye. In this case, the rotation angle of the eye can be considered as the angle along the two axes of rotation of the eye. Therefore, the optical system described herein can thus be configured to converge light to at least one convergence point located on an arcuate surface including the converging arc described on page 12 / 29 of the specification, CN 121219620 A. In other words, the “converging surface” can be defined by two converging arcs in two different dimensions, each arc having a center of curvature substantially coincident with the rotation axis of the eye, and each arc being on or near the pupil of the eye. Therefore, the defined surface can be substantially parallel to the outer surface of the cornea, iris, or lens of the eye.

[0151] As explained below, the components of the optical system (e.g., eye tracker, computing unit, projection section, etc.) can be communicatively connected to each other. The connection can be the establishment of one or more suitable communication links. Each communication link can be, for example, a wireless communication link (e.g., Wi-Fi, Bluetooth, or Bluetooth Low Energy (BLE), Near Field Communication (NFC)) or a wired communication link, such as a serial communication link (e.g., I2C, SPI, RS232, RS422, RS432), a communication bus, etc. Each communication link may not be permanent.

[0152] Figures 1A and 1B illustrate schematic diagrams of a conventional projection device 1 for projecting an image into an eye. The projection device 1 may also be referred to as a near-eye display. The projection device 1 includes a projection section (not shown) configured to generate and modulate light such that the projected light encodes an image in the optical system 2. The optical system 2 includes one or more optical elements configured to converge the projected light onto an eye 6, such that an image 5 is formed on the retina of the eye 6. In Figure 1A, device 1 is shown converging light to a first convergence point 3a, which is located on a convergence plane 4 just outside the eye (although in some configurations the convergence plane 4 is located on or inside the surface of the eye). Projection device 1 also includes an eye tracker (not shown) that tracks the movement of the eye 6 about a rotation axis 7 of the eye 6.

[0153] Figure 1B shows device 1 in a second configuration. The eye 6 rotates about axis 7 to a new position. The eye tracker measures the position of the eye 6, and the optical system 2 dynamically changes to converge light to a second convergence point 3b, which is shifted along the convergence plane 4 from the first convergence point 3a. The position of the convergence point is selected to correspond to the position of the pupil of the eye 6, such that...Light enters the eye and image 5 is formed on the retina. The window E of device 1 is defined by the maximum displacement position of the convergence point 3 that device 1 can generate.

[0154] The above system provides a dynamic window in which only the portion of the window that enters the pupil is projected into the eye. This saves energy of the projection device 1 because light is not projected onto the point on the convergence plane that does not enter the pupil of the eye 6. However, as can be seen from the configuration shown in FIG1B, when the eye rotates away from the center of the convergence plane 4, image 5 is laterally shifted on the retina, such that the center C' of image 5 is not aligned with the center C of the retina. Therefore, the viewer of projection device 1 perceives the edge of the window (or the edge of the FOV) and is able to move the eye 6 so that the retina focuses on the edge of the window. The resulting effect is that the user perceives the projected virtual scene as being displayed through a window with edges. This is shown in FIG1C, which shows the image of projection device 1 as perceived by the user. When the user's eye 6 falls on the center of the window, the user's central vision, indicated by F1, is contained within the edge of the image, so that the user is unaware of any edges. When the user's eye 6 moves away from the periphery of the viewing window, the user's central vision, indicated by F1, makes the edges of the image perceptible, causing the user to generally perceive that they are viewing the image through a fixed window (also referred to herein as the window effect).

[0155] To overcome this problem, a projection device 1, such as that shown in Figures 2A and 2B, can be provided. The projection device 1 includes a projection portion (not shown) configured to project light that encodes an image in an optical system 2. The optical system 2 includes one or more optical elements configured to converge the projected light onto the eye 6, such that an image 5 is formed on the retina of the eye 6. Unlike the device shown in Figure 1A, the device 1 converges the light to a convergence point on or near the rotation axis 7 of the eye 6. The projection device 1 also includes an eye tracker (not shown) that tracks the movement of the eye 6 about the rotation axis 7 of the eye 6. Figure 2B shows the device 1 in a second configuration. The eye 6 rotates about the axis 7 to the new position shown in Figure 2B. An eye tracker measures the position of eye 6, and optical system 2 dynamically changes the rotation of the projected light to match the rotation of eye 6 around axis 7, so that light enters the eye and image 5 is formed on the retina. The viewing window E is defined by the maximum rotational position of the projected light that device 1 can produce.

[0156] Unlike the systems shown in Figures 1A and 1B, image 5 remains centered on the retina, thus eliminating the windowing effect. However, the image 5 formed on the retina is limited by the optical aperture of the pupil, and therefore smaller in size compared to the devices shown in Figures 1A and 1B, thereby limiting the field of view for the user at a given time.

[0157] Figures 3A and 3B illustrate schematic diagrams of a projection device 10 according to one or more embodiments. The projection device 10 includes a projection portion (not shown) configured to project light that encodes an image in an optical system 20. The optical system 20 includes one or more optical elements configured to converge the projected light onto an eye 6, such that an image 5 is formed on the retina of the eye 6. Unlike the devices shown in Figures 1A, 1B, 2A, and 2B, device 10 converges light to a point on a converging arc 41 located in front of the rotation axis 7 of the eye 6 (i.e., the arc lies between the rotation axis 7 and the optical system 20), and more specifically at or near the pupil of the eye 6 (inside or outside the eye), while the image still rotates about a rotation axis 40 that is close to or coincides with the rotation axis 7 of the eye. In other words, the distance between the point of convergence of light and the rotation axis 7 of the eye (and the image) is chosen to be approximately equal to the distance between the pupil or lens of the eye 6 and the rotation axis 7. For example, the distance between the convergence point and the rotation axis 7 (which may correspond to the radius of curvature of the convergence arc) may be between 1 cm and 2 cm.

[0158] The projection device 10 may also include an eye tracker (not shown) that tracks the movement of the eye 6 about the rotation axis 7 of the eye 6. FIG3B shows the device 10 in a second configuration. The eye 6 rotates about the axis 7 to the new position shown in FIG3B. The eye tracker measures the position of the eye 6, and the optical system 20 dynamically changes to rotate the image 5 to match the rotation of the eye 6 about the axis 7, such that light enters the eye and the image 5 is formed to be centered on the retina. The window E of the device 10 may correspond to the area defined by the maximum rotational position of the projected light that the device 10 is capable of producing.

[0159] Unlike the device 1 shown in FIG2A and FIG2B, the device 10 allows a wider field of view because the convergence point of the projected light is near the pupil, while still keeping the image centered on the retina of the eye 6 (i.e., focused on or near the central region of the retina) to avoid the window effect. Preferably, image 5 may be focused on or near the macula of the retina. More preferably, image 5 is focused on or near the area forming the macula, such as the perifoveal region, foveal periapical region, fovea, avascular fovea region, fovea minor region, or fovea fovea.

[0160] In three-dimensional space, the rotation axis 40 may be, for example, a vertical axis that allows image 5 to remain focused on the central region of the retina during lateral eye movement, or a horizontal axis that allows image 5 to remain focused on the central region of the retina as the eye moves up or down.

[0161] FIG4 illustrates a projection device 10 according to one or more embodiments. The projection device 10 includes an eye tracker 60, a computing unit 50, and an optical system 20. The projection device may also include any suitable power source, such as a battery or other AC orA DC power supply may be included, and / or a power connector may be included for connecting to an external AC or DC power supply. The eye tracker 60 may be any suitable eye tracker configured to track the positioning of a user's pupil. The eye tracker 60 is communicatively connected to the computing unit 50 (e.g., via a communication line or via a wireless communication link) and is configured to output a value to the computing unit 50 indicating the positioning of the pupil 6a of the user's eye 6. The computing unit 50 is configured to render a scene (i.e., a 2D or 3D image or video, which may include one or more virtual elements) and is configured to render the scene based on the position of the eye 6. More specifically, the computing unit 50 is configured to calculate which direction the pupil 6a is pointing within the scene and render that portion of the scene.

[0162] The eye tracking element 60 may, for example, include an IR (infrared) and / or RGB camera and an IR source for the eye. The IR source illuminates the eye to enable tracking of the eye in low-light conditions without being seen by the user (because the frequency of the light is outside the human visual spectrum). The eye-tracking element 60 can be mounted separately from the optical system 20 to the device 100, or one or more optical elements within the optical system 20 can be used to direct illumination light to the eye 6 and reflect reflected light back to the eye-tracking element 60. In some cases where both eyes are to be tracked, the same eye-tracking element 60 can be configured to track each eye, or separate eye-tracking elements can each track their respective eyes.

[0163] The projection device also includes the optical system 20. The optical system 20 includes a projection portion 21 configured to project light that encodes an image of the rendered scene generated by the computing unit 50. The optical system also includes a steering portion 22 configured to deflect light emitted from the optical system 20 around a rotation axis 40 based on the positioning of the pupil 6a indicated by the computing unit 50 (or, alternatively, based on direct communication with the eye tracker 60) (see Figures 3A and 3B). Finally, the optical system includes a focusing section 23 configured to converge the light rays encoding the image to a point on the converging arc 41. The projection section 21, the steering section 22, and the focusing section 23 can be formed as separate optical elements, or two or more of these elements can be included in a single optical element that performs two or more of the functions of elements 21, 22, and 23. Separate projection sections 21, steering elements 22, and focusing elements 23 can be provided for each eye, or the same element 21, the same element 22, and / or the same element 23 can be used to project an image into both eyes (e.g., sequentially in time or space). In this case, the image projected into each eye may be different.This is different. For example, offset can be used to provide a stereoscopic image, or a portion (e.g., half) of a larger image can be projected into each eye.

[0164] The computing unit 50 can control all elements, or it can provide a separate computing unit to control different optical components for each eye (i.e., the projection device includes a first eye tracker, a first computing unit, and / or a first optical system for the left eye, and a second eye tracker, a second computing unit, and / or a second optical system for the right eye).

[0165] In some embodiments, the projection portion 21 may include a light generating element and a light modulation element, the light modulation element modulating the light generated by the light generating element to encode the image.

[0166] The focusing element 23 may be an optical combiner that redirects and converges light so that an image is projected onto the eye while allowing ambient light to pass through, so that a virtual image formed by the optical system 20 is superimposed onto the real-world environment (augmented reality). In embodiments where the projection device is a pair of smart glasses, the optical combiner may be formed as part of one (or more) lenses of the smart glasses. In other embodiments, the focusing element 23 may be combined with an opaque lens such that light from the environment is not combined with the image, thus providing a completely virtual view (virtual reality).

[0167] In some embodiments, the focusing element 23 or optical combiner may be a curved transparent reflector or an off-axis concave (or parabolic) mirror. The projection portion 21 and the steering element 22 may be mounted in or on the arm of the eyeglasses, and light may be projected through air toward the eyeglass lens, which is reflected by a concave reflector on the lens and converged to the eye, acting as a converging lens.

[0168] In some embodiments, the focusing element 23 or optical combiner may include one or more flat or curved holographic optical elements (HOEs), each of which may be a photopolymer formed on a glass substrate. The HOEs may be recorded to act as off-axis concave mirrors, such that they behave similarly to curved reflectors. The HOEs may be manufactured to have the same optical properties as curved reflectors, but are much thinner.

[0169] In some embodiments, the focusing element 23 or the optical combiner may include one or more holographic polymer-dispersed liquid crystals (HPDLCs) or switchable Bragg gratings, each of which acts as a HOE that can be turned on and off (when off, the HPDLC or Bragg grating is transparent and allows light to pass through without substantially changing the angular size of the light).

[0170] When using a HOE or a single HPDLC, there may be lower tolerances for incident light with an angle of incidence different from the recording angle. Therefore, this may limit the viewport of the projection device. Therefore, in some embodiments, the optical combiner may include a needle.Multiple HPDLC layers are recorded for different steering angles, and the calculation unit 50 can be configured to turn on the HPDLC corresponding to the desired steering angle (page 15 / 29 of specification 20 CN 121219620 A) so that the light is correctly focused onto the pupil.

[0171] In some embodiments, the focusing element 23 or optical combiner may include a waveguide through which light is projected and focused onto the eye by means of total internal reflection (TIR). It should be understood that many suitable waveguide configurations can be used. In some embodiments, a holographic waveguide including an input coupler and an output coupler is used.

[0172] In some embodiments, at least one of the projection portion 21 and the focusing portion 23 may be movable by the steering portion 22 to steer the light about the rotation axis 40. In some embodiments, the projection portion 21 may be movable to move the convergence point along one dimension of the convergence surface, and the focusing portion may be movable to move the convergence point along another dimension of the convergence surface. Thus, by moving both the projection portion 21 and the focusing portion 23, the convergence point can be moved according to any movement of the eye.

[0173] In some embodiments, the eye tracker 60 may be omitted. Instead, the projection portion 21 may be configured to project light that encodes multiple portions of the image 5, and the focusing portion 23 may be configured to converge each light to a corresponding convergence point among multiple convergence points located on a convergence arc (or convergence surface). The location of each convergence point corresponds to a corresponding rotation angle of the eye 6 about the eye rotation axis 4. Thus, when the eye rotates at an angle corresponding to one of the convergence points, the light converged at that convergence point enters the eye and forms a centered image on the retina of the eye. Light converged to adjacent convergence points can encode adjacent portions of the image, allowing the eye to rotate from an angle corresponding to one convergence point to an angle corresponding to an adjacent convergence point without causing a visible transition in the image formed on the retina.

[0174] In some embodiments, the eye tracker 60 may include a light source. The light source may be configured to be undetectable by the eye. Thus, the light source can be prevented from affecting the image to be projected onto the eye. As a non-limiting example, the light source may be configured such that light emitted by the light source does not enter the eye due to the position and / or orientation of the light source, or light from the light source is not projected onto the retina or is not visible (i.e., detectable by retinal cells).

[0175] In some embodiments, the projection device will be held or worn by a user. As a non-limiting example, the projection device may include a head-mounted device to be mounted on the user's head, wherein elements of the optical system are positioned near and in front of the user's eyes (or both eyes).

[0176] In some embodiments, the projection device includes a housing for storing elements of the optical system. The housing may, for example, be located in...On the head-mounted device of the projection device. Therefore, the components stored in the housing can be protected from the influence of the environment, for example, by avoiding the light that encodes the image to be projected from unwanted light from the environment.

[0177] FIG5 illustrates a computing unit 50 for controlling the projection device according to one or more embodiments. The computing unit 50 includes one or more communication interfaces 510 for communicating with components of the eye tracker 60 and the optical system 20. The communication interfaces may allow the computing unit 50 to communicate with other computing units outside the projection device 10, for example, to exchange data related to the image to be projected. Each communication interface may use any suitable communication link as described herein.

[0178] The computing unit 50 also includes a processor 515. The processor 515 may include one or more processing units, such as a microprocessor, GPU, CPU, multi-core processor or similar computer processing unit.

[0179] The computing unit 50 also includes a memory 520. The memory 520 can be any suitable storage medium, including but not limited to optical discs, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archiving / warehouse devices, and / or any other type of device suitable for storing instructions and / or data. The memory 520 can store two-dimensional or three-dimensional image data and any other data to be displayed to the user, and can store data received from components of the eye tracker 60 and / or optical system 20.

[0180] The computing unit 50 may also include an eye tracking module 525, a rendering module 530, a display module 535, and a steering module 540. All modules of the computing unit 50 can be implemented in hardware, software, or a combination of both. For example, modules can be stored in the memory 520 as software. Modules may include any suitable instruction sequences stored in the computing unit 50, which is configured to perform the methods disclosed herein.

[0181] In operation, the eye tracker 60 tracks the positioning of the user's pupils (or both pupils) and outputs a value indicating the rotation angle of the pupils, which is received by the interface 510. The eye tracking module 525 can convert this data into coordinates in a predetermined coordinate system, which describes the direction the pupils are pointing. Then, the rendering module 530 can receive the coordinates describing the direction the pupils are pointing, receive image data from the memory 520, and render a 2D or 3D image to be displayed to the pupils. The rendering module 530 can render different images for each pupil, taking into account parallax, so that the user perceives a 3D environment. The rendered image data is sent to the display module 535, which generates instructions to the projection section 21, causing the projection section 21 to project the rendered image data onto the pupils.The image is encoded with light. The steering module 540 calculates the steering angle of the steering element 22 corresponding to the direction of the pupil and instructs the steering element 22 to steering the image to the correct steering angle. The rendered image is then converged by the focusing element 23 and formed on the retina of the user's eye. The value output by the eye tracker 60 can specify the rotation angle of the pupil (e.g., the output of the eye tracker 60 can include an angle value in degrees or radians), or the output of the eye tracker can be an image of the eye (e.g., from a camera) or a sensor value (e.g., from an ECG sensor), and the eye tracking module can determine the rotation angle of the pupil based on the received image or sensor value.

[0182] FIG6A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The projection device includes a self-emitting display 201 (e.g., a micro LED display) that emits an image to be displayed with collimated light (or a standard display can be used with collimating optics placed in front of the display). The device also includes a focusing element 203 that converges the light. The device also includes a steerable mirror 205, which may be, for example, a MEMS mirror (i.e., a mirror including a microelectromechanical system MEMS), and is positioned after the convergence point F of the focused light. The steerable mirror 205 reflects light toward the focusing element 207, which converges the light to the convergence point on arc 41. An image is formed on the retina of the eye 6. Turning the mirror 205 as indicated by arrow R rotates the virtual image V of the convergence point F, which in turn translates the convergence point of the light along arc 41. Thus, the mirror 205 can be turned to follow the position of the eye 6, such that the image projected onto the retina remains centered on the retina.

[0183] Note that although the focusing elements 203 and 207 are shown as transmissive lenses, they could alternatively be concave mirrors or HOEs with the same focal length. For example, Figure 6B shows an alternative embodiment in which a curved mirror or HOE is used instead of a lens for focusing element 207. This curved mirror or HOE can also be an optical combiner configured to allow light from the environment to pass through the mirror / HOE. If coherence of the light is required, for example, to use the HOE for components 203 or 207, a pinhole filter 204 (which may be referred to as a spatial or Fourier filter) can be added to the optical system.

[0184] For the optical system 20 shown in Figures 6A and 6B, display 201 and focusing element 203 can be considered as projection portion 21, steerable mirror 205 can be considered as steering element 22, and focusing element 207 can be considered as focusing element 23 of Figure 4 (and both elements 21 and 22 can be mounted in or on the arm of a pair of smart glasses, while element 23 can be integrated or coupled to the smart glasses lens).

[0185] Figure 7A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system includes a laser 210 comprising one or more laser diodes (e.g., individual RGB laser diodes or a single RGB diode) configured to emit light along red, green, and blue light. It should be understood that the laser may also emit IR light or light of other wavelengths in the visible spectrum. The light emitted by the laser 210 is collimated by a collimating element 211 (shown as a lens, alternatively a concave mirror or HOE, as shown in Figure 7B). The collimated laser is incident on a phase or amplitude spatial light modulator (SLM) 212, which modulates the light such that the image to be projected is encoded in the light transmitted by the SLM 212. Any suitable SLM, such as a transmissive or reflective amplitude and / or phase modulator based on liquid crystal (LC) or liquid crystal on silicon (LCoS) or digital micromirror device (DMD), can be used. It should be noted that although SLM 212 in Figure 7A is shown as a reflective SLM, a transmissive SLM can alternatively be used. The modulated light is then converged to a convergence point F by a focusing element 213 (shown as a lens, but could be a concave mirror or HOE) before incident on the steerable mirror 205. The steerable mirror 205 can be any suitable steerable mirror, such as a MEMS mirror. The light is reflected by the steerable mirror 205 toward a focusing element 207 (a curved reflector or HOE), which converges the light to a point on the convergence arc 41. In other embodiments, the focusing element 207 is a focusing lens, similar to the embodiment shown in Figure 6A. An image is projected onto the retina of the eye 6. As the steerable mirror 205 is turned, as indicated by arrow R, the virtual image V of the convergence point F rotates around the arc, and the convergence point at the pupil of the eye 6 shifts along arc 41. Therefore, the steerable mirror 205 is controllable to rotate the image projected onto the retina so that the image remains centered on the retina. Collimating lenses 211 and 213 can be replaced by a single focusing element 215 comprising a concave mirror (or one or more lenses), as shown in FIG7B. The focusing element 215 is configured to guide the laser light from laser 210 to SLM 212 such that the light is incident on SLM 212 in a non-collimated manner and converges the light to a convergence point F in front of the steerable mirror 205. SLM 212 modulates the non-collimated light so that an image is encoded into the light. Once the image has been encoded into the light, the focusing element 215 can also be placed after SLM 212.

[0186] For the optical system 20 shown in FIG7A, elements 210, 211, 212 and 213 can be considered as projection section 21.The steerable mirror 205 can be considered as steering element 22, and the focusing element 207 can be considered as focusing element 23 of FIG. 4 (and both elements 21 and 22 can be mounted in or on the arm of a pair of smart glasses). For the optical system 20 shown in FIG. 7B, elements 210, 215, and 212 can be considered as projection portion 21, the steerable mirror 205 can be considered as steering element 22, and the focusing element 207 can be considered as focusing element 23 of FIG. 4 (and both elements 21 and 22 can be mounted in or on the arm of a pair of smart glasses).

[0187] FIG. 8A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system includes a laser 210, such as an RGB laser diode configured to emit red, green, and blue light. The laser 210 is configured to emit a laser beam to the steerable mirror 205' (such as a MEMS mirror, etc.). The steerable mirror 205' is steerable (indicated by arrow R'), and the laser 210 and the steerable mirror 205' together emit light that encodes the image the user wants to view (selective emission of the laser from the laser 210 and selective positioning of the steerable mirror 205' together provide the necessary light generation and modulation to generate the image). Light is guided from the steerable mirror 205' to the steerable mirror 205 (any suitable mirror, such as a MEMS mirror), which is configured to rotate (indicated by arrow R). The light reflected by the steerable mirror 205 corresponds to the virtual image V and is guided toward the focusing element 207. In FIG. 8A, the focusing element 207 is a lens, while in other embodiments (as shown in FIG. 8B), the focusing element is a concave mirror, HOE, HPDLC, or a stack of HPDLCs. The focusing element 207 converges the light to a convergence point on the convergence arc 41, and the image is formed on the retina of the eye 6. Rotation of the steerable mirror 205 shifts the convergence point along the convergence arc 41 and rotates the projected image about an axis 40 that coincides with or is near the eye's rotation axis, such that the image projected onto the retina remains centered on the retina as the user's eye rotates.

[0188] For the optical system 20 shown in FIG8A, elements 210 and 205' can be considered as projection portion 21, the steerable mirror 205 can be considered as steering element 22, and the focusing element 207 can be considered as focusing element 23 of FIG4 (and both elements 21 and 22 can be mounted in or on the arm of a pair of smart glasses).

[0189] FIG9A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a self-emitting display 217 (e.g., a micro LED display) that emits collimated light.The image to be displayed is emitted (or a standard display can be used with collimating optics placed in front of the display). The optical system also includes a diverging element 218, which is configured to form a virtual image V of the converging arc 41. Although element 218 is illustrated as a diverging lens on pages 18 / 29 of the diverging specification 23 CN 121219620 A, in other embodiments, a convex mirror, HOE, or HPDLC with the same optical properties as a diverging lens can be used. The self-emitting display 217 and the diverging element 218 are mounted to a steerable element (not shown) such as a MEMS, such that these components can be steered by rotation, the direction of rotation indicated by arrow R. The light emitted by the diverging element is directed to a focusing element 207, shown as a focusing lens, which converges the light to a convergence point on the arc 41, and the image is projected onto the retina of the eye 6. The display 217 and the diverging element 218 are rotatable, such that the convergence point is moved to a point on the converging arc corresponding to the position of the eye 6. Therefore, as the eye 6 moves, the image remains centered on the retina of the eye 6. The display 217 and the diverging element 2018 can be integrated into a single self-emitting display that emits diverging light.

[0190] In an alternative embodiment, the self-emitting display 217 may include a laser configured to illuminate an optical modulation element (such as a spatial light modulator (SLM) or a DLP) with diverging light. The diverging light is transmitted or reflected to the focusing element 207. The display 217, including the laser and the optical modulator, is mounted to a steerable element such as a MEMS such that the projected image follows the position of the eye and remains centered on the retina. In embodiments using a laser configured to emit diverging light, the diverging element 218 is not required because the laser already emits a diverging beam.

[0191] In some embodiments, the focusing lens of the focusing element 207 may alternatively be a concave mirror, HOE, HPDLC, or HPDLC stack having the same optical properties as the focusing lens, as shown in FIG9B.

[0192] In the embodiments described with reference to Figures 9A and 9B, the display 217 and the diverging element 218 (or SLM) mounted on the steerable element can be considered as both the projection portion 21 and the steering element 22, and the focusing element 207 can be considered as the focusing element 23 of Figure 4 (and both elements 21 and 22 can be mounted in or on the arm of a pair of smart glasses).

[0193] Figure 10A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a display 219. The display 219 can be a self-emitting display configured to emit coherent light, or a reflective or transmissive light modulation element, such as an air-filled display illuminated with substantially coherent light (e.g., coherent point light or collimated light).Inter-light modulator (SLM) or DLP. Coherent light emitted by display 219 is incident on an HPDLC stack 220 comprising multiple HPDLC layers. Each layer of HPDLC is configured to act as a diverging lens, with each stack being recorded at a different angle. Therefore, the angle at which the diverging beam is emitted depends on which HPDLC layer (or combination thereof) is switched on. The diverging beam is directed to focusing element 207, which, in the illustrated embodiment, is a focusing lens. The light converges to a convergence point on a convergence arc 41, and an image is projected onto the retina of the eye. The layers of HPDLC stack 220 can be selectively switched on in different combinations to emit diverging light to focusing element 207 at different selectable angles. Depending on the selected angle, the light converges to different points on the convergence arc 41. Therefore, the angle at which the image is projected onto the retina can be rotated about axis 40, such that the image projected onto the retina remains centered on the retina. Advantageously, this embodiment does not include moving parts, thereby simplifying the structure of the projection device.

[0194] In some embodiments, the light emitted by the display 219 may not be coherent, but may instead be divergent. In this case, the HPDLC stack 220 may modify the divergent light from the display 219 substantially as described above.

[0195] The focusing element 207 may alternatively comprise a concave mirror, HOE, or HPDLC or HPDLC stack having the same optical properties as the focusing lens, as shown in FIG10B.

[0196] In the embodiments shown in FIG10A and FIG10B, the display 219 may be considered as projection portion 21, the HPDLC stack 220 may be considered as steering element 22, and the focusing element 207 may be considered as focusing element 23 of FIG4 (and both elements 21 and 22 may be mounted in or on the arm of a pair of smart glasses).

[0197] FIG11A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a laser 210, such as an RGB laser, which is configured to emit divergent light toward a phase modulator 212. The phase modulator can be, for example, a reflective or transmissive SLM or DLP. Phase modulator 212 is configured to spatially modulate the phase of light (and optionally, the amplitude of the light) to encode an image. Phase modulator 212 is also configured to modulate the light to direct the beam in a given direction. The light is guided by phase modulator 212 to focusing element 207, which, in the illustrated embodiment, is a focusing lens. The light is converged by focusing element 207 to a convergence point on convergence arc 41, and an image is projected onto the retina of the eye. The phase modulation of the light by phase modulator 212 can be selected for different directions.The image is projected such that light converges to different points along the convergence arc 41. Therefore, phase modulation can be controlled to project the image onto a point on the convergence arc 41 corresponding to the position of the eye 6, such that the projected image remains centered on the retina of the eye 6.

[0198] In some embodiments, the focusing element 207 may be a concave mirror, HOE, HPDLC, or a stack of HPDLCs, as shown in FIG11B. The phase modulator 212 may also be reflective, as shown in FIG11B.

[0199] In the embodiments shown in FIG11A and FIG11B, the laser 210 and the phase modulator 212 may be considered together as the projection portion 21 and the steering element 22, and the focusing element 207 may be considered as the focusing element 23 of FIG4 (and both elements 21 and 22 may be mounted in or on the arm of a pair of smart glasses).

[0200] FIG12A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a self-emitting steerable display 221 configured to generate and modulate light. The display 221 is configured to emit light that encodes an image to be projected onto the retina of the eye 6. The light is transmitted to a focusing element 207, which in the illustrated embodiment is a focusing lens, which converges the light to a point on a convergence arc 41, and the image is projected onto the retina of the eye 6. The display 221 is also configured to spatially modulate the phase of the light such that the image is projected in a specific direction and converges to a point along the convergence arc 41. Therefore, the display 221 is controllable to steer the projected light so that the convergence point moves to a point on the convergence arc 41 corresponding to the position of the eye, thereby keeping the projected image centered on the retina of the eye 6. In some embodiments, the focusing element 207 may be a concave mirror, a holoeurized mirror (HOE), an HPDLC, or an HPDLC stack, as shown in FIG12B.

[0201] In the embodiments shown in Figures 12A and 12B, the self-emitting steerable display 221 can be considered as both the projection portion 21 and the steerable element 22, and the focusing element 207 can be considered as the focusing element 23 of Figure 4 (and both elements 21 and 22 can be mounted in or on the arm of a pair of smart glasses).

[0202] For any of the embodiments disclosed herein, it should be noted that the projected light can be transmitted between the optical components via waveguides rather than via air. Figures 13A through 13C show schematic diagrams of an optical system 20 using waveguides.

[0203] Although the following exemplary embodiments are described with a single waveguide, this is not limiting, as any number of waveguides can be used. For example, in the embodiment shown in Figure 7B, a first waveguide can be provided between the laser 210 and the focusing element 215, and a second waveguide can be provided after the rotatable mirror 205. Each waveguide can have a corresponding input coupling.The output coupler and the output coupler of the waveguide can be used instead of the concave mirror 207 shown in FIG. 7B.

[0204] In the following exemplary embodiments, the output coupler of the waveguide is located on the surface of the waveguide away from the eye, and the TIR is used to reflect the light from the output coupler of the waveguide and converge the light to the eye. However, these are non-limiting examples, as the output coupler can alternatively be located on the surface of the waveguide near the eye, and the light is converged to the eye by focusing the light.

[0205] In the embodiment shown in FIG. 13A, the self-emitting steerable display 221 generates light and modulates the light. The light is transmitted to the input coupler 250 of the waveguide 251. The light is guided through the waveguide 251 to the output coupler 252 of the waveguide 252, which converges the light to the eye through the waveguide 251 by means of the TIR.

[0206] In the embodiment shown in FIG. 13B, the self-emitting display 201 (emitting collimated light or including collimating optics) emits an image to be displayed with collimated light. Collimated light rays pass through focusing element 203, which converges the light rays through waveguide 251. The light rays are then reflected by a steerable mirror 205 (e.g., a MEMS mirror). The light rays reflected from the steerable mirror 205 are transmitted to the input coupler 250 of waveguide 251. As in the embodiment shown in FIG13A, the light rays from the input coupler 250 are guided through waveguide 251 and converged to the eye by means of output coupler 252 via TIR.

[0207] In the embodiment shown in FIG13C, a coherent light source 210 (e.g., a laser, an LED emitting sufficiently coherent light, etc.) (e.g., at red, blue, and green wavelengths) emits light along the visible spectrum, which is converged by collimating element 211 (e.g., a lens as shown in FIG13C, or alternatively, a concave mirror or HOE). The converging light rays pass through waveguide 251 and are reflected by SLM 212. The light rays reflected by SLM 212 are transmitted to input coupler 250 of waveguide 251. Then, as shown in Figures 13A and 13B, the light rays are guided through waveguide 251 to output coupler 252 and then converge into the eye. The coherent light source can be replaced by a self-emitting display that emits amplitude-encoded images or by a reflective display (e.g., a laser / coherent light source that emits light reflected by an amplitude modulator).

[0208] In the previously disclosed embodiments, an increased field of view (as shown when comparing the image 5 formed in Figures 3A and 3B with the image formed by the system shown in Figures 2A and 2B) is obtained by converging the projected light at a point in front of the eye's rotation center. The projection system according to the invention allows a monocular field of view of 150° or greater. The natural resolution of the human eyeWith approximately 60 pixels per degree, this means a display with a resolution higher than 9000*9000 pixels would be required. While this is possible, rendering such a resolution would result in an expensive, bulky, and computationally demanding display device. However, the retina has the highest sensitivity in the central foveal region and lower sensitivity in the peripheral region. Therefore, some projection devices can combine a foveal portion that uses a low-resolution image for the periphery of the user's field of view with a high-resolution image for the foveal region. This reduces the requirements for resolution, bandwidth, optical power, and computation. Typical display devices use two displays (one for the high-resolution image and one for the low-resolution image), which also makes the display device more expensive and bulky. According to one aspect of this disclosure, a foveal portion 24 is provided in the optical system 20 of the display device 10, which uses a single projection portion 21 for both images, reducing the cost and size of the system. The foveal portion 24 is actuable to switch between a foveal mode and a peripheral mode, in which the high-resolution image is projected onto the retina with a small field of view, and in the peripheral mode, the lower-resolution image is projected onto the periphery of the retina. The concave portion 24 can be controlled by the computing unit 50 to switch between a concave mode and a peripheral mode. The computing unit 50 is configured to also cause the projection portion 21 to switch between the concave mode and the peripheral mode in a consistent manner with the concave portion 24, such that the projection portion 21 projects an image in a high-resolution, low-field-of-view concave mode or a lower-resolution, high-field-of-view peripheral mode. In such an embodiment, the computing unit 50 includes a concave module 545 that can be implemented in hardware, software, or a combination of both (e.g., stored as software in memory 520). The concave module includes a sequence of instructions stored in the computing unit 50 to consistently control the projection portion 21 and the concave portion 24 as described herein. The concave portion 24 can be placed at any suitable point along the optical path. For example, the concave portion 24 can be placed after the projection portion 21 and before the steering element 22, or it can be placed after the steering element 22 and before the focusing element 23.

[0209] FIG14A shows a schematic diagram of the optical system 20 of the projection device operating in peripheral mode. The optical system (when it includes the concave portion 24) can be, for example, the optical system 20 of the projection device of FIG. 4. In peripheral mode, the optical system 20 projects a peripheral image 5a onto the retina of the eye. The peripheral image is a lower-resolution image projected onto the periphery of the retina when the eye is centered as shown. In embodiments where the steering element 22 is included in the projection device, the center of the peripheral image 5a remains on the center of the retina as the eye moves, such that the peripheral image remains on the periphery of the retina.Above. The center C of the peripheral image 5a remains blank (i.e., no light or image is projected onto the retina in the central portion C in peripheral mode). Figure 14A also shows the peripheral image Ip that the user will observe when using the projection device.

[0210] Figure 14B shows a schematic diagram of an optical system operating in foveal mode. The optical system (when it includes the central foveal portion 24) can be, for example, the optical system 20 of Figure 4. In foveal mode, the optical system 20 projects a foveal image 5b onto the retina of the eye. A foveal image is a higher resolution image projected onto the foveal portion of the retina with a lower field of view when the image is focused on the central portion of the retina (as shown). In embodiments in which the steering element 22 is included in the projection device, the center of the foveal image 5b remains on the center of the retina as the eye moves, such that the foveal image remains on the fovea of ​​the retina. Figure 14B also shows the peripheral image If that the user will observe when using the projection device.

[0211] The optical system 20 is controlled by a computing unit 50, such as the computing unit 50 shown in FIG. 5. The computing unit 50 controls the projection section 21 and the optical system 20 to switch between a foveal mode and a peripheral mode. The switching is performed at a sufficiently high rate so that the user does not perceive the switching but perceives the complete image 5, such as the image shown in FIG. 14C, which is a combination of a foveal image 5b and a peripheral image 5a. For example, the projection section 21 is configured to switch between the peripheral mode and the foveal mode at a rate of 60 Hz or higher (i.e., each of the foveal image 5b and the peripheral image 5a is shown at a rate of 30 frames per second or higher).

[0212] Note that the resolution and field of view of the peripheral image and the foveal image can vary depending on the requirements of the projection device. For example, as shown in FIG. 14D, the foveal image may include a wider field of view and a lower resolution than shown in FIG. 14C. Furthermore, the projection device may include three or more foveal modes. For example, as shown in FIG14E, the image may include: an inner region image having a first resolution; an inner periphery image surrounding the inner region image having a lower resolution than the inner region image; and an outer periphery image surrounding the inner periphery image having a lower resolution than the inner periphery image. In such an embodiment, the foveal portion 24 may switch between three or more modes, each mode projecting an image onto a retina with a different field of view.

[0213] FIG15A shows a schematic diagram of an optical system 20 incorporating a foveal portion 24 according to one or more embodiments. The optical system 20 includes a projection portion 21 configured to project light onto the retina.The peripheral image is encoded. The projection portion 21 can be any suitable projection portion for projecting the image onto the retina of the eye (such as any projection portion shown in Figures 6A, 6B, 7A, 7B, 8A, 8B, 9A, 10A, 10B, 11A, 11B, or 12A). In embodiments where the steering element 22 is incorporated into the optical system 20 and includes mechanically steerable elements (such as mirror 205), the projection portion 21 can be, for example: a display 201 (and optional focusing element 203) as described with reference to Figures 6A and 6B; a laser 210, collimating element 211, and SLM 212 (and optional focusing element 213) as described with reference to Figure 7A; a laser 210, focusing element 215, and SLM 212 as described with reference to Figure 7B; or a laser 210 and a steerable mirror 205' as described with reference to Figures 8A and 8B.

[0214] In such an embodiment, the foveal portion 24 includes one or more switchable focusing and / or diverging elements (one or more switchable lenses or HPDLC layers) positioned between the projection portion 21 and the steering element 23 (i.e., the steerable mirror 205). In the peripheral mode shown in FIG15A, the switchable element and the projection portion 21 are switched to a peripheral mode in which the projection portion 21 projects a peripheral image 5a onto the periphery of the retina with a lower resolution and a higher field of view. FIG15B shows a schematic diagram when the optical system of FIG15A is in foveal mode. In foveal mode, the switchable element and the projection portion 21 are switched to a foveal mode in which the projection portion 21 projects a foveal image onto the fovea of ​​the retina with a higher resolution and a lower field of view. Note that by switching between these two modes, the resolution of the combined image viewed by the user is twice that of the resolution output by the projection portion 21, because the combined image is a combination of two images projected by the projection portion 21 (and the perceived resolution can be much higher, taking into account the non-uniform distribution of photoreceptors on the human retina). A steerable mirror 205 steers the image to follow the position of the retina, as described in the previous embodiments. Note that in the illustrated embodiment, the light from both the foveal and peripheral modes converges to the same convergence point F, causing the light from both the peripheral image 5a and the foveal image 5b to converge to a point on the convergence arc 41 and rotate with the position of the eye. This can be achieved through many different combinations of switchable elements. In the illustrated embodiment, in the peripheral mode, only the first focusing element 24c is activated, which converges the collimated light from the projection portion 21 at point F; in the foveal mode, the second focusing element 24a and the collimating element 24b are also activated, causing a narrower collimated beam to be guided to the firstFocusing element 24c, the first focusing element 24c again converges the light to the convergence point F. It should be understood that any number of elements (e.g., two elements, such as a first diverging lens and a subsequent converging lens, or any other suitable combination of optical elements) can be switched on in the peripheral mode.

[0215] FIG16A illustrates an optical system 20 for a projection device according to one or more embodiments, including a concave portion 24. The optical system 20 includes a projection portion 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection portion 21 can be any suitable projection portion for projecting an image onto the retina of the eye (such as any projection portion shown on FIG6A, FIG6B, FIG7A, FIG7B, FIG8A, FIG8B, FIG9A, FIG10A, FIG10B, FIG11A, FIG11B, or FIG12A). The illustrated embodiment shows an optical system 20 that includes only a projection portion 21 and a concave portion 24 that converges the projected light directly to a convergence point F near the pupil of the eye (i.e., a projection device excluding the steering element 22). In such an embodiment, the concave portion 24 can also be considered as a focusing element 23. It should be understood that the concave portion 24 described with reference to FIG. 16A can be interchanged with the concave portion 24 described with reference to FIG. 15A and FIG. 15B (and vice versa). In this case, as discussed with respect to FIG. 15A, the concave portion 24 converges the projected light to a convergence point F before the steerable reflector 205, as shown in FIG. 15A and FIG. 15B. The concave portion 24 includes a first switchable focusing element 230 and a second switchable focusing element 231 (such as a switchable lens, a reflector, and / or an HPDLC layer), which are configured to receive collimated light from the projection portion 21. The first lens 230 and the second lens 231 have different focal lengths and are positioned away from the projection portion 21 and separated from each other, such that the convergence point F of the two lenses 230, 231 coincides. In FIG16A, the optical system 20 is shown in foveal mode, in which the projection portion 21 is controlled to project light encoding the foveal image 5b, the first switchable focusing element 230 is turned on and the second switchable focusing element 231 is turned off. The light converges to the convergence point F and the image is formed on the retina.

[0216] FIG16B shows the optical system 20 of FIG16A in peripheral mode, in which the projection portion 21 is controlled to project light encoding the peripheral image 5a, the second switchable focusing element 231 is turned on and the first switchable focusing element 230 is turned off. The peripheral image is projected to the convergence point F and the image is formed on the retina. Due to the shorter focal length of the second switchable focusing element 231, the field of view of the peripheral image is widened. Note that by switching between these two modesIn other words, the resolution of the combined image viewed by the user is twice the resolution output by the projection portion 21, because the combined image is a combination of two images projected by the projection portion 21. It should be understood that the field of view of the foveal image and the peripheral image depends on the size of the collimating light projected from the projection portion, the focal length of the focusing elements 230 and 231, and the distance from the convergence point F to the retina.

[0217] FIG17A illustrates an optical system 20 for a projection device according to one or more embodiments, including a foveal portion 24. The optical system 20 includes a projection portion 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection portion 21 can be any suitable projection portion for projecting an image onto the retina. The illustrated embodiment shows an optical system 20 that includes only the projection portion 21 and the foveal portion 24 that converges the projected light directly to the convergence point F near the pupil of the eye (i.e., a projection device excluding the steering element 22). In such an embodiment, the foveal portion 24 can also be considered as the focusing element 23. It should be understood that the central concave portion 24 described with reference to FIG. 17A can be interchanged with the central concave portion 24 described with reference to FIG. 15A and FIG. 15B (and vice versa). In this case, as discussed with respect to FIG. 15A, the central concave portion 24 converges the projected light to a convergence point F in front of the steerable reflector 205, as shown in FIG. 15A and FIG. 15B. The central concave portion 24 includes a switchable diverging element 232 (e.g., a switchable lens, a convex reflector, or an HPDLC layer as described on pages 23 / 29 of the specification, CN 121219620 A), and a first switchable focusing element 233 and a second switchable focusing element 234 (e.g., a switchable lens, a convex reflector, or an HPDLC layer, or any combination thereof). The switchable diverging element 232 is positioned away from and in front of the first switchable focusing element 233. In Figure 17A, the optical system 20 is shown in foveal mode, in which the projection section 21 is controlled to project light encoding the foveal image 5b, the second switchable focusing element 234 is turned on, and the first switchable focusing element 233 and the diverging element 232 are turned off. The collimated light encoding the image is guided from the projection section 21 to the second focusing element 234, the light converges to the convergence point F, and the image is formed on the retina.

[0218] Figure 17B shows the optical system 20 of Figure 17A in peripheral mode, in which the projection section 21 is controlled to project light encoding the peripheral image 5a. In peripheral mode, the second switchable focusing element 234 is turned off, and the first switchable focusing element 233 and the diverging element 232 are turned on. The projection section 21 encodes the peripheral image...The collimated light is projected onto a switchable diverging element 232. The diverging light is directed to a first switchable focusing element 233, which converges the light to a convergence point F. The peripheral image 5a is then projected onto the retina. Because the first switchable focusing element 233 has a shorter focal length than the second switchable focusing element 231, the field of view of the peripheral image is wider. It should be understood that the exact positions of elements 232, 233, and 234 depend on the focal length. It should be understood that the field of view of the foveal image and the peripheral image depends on the size of the collimated light projected from the projection section, the focal lengths of elements 232, 233, and 234, and the distance from the convergence point F to the retina. Note that by switching between these two modes, the resolution of the combined image viewed by the user is twice that of the resolution output by the projection section 21, because the combined image is a combination of the two images projected by the projection section 21. By using the diverging element 232, the minimum space required for the optical system can be advantageously reduced. Specifically, the optical system requires a certain amount of space, which is partly determined by the focal length of each optical element used. In particular, in the systems shown in Figures 16A and 16B, the switchable focusing elements 230 and 231 should be located at distances to the eye corresponding to their respective focal lengths. Therefore, these switchable focusing elements need to be spaced apart from each other, which may be a major factor in determining how compact the optical system 20 can be. By using a diverging element 232 and a separate switchable focusing element 234, the two switchable focusing elements 233 and 234 do not need to be spaced apart from each other.

[0219] Figure 18A illustrates an optical system 20 for a projection device according to one or more embodiments, including a concave portion 24. The optical system 20 includes a projection portion 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection portion 21 can be any suitable projection portion for projecting an image onto the retina. The illustrated embodiment shows an optical system 20 that includes only a projection portion 21 and a concave portion 24 that directly converges the projected light to a convergence point F near the pupil of the eye (i.e., a projection device excluding the steering element 22). In such an embodiment, the concave portion 24 can also be considered as a focusing element 23. It should be understood that the concave portion 24 described with reference to FIG. 18A can be interchanged with the concave portion 24 described with reference to FIGS. 15A and 15B (and vice versa). In this case, as discussed with respect to FIG. 15A, the concave portion 24 converges the projected light to a convergence point F before the steerable reflector 205, as shown in FIGS. 15A and 15B. The concave portion 24 includes a first switchable diverging element 235a and a second switchable diverging element 235b, and a first switchable focusing element 236a and a second switchable focusing element 236b. The diverging and focusing elements can beThis can be, for example, a switchable lens, a switchable mirror, or an HPDLC layer, or any combination thereof. The first focusing element 236a is positioned away from and in front of the second diverging element 235b. FIG18A shows the optical system 20 in foveal mode. In foveal mode, the first focusing element 236a and the second diverging element 236b are turned on, and the first diverging element 235a and the second focusing element 236a are turned off. The projection section 21 is controlled to project collimated light encoding the foveal image 5b onto the first focusing element 236a. The first focusing element 236a converges the light toward the second diverging element 235b, and the second diverging element 235b diverges the light toward the convergence point F. The foveal image 5B is formed on the retina.

[0220] FIG18B shows the optical system 20 of FIG18A in peripheral mode. In peripheral mode, the first diverging element 235a and the second focusing element 236b are turned on, and the first focusing element 236a and the second diverging element 235b are turned off. The projection portion is controlled to project collimated light encoding the peripheral image 5a onto the first diverging element 235a. The first diverging element 235a diverges the light and directs it to the second focusing element 235b, which converges the light to the same convergence point F as in the foveal mode. The peripheral image 5a is formed on the retina.

[0221] In all embodiments described with reference to FIGS. 14A to 18B, the convergence point F of the projected light in the foveal mode and the peripheral mode may not coincide. FIG19A shows the optical system 20 in peripheral mode. In peripheral mode, projection portion 21 projects collimated light onto concave portion 24, which converges the light to a convergence point F1 before the steerable mirror 205. The light is guided to focusing element 207, and peripheral image 5a is formed on the retina. Note that concave portion 24 includes a first switchable focusing element 237 and a second switchable focusing element 238, which operate in a similar manner to concave portion 24 shown in Figures 16B and 16B, except that in the current case the focusing elements are positioned such that the convergence points do not coincide. In peripheral mode, the first focusing element 237 is turned on and the second focusing element 237 is turned off. Note that the concave portion 24 shown can be replaced by the concave portion of Figures 17A and 17B or Figures 18A and 18B, wherein the lens has optical properties and is positioned such that the convergence points in concave mode and peripheral mode are offset from each other.

[0222] Figure 19B shows the optical system 20 of Figure 19A in concave mode. Projection section 21 projects collimated light ontoA concave portion 24 converges light to a convergence point F2, which may, for example, coincide with the surface of a steerable mirror 205. The steerable mirror 205 directs the light to a focusing element 207, which converges the light such that a concave image 5b is projected onto the retina.

[0223] FIG20A shows a schematic diagram of an optical system 20 incorporating a concave portion 24 according to one or more embodiments. The optical system 20 includes a display 217 configured to emit collimated light that encodes an image to be projected onto the eye. The optical system also includes a concave portion 24, which may be any previously disclosed concave portion. The display 217 and the concave portion 24 are mounted to a steerable element such as a MEMS. In FIG20A, the optical system 20 is shown operating in a peripheral mode. In this mode, the display 217 projects light that encodes a peripheral image. The foveal portion 24 switches to peripheral mode and diffuses light to a wider angle, and the focusing element 207 converges the light to a convergence point F, projecting the peripheral image 5a onto the retina. A steerable element allows the peripheral image 5a to remain centered on the retina, as described in previous embodiments.

[0224] FIG20B illustrates the operation of the optical system 20 of FIG20A in foveal mode. The display 217 projects light that encodes the foveal image. The foveal portion 24 is switched to foveal mode and diffuses light to a narrower angle than in peripheral mode, and the focusing element 207 converges the light to a convergence point F (coinciding with the convergence point in peripheral mode), projecting the foveal image 5b onto the retina. A steerable element allows the foveal image to remain centered on the retina, as described in previous embodiments.

[0225] As previously stated, the light projected by the focusing element 207 in both peripheral and foveal modes does not need to have a coinciding convergence point. Figures 21A and 21B illustrate an optical system 20 similar to those of Figures 20A and 20B, except that the optical elements for each mode of the concave portion 24 are selected such that in the peripheral mode, the projected light converges to a convergence point F1, while in the concave mode, the projected light converges to a convergence point F2 offset from F1. The offset can be defined as substantially around the optical axis of the eye, the pupillary axis of the eye, or the axis formed by the pupil and the center of rotation of the eye.

[0226] In the exemplary embodiments described above, the optical system 20 includes a concave portion 24 that uses elements separate from the projection portion 21 and the focusing portion. However, this is not limiting, as a concave display can be provided without the need for separate optical elements for the concave portion (which can therefore be omitted). Specification 25 / 29 pages 30 CN 121219620 A

[0227] Figures 22A and 22B illustrate schematic diagrams of an optical system 20 configured to provide a concave display according to one or more embodiments.

[0228] Specifically, as described above in conjunction with FIG. 11A, laser 210 is configured to emit diverging light to phase modulator 212 (e.g., reflective or transmissive SLM or DLP), and phase modulator 212 is configured to spatially modulate the light to direct the beam in a given direction.

[0229] FIG. 22A illustrates an optical system 20 in foveal mode. Laser 210 and phase modulator 212 transmit light to the central portion of focusing element 207 (shown by way of a non-limiting example as a lens). Focusing element 207 converges the light from its central portion to a convergence point located near the pupil of the eye. The converged light enters the eye and projects a foveal image 5a onto the central portion 5a of the retina (e.g., the fovea).

[0230] FIG. 22B illustrates an optical system 20 in peripheral mode. Laser 210 and phase modulator 212 transmit light to a wider portion of focusing element 207 (wider than the central portion used in foveal mode). The focusing element 207 converges light from the wider portion into the eye, and this light projects the peripheral image 5b onto the retina.

[0231] It should be understood that the implementations shown in Figures 22A and 22B are non-limiting examples. Alternatively, the focusing element 207 may converge light in foveal mode to a point closer to the eye's rotation center (i.e., the center of the sphere representing the eye) and in foveal mode to a point closer to the pupil (i.e., the location of the convergence point in foveal and peripheral modes may be opposite to the examples shown in Figures 22A and 22B). In other examples, one of the convergence points (in either foveal or peripheral mode) may be located in front of the eye, and the other convergence point (in the other of foveal and peripheral modes) may be located between the pupil and the eye's rotation center.

[0232] Advantageously, with this optical system, no movable element is required, and no additional optical element is needed to provide a foveal display. Therefore, the optical system 20 can be made more compact and requires fewer resources to operate.

[0233] As with the exemplary embodiments shown in Figures 10A and 10B, the optical system 20, without moving parts, can project different images onto the central portion and the peripheral portion of the retina.

[0234] Figures 23A and 23B illustrate schematic diagrams of an optical system 20 configured to provide a foveal display according to one or more embodiments.

[0235] The optical system 20 includes a display 219, a foveal portion 24, a steering portion (shown by way of non-limiting example as a stack 220 of HPDLCs), and a focusing element 207 (shown by way of non-limiting example as a lens). The foveal portion includes one or more switchable optical elements (e.g., converging or diverging elements).

[0236] Figure 23A shows the optical system 20 in peripheral mode. The display 219 emits collimated light onto the concave portion 24. In peripheral mode, the switchable element of the concave portion 24 is turned off, and thus the collimated light from the display 219 does not substantially change the angular size of the light (i.e., the light remains collimated).

[0237] The collimated light is transmitted to the deflection portion 220, which deflects the light as explained above in conjunction with Figures 10A and 10B, and for the sake of brevity, its explanation will be omitted here.

[0238] Figure 23B shows the optical system 20 in concave mode. In concave mode, the switchable element of the concave portion 24 is turned on. Therefore, the focusing element 24a focuses the collimated light from the display 219. The collimating element 24b collimates the focused light, which is then transmitted to the deflection portion 220. In the concave mode, the light collimated by the collimating element 24b reaches a narrower portion of the deflecting portion 220 than in the peripheral mode (e.g., the collimated light is projected onto the central portion of the HPDLC stack). Therefore, the light reaches the focusing element 207 and thus the eye in the concave mode with a narrower angular range than in the peripheral mode.

[0239] Although in the examples shown in Figures 23A and 23B, the concave portion 24 includes the focusing element 24a and the collimating element 24b, this is not limiting, as it should be understood that any number of switchable optical elements can be provided. More generally, the central concave portion may alternatively correspond to the central concave portion described in any exemplary embodiment herein, such as the central concave portion shown in any of Figures 15A-15B, 16A-16B, 17A-17B, 18A-18B, 19A-19B, 20A-20B, 21A-21B, etc.

[0240] The above embodiments use examples of displays that emit light that encodes an image. In some embodiments, the display may be a light field display configured to project light that encodes an image of multiple elements into a three-dimensional light field.

[0241] Figure 24A shows a schematic diagram of an optical system 20 configured to provide a light field display according to one or more embodiments.

[0242] The optical system 20 includes a light field display 260 and a focusing element 203 (as an example of a projection portion 21), a rotatable mirror 205 (as an example of a steering portion 22), and a focusing element 207 (as an example of a focusing portion 23).

[0243] The focusing element 203, the rotatable mirror 205, and the focusing element 207 operate as explained above in conjunction with FIG. 6A; therefore, for the sake of brevity, some explanations of these elements will be omitted here.

[0244] A light field display 260 emits rays that encode multiple element images in a three-dimensional light field. Each element image is rendered so that it is perceived by the user as arriving from a virtual point in the environment. Figure 24A shows two exemplary virtual points V1 and V2, although it should be understood that any other number of virtual points may be provided.

[0245] Rays corresponding to virtual point V1 are reflected off the rotatable mirror 205 and pass through the focusing element 207 to converge at point V1' on the retina. Because these rays converge to a point located on the retina, the element images encoded in these rays appear to be in focus to the user.

[0246] Similarly, rays corresponding to virtual point V2 are reflected off the rotatable mirror 205 and pass through the focusing element 207 to be projected into the eye. However, rays corresponding to virtual point V2 converge at point V2' in front of the retina. Therefore, these rays are projected onto the retinal area indicated by V2”. Because these rays are not focused at a single point when they reach the retina, the element images encoded by these rays are blurred (i.e., they appear out of focus to the user).

[0247] Therefore, the user will perceive that the element images encoded by the rays corresponding to virtual point V2 are at a different depth than the element images encoded by the rays corresponding to virtual point V1. Thus, the image displayed by the light field display acquires a third dimension with the depth perceived by the user.

[0248] It should be understood that in the example shown in FIG24C, the rays focused by the focusing element 207 are focused at multiple convergence points on a convergence arc near the pupil. As a non-limiting example, FIG24A shows three convergence points C1, C2, and C3.

[0249] FIG25A-FIG.25E show schematic diagrams illustrating different combinations of element images that can be encoded in the light from the light field display.

[0250] As shown in Figure 25A, nine element images with different perceptual depths can be combined to form a three-dimensional light field. Specifically, the images in the middle column can correspond to a first perceptual depth (as shown by solid lines), the images in the left column can correspond to a second perceptual depth (as shown by dotted lines), and the images in the right column can correspond to a third perceptual depth (as shown by dashed lines), wherein the first, second, and third perceptual depths are different from each other. The nine element images have partial overlap with adjacent images, allowing the user to perceive the image, where each element has a different depth.

[0251] The number of element images, the configuration of each image, or the perceptual depth is not limited to the example of Figure 25A. Specifically, the number of element images can be different, for example, four element images as shown in Figure 25B and / or different configurations (i.e., the element images do not need to be configured in columns with an equal number of element images).

[0252] Furthermore, some images do not need to overlap. For example, as shown in Figure 25D, the combination description of each element image can be omitted.Page 27 / 29 32 CN 121219620 A The corresponding portion that will overlap with other element images (the shaded portion on FIG25D). In this case, a single element image among multiple element images (e.g., the element image in the upper right corner) may include the corresponding shaded portion, so the combined image is complete.

[0253] As shown in FIG25E, the light field display can also be used to display different three-dimensional light fields to the central portion and the peripheral portion of the retina. For example, a light field using five element images (with the configuration shown in FIG25C) can be projected onto the central portion, and a light field using four element images (with the configuration shown in FIG25D) can be projected onto the peripheral portion. It should be understood that the number of element images described herein is merely exemplary, and any number of element images may be used alternatively.

[0254] Various examples of three-dimensional light fields that can be provided have been described, and for the sake of brevity, other details of the light field display that will now become apparent to those skilled in the art will be omitted here.

[0255] Modifications and variations are possible in many ways to the above example embodiments.

[0256] Although FIG. 4 illustrates an optical system 20 having a steering portion 22 and a concave portion 24, in some embodiments, either or both of the steering portion 22 and the concave portion 24 may be omitted.

[0257] For example, as shown in any of FIG. 6A to FIG. 13C, the optical system 20 does not need to project different images onto the central and peripheral regions of the retina.

[0258] As another example, as shown, for example, FIG. 10A or FIG. 10B, the projection portion 21 may be configured to emit light in different directions, thereby eliminating the need for the steering portion 22.

[0259] Furthermore, in the case where the optical system 20 includes a concave portion, the optical system 20 does not need to move the convergence point along the convergence arc (in the case of a dynamic view).

[0260] Although the above example embodiments use specific optical elements, it should be understood that each of these optical elements can be replaced by other interchangeable optical elements. For example, each diverging lens can be replaced by a convex mirror, each converging lens can be replaced by a concave mirror, or each lens or mirror can be replaced by HPDLC, HOE, stacked HPDLC, etc.

[0261] Although in the above example embodiment, the mirror or other rotatable element is shown as having one axis of rotation, these elements may have two different axes of rotation (or the optical system 20 may include additional rotatable elements configured to rotate along a second different axis of rotation), thereby allowing the optical system to control the positioning of the convergence point of light along two dimensions.

[0262] In the above example embodiment, the image projected onto the eye may be part of a scene (such as shown in a...)(Still images or videos of one or more virtual elements). The projected portion of the scene may correspond to the rotation angle of the eye (i.e., the orientation of the pupil), such that the projected portion changes as the eye moves.

[0263] In other cases, such as when a dynamic view is used (and the optical system moves the convergence point along a convergence arc or convergence surface), the displayed image may alternatively remain fixed in the user's field of view. In other words, the image seen by the user may be static and does not move in the user's vision.

[0264] In any of the example embodiments described above that include a phase modulator, the optical system may be configured to project light that encodes the holographic image. For example, in FIG7A, phase modulator 212 may be configured to project light that encodes the holographic image by combining wavefronts. For the sake of brevity, details of holographic image generation and projection that will now become apparent to those skilled in the art will be omitted.

[0265] In the example embodiment described above with reference to Figures 17A and 17B, two switchable focusing elements 233 and specification page 28 / 29 (CN 121219620 A 234) are included, each switchable focusing element 233 and 234 being used to focus light in one of a concave mode and a peripheral mode. Alternatively, a single switchable focusing element can be used and controlled to switch between a first state and a second state (e.g., in the first state when the concave portion is in the concave mode and in the second state when the concave portion is in the peripheral mode, or vice versa), wherein the single switchable focusing element may have different focal lengths in the first state and the second state.

[0266] In the exemplary embodiment described above with reference to Figures 18A and 18B, the first switchable diverging element 235a and the first focusing element 236a may be replaced by a single switchable optical element (wherein, in the concave mode, the single switchable optical element has a focal length corresponding to the first switchable diverging element 235a, and in the peripheral mode, the single switchable optical element has a focal length corresponding to the first focusing element 236a). Additionally or alternatively, elements 235b and 236b may be replaced by a single switchable optical element.

[0267] In the foregoing description, exemplary aspects have been described with reference to several exemplary embodiments. Therefore, the specification should be considered illustrative rather than restrictive. Similarly, the accompanying drawings, which highlight the functionality and advantages of the exemplary embodiments, are presented merely for illustrative purposes. The architecture of the exemplary embodiments is flexible and configurable enough that it can be utilized in ways other than those shown in the drawings.

[0268] Some embodiments may also be implemented by preparing application-specific integrated circuits, field-programmable gate arrays, or by making appropriate network interconnects of conventional component circuits.

[0269] The apparatus described herein can be implemented in other specific forms without departing from its characteristics. The foregoing embodimentsExamples are illustrative and not limiting of the systems and methods described. Therefore, the scope of the apparatus described herein is indicated by the appended claims rather than the foregoing description, and thus variations falling within the meaning and scope of the equivalents of the claims are included therein. Instruction Manual Page 29 / 29 34 CN 121219620 A Figure 1A Figure 1B Instruction Manual Drawings 1 / 25 Page 35 CN 121219620 A Figure 1C Figure 2A Figure 2B Instruction Manual Drawings 2 / 25 Page 36 CN 121219620 A Figure 3A Figure 3B Figure 4 Instruction Manual Drawings 3 / 25 Page 37 CN 121219620 A Figure 5 Figure 6A Instruction Manual Drawings 4 / 25 Page 38 CN 121219620 A Figure 6B Figure 7A Instruction Manual Drawings 5 / 25 Page 39 CN 121219620 A Figure 7B Figure 8A Instruction Manual Drawings 6 / 25 Page 40 CN 121219620 A Figure 8B Figure 9A Instruction Manual Drawings 7 / 25 Page 41 CN 121219620 A Figure 9B Instruction Manual Drawings 8 / 25 Page 42 CN 121219620 Figure 10A Figure 10B Instruction Manual Drawings 9 / 25 Page 43 CN 121219620 Figure 11A Figure 11B Instruction Manual Drawings 10 / 25 Page 44 CN 121219620 Figure 12A Figure 12B Instruction Manual Drawings 11 / 25 Page 45 CN 121219620 Figure 13A Figure 13B Instruction Manual Drawings 12 / 25 Page 46 CN 121219620 Figure 13C Figure 14A Figure 14B Instruction Manual Drawings 13 / 25 Page 47 CN 121219620 Figure 14C Figure 14D Figure 14E Instruction Manual Drawings 14 / 25 Page 48 CN 121219620 Figure 15A Instruction Manual Drawings 15 / 25 Page 49 CN 121219620 Figure 15B Figure 16A Instruction Manual Drawings 16 / 25 Page 50 CN 121219620 A Figure 16B Figure 17A Figure 17B Instruction Manual Drawings 17 / 25 Page 51 CN 121219620 A Figure 18A Figure 18B Instruction Manual Drawings 18 / 25 Page 52 CN 121219620 A Figure 19A Figure 19B Instruction Manual Drawings 19 / 25 Page 53 CN 121219620 A Figure 20A Figure 20B Instruction Manual DrawingsFigure 20 / 25, page 54, CN 121219620 A; Figure 21A, Figure 21B, Instruction Manual Drawings, page 21 / 25, page 55, CN 121219620 A; Figure 22A, Figure 22B, Instruction Manual Drawings, page 22 / 25, page 56, CN 121219620 A; Figure 23A, Figure 23B, Instruction Manual Drawings, page 23 / 25, page 57, CN 121219620 A; Figure 24A, Figure 25A, Instruction Manual Drawings, page 24 / 25, page 58, CN 121219620 A; Figure 25B, Figure 25C, Figure 25D, Figure 25E, Instruction Manual Drawings, page 25 / 25, page 59, CN 121219620 A

Claims

1. An optical system for projecting an image near the eye onto a user's eye, the optical system comprising: - A projection section configured to project light that encodes the image; as well as - A central recessed portion, the central recessed portion including at least one switchable optical element, the at least one switchable optical element being controllable to switch between a central recessed mode and a peripheral mode, wherein -- In the foveal pattern, the foveal portion is configured to project light from the projection portion onto the central region of the retina of the eye, and -- In the peripheral mode, the concave portion is configured to project light from the projection portion onto a peripheral region of the retina, the peripheral region being larger than and including the central region.

2. The optical system according to claim 1, wherein, The projection portion is configured to project the light as substantially collimated light.

3. The optical system according to claim 1 or 2, wherein, Each of the at least one switchable element is configured to allow light to pass through when switched to a first state without substantially changing the angular size of the light.

4. The optical system according to any one of claims 1 to 3, wherein, The central concave portion is configured to increase the angle of light incident on the eye in the peripheral mode.

5. The optical system according to any one of claims 1 to 4, wherein, The concave portion is configured to increase the focal length of light incident on the eye in the concave pattern.

6. The optical system according to claim 5, wherein, The central recess includes at least one focusing element with a controllable focal length.

7. The optical system according to any one of claims 1 to 6, wherein, The at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length different from the first focal length.

8. The optical system according to claim 7, wherein, The first switchable element is arranged at a first distance from the display, and the second switchable element is arranged at a second distance from the display.

9. The optical system according to claim 8, wherein, The difference between the first distance and the second distance makes the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

10. The optical system according to any one of claims 7 to 9, wherein, In the central recessed mode, the first switchable element is configured to switch to the second state, and the second switchable element is configured to switch to the first state. In the peripheral mode, the first switchable element is configured to switch to a first state, and the second switchable element is configured to switch to a second state.

11. The optical system according to any one of claims 1 to 6, wherein, The at least one switchable optical element includes a first switchable diverging element, a first switchable converging element, and a second switchable converging element. The first switchable diverging element is arranged at a first distance from the projection portion, and the first switchable converging element is arranged at a second distance from the projection portion, the second distance being greater than the first distance.

12. The optical system according to claim 11, wherein, In the foveal mode, the first switchable diverging element is configured to switch to a first state, the first switchable converging element is configured to switch to a first state, and the second switchable converging element is configured to switch to a second state, such that light from the projection portion is converged by the second switchable converging element to form an image on the central region of the retina. In the peripheral mode, the first switchable diverging element is configured to switch to a second state, the first switchable converging element is configured to switch to a second state, and the second switchable converging element is configured to switch to a first state, such that light from the projection portion is diverged by the first switchable diverging element onto the first switchable converging element and converged by the first switchable diverging element to form an image on the peripheral region of the retina.

13. The optical system according to claim 11 or claim 12, wherein, The second switchable converging element is arranged at a third distance from the projection portion such that the convergence point of the first switchable converging element and the convergence point of the second switchable converging element substantially coincide with each other.

14. The optical system according to claim 11 or claim 12, wherein, The at least one switchable optical element further includes a second switchable divergence element. The second switchable diverging element is configured to switch to a second state in the concave mode and increase the angle of light focused by the first switchable converging element. The second switchable converging element is arranged at a third distance from the projection portion, and the second switchable diverging element is arranged at a fourth distance from the projection portion. The first distance, the second distance, the third distance, and the fourth distance are configured such that the convergence point of light in the concave mode and the convergence point of light in the peripheral mode substantially coincide with each other.

15. The optical system according to any one of claims 1 to 14, wherein, The concave portion is configured to converge the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina. The at least one convergence point is located on or near the pupil of the eye on a convergence arc, wherein the corresponding location of each convergence point on the convergence arc corresponds to the rotation angle of the eye about the eye's rotation axis, and the convergence arc has a center of curvature that substantially coincides with the eye's rotation axis.

16. The optical system according to claim 15, wherein, The optical system is configured to move the at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably substantially coincides with the macula of the retina, and more preferably substantially coincides with one of the perifovea, parafovea, fovea, avascular area of ​​the fovea, fovea fovea minor, and fovea fovea of ​​the retina.

17. The optical system of claim 16, further comprising a steering portion configured to move the at least one convergence point about the convergence arc in response to rotation of the eye about an eye rotation axis, the arc having a center of curvature substantially coinciding with the eye rotation axis.

18. The optical system according to claim 17, wherein, The steering portion is configured to move at least one element of the projection portion and the central recessed portion.

19. The optical system according to claim 17 or claim 18, wherein, The deflecting portion includes at least one optical element for deflecting light from the projection portion toward the central concave portion.

20. The optical system according to claim 19, wherein, The at least one optical element includes a steerable reflector.

21. The optical system according to claim 19 or claim 20, wherein, The at least one optical element includes a plurality of switchable lenses, each switchable lens having a corresponding orientation and configured to direct light from the projection portion onto different portions of the central concave portion, such that the central concave portion converges the light to different convergence points on the convergence arc.

22. The optical system according to any one of claims 19 to 21, wherein, The at least one optical element includes a phase modulator with controllable phase modulation.

23. The optical system according to any one of claims 1 to 15, wherein, The projection portion is configured to project multiple lights, each light encoding a corresponding portion of the image, and the central concave portion is configured to converge each light to a corresponding convergence point among multiple convergence points, the positioning of each convergence point on the convergence arc corresponding to a corresponding rotation angle of the eye about the eye's rotation axis, such that light converging at adjacent convergence points encodes adjacent portions of the image.

24. The optical system according to any one of claims 15 to 23, wherein, The at least one convergence point is located on a convergence surface that is substantially parallel to the pupil of the eye and includes the convergence arc.

25. The optical system according to any one of claims 1 to 24, wherein, The projection section is configured to project a first light encoding a first image onto the user's first eye, and a second light encoding a second image onto the user's second eye. The central recess portion includes two central recess components, each corresponding to one of the user's eyes. Each foveal component is configured to switch between a foveal pattern that projects light onto a central region of the retina of the corresponding eye and a peripheral pattern that projects light onto a peripheral region of the retina of the corresponding eye, the peripheral region being larger than and encompassing the central region.

26. The optical system according to any one of claims 1 to 25, wherein, The projection portion includes a display and a focusing element. The display is configured to emit an image to be displayed using substantially collimated light, and the focusing element is used to converge the light towards the central concave portion. Optionally, the projection section includes a spatial filter for receiving the converged light rays.

27. The optical system according to any one of claims 1 to 26, wherein, The projection section includes a coherent light source, preferably a laser, which is configured to emit substantially coherent light. The projection portion may optionally include at least one of the following: A collimating element configured to collimate the substantially coherent light. A phase modulator, configured to encode the image by modulating the substantially coherent light, and A laser beam scanning system, the laser beam scanning system including at least one steerable reflector.

28. The optical system according to any one of claims 1 to 27, wherein, The projection portion includes a light field display configured to project light encoding multiple element images into a three-dimensional light field, each element image forming a portion of the image.

29. The optical system according to any one of claims 1 to 27, wherein, The projection section includes a phase modulator configured to project light that encodes the holographic image.

30. A projection device comprising an optical system according to any one of claims 1 to 29 and an eye tracker for determining the rotation angle of the pupil of an eye. in, Optionally, the eye tracker includes at least one camera for capturing images of the eye, the eye tracker being configured to determine a rotation angle of the pupil of the eye based on the images, the pupil rotation angle being used to determine the positioning of the at least one convergence point on the convergence arc. Optionally, the eye tracker is configured to determine the focal length of the lens of the eye, and Optionally, the eye tracker includes at least one light source for illuminating the eye.

31. A method for projecting an image near the eye onto a user's eye, the optical system comprising: Projecting light that encodes the image, and Control at least one switchable optical element to switch between a central concave mode and a peripheral mode, wherein In the foveal mode, light encoding the image is projected onto the central region of the retina of the eye by the at least one switchable optical element, and In the peripheral mode, light encoding the image is projected onto a peripheral region of the retina by the at least one switchable optical element, the peripheral region being larger than and including the central region.

Images