Projection device

Abstract

An optical system for myopic projection of an image into a user's eye, comprising: a projection unit configured to project light encoding an image; and a foveation unit including at least one switchable optical element controllable to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the foveation unit is configured to project light from the projection unit onto a central region of the retina of the eye; and in the peripheral mode, the foveation unit is configured to project light from the projection unit onto a peripheral region of the retina, the peripheral region being larger than and including the central region. Corresponding methods and projection devices are also provided.

Description

Technical Field

[0001] Technical Field Exemplary aspects herein relate to near-eye projection of images into a user's eye, particularly optical systems, projection devices, and methods.

Background Art

[0002] Background Optical systems for projecting light into a user's eye (or both eyes) are used in extended reality (XR) settings such as augmented reality, virtual reality, or mixed reality. These optical systems are used to form virtual images that can be combined with incident light from the environment the user views (in the case of augmented reality or mixed reality) or reproduce a virtual environment the user views (in the case of virtual reality).

[0003] Typically, the elements of the optical system are placed near the user's eyes (i.e., they are optical systems for near-eye projection), so that image projection can be performed with a more compact optical system, thereby assisting portability and user convenience. For example, the elements of these optical systems can be part of a device that the user wears or holds, such as a head-mounted display.

[0004] To provide an immersive user experience, the virtual image projected onto the user's eye should be adapted to the field of view the user is currently looking at. In particular, the light projected onto the user's eye should reach a specific part of the retina, which will then determine how the user sees the virtual image, so it is necessary to adapt the direction of the projected light, particularly the angle at which the light enters the user's eye, to the user.

[0005] Therefore, there is a need for optical systems and methods that enhance the user experience, particularly provide a more immersive user experience, by improving the adaptability of the virtual image projected onto the user's eye.

[0006] Specifically, by projecting light so that the point of convergence of light within the eye coincides with the eye's center of rotation, it is possible to adapt the virtual image to the user's field of view, thereby keeping the image always centered on the retina of the eyeball. However, because the angle of incidence of light is limited by the pupil, the light is projected onto only a small portion of the retina. As a result, the size of the virtual image seen by the user is limited. [Overview of the project] [Problems that the invention aims to solve]

[0007] Therefore, there is a need for optical systems and methods that improve adaptability to the user's field of vision while enabling the projection of light onto a wider area of ​​the retina.

[0008] In addition, optical systems are resource-constrained in terms of size, battery life, and computing resources. Therefore, there is a need for optical systems and methods that improve compactness, suppress the increase in required resources, and enhance the user experience. [Means for solving the problem]

[0009] overview According to a first exemplary embodiment, an optical system is provided for myopic projection of an image into the eye of a user, the optical system comprising a projection unit and a focusing unit, the projection unit configured to project light encoding an image, and the focusing unit configured to focus the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina, the at least one convergence point located on or near the pupil of the eye, the respective position of each convergence point on the convergence arc corresponding to the angle of rotation of the eye around the axis of rotation of the eyeball, and the convergence arc having a center of curvature substantially coincides with the axis of rotation of the eyeball.

[0010] By focusing the projected light onto a point on the convergence arc, the projected light can enter the eye at a larger angle of incidence than when the light is focused onto a point near the center of eye rotation, thereby enabling a larger image to be displayed to the user, and the projected light can enter the eye within an angular range that can be held in the center of the retina, thereby avoiding the so-called window effect.

[0011] This allows the virtual image projected into the user's eye to be more adapted to the user's field of vision, thereby improving the user experience.

[0012] Preferably, the distance between the center of curvature and the center of eye rotation is 1.5 cm or less, more preferably 1 cm or less, and most preferably 0.5 cm or less.

[0013] Reducing the distance between the center of curvature of the converging arc and the center of eye rotation makes it possible to increase the field of view provided by the optical system.

[0014] Preferably, the distance between the arc and the pupil is kept substantially constant regardless of the angle of rotation of the eye.

[0015] Therefore, the range of incident angles of projected light rays entering the eye can be kept substantially constant regardless of the eye's rotation angle (i.e., the direction of the pupil), and the image can be projected onto the same area of ​​the retina, allowing the user to see a virtual image of substantially constant size.

[0016] Preferably, the optical system is configured to move at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably the macula of the retina, more preferably one of the perifovea, parafovea, fovea, foveal avascular zone, foveal fovea and foveal fovea of ​​the retina.

[0017] By shifting the convergence point along the convergence arc, light can continue to enter the eye even as the eye rotates, while avoiding the increase in energy required to project light into the eye (compared to the case where light converges on multiple separate convergence points corresponding to different rotation angles of the eye).

[0018] Preferably, the optical system further includes an operating unit configured to move at least one convergence point around a convergence arc in response to rotation of the eye around the eye rotation axis, the arc having a center of curvature that substantially coincides with the eye rotation axis.

[0019] Preferably, the operating unit is configured to move at least one element of the projection unit and the condenser unit.

[0020] Preferably, the operating unit includes at least one optical element for manipulating light from the projection unit towards the condenser unit.

[0021] Preferably, the at least one optical element includes an operable mirror.

[0022] Preferably, the at least one optical element includes a plurality of switchable lenses, each switchable lens having a respective orientation and being configured to direct light from the projection unit onto different portions of the condenser unit, whereby the condenser unit converges the light onto different convergence points on the convergence arc.

[0023] Thus, the position of the convergence point light can be changed by controlling the state of each switchable lens.

[0024] Preferably, the at least one optical element includes an element having controllable phase modulation.

[0025] Preferably, the projection unit is configured to project a plurality of lights that respectively encode respective portions of an image, and the condenser unit is configured to converge each light onto a corresponding one of a plurality of convergence points, the position of each convergence point on the convergence arc corresponding to a respective rotation angle of the eye around the eye rotation axis, whereby lights converging on adjacent convergence points encode adjacent portions of the image.

[0026] Thus, depending on the rotation angle of the eye, the light focused on one of the different convergence points coincides with the pupil and enters the eye, forming a (virtual) image on the retina.

[0027] By projecting light that converges simultaneously on a plurality of convergence points (which may be referred to as a static eye box), the optical system can be simplified and thus become more compact and require fewer resources, for example by omitting elements that move the convergence points or elements that track the position of the eye.

[0028] Preferably, at least one convergence point is located on a convergence surface, the convergence surface is substantially parallel to the pupil of the eye, and includes a convergence arc.

[0029] The convergence surface may have a center of curvature that substantially coincides with the center of rotation of the eyeball. The convergence surface may substantially coincide with the surface of a sphere approximating the eye, or the convergence surface may be substantially parallel to the surface of a sphere having a larger or smaller radius of curvature, as described above in the case of a convergence arc, with the center of curvature shifted relative to the center of the sphere, or both.

[0030] The optical system may be configured to move at least one convergence point on the convergence surface along both dimensions on the surface, or the optical system may be configured to project light onto a plurality of convergence points (not necessarily simultaneously) on the convergence surface, in which case the convergence points are spaced apart (or dispersed) from each other along both dimensions on the convergence surface.

[0031] Thus, even when the eye rotates around both axes of rotation of the eyeballs (or the centers of rotation of the eyeballs), the projected light can continue to enter the eye.

[0032] Preferably, the projection unit is configured to project a first light that encodes a first image for the user's first eye and a second light that encodes a second image for the user's second eye, and the focusing unit is configured to focus the first light to at least one first convergence point so that the projected light enters the first eye and forms an image on the retina of the first eye, and to focus the second light to at least one second convergence point so that the projected light enters the second eye and forms an image on the retina of the second eye.

[0033] Therefore, the optical system can project a stereoscopic image, thereby giving the user a sense of depth in the virtual image.

[0034] Preferably, the projection unit includes a first projection unit configured to project a first light and a second projection unit configured to project a second light, and / or the focusing unit includes a first focusing unit and a second focusing unit, the first focusing unit configured to focus the first light to at least one first convergence point, and the second focusing unit configured to focus the second light to at least one second convergence point.

[0035] Therefore, each part of the projection and / or light-gathering section can be individually controlled to project the image into the user's eye.

[0036] Preferably, the projection unit includes a display configured to emit the image to be displayed and a focusing element for converging light rays toward a focusing unit, and optionally the projection unit includes a spatial filter for receiving the focused light rays.

[0037] Preferably, the projection unit includes a coherent light source, preferably a laser, configured to emit substantially coherent light, and optionally includes at least one laser beam scanning system, which includes a collimating element configured to collimate substantially coherent light, a phase modulator configured to encode an image by modulating substantially coherent light, and at least one operable mirror.

[0038] Preferably, the optical system further includes a foveal unit configured to switch between a foveal mode and a peripheral mode, wherein in the foveal mode, the foveal unit is configured to project light from a projection unit onto the central region of the retina of the eye, and in the peripheral mode, the foveal unit is configured to project light from a projection unit onto the peripheral region of the retina, wherein the peripheral region is larger than and includes the central region.

[0039] Because the central region of the retina has higher sensitivity, images projected in foveal mode may have higher resolution than images projected in peripheral mode, and this does not result in a significant change in the quality of the image perceived by the user.

[0040] Therefore, the resource requirements of the optical system (e.g., resolution, bandwidth, optical power, and computation) can be reduced while avoiding a significant impact on the quality of the projected image.

[0041] Preferably, the projection section of the optical system, including the foviation section, is configured to project light as substantially collimated light.

[0042] Preferably, the foviation unit is configured to increase the angular size of light incident on the eye in peripheral mode.

[0043] In other words, the angular size of light projected onto the eye is larger in peripheral modes than in foveal modes (i.e., in peripheral modes, light enters the pupil from a wider angular range).

[0044] Preferably, the foveation portion is configured to increase the focal length of light incident on the eye in the foveal mode.

[0045] Preferably, the foviation section includes at least one light-gathering element having a controllable focal length.

[0046] Preferably, the foviation section includes at least one switchable optical element, each of which is optionally configured to allow light to pass through without substantially changing the angular size of the light when switched to a first state.

[0047] Preferably, at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length, wherein the second focal length is different from the first focal length.

[0048] Preferably, the first switchable element is positioned at a first distance from the display, and the second switchable element is positioned at a second distance from the display.

[0049] Preferably, the difference between the first distance and the second distance is such that the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

[0050] Preferably, in the foveal mode, the first switchable element is configured to be switched to a second state and the second switchable element is configured to be switched to a first state, and in the peripheral mode, the first switchable element is configured to be switched to a first state and the second switchable element is configured to be switched to a second state.

[0051] Preferably, at least one switchable optical element includes a first switchable divergent element, a first switchable convergent element, and a second switchable convergent element, wherein the first switchable divergent element is positioned at a first distance from the projection area, and the first switchable convergent element is positioned at a second distance from the projection area, the second distance being greater than the first distance.

[0052] Preferably, in the foveal mode, the first switchable divergent element is configured to be switched to a first state, the first switchable converging element is configured to be switched to a first state, and the second switchable converging element is configured to be switched to a second state, thereby the light from the projection unit is focused by the second switchable converging element to form an image on the central region of the retina, and in the peripheral mode, the first switchable divergent element is configured to be switched to a second state, the first switchable converging element is configured to be switched to a second state, and the second switchable converging element is configured to be switched to a first state, thereby the light from the projection unit is diverged by the first switchable divergent element onto the first switchable converging element and focused by the first switchable converging element to form an image on the peripheral region of the retina.

[0053] Preferably, the second switchable convergence element is positioned at a third distance from the projection such that the convergence point of the first switchable convergence element and the convergence point of the second switchable convergence element substantially coincide with each other.

[0054] Preferably, at least one switchable optical element further includes a second switchable divergent element, the second switchable divergent element being switched to a second state in foveal mode and configured to increase the angular size of the light focused by the first switchable converging element, the second switchable converging element being positioned at a third distance from the projection area, and the second switchable divergent element being positioned at a fourth distance from the projection area, the first, second, third and fourth distances being configured such that the point of convergence of light in foveal mode and the point of convergence of light in peripheral mode substantially coincide with each other.

[0055] Preferably, the projection unit includes a light field display configured to project light that encodes a plurality of elemental images in a three-dimensional light field, and each elemental image forms a part of the image.

[0056] Preferably, the projection unit includes a phase modulator configured to project light that encodes a holographic image.

[0057] According to a second exemplary embodiment, an optical system is provided for myopic projection of an image into the eye of a user, the optical system comprising a projection unit configured to project light encoding an image, and a foveation unit comprising at least one switchable optical element controllable to switch between a foveal mode and a peripheral mode, in which the foveal mode, the foveation unit is configured to project light from the projection unit onto a central region of the retina of the eye, and in which peripheral mode, the foveation unit is configured to project light from the projection unit onto a peripheral region of the retina, the peripheral region being larger than and including the central region.

[0058] By using at least one switchable optical element, it becomes possible to distinguish between a more sensitive image projected onto the central region of the retina (e.g., the fovea) and an image projected onto the peripheral region of the retina, without requiring any additional elements.

[0059] This allows images with higher resolution to be projected onto the central region of the retina, ensuring an improved image quality perceived by the user. Conversely, images with lower resolution can be projected onto the peripheral region of the retina without affecting the image quality perceived by the user.

[0060] Therefore, the optical system improves the user experience while improving compactness and suppressing the increase in required resources.

[0061] Preferably, the projection unit is configured to project light as substantially collimated light.

[0062] Therefore, the focal length of the light projected into the eye can be determined by the focal length of the switchable optical element.

[0063] Preferably, each of at least one switchable element is configured such that, when switched to the first state, it allows light to pass through without substantially changing the angular size of the light.

[0064] Preferably, the foviation unit is configured to increase the angular size of light incident on the eye in peripheral mode.

[0065] Preferably, the foveation portion is configured to increase the focal length of light incident on the eye in the foveal mode.

[0066] Preferably, the foviation section includes at least one light-gathering element having a controllable focal length.

[0067] Preferably, at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length, wherein the second focal length is different from the first focal length.

[0068] Preferably, the first switchable element is positioned at a first distance from the display, and the second switchable element is positioned at a second distance from the display.

[0069] Preferably, the difference between the first distance and the second distance is such that the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

[0070] For example, the distance separating the second switchable element from the first switchable element (i.e., the difference between the second distance and the first distance) can be set based on the difference between the first focal length and the second focal length.

[0071] Therefore, the light converges to substantially the same point, regardless of whether the focal length of the light is changed by the first switchable element or by the second switchable element.

[0072] Preferably, in the foveal mode, the first switchable element is configured to be switched to a second state and the second switchable element is configured to be switched to a first state, and in the peripheral mode, the first switchable element is configured to be switched to a first state and the second switchable element is configured to be switched to a second state.

[0073] Preferably, at least one switchable optical element includes a first switchable divergent element, a first switchable convergent element, and a second switchable convergent element, wherein the first switchable divergent element is positioned at a first distance from the projection area, and the first switchable convergent element is positioned at a second distance from the projection area, the second distance being greater than the first distance.

[0074] Preferably, in the foveal mode, the first switchable divergent element is configured to be switched to a first state, the first switchable converging element is configured to be switched to a first state, and the second switchable converging element is configured to be switched to a second state, thereby the light from the projection unit is focused by the second switchable converging element to form an image on the central region of the retina, and in the peripheral mode, the first switchable divergent element is configured to be switched to a second state, the first switchable converging element is configured to be switched to a second state, and the second switchable divergent element is configured to be switched to a first state, thereby the light from the projection unit is diverged by the first switchable divergent element onto the first switchable converging element and focused by the first switchable divergent element to form an image on the peripheral region of the retina.

[0075] Preferably, the second switchable convergence element is positioned at a third distance from the projection such that the convergence point of the first switchable convergence element and the convergence point of the second switchable convergence element substantially coincide with each other.

[0076] For example, the distance separating the second switchable convergence element from the first switchable convergence element (i.e., the difference between the third distance and the first distance) can be set based on the difference between the focal length of the second switchable convergence element and the focal length of the first switchable convergence element.

[0077] Preferably, at least one switchable optical element further includes a second switchable divergent element, the second switchable divergent element being switched to a second state in foveal mode and configured to increase the angular size of the light focused by the first switchable converging element, the second switchable converging element being positioned at a third distance from the projection area, and the second switchable divergent element being positioned at a fourth distance from the projection area, the first, second, third and fourth distances being configured such that the point of convergence of light in foveal mode and the point of convergence of light in peripheral mode substantially coincide with each other.

[0078] Preferably, the foviation unit is configured to focus the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina, the at least one convergence point being located on or near the pupil of the eye on a convergence arc, the position of each convergence point on the convergence arc corresponding to the angle of rotation of the eye around the axis of rotation of the eyeball, and the center of curvature of the convergence arc substantially coincides with the axis of rotation of the eyeball.

[0079] Preferably, the optical system is configured to move at least one convergence point such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably the macula of the retina, more preferably one of the perifovea, parafovea, fovea, foveal avascular zone, foveal fovea and foveal fovea of ​​the retina.

[0080] Preferably, the optical system further includes an operating unit configured to move at least one convergence point around a convergence arc in response to the rotation of the eye around an axis of ocular rotation, wherein the center of curvature of the arc substantially coincides with the axis of ocular rotation.

[0081] Preferably, the operating unit is configured to move at least one element of the projection unit and the foviation unit.

[0082] Preferably, the operating unit includes at least one optical element for directing light from the projection unit toward the foviation unit.

[0083] Preferably, at least one optical element includes an operable mirror.

[0084] Preferably, at least one optical element includes a plurality of switchable lenses, each switchable lens having a respective orientation and configured to direct light from the projection section onto different portions of the foveation section, thereby causing the foveation section to focus the light onto different convergence points on the convergence arc.

[0085] Preferably, at least one optical element includes a phase modulator having controllable phase modulation.

[0086] Preferably, the projection unit is configured to project multiple lights, each encoding a different portion of the image, and the foviation unit is configured to converge each light onto a corresponding point among a plurality of convergence points, the position of each convergence point on the convergence arc corresponding to the respective rotation angles of the eye around the axis of eye rotation, so that the light converging on adjacent convergence points encodes adjacent portions of the image.

[0087] Preferably, at least one convergence point is located on a convergence plane, which is substantially parallel to the pupil of the eye and includes a convergence arc.

[0088] Preferably, the multiple convergence points are distributed along both dimensions of the convergence plane.

[0089] Preferably, the optical system is configured to move at least one convergence point along both directions of the convergence plane.

[0090] Preferably, the projection unit is configured to project a first light encoding a first image for the user's first eye and a second light encoding a second image for the user's second eye, and the foveation unit includes two foveation portions corresponding to each of the user's eyes, each foveal portion being configured to switch between a foveal mode for projecting light onto the central region of the retina of the corresponding eye and a peripheral mode for projecting light onto the peripheral region of the retina of the corresponding eye, the peripheral region being larger than and including the central region.

[0091] Preferably, the projection unit includes a first projection unit configured to project a first light and a second projection unit configured to project a second light.

[0092] Preferably, the projection unit includes a display configured to emit substantially collimated rays of light for displaying the image, and a focusing element for converging the rays toward the foveal region, and optionally the projection unit includes a spatial filter for receiving the focused rays.

[0093] Preferably, the projection unit includes a coherent light source, preferably a laser, configured to emit substantially coherent light, and optionally includes at least one laser beam scanning system, which includes a collimating element configured to collimate substantially coherent light, a phase modulator configured to encode an image by modulating substantially coherent light, and at least one operable mirror.

[0094] Preferably, the projection unit includes a light field display configured to project light that encodes a plurality of elemental images in a three-dimensional light field, and each elemental image forms a part of the image.

[0095] Preferably, in foveal mode, the foveation unit is configured to project light encoding a first plurality of elemental images from a light field display onto the central region of the retina of the eye, and in peripheral mode, the foveation unit is configured to project a second plurality of elemental images from a light field display onto the peripheral region of the retina, wherein the second plurality of elemental images are different from the first plurality of elemental images.

[0096] Preferably, the projection unit includes a phase modulator configured to project light that encodes a holographic image.

[0097] Preferably, in the foveal mode, the foveation unit is configured to project light encoding a first portion of the holographic image onto the central region of the retina of the eye, and in the peripheral mode, the foveation unit is configured to project light encoding a second portion of the holographic image onto the peripheral region of the retina, wherein the first and second portions of the holographic image are distinct from each other.

[0098] According to a third embodiment, a projection device is provided which includes an optical system according to the first or second embodiment of this specification, and at least one of one or more eye trackers and a computing unit for determining the rotation angle of the pupil of the eye.

[0099] Optionally, each eye tracker includes at least one camera for capturing an image of the eye, and the eye tracker is configured to determine the rotation angle of the pupil of the eye based on the image, and the rotation angle of the pupil is used to determine the position of at least one convergence point on the convergence arc.

[0100] By tracking the rotation angle of the pupil, an optical system can determine the position on the convergence arc (or convergence plane) that allows light to enter the eye. This can be used to determine the position on the convergence arc from which the convergence point is moved (e.g., in the case of a dynamic eyebox), or to determine the convergence point from which light is moved.

[0101] In the case of a static eyebox, the optical system can determine, based on the rotation angle of the pupil, a subset of one or more convergence points where the light encoding the image (for that light to enter the eye) converges, and selectively project only the light converging on the convergence points within that subset, thus reducing the energy required to project the image. In other words, the optical system can block the projection of light converging on convergence points outside the subset.

[0102] Optionally, the eye tracker is configured to determine the focal length of the lens of the eye.

[0103] The tracker can determine the instantaneous focal length, for example, when it detects a predetermined trigger, such as eye movement, or when it repeatedly occurs (e.g., periodically, such as every millisecond).

[0104] Therefore, the image projected onto the eye can be adapted to the eye's focal length. This makes it possible, for example, to introduce artifacts (or digital blur) into the image to improve the perceived depth of elements within the virtual image.

[0105] Optionally, the eye tracker includes at least one light source for illuminating the eye.

[0106] Preferably, the light source is not detectable by the eye in order to avoid affecting the image projected into the eye.

[0107] Preferably, the projection device is held or attached by the user.

[0108] Preferably, the projection device includes a headset mounted on the user's head.

[0109] Preferably, at least one element of the optical system is located within the housing on the headset.

[0110] Preferably, the calculation unit is configured to obtain a value indicating the rotation angle of the eye and to generate one or more first control signals for projecting light to render an image based on the indicated rotation angle of the eye.

[0111] Preferably, the calculation unit is configured to acquire values ​​from one or more eye trackers.

[0112] Preferably, the calculation unit is configured to determine a position on the convergence arc corresponding to an indicated eye rotation angle based on the acquired value, and to generate one or more second control signals for controlling at least one of the projection unit and the light-gathering unit, the one or more second control signals causing the light to converge to one or more convergence points corresponding to the determined position.

[0113] Preferably, the calculation unit is configured to determine the direction in the scene that the pupil is pointing based on the acquired values, and to render the portion of the scene corresponding to that direction.

[0114] Preferably, the optical system includes a computing unit configured to generate one or more first control signals to cause the projection unit to project light to render a first portion of the image for the central region of the retina and to switch the foveal unit to foveal mode, and to generate one or more second control signals to cause the projection unit to project light to render a second portion of the image for the peripheral region of the retina and to switch the foveal unit to peripheral mode.

[0115] Preferably, the calculation unit is configured to generate one or more first control signals and one or more second control signals to switch the projection unit and the foveation unit substantially simultaneously, the projection unit switching between projection light rendering a first portion and light rendering a second portion, and the foveation unit switching between foveal mode and peripheral mode.

[0116] According to a fourth exemplary embodiment, a method for myopic projection of an image into the eye of a user is provided, the method comprising projecting light encoding an image and focusing the light to at least one convergence point so that the projected light enters the eye and forms an image on the retina, the at least one convergence point located on or near the pupil of the eye on a convergence arc, the respective position of each convergence point on the convergence arc corresponding to the angle of rotation of the eye around the axis of rotation of the eyeball, and the convergence arc having a center of curvature substantially coincides with the axis of rotation of the eyeball.

[0117] According to a fifth exemplary embodiment, a method for myopic projection of an image into a user's eye is provided, the optical system comprising projecting light encoding an image and controlling at least one switchable optical element to switch between a foveal mode and a peripheral mode, in which the light encoding the image is projected onto a central region of the retina of the eye by at least one switchable optical element, and in which the light encoding the image is projected onto a peripheral region of the retina by at least one switchable optical element, the peripheral region being larger than and including the central region.

[0118] Preferably, any of the optical systems according to the first exemplary embodiment, the optical systems according to the second exemplary embodiment, the projection apparatus according to the third exemplary embodiment, the method according to the fourth exemplary embodiment, or the method according to the fifth exemplary embodiment is for projecting augmented reality (XR) images, more preferably for projecting one of augmented reality (AR), virtual reality (VR), or mixed reality (MR) images.

[0119] Preferably, the XR image includes at least one virtual element representing digital information.

[0120] Preferably, each virtual element is either captured by imaging means, superimposed on the user's physical environment as seen by the user, or forms part of the virtual environment.

[0121] For simplicity, if we approximate the eye as a sphere, the center of the sphere can be defined as the center of rotation of the eye, and therefore the rotation of the eye can be defined as a rotation around the center of the sphere, i.e., the center of rotation of the eye. A rotation around the center of rotation of the eye can also be defined as a rotation around two separate axes of rotation of the eye that intersect at the center of rotation of the eye and lie in a plane parallel to the pupil of the eye. The axes of rotation of the eye can be two axes defined according to List's law (e.g., the vertical and transverse axes of the eye), i.e., two axes that lie in the List plane of the eye.

[0122] In certain embodiments of this specification, the light projected into the eye converges to at least one convergence point, which may be a point located anterior to the center of rotation of the eyeball and substantially along the pupillary axis or foveal axis of the eye (i.e., a point that is likely to coincide with the pupil of the eye such that the light converging on the convergence point enters the eye and is projected onto the retina).

[0123] In certain embodiments of this specification, the point of convergence lies on a convergence arc. The center of curvature of the convergence arc may be defined by an axis coinciding with one of the axes of ocular rotation and a radius of curvature corresponding to the radius of the sphere approximating the eye. However, it will be understood that there may be a difference between the center of curvature of the convergence arc and the center of ocular rotation (i.e., the center of the sphere), a difference between the radius of curvature of the convergence arc and the radius of the sphere (preferably a difference of 1 cm or less), or both, and these may be caused by variations in the dimensions of the human eye.

[0124] If the center of curvature of the converging arc does not coincide with the center of the sphere, it may be located between the pupil and the center of rotation of the eye (for example, within the volume around the pupillary axis or macular axis of the eye, such as a cone with the iris as the base and the center of rotation of the eye as the apex) or between the center of rotation of the eye and the retina (for example, within the volume around the pupillary axis or foveal axis of the eye, such as a cone with the central retina as the base and the center of rotation of the eye as the apex). If the converging arc has a radius of convergence greater than the radius of the sphere, the converging arc will be outside the eye and in front of the pupil.

[0125] The term "angle of rotation of the eye" can also be defined as the orientation of the eye (or pupil), the angle of rotation of the pupil, or the position of the pupil. The angle of rotation can be defined relative to the default axis corresponding to the orientation of the pupil when the eye is considered to be at rest.

[0126] In certain embodiments of this specification, an optical system moves at least one convergence point based on the rotation angle of the eye. This may be done, for example, to focus light onto a point substantially coinciding with the center of the pupil of the eye. Since the convergence point moves dynamically with the rotation of the eye, this may be called a dynamic eyebox.

[0127] In certain embodiments of this specification, one or more operating parts may be provided for moving at least one element of the projection unit and the light-gathering unit. The operating part (or each operating part) may include electromechanical means such as actuators and motors configured to move the components of the projection unit, the light-gathering unit, or both.

[0128] In certain embodiments of this specification, the operating section (or each operating section) may include at least one optical element. The optical element of the operating section may be positioned along the optical path of light between the projection section and the focusing section. The optical element of the operating section may be movable to change the optical path of light toward the focusing section, thereby causing a shift in the convergence point.

[0129] In certain embodiments of this specification, at least one optical element may have a controllable phase modulator, such as a phase or amplitude spatial light modulator (SLM), a lens having a controllable phase modulator, etc.

[0130] In certain embodiments of this specification, at least one convergence point is located on a convergence plane. In other words, the optical system may move the convergence point along two dimensions on the convergence plane, or the optical system may be configured to focus light onto convergence points distributed along two dimensions on the convergence plane. The two dimensions are understood to be separate (e.g., orthogonal) dimensions.

[0131] In certain embodiments of this specification, multiple beams of light encoding different parts of an image are converged onto a corresponding point of convergence. These convergence points may be arranged (or dispersed) at equal intervals along a convergence arc or plane, or the distance between adjacent convergence points may be varied along the arc or plane, for example, by providing a higher density in the arc portion corresponding to the rotation angle when the eye is stationary. In some cases, adjacent convergence points may be close enough to each other that light converged at two or more convergence points enters the eye simultaneously, projecting different parts of the image onto different parts of the retina (which may overlap).

[0132] In certain embodiments of this specification, one or more eye trackers may be used to track one or both of a user's eyes. Each eye tracker may be a camera-based tracker, a tracker using a physiological sensor (e.g., based on an ECG signal), or other sensing methods such as LiDAR that can track a pupil or a protrusion formed by the cornea. Each tracker may be a camera-based tracker, in which case the camera may be a camera that captures images in the visible wavelength range (e.g., a red-green-blue (RGB) camera) or the infrared (IR) range (e.g., an IR camera, a near-IR camera). Each tracker may be an IR light source or visible light (e.g., white light) that is off-center so as not to be detected by the eye, or may include at least one light source having a luminance below a threshold to avoid or reduce its effect on the image projected into the eye.

[0133] In certain aspects of this specification, an optical system may project an image into the user's eyes by including, for example, separate parts for each eye (e.g., separate projection units, focusing units, operating units and / or eye trackers), and each eye's part may be individually controlled.

[0134] In certain embodiments of this specification, displays and focusing elements may be used. For example, this may be a self-emissive display that emits light rays. The display (e.g., a self-emissive display) may emit collimated rays or include a collimating optical system for collimating rays, or the rays emitted by the display may diverge if, for example, the focusing element is configured to receive divergent rays at its position and / or focal length and converge them toward the focusing part. Spatial filters may be, for example, pinhole (or Fourier) filters.

[0135] In certain embodiments of this specification, a coherent light source may be used. A coherent light source may be any light source that emits light having at least a certain level of coherence, such as sufficient coherence to be modulated by a phase modulator (e.g., an SLM such as a device including an array of movable micromirrors, such as a digital photoprocessing (DLP) or other type of digital micromirror device (DMD)). A coherent light source may include a laser.

[0136] When the coherent light source is a laser, the laser may include one or more laser diodes, each of which may be configured to emit light at a given wavelength (e.g., a diode that emits light at red, green, blue, and infrared wavelengths). Preferably, the laser may include multiple diodes (e.g., at least red, green, and blue diodes) to emit light substantially across the visible spectrum.

[0137] In certain embodiments of this specification, a light field display may be used to project light encoding elemental images. Each elemental image in the light field has a corresponding perceptual depth (e.g., corresponding to the distance between the retina and the focal point where the light rays associated with the elemental image converge), and the light of each elemental image may be projected onto different parts of the retina. The portion formed by each elemental image may overlap with at least one portion formed by adjacent elemental images.

[0138] In certain embodiments of this specification, the optical system may include a foveation unit and a projection unit having a light field display. In such a case, the light field display may be configured to project light encoding different elemental images for foveal and peripheral modes.

[0139] In certain embodiments of this specification, the optical system may include a foveation unit and a projection unit including a phase modulator for projecting light encoding a holographic image. In such a case, the phase modulator may project light encoding different holographic images for foveal and peripheral modes.

[0140] In certain embodiments of this specification, at least one switchable optical element may be used. In such cases, each switchable optical element may be, for example, a converging or diverging element configured to change the angular size of the light passing through it. For example, each switchable optical element may be a polarizing switchable lens such as a converging / diverging lens, a mirror, a holographic polymer-dispersed liquid crystal (HPDLC) layer, an Alvarez lens, or a polarizing volume diffraction grating.

[0141] In certain aspects of this specification, a switchable optical element (hereinafter also referred to as SOE) may be defined as being switchable between a first state and a second state. The switchable optical element may have different focal lengths in the first state and the second state.

[0142] For example, the first state may correspond to a state in which the SOE allows light to pass through without substantially changing the angular size of the light, while in the second state, the SOE may change the angular size of the light (for example, by converging or diverging the incident rays, or by collimating the converging / diverging incident rays). In other words, in one of the states (for example, the first state), the SOE may correspond to a neutral lens.

[0143] In another example, in both the first and second states, the SOE can change the angular size of the incident ray.

[0144] In certain aspects of this specification, the SOE may be switched between a first state and a second state by controlling the electrical signals (e.g., control voltage) that the SOE receives. The SOE switches to the second state when it receives an electrical signal, and switches back to the first state when it no longer receives an electrical signal (and vice versa). Thus, the first and second states may be defined as the "off" state and the "on" state, or the "default" state and the "operating" state.

[0145] As used herein, the term “central region of the retina” may mean, for example, the macula of the retina, preferably one of the perifovea, parafovea, fovea, fovea avascular zone, foveal fovea, and foveal fovea of ​​the retina.

[0146] As used herein, the term “eyebox” may mean the area in front of the projection device where the user’s eyes can be positioned to view the image projected by the projection device without distortion or loss of any part of the projected image.

[0147] As used herein, the term “field of view” may mean the angular range of images displayed to the user’s eyes. [Brief explanation of the drawing]

[0148] Brief explanation of the drawing [Figure 1A] This is a schematic diagram of a conventional projection device. [Figure 1B] This is a schematic diagram of a conventional projection device. [Figure 2A] This is a schematic diagram of another conventional projection device. [Figure 2B] This is a schematic diagram of another conventional projection device. [Figure 3A] This is a schematic diagram of a projection device in an exemplary embodiment. [Figure 3B] This is a schematic diagram of a projection device in an exemplary embodiment. [Figure 4] This is a schematic diagram of a projection device in an exemplary embodiment. [Figure 5] A computing unit for controlling an optical system in an exemplary embodiment is shown. [Figure 6A] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 6B] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 7A] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 7B] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 8A] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 8B] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 9A] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 9B] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 10A] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 10B] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 11A] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 11B] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 12A] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 12B] This is a schematic diagram of an optical system in an exemplary embodiment. [Figure 13A] This is a schematic diagram of an optical system using an optical waveguide in an exemplary embodiment. [Figure 13B] This is a schematic diagram of an optical system using an optical waveguide in an exemplary embodiment. [Figure 14A]This is a schematic diagram of a peripheral image projected by an optical system in an exemplary embodiment. [Figure 14B] This is a schematic diagram of a foveal image projected by an optical system in an exemplary embodiment. [Figure 14C] This is a schematic diagram of the foveal and peripheral images projected by the optical system in an exemplary embodiment. [Figure 14D] This is a schematic diagram of the foveal and peripheral images projected by the optical system in an exemplary embodiment. [Figure 15A] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 15B] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 16A] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 16B] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 17A] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 17B] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 18A] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 18B] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 19A]This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 19B] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 20A] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 20B] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 21A] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 21B] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 22A] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 22B] This is a schematic diagram of an optical system in which the foveal mode and peripheral mode are used, respectively, in an exemplary embodiment. [Figure 23A] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 23B] This is a schematic diagram of an optical system in which the foveation section is used in peripheral mode and foveal mode, respectively, in an exemplary embodiment. [Figure 24A] This is a schematic diagram of an optical system using a light field display in an exemplary embodiment. [Figure 25A] This is a schematic diagram of an elemental image projected by an optical system using a light field display in an exemplary embodiment. [Figure 25B] This is a schematic diagram of an elemental image projected by an optical system using a light field display in an exemplary embodiment. [Figure 25C] This is a schematic diagram of an elemental image projected by an optical system using a light field display in an exemplary embodiment. [Figure 25D] This is a schematic diagram of an elemental image projected by an optical system using a light field display in an exemplary embodiment. [Figure 25E] This is a schematic diagram of an elemental image projected by an optical system using a light field display in an exemplary embodiment. [Modes for carrying out the invention]

[0149] Detailed explanation For simplicity, the following description refers to the eye rotating around an axis of rotation in two-dimensional space, which is understood to be an axis perpendicular to the plane shown and located near the center of the sphere representing the eye. Thus, the optical systems described herein can adapt the image projected into the eye to the movement of the eye along one of the eye's axes of rotation (e.g., rotation around a vertical axis as the eyeball moves from side to side, or rotation around a horizontal axis as the eye moves up and down).

[0150] However, it will be understood that the optical systems described herein can substantially adapt images to eye movements along the axis of rotation of the eye (e.g., both the vertical and horizontal axes described above), thereby allowing the images to adapt to any movement of the eye. In this case, the angle of rotation of the eye can be considered as an angle along the two axes of rotation of the eye. Thus, the optical systems described herein can be configured to focus light onto at least one convergence point located on an arcuate plane containing a convergence arc. In other words, this “convergence plane” can be defined by two convergence arcs in two separate dimensions, each arc having a center of curvature substantially coinciding with the axis of rotation of the eyeball, and each arc being on or near the pupil of the eye. Thus, the defined plane may be substantially parallel to the outer surface of the cornea, iris, or lens of the eye.

[0151] As will be described later, elements of an optical system (e.g., eye tracker, computing unit, projection unit, etc.) can be connected to communicate with each other. Connection may be by establishing one or more suitable communication links. Each communication link may be a wired communication link such as a wireless communication link, e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), near-field communication (NFC), or serial communication link (e.g., I 2 This could include C, SPI, RS232, RS422, RS432, communication buses, etc. Each communication link may not be permanent.

[0152] Figures 1A and 1B show schematic diagrams of a conventional projection device 1 for projecting an image into the eye. The projection device 1 may also be known as a myopia display. The projection device 1 includes a projection unit (not shown) configured to generate and modulate light such that the projected light encodes an image in an optical system 2. The optical system 2 includes one or more optical elements configured to focus the projected light onto the eye 6 such that an image 5 is formed on the retina of the eye 6. Figure 1A shows the device 1 focusing the light onto a first convergence point 3a, which lies on a convergence surface 4 just outside the eye (however, in some configurations, the convergence surface 4 is located on or inside the surface of the eye). The projection device 1 further includes an eye tracker (not shown) that tracks the movement of the eye 6 around an axis of rotation 7 of the eye 6.

[0153] Figure 1B shows the apparatus 1 in the second configuration. The eye 6 is rotated around axis 7 to a new position. The eye tracker measures the position of the eye 6, and the optical system 2 dynamically changes to focus the light along the focusing plane 4 from the first focusing point 3a to a second focusing point 3B which is displaced. The position of the focusing point is selected to correspond to the position of the pupil of the eye 6 so that light enters the eye and an image 5 is formed on the retina. The eye box E of the apparatus 1 is defined by the maximum displacement position of the focusing point 3 that the apparatus 1 can generate.

[0154] The system described above provides a dynamic eyebox in which only a portion of the eyebox that enters the pupil is projected into the eye. This saves energy in the projection device 1 because light is not projected onto points on the converging plane that do not enter the pupil of eye 6. However, as may be evident from the configuration shown in Figure 1B, when the eye rotates away from the center of the converging plane 4, the image 5 shifts laterally on the retina, so that the center C' of the image 5 no longer aligns with the center C of the retina. Thus, the viewer of the projection device 1 can perceive the edge of the eyebox (or the edge of the field of view) and move eye 6 so that the center of the retina is on the edge of the eyebox. As a result, the user perceives the projected virtual scene as if it were being displayed through a window with an edge. This is shown in Figure 1C, which shows the image of the projection device 1 as perceived by the user. When the center of the user's eye 6 is on the eyebox, the user's central field of view, indicated by F1, fits within the edge of the image, so the user does not perceive any edges. As the user's eye 6 moves away to the periphery of the eye box, the user's central field of vision, indicated by F1, begins to perceive the edges of the image, and as a result, the user generally perceives the image as being viewed through a fixed window (also known as the window effect).

[0155] To solve this problem, a projection device 1 as shown in Figures 2A and 2B may be provided. The projection device 1 includes a projection unit (not shown) configured to project light that encodes an image in an optical system 2. The optical system 2 includes one or more optical elements configured to focus the projected light onto the eye 6 so that an image 5 is formed on the retina of the eye 6. Unlike the device shown in Figure 1A, the device 1 focuses the light onto a convergence point on or near the axis of rotation 7 of the eye 6. The projection device 1 further includes an eye tracker (not shown) that tracks the movement of the eye 6 around the axis of rotation 7 of the eye 6. Figure 2B shows the device 1 in a second configuration. As shown in Figure 2B, the eye 6 is rotated to a new position around the axis 7. The eye tracker measures the position of the eye 6, and the optical system 2 dynamically changes to rotate the projected light to match the rotation of the eye 6 around the axis 7, so that the light enters the eye and an image 5 is formed on the retina. The eyebox E of device 1 is defined by the maximum rotational position of the projected light that device 1 can generate.

[0156] Unlike the systems shown in Figures 1A and 1B, the window effect is eliminated because image 5 is kept in the center of the retina. However, because image 5 formed on the retina is limited by the optical aperture of the pupil, its size is reduced compared to the devices shown in Figures 1A and 1B, and it limits the field of view that the user sees in a given time.

[0157] Figures 3A and 3B show schematic diagrams of projection devices 10 according to one or more embodiments. The projection device 10 includes a projection unit (not shown) configured to project light that encodes an image in an optical system 20. The optical system 20 includes one or more optical elements configured to focus the projected light onto the eye 6 so that an image 5 is formed on the retina of the eye 6. Unlike the devices shown in Figures 1A, 1B, 2A, and 2B, the device 10 focuses the light onto a point on a convergence arc 41 in front of the rotation axis 7 of the eye 6 (i.e., the arc is located between the rotation axis 7 and the optical system 20), more specifically onto the pupil of the eye 6 or its vicinity (inside or outside), while the image still rotates around a rotation axis 40 that is close to or coincides with the rotation axis 7 of the eye. In other words, the distance between the point of convergence of the light and the rotation axis 7 of the eye (and image) is selected to be approximately equal to the distance between the pupil or lens of the eye 6 and the rotation axis 7. For example, the distance between the convergence point and the axis of rotation 7 (which may correspond to the radius of curvature of the convergence arc) can be 1 cm to 2 cm.

[0158] The projection device 10 may further include an eye tracker (not shown) that tracks the movement of the eye 6 around the rotation axis 7 of the eye 6. Figure 3B shows the device 10 in a second configuration. As shown in Figure 3B, the eye 6 is rotated to a new position around the axis 7. The eye tracker measures the position of the eye 6, and the optical system 20 dynamically changes to rotate the image 5 to match the rotation of the eye 6 around the axis 7, thereby allowing light to enter the eye and forming the image 5 in the center of the retina. The eye box E of the device 10 may correspond to a region defined by the maximum rotational position of the projected light that the device 10 can generate.

[0159] Unlike apparatus 1 shown in Figures 2A and 2B, apparatus 10 allows for a wider field of view because the point of convergence of the projected light is close to the pupil, while still having an image centered on the retina of eye 6 (i.e., on or near the center of the central region of the retina) to avoid the window effect. Preferably, image 5 may be centered on or near the macula of the retina. More preferably, image 5 is centered on or near the region that forms the macula, such as the perifovea, parafovea, fovea, foveal avascular zone, fovea centralis, and foveal fovea.

[0160] In three-dimensional space, the axis of rotation 40 could be, for example, a vertical axis that allows image 5 to remain centered in the central retinal region during lateral movement of the eye, or a horizontal axis that allows image 5 to remain centered in the central retinal region when the eye is moved up and down.

[0161] Figure 4 shows a projection device 10 according to one or more embodiments. The projection device 10 includes an eye tracker 60, a computing unit 50, and an optical system 20. The projection device may further include any suitable power source, such as a battery or other AC or DC power source, and / or a power connector for connecting to an external AC or DC power source. The eye tracker 60 may be any suitable eye tracker configured to track the position of the user's pupil. The eye tracker 60 is communicatively connected to the computing unit 50 (e.g., via a communication line or wireless link) and is configured to output a value to the computing unit 50 indicating the position of the pupil 6a of the user's eye 6. The computing unit 50 is configured to render a scene (i.e., a 2D or 3D image or video that may include one or more virtual elements) and to render the scene based on the position of the eye 6. More specifically, the computing unit 50 is configured to calculate the direction in the scene that the pupil 6a is pointing to and to render the scene for that portion.

[0162] The eye-tracking element 60 may include, for example, an infrared (IR) camera and / or an RGB camera, as well as an IR light source for the eye. The IR light source illuminates the eye so that it can be tracked even under low-light conditions, but is invisible to the user (because the light frequency is outside the human visible spectrum). The eye tracker 60 may be mounted in the device 100 separately from the optical system 20, or one or more optical elements within the optical system 20 may be utilized to direct illumination light to the eye 6 and reflect the reflected light back to the eye-tracking element 60. In some cases where both eyes are tracked, the same eye-tracking element 60 may be configured to track each eye, or separate eye-tracking elements may track each eye separately.

[0163] The projection device further includes an optical system 20. The optical system 20 includes a projection unit 21 configured to project light that encodes an image of the rendered scene, which is generated by the computing unit 50. The optical system further includes an operating unit 22 configured to manipulate the light rays emitted from the optical system 20 around a rotation axis 40 based on the position of the pupil 6a (or alternatively based on direct communication with the eye tracker 60) as directed by the computing unit 50 (see Figures 3A and 3B). Finally, the optical system includes a focusing unit 23 configured to converge the light rays encoding the image to a point on a convergence arc 41. The projection unit 21, the operating unit 22, and the focusing unit 23 may be formed as separate optical elements, or two or more elements may be contained within a single optical element that performs two or more functions of elements 21, 22, and 23. Individual projection units 21, operating elements 22, and light-gathering elements 23 may be provided for each eye, or the same elements 21, 22, and / or 23 may be used to project images into both eyes (e.g., sequentially in time or spatially). In such cases, the images projected into each eye may differ from one another. For example, with offsets to provide a stereoscopic image, a portion (e.g., half) of a larger image may be projected into each eye.

[0164] The computing unit 50 may be able to control all elements, or separate computing units may be provided to control different optical components for each eye (i.e., the projection device includes a first eye tracker, computing unit and / or optical system for the left eye and a second eye tracker, computing unit and / or optical system for the right eye).

[0165] In some embodiments, the projection unit 21 may include a light-emitting element and a light-modulating element that modulates the light generated by the light-emitting element to encode an image.

[0166] The light-gathering element 23 may be an optical coupler that redirects and focuses light to project an image onto the eye while allowing ambient light to pass through, thereby superimposing the virtual image formed by the optical system 20 onto the real-world environment (augmented reality). In embodiments where the projection device is a pair of smart glasses, the optical coupler may be formed as part of the lens of the smart glasses. In other embodiments, the light-gathering element 23 may be provided in combination with an opaque lens, thereby providing a completely virtual field of view by preventing light from the environment from merging with the image (virtual reality).

[0167] In some embodiments, the focusing element 23 or optical coupler may be a curved transparent reflector or an off-axis concave (or parabolic) mirror. The projection unit 21 and operating element 22 may be mounted within or on the arm of the eyeglasses, and light may be projected through the air toward the eyeglass lens and reflected and focused toward the eye by a concave reflector on the lens that functions as a focusing lens.

[0168] In some embodiments, the focusing element 23 or optical coupler includes one or more flat or curved holographic optical elements (HOEs), each of which may be a photopolymer formed on a glass substrate. The HOEs may be recorded to function as off-axis concave mirrors, behaving similarly to curved reflective mirrors. The HOEs can be manufactured to be much thinner while having the same optical properties as curved reflective mirrors.

[0169] In some embodiments, the light-gathering element 23 or optical coupler may include one or more holographic polymer-dispersed liquid crystals (HPDLCs) or switchable Bragg gratings, each of which functions as a switchable HOE (Hot Eye Energy). (When switched off, the HPDLC or Bragg grating becomes transparent, transmitting light without substantially changing the angular aspect ratio of the light.)

[0170] When an HOE or a single HPDLC is used, the tolerance for incident light that differs from the recorded angle may be low. This can therefore limit the eyebox of the projection device. Accordingly, in some embodiments, the optical coupler may include multiple HPDLC layers recorded for different operating angles, and the calculation unit 50 may be configured to switch on the HPDLC corresponding to the desired operating angle, thereby allowing the light to be correctly focused into the pupil.

[0171] In some embodiments, the focusing element 23 or optical coupler may include a waveguide through which light is projected and focused to the eye by total internal reflection (TIR). It will be understood that many suitable waveguide configurations can be used. In some embodiments, a holographic waveguide including in-couplers and out-couplers is used.

[0172] In some embodiments, at least one of the projection unit 21 and the focusing unit 23 may be operable by an operating unit 22 to manipulate the light around the rotation axis 40. In some embodiments, the projection unit 21 is movable to move the convergence point along one of the dimensions of the convergence plane, and the focusing unit is movable independently to move the convergence point along the other dimension of the convergence plane. Thus, by moving both the projection unit 21 and the focusing unit 23, the convergence point may be moved in accordance with any movement made by the eye.

[0173] In some embodiments, the eye tracker 60 may be omitted. Instead, the projection unit 21 may be configured to project light encoding multiple portions of the image 5, and the light-gathering unit 23 may be configured to focus each light onto a corresponding one of a plurality of convergence points located on a convergence arc (or convergence plane). The position of each convergence point corresponds to the respective rotation angle of the eye 6 around the rotation axis 4 of the eyeball. Thus, when the eye rotates by an angle corresponding to one of the convergence points, the light focused on that convergence point enters the eye and forms an image centered on the retina of the eye. Since light converging on adjacent convergence points can encode adjacent portions of the image, the eye can rotate from an angle corresponding to one convergence point to an angle corresponding to an adjacent convergence point without causing a visible transition in the image formed on the retina.

[0174] In some embodiments, the eye tracker 60 may include a light source. The light source may be configured not to be detectable by the eye. Thus, the light source may be prevented from affecting the image projected into the eye. In a non-limiting example, the light source may be configured, due to its position and / or orientation, such that the rays emitted from the light source do not enter the eye, or the rays from the light source are not projected onto the retina, or are invisible (i.e., not detectable by retinal cells).

[0175] In some embodiments, the projection device is held or worn by the user. In a non-limiting example, the projection device may include a headset mounted on the user's head, with elements of the optical system provided near and in front of the user's eyes (or both eyes).

[0176] In some embodiments, the projection device includes a housing that accommodates elements of the optical system. The housing may, for example, be located on the headset of the projection device. Thus, the elements housed within the housing can be protected from the environment, for example, by avoiding unwanted light from the environment that would affect the light encoding the projected image.

[0177] Figure 5 shows a computing unit 50 for controlling a projection device according to one or more embodiments. The computing unit 50 includes one or more communication interfaces 510 for communicating with the eye tracker 60 and components of the optical system 20. The communication interfaces may allow the computing unit 50 to communicate with other computing units outside the projection device 10, for example, to exchange data related to the projected image. Each communication interface may use any suitable communication link described herein.

[0178] The computing unit 50 further includes a processor 515. The processor 515 may include one or more processing units such as a microprocessor, GPU, CUP, multicore processor, or similar computer processing unit.

[0179] The computing unit 50 further includes memory 520. Memory 520 may be any suitable storage medium, including but not limited to optical discs, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory, flash cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAIDs, remote data storage / archives / warehousing, and / or other types of devices suitable for storing instructions and / or data. Memory 520 may store two-dimensional or three-dimensional image data and other data displayed to the user, and may store data received from the eye tracker 60 and / or components of the optical system 20.

[0180] The computing unit 50 may further include an eye-tracking module 525, a rendering module 530, a display module 535, and an operation module 540. All modules of the computing unit 50 may be implemented in hardware, software, or a combination of both. For example, a module may be stored as software in memory 520. A module may include any suitable instruction sequence stored in the computing unit 50, configured to perform the methods disclosed herein.

[0181] During operation, the eye tracker 60 tracks the position of the user's pupil (or both pupils) and outputs a value indicating the rotation angle of the pupil, which is received by the interface 510. The eye tracking module 525 can convert this data into coordinates in a predetermined coordinate system representing the direction the pupil is pointing. The rendering module 530 receives the coordinates representing the direction the pupil is pointing, receives image data from memory 520, and can render a 2D or 3D image to be displayed on the pupil. The rendering module 530 may render different images for each pupil, taking parallax into account, so that the user perceives a 3D environment. The rendered image data is sent to the display module 535, which generates instructions to the projection unit 21 to project light that encodes the rendered image. The operation module 540 calculates the operation angle of the operation element 22 corresponding to the direction of the pupil and instructs the operation element 22 to manipulate the image to the correct operation angle. The rendered image is then focused by the light-gathering element 23 and formed on the retina of the user's eye. The value output by the eye tracker 60 may specify the pupillary rotation angle (for example, the output of the eye tracker 60 may include an angle value in degrees or radians), or the output of the eye tracker may be an image of the eye (e.g., from a camera) or a sensor value (e.g., from an electrocardiogram sensor), and the eye tracking module may determine the pupillary rotation angle based on the received image or sensor value.

[0182] Figure 6A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The projection device includes a self-emissive display 201, which is, for example, a micro-LED display, which emits the image to be displayed as a collimating ray (or a standard display with a collimating optical system positioned in front of the display may be used). The device further includes a focusing element 203 that focuses the ray. The device further includes an operable mirror 205, which may be, for example, a MEMS mirror (i.e., a mirror containing a micro-electromechanical system (MEMS)), which is positioned behind the convergence point F of the focused ray. This operable mirror 205 reflects light toward the focusing element 207, which converges the light to a convergence point on the arc 41. The image is formed on the retina of the eye 6. Manipulating the mirror 205 as indicated by arrow R rotates the virtual image V at the convergence point F, which then moves the convergence point of the ray along the arc 41. Therefore, the mirror 205 can be operated to follow the position of the eye 6 so that the image projected onto the retina remains centered on the retina.

[0183] Although the light-gathering elements 203 and 207 are shown as transmissive lenses, it should be noted that they can be replaced with concave reflective mirrors or HOEs (holographic optical elements) having the same focal length. For example, Figure 6B shows an alternative embodiment in which a curved mirror or HOE may be used instead of a lens for the light-gathering element 207, which can also function as an optical coupler configured to allow light from the environment to pass through the mirror / HOE. For example, if ray coherence is required, a pinhole filter 204 (which may be called a spatial Fourier filter) may be added to the optical system, for example, by using an HOE for element 203 or 207.

[0184] In the optical system 20 shown in Figures 6A and 6B, the display 201 and the light-gathering element 203 may be considered as the projection unit 21 in Figure 4, the operable mirror 205 may be considered as the operable element 22, and the light-gathering element 207 may be considered as the light-gathering element 23 (and both elements 21 and 22 may be mounted in-unit or on the arms of the smart glasses, and element 23 may be integrated or coupled to the smart glasses lens).

[0185] Figure 7A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system includes a laser 210 comprising one or more laser diodes (e.g., separate RBG laser diodes or a single RBG diode) configured to emit red, green, and blue light. It will be understood that the laser may also emit IR light or light of other wavelengths in the visible spectrum. The light emitted from the laser 210 is collimated by a collimating element 211 (shown as a lens, but alternatively, it may be a concave reflective mirror or holographic optical element (HOE) as shown in Figure 7B). The collimated laser light is incident on a phase or amplitude spatial light modulator (SLM) 212, which modulates the light so that the projected image is encoded in the light transmitted by the SLM 212. Any suitable SLM may be used, such as a transmissive or reflective amplitude and / or phase modulator based on liquid crystal (LC), liquid crystal on silicon (LCoS), or a digital micromirror device (DMD). Note that while SLM212 in Figure 7A is shown as a reflective SLM, a transmissive SLM may be used as an alternative. The modulated light is then focused to a convergence point F by a focusing element 213 (shown as a lens, but may be a concave mirror or HOE), and then incident on a controllable mirror 205, which may be any suitable controllable mirror, such as a MEMS mirror. The light is reflected by the controllable mirror 205 toward a focusing element 207 (a curved reflective mirror or HOE), causing the light to converge to a point on the convergence arc 41. In other embodiments, the focusing element 207 is a focusing lens, similar to the embodiment shown in Figure 6A. The image is projected onto the retina of eye 6. When the controllable mirror 205 is operated as indicated by the arrow R, the virtual image V at the convergence point F is rotated around the arc, and the convergence point in the pupil of eye 6 is shifted along the arc 41. Therefore, the operable mirror 205 can be controlled to rotate the image projected onto the retina and keep it centered on the retina. As shown in Figure 7B, the collimating lenses 211 and 213 can be replaced by a single focusing element 215 that includes a concave reflective mirror (or one or more lenses).The focusing element 215 is configured to guide laser light from the laser 210 to the SLM 212, thereby causing the light to be incident on the SLM 212 in a non-collimated state and focused to a convergence point F in front of the operable mirror 205. The SLM 212 modulates the non-collimated light and encodes an image into light. Once the image is encoded into light, the focusing element 215 may also be positioned behind the SLM 212.

[0186] In the optical system 20 shown in Figure 7A, elements 210, 211, 212, and 213 can be considered as the projection unit 21 in Figure 4, the operable mirror 205 can be considered as the operating element 22, and the light-gathering element 207 can be considered as the light-gathering element 23 (and both elements 21 and 22 can be mounted inside or on the arms of the smart glasses). In the optical system 20 shown in Figure 7B, elements 210, 215, and 212 can be considered as the projection unit 21 in Figure 4, the operable mirror 205 can be considered as the operating element 22, and the light-gathering element 207 can be considered as the light-gathering element 23 (and both elements 21 and 22 can be mounted inside or on the arms of the smart glasses).

[0187] Figure 8A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system includes a laser 210, such as an RGB laser diode configured to emit red, green, and blue light. The laser 210 is configured to emit a laser beam onto a movable mirror 205' (such as a MEMS mirror). The movable mirror 205' is movable (indicated by arrow R'), and the laser 210 and the movable mirror 205' together emit light that encodes the image viewed by the user (selective emission of laser light from the laser 210 and selective positioning of the movable mirror 205' together provide the generation and modulation of light necessary to produce an image). The light is directed from the movable mirror 205', which is configured to rotate (indicated by arrow R), to a movable mirror 205 (such as a MEMS mirror or any suitable mirror). The light reflected by the movable mirror 205 corresponds to a virtual image V and is directed toward a focusing element 207. In Figure 8A, the focusing element 207 is a lens, but in other embodiments (as shown in Figure 8B), the focusing element is a concave mirror, HOE, HPDLC, or stacked HPDLC. The focusing element 207 focuses light to a convergence point on the convergence arc 41, and an image is formed on the retina of the eye 6. By rotating the operable mirror 205, the convergence point is shifted along the convergence arc 41, and the projected image is rotated around an axis 40 that coincides with or is close to the axis of rotation of the eye, so that the image projected onto the retina remains centered on the retina even when the user's eye rotates.

[0188] In the case of the optical system 20 shown in Figure 8A, elements 210 and 205' can be considered as the projection unit 21 in Figure 4, the operable mirror 205 can be considered as the operable element 22, and the light-gathering element 207 can be considered as the light-gathering element 23 (and both elements 21 and 22 can be mounted in-unit or on the arms of smart glasses).

[0189] Figure 9A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a self-emissive display 217, which is, for example, a micro-LED display, which emits the image to be displayed as a collimating ray (or a standard display with a collimating optical system positioned in front of the display may be used). The optical system further includes a diverging element 218 configured to form a virtual image V of a converging arc 41. The diverging element 218 is shown as a diverging lens, but in other embodiments, a convex mirror, HOE, or HPDLC having the same optical properties as a diverging lens may be used. The self-emissive display 217 and the diverging element 218 are mounted on a movable element (not shown), such as a MEMS, thereby making these components operable by rotation (the direction of rotation is indicated by arrow R). Light emitted from the diverging element is directed toward a condensing element 207, shown as a condensing lens, which focuses the light to a convergence point on the arc 41, and an image is projected onto the retina of the eye 6. The display 217 and the divergent element 218 are rotatable such that the convergence point moves to a point on the convergence arc corresponding to the position of eye 6. Therefore, even if eye 6 moves, the image remains centered on the retina of eye 6. The display 217 and the divergent element 218 can be integrated into a single self-emissive display that emits divergent light.

[0190] In an alternative embodiment, the self-emissive display 217 may include a laser configured to illuminate an optical modulation element, such as a spatial light modulator (SLM) or DLP, with divergent light. The divergent light is transmitted to or reflected by a focusing element 207. The display 217, including the laser and optical modulator, is mounted on a movable element such as a MEMS, so that the projected image follows the position of the eye and is kept centered on the retina. In embodiments where a laser configured to emit divergent light is used, the divergent element 218 is not necessary because the laser already emits a divergent beam.

[0191] In some embodiments, as shown in Figure 9B, the focusing lens of the focusing element 207 may instead be a concave reflective mirror, an HOE, an HPDLC, or an HPDLC laminate having the same optical properties as the focusing lens.

[0192] In the embodiments described with reference to Figures 9A and 9B, the display 217 and divergent element 218 (or SLM) mounted on the movable element may be considered as both the projection unit 21 and the operating element 22 in Figure 4, and the light-gathering element 207 may be considered as the light-gathering element 23 (and both elements 21 and 22 may be mounted in-unit or on the arms of the smart glasses).

[0193] Figure 10A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a display 219. The display 219 may be a self-emissive display configured to emit coherent light, or a reflective or transmissive optical modulation element such as a spatial light modulator (SLM) or DLP, illuminated with substantially coherent light (e.g., a coherent point light source or coherent light). Coherent light emitted from the display 219 is incident on an HPDLC laminate 220 comprising multiple HPDLC layers. Each layer of HPDLC is configured to function as a diverging lens, and each laminate is recorded at a different angle. Thus, the angle at which the diverging beam is emitted depends on which HPDLC layer (or combination thereof) is switched on. The diverging beam is directed toward a focusing element 207, which is a focusing lens in the illustrated embodiment. The light is focused to a convergence point on a converging arc 41, and an image is projected onto the retina of the eye. The layers of the HPDLC laminate 220 can be selectively switched on in different combinations to radiate divergent light to the focusing element 207 at different selectable angles. Depending on the selected angle, the light is focused to different points on the converging arc 41. Thus, the angle of the image projected onto the retina can be rotated around the axis 40 so that the image projected onto the retina is kept centered on the retina. Advantageously, this embodiment simplifies the structure of the projection device by not including any moving parts.

[0194] In some embodiments, the light emitted from the display 219 may not be coherent but instead diverge. In such cases, the HPDLC laminate 220 may modify the divergent light from the display 219 substantially in the same manner as described above.

[0195] As may be shown in Figure 10B, the light-gathering element 207 may instead include a concave mirror, HOE, HPDLC, or HPDLC laminate having the same optical properties as the light-gathering lens.

[0196] In the embodiments shown in Figures 10A and 10B, the display 219 may be considered as the projection unit 21 in Figure 4, the HPDLC laminate 220 may be considered as the operating element 22, and the light-gathering element 207 may be considered as the light-gathering element 23 (and both elements 21 and 22 may be mounted in-unit or on the arms of the smart glasses).

[0197] Figure 11A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a laser 210, which is, for example, an RGB laser, configured to emit divergent light to a phase modulator 212. The phase modulator may be, for example, a reflective or transmissive SLM or DLP. The phase modulator 212 is configured to encode an image by spatially modulating the phase of light (and optionally modulating the amplitude of light as well). The phase modulator 212 is further configured to modulate the light to manipulate the beam in a given direction. The light is guided by the phase modulator 212 to a focusing element 207, which is a focusing lens in the illustrated embodiment. The light is focused by the focusing element 207 to a convergence point on a convergence arc 41, and the image is projected onto the retina of the eye. The phase modulation of the light by the phase modulator 212 may be selected to project the image in different directions so that the light converges to different points along the convergence arc 41. Therefore, phase modulation can be controlled to project the image onto a point on the converging arc 41 corresponding to the position of eye 6, so that the projected image is kept central on the retina of eye 6.

[0198] In some embodiments, as shown in Figure 11B, the focusing element 207 may be a concave reflective mirror, HOE, HPDLC, or HPDLC laminate. The phase modulator 212 may also be reflective, as shown in Figure 1IB.

[0199] In the embodiments shown in Figures 11A and 11B, the laser 210 and the phase modulator 212 can be considered as both the projection unit 21 and the operating element 22 in Figure 4, and the focusing element 207 can be considered as the focusing element 23 (and both elements 21 and 22 can be mounted in-unit or on the arms of the smart glasses).

[0200] Figure 12A shows a schematic diagram of an optical system 20 for a projection device according to one or more embodiments. The optical system 20 includes a self-luminous operable display 221 configured to generate and modulate light. The display 221 is configured to emit light that encodes an image to be projected onto the retina of eye 6. The light is transmitted to a focusing element 207, which in the illustrated embodiment is a focusing lens, and the focusing element 207 focuses the light to a point on a convergence arc 41, so that the image is projected onto the retina of eye 6. The display 221 is further configured to spatially modulate the phase of the light so that the image is projected in a particular direction and converges to a point along the convergence arc 41. Thus, the display 221 can be moved to a point corresponding to the position of the eye on the convergence arc 41, thereby controlling the projected light so that the projected image is kept in a central position on the retina of eye 6. In some embodiments, as shown in Figure 12B, the focusing element 207 may be a concave reflective mirror, HOE, HPDLC, or HPDLC laminate.

[0201] In the embodiments shown in Figures 12A and 12B, the self-illuminating movable display 221 can be considered as both the projection unit 21 and the operating element 22 in Figure 4, and the light-gathering element 207 can be considered as the light-gathering element 23 (and both elements 21 and 22 can be mounted inside or on the arms of the smart glasses).

[0202] It should be noted that in any embodiment disclosed herein, projected light may be transmitted between optical components via a waveguide rather than through air. Figures 13A–13C show schematic diagrams of an optical system 20 using a waveguide.

[0203] The following exemplary embodiments are described using a single waveguide, but this is not limiting, and any number of waveguides may be used. For example, in the embodiment shown in Figure 7B, a first waveguide may be provided between the laser 210 and the focusing element 215, and a second waveguide may be provided behind the rotating mirror 205. Each waveguide may have its own in-coupler and out-coupler, and the out-coupler of the second waveguide may be used instead of the concave mirror 207 shown in Figure 7B.

[0204] In the following exemplary embodiment, the waveguide outcoupler is located on the waveguide surface distal to the eye, and TIR is used to reflect light from the waveguide outcoupler and focus it toward the eye. However, these are non-limiting examples, and the outcoupler could instead be located on the waveguide surface proximal to the eye, focusing the light toward the eye by collecting it.

[0205] In the embodiment shown in Figure 13A, a self-illuminating movable display 221 generates and modulates light. The light is transmitted to the in-coupler 250 of waveguide 251. The light is then guided through waveguide 251 to the out-coupler 252 of waveguide 252, which uses TIR to focus the light through waveguide 251 towards the eye.

[0206] In the embodiment shown in Figure 13B, a self-emissive display 201 (either emitting collimating rays or including a collimating optical system) emits a display image with collimating rays. The collimating rays pass through a waveguide 251 and a focusing element 203 that focuses the rays. The rays are then reflected off a controllable mirror 205 (e.g., a MEMS mirror). The rays reflected from the controllable mirror 205 are transmitted to an in-coupler 250 in the waveguide 251. Similar to the embodiment shown in Figure 13A, the rays from the in-coupler 250 are guided through the waveguide 251 and focused towards the eye by a TIR (Time Infrared) by an out-coupler 252.

[0207] In the embodiment shown in Figure 13C, a coherent light source (e.g., a laser, an LED emitting sufficiently coherent light) 210 emits light along the visible spectrum (e.g., red, blue, and green wavelengths), and the light is focused by a collimating light element 211 (e.g., a lens shown in Figure 13C, or alternatively, a concave mirror or HOE). The focused light rays pass through a waveguide 251 and are reflected by an SLM 212. The light rays reflected by the SLM 212 are transmitted to an in-coupler 250 in the waveguide 251. Then, as in Figures 13A and 13B, the light rays are guided through the waveguide 251 to an out-coupler 252 and subsequently focused into the eye. The coherent light source can be replaced by a self-emissive or reflective display that emits an amplitude-coded image (e.g., a laser / coherent light source that emits light reflected by an amplitude modulator).

[0208] In the embodiments described above, an expanded field of view is obtained by focusing the projected light to a point in front of the center of eye rotation (as shown in image 5 formed in Figures 3A and 3B, compared to the image formed by the systems shown in Figures 2A and 2B). The projection system according to the present invention enables a monocular field of view of 150° or more. The innate resolution of the human eye is about 60 pixels / degree, which means that a display of more than 9000*9000 pixels would be required to achieve such a field of view. While this may be possible, the display for this would be expensive, large, and would require a computational load to render such a resolution. Nevertheless, the retina has the highest sensitivity in the foveal region and low sensitivity in the peripheral region. Therefore, some projection devices may incorporate a foveation unit that combines a high-resolution image for the foveal region with a low-resolution image for the peripheral part of the user's field of view. This reduces the requirements for resolution, bandwidth, light output, and computation. A typical display device employs two displays (one for the high-resolution image and one for the low-resolution image), which also makes the display device more effective and larger. According to one aspect of the present disclosure, the optical system 20 of the display device 10 is provided with a foveation unit 24 that uses a single projection unit 21 for both images, thereby reducing the cost and size of the system. The foveation unit 24 is operable to switch between a foveal mode in which a high-resolution image is projected onto the retina with a smaller field of view, and a peripheral mode in which a low-resolution image is projected onto the peripheral part of the retina. The foveation unit 24 is controllable by a computing unit 50 to switch between the foveal mode and the peripheral mode. The computing unit 50 is also configured to work in conjunction with the foveation unit 24 to switch the projection unit 21 between the foveal mode and the peripheral mode, so that the projection unit 21 projects an image in the high-resolution, low-field-of-view foveal mode or in the low-resolution, wide-field-of-view peripheral mode. In such embodiments, the computing unit 50 includes a foveation module 545 which can be implemented in hardware, software, or a combination of both (for example, stored as software in memory 520).The foveation module includes a sequence of instructions stored in the computing unit 50 for controlling the projection unit 21 and the foveation unit 24 in conjunction, as described herein. The foveation unit 24 may be positioned at any suitable point along the optical path. For example, the foveation unit 24 may be positioned behind the projection unit 21 and in front of the operating element 22, or behind the operating element 22 and in front of the focusing element 23.

[0209] Figure 14A shows a schematic diagram of the optical system 20 of a projection device operating in peripheral mode. The optical system may be, for example, the optical system 20 of the projection device in Figure 4, which includes a foviation unit 24. In peripheral mode, the optical system 20 projects a peripheral image 5a onto the retina of the eye. The peripheral image is a low-resolution image projected onto the peripheral part of the retina when the eye is centered, as shown. In embodiments in which the projection device includes an operating element 22, the center of the peripheral image 5a remains centered on the retina even when the eyeball moves, thereby keeping the peripheral image on the peripheral part of the retina. The center C of the peripheral image 5a remains blank (i.e., in peripheral mode, no light or image is projected onto the retina in the central portion C). Figure 14A shows the peripheral image I that the user will observe when using the projection device. p This further demonstrates the following.

[0210] Figure 14B shows a schematic diagram of an optical system operating in foveal mode. The optical system may be, for example, the optical system 20 in Figure 4, which includes a foveation unit 24. In foveal mode, the optical system 20 projects a foveal image 5b onto the retina of the eye. As shown, when the image is centered on the central part of the retina, the foveal image is a high-resolution image projected onto the foveal region of the retina with a narrower field of view. In embodiments in which the projection device includes an operating element 22, the center of the foveal image 5b is kept centered on the retina even when the eyeball moves, thereby keeping the foveal image on the fovea of ​​the retina. Figure 14B shows the peripheral image I that the user will observe when using the projection device. F This further demonstrates the following.

[0211] The optical system 20 is controlled by a computing unit 50, such as the computing unit 50 shown in Figure 5. The computing unit 50 controls the projection unit 21 and the optical system 20 to switch between foveal mode and peripheral mode. This switching is performed at a speed fast enough that the user does not perceive the switch, but instead perceives a complete image 5, for example, shown in Figure 14C, which is a combination of the foveal image 5B and the peripheral image 5A. For example, the projection unit 21 is configured to switch between peripheral mode and foveal mode at a rate of 60 Hz or higher (i.e., each of the foveal image 5b and peripheral image 5a is shown at a rate of 30 frames / second or higher).

[0212] It should be noted that the resolution and field of view of peripheral and foveal images may vary depending on the requirements of the projection device. For example, as shown in Figure 14D, the foveal image may have a wider field of view and lower resolution than that shown in Figure 14C. Furthermore, the projection device may include three or more foveal modes. For example, as shown in Figure 14E, the image may include an internal region image with a first resolution, an internal peripheral image adjacent to the internal region image with a lower resolution than the internal region image, and an external peripheral image adjacent to the internal peripheral image with a lower resolution than the internal peripheral image. In such embodiments, the foveation unit 24 is switchable between three or more modes, each of which projects an image with a different field of view onto the retina.

[0213] Figure 15A shows a schematic diagram of an optical system 20 incorporating a foviation unit 24 according to one or more embodiments. The optical system 20 includes a projection unit 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection unit 21 may be any suitable projection unit for projecting an image onto the retina of the eye (such as any of the projection units shown in Figures 6A, 6B, 7A, 7B, 8A, 8B, 9A, 10A, 10B, 11A, 11B, or 12A). In embodiments in which the operating element 22 is incorporated into the optical system 20 and includes mechanically operable elements (such as a mirror 205), the projection unit 21 may be, for example, a display 201 (and optionally a focusing element 203) as described with reference to Figures 6A and 6B, a laser 210, a collimating element 211, an SLM 212 (and optionally a focusing element 213) as described with reference to Figure 7A, a laser 210, a focusing element 215 and an SLM 212 as described with reference to Figure 7B, or a laser 210 and an operable mirror 205' as described with reference to Figures 8A and 8B.

[0214] In such embodiments, the foveation unit 24 includes one or more switchable focusing and / or diverging elements (one or more switchable lenses or HPDLC layers) positioned between the projection unit 21 and the operating element 23 (i.e., the operating mirror 205). In the peripheral mode shown in Figure 15A, the switchable elements and the projection unit 21 are switched to peripheral mode, and the projection unit 21 projects a peripheral image 5a to the peripheral part of the retina with low resolution and a wide field of view. Figure 15B shows a schematic diagram of the optical system of Figure 15A when it is in foveal mode. In foveal mode, the switchable elements and the projection unit 21 are switched to foveal mode, and the projection unit 21 projects a foveal image to the fovea of ​​the retina with higher resolution and a narrower field of view. It should be noted that by switching between these two modes, the combined image is a combination of two images projected by the projection unit 21, and therefore the resolution of the combined image viewed by the user is twice the resolution output by the projection unit 21 (and, considering the non-uniform distribution of photoreceptors on the human retina, the perceived resolution may be even higher). The operable mirror 205 manipulates the image to follow the position of the retina, as described in the embodiments above. In the illustrated embodiment, both the foveal mode and peripheral mode light converge to the same convergence point F, so both the peripheral image 5A and the foveal image 5B light converge to a point on the convergence arc 41 and rotate with the position of the eyeball. This can be achieved by many different combinations of the switchable elements. In the illustrated embodiment, in peripheral mode, only the first focusing element 24c is switched on to focus the collimated light from the projection unit 21 to point F, and in foveal mode, the second focusing element 24a and collimating element 24b are similarly switched on, guiding a narrower parallel beam toward the first focusing element 24c, thereby causing the light to converge again to the convergence point F. It will be understood that in peripheral mode, any number of elements (for example, two or other suitable combinations of optical elements, such as a focusing lens following a first scattering lens) can be switched on.

[0215] Figure 16A shows an optical system 20 including a foviation unit 24 for a projection device according to one or more embodiments. The optical system 20 includes a projection unit 21 configured to project light that encodes a peripheral image to be projected onto the retina. The projection unit 21 may be any suitable projection unit for projecting an image onto the retina of the eye (such as any of the projection units shown in Figures 6A, 6B, 7A, 7B, 8A, 8B, 9A, 10A, 10B, 11A, 11B, or 12A). The illustrated embodiments show an optical system 20 including only the projection unit 21 and a foviation unit 24 that focuses the projected light directly onto a convergence point F near the pupil of the eye (i.e., a projection device without an operating element 22). In such embodiments, the foviation unit 24 may also be considered a focal element 23. It will be understood that the foviation unit 24 described with reference to Figure 16A may be interchangeable with the foviation unit 24 described with reference to Figures 15A and 15B (and vice versa). In such a case, as discussed in relation to Figure 15A, the foviation unit 24 focuses the projected light to a convergence point F in front of an operable mirror 205, as shown in Figures 15A and 15B. The foviation unit 24 includes first and second switchable focusing elements 230, 231 (such as switchable lenses, mirrors and / or HPDLC layers) configured to receive collimated light from the projection unit 21. The first and second lenses 230, 231 have different focal lengths and are positioned away from the projection unit 21 and away from each other such that the convergence points F of both lenses 230, 231 coincide. In Figure 16A, the optical system 20 is shown in foveal mode, in which the projection unit 21 is controlled to project light encoding the foveal image 5b, with the first switchable focusing element 230 switched on and the second switchable focusing element 231 switched off. The light converges at the convergence point F, and the image is formed on the retina.

[0216] Figure 16B shows the optical system 20 of Figure 16A in peripheral mode, in which the projection unit 21 is controlled to project light encoding the peripheral image 5A, the second switchable focusing element 231 is switched on, and the first switchable focusing element 230 is switched off. The peripheral image is projected onto the convergence point F, and the image is formed on the retina. Because the second switchable focusing element 231 has a shorter focal length, the field of view of the peripheral image is wider. Note that by switching between these two modes, the combined image is a combination of the two images projected by the projection unit 21, so the resolution of the combined image viewed by the user is twice the resolution output by the projection unit 21. It will be understood that the field of view of the foveal image and the peripheral image depends on the size of the collimated light projected from the projection unit, the focal lengths of the focusing elements 230 and 231, and the distance from the convergence point F to the retina.

[0217] Figure 17A shows an optical system 20 including a foviation unit 24 for a projection device according to one or more embodiments. The optical system 20 includes a projection unit 21 configured to project light encoding a peripheral image to be projected onto the retina. The projection unit 21 may be any suitable projection unit for projecting an image onto the retina. The illustrated embodiments show an optical system 20 (i.e., a projection device without an operating element 22) including only the projection unit 21 and a foviation unit 24 that directly focuses the projected light to a convergence point F near the pupil of the eye. In such embodiments, the foviation unit 24 may also be considered a focal element 23. It will be understood that the foviation unit 24 described with reference to Figure 17A may be interchangeable with the foviation unit 24 described with reference to Figures 15A and 15B (and vice versa). In such cases, as discussed in reference to Figure 15A, the foviation unit 24 focuses the projected light to a convergence point F in front of an operable mirror 205, as shown in Figures 15A and 15B. The foveation unit 24 includes a switchable divergent element 232 (e.g., a switchable lens, a convex mirror, or an HPDLC layer) and first and second switchable focusing elements 233, 234 (e.g., a switchable lens, a convex mirror, or an HPDLC layer, or any combination thereof). The switchable divergent element 232 is positioned away from and in front of the first switchable focusing element 233. In Figure 17A, the optical system 20 is shown in foveal mode, in which the projection unit 21 is controlled to project light encoding the foveal image 5b, the second switchable focusing element 234 is switched on, and the first switchable focusing element 233 and divergent element 232 are switched off. Collimated light encoding the image is guided from the projection unit 21 to the second focusing element 234, where the light is focused at a convergence point F and an image is formed on the retina.

[0218] Figure 17B shows the optical system 20 of Figure 17A in peripheral mode, in which the projection unit 21 is controlled to project light encoding the peripheral image 5a. In peripheral mode, the second switchable focusing element 234 is switched off, and the first switchable focusing element 233 and diverging element 232 are switched on. The projection unit 21 projects collimated light encoding the peripheral image onto the diverging element 232. The diverging light is guided toward the first switchable focusing element 233, where the light is focused to a convergence point F. The peripheral image 5a is then projected onto the retina. The field of view of the peripheral image is wider because the first switchable focusing element 233 has a shorter focal length than the second switchable focusing element 231. It will be understood that the precise positions of elements 232, 233, and 234 depend on their focal lengths. It will be understood that the field of view of the foveal and peripheral images depends on the size of the collimated light projected from the projection unit, the focal lengths of elements 232, 233, and 234, and the distance from the convergence point F to the retina. Note that by switching between these two modes, the combined image is a combination of the two images projected by the projection unit 21, and therefore the resolution of the combined image viewed by the user is twice the resolution output by the projection unit 21. By using the divergent element 232, the minimum space required for the optical system can be advantageously reduced. Specifically, the optical system requires a certain space, which is partially determined by the focal length of each optical element used. In particular, in the systems shown in Figures 16A and 16B, the switchable focusing elements 230 and 231 should be at a distance to the eye corresponding to their respective focal lengths. Therefore, these switchable focusing elements need to be spaced apart from each other, which can be a major factor in determining the compactness of the optical system 20. By using the divergent element 232 and another switchable focusing element 234, it is no longer necessary to space out the two switchable focusing elements 233 and 234 from each other.

[0219] Figure 18A shows an optical system 20 including a foviation unit 24 for a projection device according to one or more embodiments. The optical system 20 includes a projection unit 21 configured to project light encoding a peripheral image to be projected onto the retina. The projection unit 21 may be any suitable projection unit for projecting an image onto the retina. The illustrated embodiments show an optical system 20 (i.e., a projection device without an operating element 22) including only the projection unit 21 and a foviation unit 24 that focuses the projected light directly to a convergence point F near the pupil of the eye. In such embodiments, the foviation unit 24 may also be considered a focal element 23. It will be understood that the foviation unit 24 described with reference to Figure 18A may be interchangeable with the foviation unit 24 described with reference to Figures 15A and 15B (and vice versa). In such cases, as discussed in reference to Figure 15A, the foviation unit 24 focuses the projected light to the convergence point F in front of an operable mirror 205, as shown in Figures 15A and 15B. The foveation section 24 includes first and second switchable divergent elements 235A and 235B and first and second switchable focusing elements 236A and 236B. The divergent and focusing elements may be, for example, switchable lenses, switchable mirrors, HPDLC layers, or any combination thereof. The first focusing element 236a is positioned away from and in front of the second divergent element 235b. Figure 18A shows the optical system 20 in foveal mode. In foveal mode, the first focusing element 236a and the second divergent element 236b are switched on, and the first divergent element 235a and the second focusing element 236a are switched off. The projection unit 21 is controlled to project collimating light encoding the foveal image 5b onto the first focusing element 236a. The first focusing element 236a focuses the light toward the second diverging element 235b, and the second diverging element 235b diverges the light toward the convergence point F. The foveal image 5B is formed on the retina.

[0220] Figure 18B shows the optical system 20 of Figure 18A in peripheral mode. In peripheral mode, the first divergent lens 235a and the second condensing lens 236b are switched on, and the first condensing lens 236a and the second divergent lens 235b are switched off. The projection unit is controlled to project collimated light encoding the peripheral image 5a onto the first divergent element 235a. The first divergent element 235a diverges the light and directs it toward the second condensing element 235b, which focuses the light to the same convergence point F as in the foveal mode. The peripheral image 5a is formed on the retina.

[0221] In all embodiments described with reference to Figures 14A to 18B, the convergence point F of the projected light in the foveal mode and the peripheral mode may not coincide. Figure 19A shows the optical system 20 in peripheral mode. In peripheral mode, the projection unit 21 projects collimated light onto a foveation unit 24 that focuses the light to a convergence point F1 in front of a light-manipulating mirror 205. The light is guided to a light-gathering element 207, and a peripheral image 5a is formed on the retina. Note that the foveation unit 24 includes a first switchable light-gathering element 237 and a second switchable light-gathering element 238, which operate similarly to the foveation unit 24 shown in Figures 16B and 16B, except that in this case the convergence points do not coincide due to the arrangement of the light-gathering elements. In peripheral mode, the first light-gathering element 237 is switched on and the second light-gathering element 237 is switched off. The foveation section 24 shown may be replaced with the foveation section in either Figures 17A and 17B or Figures 18A and 18B, and it should be noted that the lens has optical properties and is arranged such that the convergence points in the foveal mode and peripheral mode are offset from each other.

[0222] Figure 19B shows the optical system 20 of Figure 19A in foveal mode. The projection unit 21 projects collimated light onto the foveation unit 24, which focuses the light to a convergence point F2, which may coincide with, for example, the surface of an operable mirror 205. The operable mirror 205 directs the light to a focusing element 207, which focuses the light to project a foveal image 5b onto the retina.

[0223] Figure 20A shows a schematic diagram of an optical system 20 incorporating a foviation unit 24 according to one or more embodiments. The optical system 20 includes a display 217 configured to emit collimating light that encodes an image projected onto the eye. The optical system further includes a foviation unit 24, which may be one of the foviation units described above. The display 217 and the foviation unit 24 are mounted on a movable element such as a MEMS. Figure 20A shows the optical system 20 operating in peripheral mode. In this mode, the display 217 projects light that encodes a peripheral image. The foviation unit 24 is switched to peripheral mode, diverging light to a wide angle, and a focusing element 207 focuses the light to a convergence point F, so that the peripheral image 5a is projected onto the retina. The movable element keeps the peripheral image 5a centered on the retina, as described in the embodiments above.

[0224] Figure 20B shows the optical system 20 of Figure 20A operating in foveal mode. The display 217 projects light that encodes the foveal image. The foveation unit 24 is switched to foveal mode, diverging the light at a narrower angle than in peripheral mode, and the focusing element 207 focuses the light to a convergence point F (which coincides with the convergence point in peripheral mode), and the foveal image 5b is projected onto the retina. A movable element keeps the foveal image centered on the retina, as described in the embodiments above.

[0225] As mentioned above, the projected light in the peripheral and foveal modes by the focusing element 207 does not need to have a coincident convergence point. Figures 21A and 21B show an optical system 20 similar to those in Figures 20A and 20B, except that the optical elements for each mode of the foveation section 24 are selected such that in the peripheral mode the projected light converges to a convergence point F1, and in the foveal mode the projected light converges to a convergence point F2 offset from F1. The offset can be defined substantially around the optical axis of the eye, the pupillary axis of the eye, or an axis formed by the pupil and the center of rotation of the eyeball.

[0226] In the exemplary embodiment described above, the optical system 20 includes a foveation unit 24 that uses elements separate from the projection unit 21 and the light-gathering unit. However, this is not limiting, and foveal display may be provided without requiring (and therefore omitted from) the foveation unit separate from the optical elements.

[0227] Figures 22A and 22B show schematic diagrams of an optical system 20 configured to provide foveal display according to one or more embodiments.

[0228] Specifically, as described in relation to Figure 11A above, the laser 210 is configured to emit divergent light to a phase modulator 212 (e.g., a reflective or transmissive SLM or DLP), and the phase modulator 212 is configured to spatially modulate the light to manipulate the beam in a given direction.

[0229] Figure 22A shows the optical system 20 in foveal mode. The laser 210 and phase modulator 212 transmit light onto the central portion of a focusing element 207 (shown as a lens in a non-limiting example). The focusing element 207 focuses the light from its central portion onto a convergence point located near the pupil of the eye. The focused light rays enter the eye and project a foveal image 5A onto the central part of the retina 5A (e.g., the fovea).

[0230] Figure 22B shows the optical system 20 in peripheral mode. The laser 210 and phase modulator 212 transmit light onto a wider portion of the focusing element 207 (wider than the central portion used in foveal mode). The focusing element 207 focuses the light from this wider portion into the eye, and the light projects the peripheral image 5B onto the retina.

[0231] It will be understood that the embodiments shown in Figures 22A and 22B are non-limiting examples. Alternatively, the light-gathering element 207 may, in foveal mode, focus light onto a convergence point close to the center of ocular rotation (i.e., the center of the sphere representing the eye), and in foveal mode, focus light onto a convergence point close to the pupil (i.e., the positions of the convergence points in foveal and peripheral modes may be reversed from the examples shown in Figures 22A and 22B). In other examples, one of the convergence points (in either foveal or peripheral mode) may be located in front of the eye, and the other convergence point (in the other of foveal or peripheral mode) may be located between the pupil and the center of ocular rotation.

[0232] Advantageously, this optical system does not require any moving elements, nor any additional optical elements to provide foveal display. Therefore, the optical system 20 can be made more compact and requires fewer resources to operate.

[0233] Similar to the exemplary embodiments shown in Figures 10A and 10B, the optical system 20 without moving parts can project different images onto the central and peripheral parts of the retina.

[0234] Figures 23A and 23B show schematic diagrams of optical systems 20 configured to provide foveal display according to one or more embodiments.

[0235] The optical system 20 includes a display 219, a foviation unit 24, an operating unit (shown as a stack of HPDLC220 in a non-limiting example), and a light-gathering element 207 (shown as a lens in a non-limiting example). The foviation unit includes one or more switchable optical elements (e.g., a converging element or a diverging element).

[0236] Figure 23A shows the optical system 20 in peripheral mode. The display 219 emits collimating rays onto the foveation unit 24. In peripheral mode, the switchable elements of the foveation unit 24 are switched off, thereby allowing the collimating rays from the display 219 to pass through without substantially changing the angle size of the rays (i.e., keeping the rays parallel).

[0237] The collimating ray is transmitted to the control unit 220, which manipulates the ray as described above in relation to Figures 10A and 10B. For simplicity, this explanation is omitted here.

[0238] Figure 23B shows the optical system 20 in foveal mode. In foveal mode, the switchable elements of the foveation unit 24 are switched on. Thus, the focusing element 24a focuses the collimating rays from the display 219. The collimating element 24b collimates the focused rays and transmits them to the operating unit 220. In foveal mode, the rays collimated by the collimating element 24B reach a narrower portion of the operating unit 220 than in peripheral mode (for example, the collimated rays are projected onto the central portion of the HPDLC laminate). Therefore, the rays reach the focusing element 207 and consequently the eye over a narrower angular range in foveal mode than in peripheral mode.

[0239] In the example shown in Figures 23A and 23B, the favitation section 24 includes a light-gathering element 24a and a collimating element 24b, but this is not limiting, and it will be understood that any number of switchable optical elements may be provided. More generally, the favitation section may instead correspond to the favitation section described in any of the exemplary embodiments of this specification, for example, shown in Figures 15A-15B, 16A-16B, 17A-17B, 18A-18B, 19A-19B, 20A-20B, 21A-21B, etc.

[0240] The embodiments described above use examples of displays that emit light rays that encode images. In some embodiments, the display may be a light field display configured to project multiple elemental images in a three-dimensional light field.

[0241] Figure 24A shows a schematic diagram of an optical system 20 configured to provide a light field display according to one or more embodiments.

[0242] The optical system 20 includes a light field display 260 and a light-gathering element 203 (as an example of a projection unit 21), a rotating mirror 205 (as an example of an operating unit 22), and a light-gathering element 207 (as an example of a light-gathering unit 23).

[0243] Since the light-gathering element 203, the rotating mirror 205, and the light-gathering element 207 operate as described in relation to Figure 6A above, for the sake of brevity, a description of these elements will be omitted here.

[0244] The light field display 260 emits rays that encode multiple elemental images in a three-dimensional light field. Each elemental image is rendered so that the user perceives it as arriving from a virtual point in the environment. Figure 24A shows two exemplary virtual points VI and V2, but it will be understood that any other number of virtual points may be provided.

[0245] Rays corresponding to the virtual point VI are reflected from the rotatable mirror 205 and converge to point VI' on the retina via the light-gathering element 207. These rays then converge onto a point located on the retina. The elemental images encoded by these rays appear in focus to the user.

[0246] Similarly, rays corresponding to the virtual point V2 are reflected from the rotatable mirror 205 and projected into the eye via the focusing element 207. However, the rays corresponding to the virtual point V2 converge to point V2' in front of the retina. Therefore, these rays are projected onto the region on the retina indicated by V2''. Because these rays do not converge onto a point when they reach the retina, the elemental image encoded by these rays is blurred (i.e., appears out of focus to the user).

[0247] Therefore, the user perceives the elemental image encoded by the ray corresponding to virtual point V2 as being at a different depth than the elemental image encoded by the ray corresponding to virtual point VI. The image displayed by the light field display acquires a third dimension with a depth perceived by the user.

[0248] In the example shown in Figure 24C, it will be understood that the light rays focused by the focusing element 207 converge on multiple convergence points on the convergence arc near the pupil. As a non-limiting example, Figure 24A shows three convergence points C1, C2, and C3.

[0249] Figures 25A to 25E show schematic diagrams illustrating different combinations of elemental images that can be encoded into light from a light field display.

[0250] As shown in Figure 25A, nine elemental images with varying perceived depths can be combined to form a three-dimensional light field. Specifically, the images in the center column may correspond to a first perceived depth (shown by a solid line), the images in the left column may correspond to a second perceived depth (shown by a dotted line), and the images in the right column may correspond to a third perceived depth (shown by a dashed line), with the first, second, and third perceived depths being distinct from each other. The nine elemental images partially overlap with adjacent images, thereby allowing the user to perceive an image with elements having different depths.

[0251] The number of elemental images, their composition, or the perceived depth of each image are not limited to the example in Figure 25A. Specifically, the number of elemental images may vary; for example, four elemental images and / or different compositions may be used, as shown in Figure 25B (i.e., the elemental images do not have to be columns composed of an equal number of elemental images).

[0252] In addition, the sub-images do not need to overlap. For example, as shown in Figure 25D, the portion corresponding to each sub-image that would overlap with other sub-images when combined (the shaded area in Figure 25D) may be omitted. In such cases, one of the sub-images (e.g., the upper right sub-image) may contain the corresponding shaded area, so the combined image will be complete.

[0253] As shown in Figure 25E, a light field display can also be used to display different three-dimensional light fields on the central and peripheral portions of the retina. For example, a light field using five elemental images (with the configuration shown in Figure 25C) may be projected onto the central portion of the retina, and a light field using four elemental images (with the configuration shown in Figure 25D) may be projected onto the peripheral portion of the retina. The number of elemental images described herein is merely an example, and it should be understood that any number of elemental images may be used instead.

[0254] While various examples of 3D light fields that may be provided have been described, other details of light field displays are obvious to those skilled in the art and are therefore omitted here for brevity.

[0255] Modified forms and transformed forms Many variations and modifications can be made to the exemplary embodiments described above.

[0256] Figure 4 shows an optical system 20 having an operating unit 22 and a foviation unit 24, but in some embodiments, either the operating unit 22 or the foviation unit 24, or both, may be omitted.

[0257] For example, as shown in any of FIGS. 6A to 13C, the optical system 20 need not project different images onto the central and peripheral regions of the retina.

[0258] As another example, as shown in, for example, FIG. 10A or FIG. 10B, the projection unit 21 is configured to emit light in different directions, thereby eliminating the need for the operation unit 22.

[0259] Furthermore, when the optical system 20 includes a focusing unit, the optical system 20 need not move the convergence point along the convergence arc (in the case of a dynamic eye box).

[0260] Although the above exemplary embodiments use specific optical elements, it will be understood that each of these can be replaced by other interchangeable optical elements. For example, each diverging lens can be replaced by a convex mirror, each converging lens can be replaced by a concave mirror, or each lens or mirror can be replaced by HPDLC, HOE, laminated HPDLC, etc.

[0261] In the above exemplary embodiments, the mirror or other rotating element is shown as having one axis of rotation, but these can have two separate axes of rotation (or the optical system 20 can include an additional rotating element configured to rotate along a second separate axis of rotation, thereby enabling the optical system to control the convergence point of the light rays along two dimensions).

[0262] In the above exemplary embodiments, the image projected onto the eye can be part of a scene (such as a still image or a moving image showing one or more virtual elements). A part of the projected scene can correspond to the rotation angle of the eye (i.e., the orientation of the pupil), so that when the eye moves, the projected part changes.

[0263] In other cases, for example, when a dynamic eyebox is used (and when the optical system moves the point of convergence along a convergence arc or plane), the displayed image may instead remain fixed within the user's field of view. In other words, the image the user sees may be static and not move within the user's field of view.

[0264] In any of the exemplary embodiments described above, including the phase modulator, the optical system may be configured to project light encoding a holographic image. For example, in Figure 7A, the phase modulator 212 may be configured to project light encoding a holographic image by coupling wavefronts. For brevity, details of holographic image generation and projection, which will be apparent to those skilled in the art, are omitted here.

[0265] The embodiments described above with reference to Figures 17A and 17B include two switchable focusing elements 233 and 234, each used to focus light in either a foveal mode or a peripheral mode. Alternatively, a single switchable focusing element may be used and controlled to switch between a first state and a second state (for example, a first state in which the focusing portion is in foveal mode and a second state in which the foveation portion is in peripheral mode, or vice versa), and the single switchable focusing element may have different focal lengths in the first and second states.

[0266] In the embodiments described above with reference to Figures 18A and 18B, the first switchable divergent element 235a and the first condensing element 236a may be replaced by a single switchable optical element (wherein the foveal mode, the single switchable optical element has a focal length corresponding to the first switchable divergent element 235a, and in the peripheral mode, the single switchable optical element has a focal length corresponding to the first condensing element 236a). Additionally or alternatively, elements 235b and 236b may be replaced by a single switchable optical element.

[0267] In the foregoing description, exemplary embodiments are described with reference to several embodiments. Therefore, this specification should be considered exemplary, not restrictive. Similarly, the drawings illustrating the functionality and advantages of the exemplary embodiments are presented for illustrative purposes only, for example. The architecture of the exemplary embodiments is sufficiently flexible and configurable to be utilized in ways other than those shown in the accompanying drawings.

[0268] Some embodiments may also be implemented by preparing application-specific integrated circuits, field-programmable gate arrays, or by interconnecting appropriate networks of conventional component circuits.

[0269] The apparatus described herein may be embodied in other specific forms without departing from its features. The embodiments described herein are illustrative and not limiting to the systems and methods described herein. Accordingly, the scope of the apparatus described herein is indicated not by the foregoing description but by the appended claims, which include the meaning of the claims and modifications within the equivalent scope.

Claims

1. An optical system for myopic projection of images into the user's eye, A projection unit configured to project light that encodes the aforementioned image, A foveation section including at least one switchable optical element that can be controlled to switch between foveal mode and peripheral mode, Includes, In the foveal mode, the foveation unit is configured to project the light from the projection unit onto the central region of the retina of the eye, and In the peripheral mode, the foviation unit is configured to project the light from the projection unit onto the peripheral region of the retina, wherein the peripheral region is larger than and includes the central region, an optical system.

2. The optical system according to claim 1, wherein the projection unit is configured to project the light as substantially collimated light.

3. The optical system according to claim 1 or 2, wherein each of the at least one switchable element is configured to allow light to pass through without substantially changing the angular size of the light when switched to the first state.

4. The optical system according to any one of claims 1 to 3, wherein the foviation unit is configured to increase the angular size of the light incident on the eye in the peripheral mode.

5. The optical system according to any one of claims 1 to 4, wherein the foveation portion is configured to increase the focal length of the light incident on the eye in the foveal mode.

6. The optical system according to claim 5, wherein the foviation section includes at least one light-gathering element having a controllable focal length.

7. The optical system according to any one of claims 1 to 6, wherein the at least one switchable optical element includes a first switchable element having a first focal length and a second switchable element having a second focal length, the second focal length being different from the first focal length.

8. The optical system according to claim 7, wherein the first switchable element is positioned at a first distance from the display, and the second switchable element is positioned at a second distance from the display.

9. The optical system according to claim 8, wherein the difference between the first distance and the second distance is such that the convergence point of the first switchable element and the convergence point of the second switchable element substantially coincide with each other.

10. In the foveal mode, the first switchable element is configured to be switched to a second state, and the second switchable element is configured to be switched to a first state, and The optical system according to any one of claims 7 to 9, wherein in the peripheral mode, the first switchable element is configured to be switched to a first state, and the second switchable element is configured to be switched to a second state.

11. The at least one switchable optical element includes a first switchable divergent element, a first switchable convergent element, and a second switchable convergent element. The optical system according to any one of claims 1 to 6, wherein the first switchable divergent element is located at a first distance from the projection section, and the first switchable converging element is located at a second distance from the projection section, the second distance being greater than the first distance.

12. In the foveal mode, the first switchable divergent element is configured to be switched to a first state, the first switchable convergent element is configured to be switched to a first state, and the second switchable convergent element is configured to be switched to a second state, thereby the light from the projection unit is focused by the second switchable convergent element to form an image on the central region of the retina, and The optical system according to claim 11, wherein in the peripheral mode, the first switchable divergent element is configured to be switched to a second state, the first switchable converging element is configured to be switched to a second state, and the second switchable converging element is configured to be switched to a first state, thereby causing the light from the projection unit to diverge onto the first switchable converging element by the first switchable divergent element and to be converged by the first switchable divergent element to form an image on the peripheral region of the retina.

13. The optical system according to claim 11 or 12, wherein the second switchable focusing element is positioned at a third distance from the projection section such that the convergence point of the first switchable focusing element and the convergence point of the second switchable focusing element substantially coincide with each other.

14. The at least one switchable optical element further includes a second switchable divergent element, The second switchable divergent element is configured to be switched to a second state in the foveal mode and to increase the angular size of the light focused by the first switchable converging element, The second switchable convergence element is positioned at a third distance from the projection section, and the second switchable divergence element is positioned at a fourth distance from the projection section. The optical system according to claim 11 or 12, wherein the first distance, the second distance, the third distance, and the fourth distance are configured such that the point of convergence of the light in the foveal mode and the point of convergence of the light in the peripheral mode substantially coincide with each other.

15. The foviation unit is configured to focus the projected light to at least one convergence point in order to cause the projected light to enter the eye and form an image on the retina. The optical system according to any one of claims 1 to 14, wherein the at least one convergence point is located on or near the pupil of the eye on a convergence arc, the position of each convergence point on the convergence arc corresponds to the rotation angle of the eye around the axis of rotation of the eyeball, and the convergence arc has a center of curvature substantially coincides with the axis of rotation of the eyeball.

16. The optical system according to claim 15, wherein the at least one convergence point is moved such that the central portion of the image substantially coincides with the central region of the retina of the eye, preferably the macula of the retina, more preferably one of the perifovea, parafovea, fovea, foveal avascular zone, foveal fovea and foveal fovea of ​​the retina.

17. The optical system according to claim 16, further comprising an operating unit configured to move the at least one convergence point around the convergence arc in response to the rotation of the eye around the axis of eye rotation, wherein the arc has a center of curvature substantially coinciding with the axis of eye rotation.

18. The optical system according to claim 17, wherein the operating unit is configured to move at least one element of the projection unit and the foviation unit.

19. The optical system according to claim 17 or 18, wherein the operating unit includes at least one optical element for directing the light from the projection unit toward the foviation unit.

20. The optical system according to claim 19, wherein the at least one optical element includes an operable mirror.

21. The optical system according to claim 19 or 20, wherein the at least one optical element includes a plurality of switchable lenses, each switchable lens having a respective orientation and configured to direct the light from the projection portion onto different portions of the foviation portion, so that the foviation portion focuses the light onto different convergence points on the convergence arc.

22. The optical system according to any one of claims 19 to 21, wherein the at least one optical element includes a phase modulator having controllable phase modulation.

23. The optical system according to any one of claims 1 to 15, wherein the projection unit is configured to project a plurality of lights, each encoding a portion of the image, and the foviation unit is configured to converge each light onto a corresponding point of convergence, the position of each convergence point on the convergence arc corresponds to the respective rotation angles of the eye around the eyeball rotation axis, so that the light converging on adjacent convergence points encodes adjacent portions of the image.

24. The optical system according to any one of claims 15 to 23, wherein the at least one convergence point is located on a convergence plane, the convergence plane is substantially parallel to the pupil of the eye, and includes the convergence arc.

25. The projection unit is configured to project a first light that encodes a first image for the user's first eye, and a second light that encodes a second image for the user's second eye, and The foviation unit includes two foviation sections, each corresponding to one of the user's eyes. The optical system according to any one of claims 1 to 24, wherein each foveation portion is configured to switch between a foveal mode for projecting light onto a central region of the retina of the corresponding eye and a peripheral mode for projecting the light onto a peripheral region of the retina of the corresponding eye, the peripheral region being larger than and including the central region.

26. The projection unit includes a display configured to emit the displayed image with substantially collimated rays, and a focusing element for converging the rays toward the foviation unit. The optical system according to any one of claims 1 to 25, wherein the projection unit optionally includes a spatial filter for receiving the focused light rays.

27. The projection unit includes a coherent light source, preferably a laser, configured to emit substantially coherent light. The projection unit is A collimating element configured to collimate the substantially coherent light, A phase modulator configured to encode the image by modulating the substantially coherent light, and Laser beam scanning system including at least one operable mirror An optical system according to any one of claims 1 to 26, optionally comprising at least one of the above.

28. The optical system according to any one of claims 1 to 27, wherein the projection unit includes a light field display configured to project light that encodes a plurality of elemental images in a three-dimensional light field, and each elemental image forms a part of the image.

29. The optical system according to any one of claims 1 to 27, wherein the projection unit includes a phase modulator configured to project the light that encodes a holographic image.

30. A projection device comprising an optical system according to any one of claims 1 to 29 and an eye tracker for determining the rotation angle of the pupil of the eye, Optionally, the eye tracker includes at least one camera for capturing an image of the eye, the eye tracker is configured to determine the rotation angle of the pupil of the eye based on the image, and the rotation angle of the pupil is used to determine the position of the at least one convergence point on the convergence arc. Optionally, the eye tracker is configured to determine the focal length of the lens of the eye, and Optionally, the eye tracker is a projection device comprising at least one light source for illuminating the eye.

31. A method for myopic projection of an image into the user's eye, wherein the optical system is: Projecting light to encode the aforementioned image, Controlling at least one switchable optical element to switch between foveal mode and peripheral mode. Includes, In the foveal mode, the light encoding the image is projected onto the central region of the retina of the eye by the at least one switchable optical element, and In the peripheral mode, the light encoding the image is projected onto the peripheral region of the retina by the at least one switchable optical element, wherein the peripheral region is larger than and includes the central region.

Images