Image projection system

Abstract

A wearable / head-mounted retinal projection system is provided for providing a pure, augmented or virtual reality experience to a user. A system (100) for use in retinal image projection includes at least a first image projection unit (130) and a second image projection unit (140) and an ocular projection optical module (120). The projection units (130, 140) are configured and operable to project at least first and second image portions, respectively. The ocular projection optical module (120) is optically coupled to the image projection units (130, 140) and configured to combine the projection optical paths of the at least first and second image projection units (130, 140) along a common optical path to project a composite image of the first and second image portions onto the retina (12, 14) of a user's eye (10).

Description

Technical Field

[0001] The present invention relates to the field of image projection systems, and more particularly to a wearable / head-mounted retinal projection system for providing a pure, augmented or virtual reality experience to a user.

Background Art

[0002] Head-mounted or generally wearable image projection systems are used to provide virtual reality and / or augmented reality experiences by directly displaying an image onto a user's eye. Various types of head-mounted projection systems using image projection in front of or into the user's eye are known. Such projection systems are often configured as glasses that can be attached to the user's head and project an image onto the user's eye to provide a realistic and convincing display.

[0003] Similar to standard display systems, head-mounted display systems aim to provide high-resolution images while utilizing limited computing power. To simplify the complexity of image rendering, specific retinal / foveal display systems have been developed that utilize separate image projections for the foveal region of the user's eye and lower-resolution image projections directed towards the peripheral region of the retina to provide a wide field of view.

[0004] US2008 / 002262 discloses a head-mounted display device comprising a mount for attaching the device to a user's head, a beam splitter attached to the mount by an actuating device, an image projector for projecting an image onto the beam splitter, an eye tracker for tracking the user's line of sight, and one or more processors. This device uses the eye tracker and the actuating device, and optionally a head tracker, to move the beam splitter around the center of rotation of the eye and to keep the beam splitter within the direct line of sight of the eye. The user can view the image and the environment behind the image simultaneously. A second beam splitter, eye tracker, and projector can be used for the user's other eye to create a stereoscopic virtual environment. This display can accommodate the rotational ability of the human eye. This invention presets high-resolution images wherever the user looks.

[0005] US2012 / 0105310 describes a head-mounted display system having one or more retinal display units with a curved reflector disposed in front of one or both eyes of a wearer. The unit comprises a first set of three modulated visible light lasers co-arranged and adapted to provide a selectable color laser beam, and a first scanner for scanning the laser beam horizontally and vertically across a portion of the reflector curved in multiple directions to produce a reflection of the color laser beam sufficient to surround the fovea through the pupil of the eye onto a portion of the retina. The unit also comprises a second set of three modulated visible light beam lasers and an infrared laser, all lasers being co-arranged and adapted to provide color and infrared peripheral vision laser beams, and a second scanner unit scans the visible light beam and the infrared laser beam horizontally and vertically across a portion of the reflector curved in multiple directions to reflect the scanned color and infrared laser beams through the pupil of the eye onto a portion of the retina corresponding to a field of view of at least 30 degrees × 30 degrees.

[0006] US2005 / 185281 describes a viewing device having a screen. This device includes means for detecting the fixation point of the observer's eye in the image on the screen. This device has means for displaying a foveal inset image of the image on the screen around the fixation point, such that the observer's fovea will view the foveal image while the rest of the eye views the image. This method includes the step of detecting the fixation point of the observer's eye on the image on the screen. By the step of displaying on the screen a foveal inset image of the image around the fixation point, the observer's fovea will view the foveal image while the rest of the eye views the image.

[0007] US2009 / 189830 describes a display device mounted on and / or inside the eye. This eye-mounted display includes a plurality of sub-displays, each projecting light to a different retinal position within the portion of the retina corresponding to that sub-display. The projected light propagates through the pupil but does not spread across the entire pupil. In this way, the plurality of sub-displays can each project their light to the associated portion of the retina. As the projection moves from the pupil to the cornea, the projection of the pupil onto the cornea is called the corneal aperture. The projected light propagates through a portion smaller than the full aperture of the cornea. The sub-displays utilize spatial multiplexing on the corneal surface. SUMMARY OF THE INVENTION

[0008] There is a need in the art for a novel configuration of a display system that provides retinal image projection with desirable high image / projection quality for a given image rendering capability.

[0009] In conventional projection systems, the maximum image resolution is generally limited by several factors, such as the processing capabilities provided by control units such as image generation elements (projection units), graphics processing units (GPUs), and the data transmission bandwidth from the GPU to the projection unit. Therefore, to provide image projection with a pixel density equal to the spatial resolution of human vision using a conventional eye projection system, extremely high computing power and typically an array of small projection / display units may be required.

[0010] More specifically, to provide an image at the maximum resolution of the human eye, it is typically necessary to project an image frame of about 20 megapixels or more for each eye. Furthermore, to provide a temporal resolution that matches human perception (so that the movement of the image is smooth and seamless), it is necessary to render the displayed image at 60 Hz or more. This requires speeding up image rendering and data transfer between the control unit and the projection unit, and between the memory utility and the control unit (for example, on the order of 28 GB / second considering the projection of an image with a color depth of 24-bit color). Such high data transfer speeds generally exceed the capabilities of the latest eye projection devices, and in any case, the weight, size, cost, and energy consumption of the system will increase.

[0011] The present invention provides a novel image projection system that uses two or more image projection modules / units to project an image having spatially varying image projection quality onto the retina. In this regard, the term "image projection quality" is used in this document to refer to the pixel density of the image projection onto the retina (e.g., DPI or dots per unit solid angle), and optionally the color depth level in the projected image. For this purpose, in some embodiments, two or more projection modules each provide an image portion having two or more levels of color depth.

[0012] In certain embodiments, the technology of the present invention utilizes a high pixel density image portion to the foveal region of the user's eye, i.e., a projection with a high angular resolution and an equivalently high dots per inch (DPI) per projection area, and a projection of an image portion with a lower pixel density (lower angular resolution / DPI) to the peripheral retina of the user (i.e., the parafoveal region). Thereby, while reducing the requirements for image rendering, data transfer, and storage of the projection system, an effective high-resolution perception of the projected image by the user's eye is achieved. In this way, a high pixel density image is provided to the retinal region (fovea) that can collect the details of the image and convert them into the user's brain, while an image with a low pixel density (angular resolution) is provided to the retinal region (parafovea) with low perception ability.

[0013] Similarly, certain embodiments of the present invention utilize the fact that the perception of color depth is much more prominent in the foveal region of the retina of the eye than in other (parafoveal) regions. In these embodiments, the image portion projected onto the fovea is projected with a higher color depth than the image portion projected onto the periphery.

[0014] Accordingly, according to certain embodiments of the present invention, a specific portion of the image is projected with a high image projection quality (high angular resolution and / or high color depth) onto a specific region of the retina (i.e., the fovea) that can perceive a projected image with a high DPI and / or a high color depth, and a specific other portion of the image is projected with a lower image projection quality onto a region of the retina (i.e., the peripheral / parafoveal region of the retina) where the perception is limited to a lower DPI and / or a lower color depth.

[0015] Accordingly, some embodiments of the present invention utilize two or more image projection modules / units each having a different, wide or narrow, angular spread. An image projection module having a narrow angular spread (e.g., covering a solid angle of 3° to 10° along each of the horizontal and vertical axes) projects an image of higher image projection quality (higher angular resolution / DPI and / or higher color depth) onto the central (fovea) region of the retina and is configured such that a user can perceive a high-quality image. An image projection module having a wide angular spread (e.g., covering a solid angle of 60° to 170° along each of the horizontal and vertical axes) is configured to project an image portion having a lower image projection quality onto the peripheral part of the retina (the so-called parafoveal region). Thereby, by utilizing the anatomical characteristics of the human eye, while projecting a high-quality image closer, it is possible to reduce the data volume and processing requirements, and / or the size / weight and / or cost of the system that would be required if the image were projected evenly at the same high quality across the entire retina.

[0016] Accordingly, the technology of the present invention dramatically reduces the data transfer and processing requirements of the eye projection system while maximizing the user experience from the projection system (the user can still recognize a high-resolution image through the regions of the retina where it is possible).

[0017] As is known, the inner coating of the human eye's retina has photosensitive tissue. The retinal region called the fovea has a high density of conical photosensitive nerve cells and is responsible for sharp vision. For this reason, the technology of the present invention uses a high-resolution image directed at the user's fovea while providing a peripheral image directed at the retina at a lower image resolution, reducing the rendering complexity while ensuring a larger field of view. Accordingly, the technology of the present invention focuses on high-resolution image projection onto the fovea and provides projection at a lower resolution, thereby providing high-resolution projection with reduced processing and data transmission requirements compared to uniform pixel density rendering.

[0018] The eye projection system of the present invention comprises an optical module configured to direct images (also referred to herein as image portions) from at least two (e.g., first and second) image projection units towards the user's eye (towards at least one of the eyes). This optical module is configured to direct the image portion provided by the first projection unit towards a first region (fovea) of the user's eye, and to direct the image portion projected from another projection unit (e.g., the second projection unit, or an additional projection unit, if used) towards the peripheral / peripheral region (parafovea) of the retina.

[0019] According to some embodiments, the optical module generally comprises a combining unit (e.g., a beam combiner) and a relay unit (optical relay), which are arranged in a cascaded order along the optical path of the optical module to direct the image projection from the image projection units and combine them to project (simultaneously or otherwise) onto the user's eye. More specifically, the combining unit combines the light beams associated with the projected image portions generated by at least the first and second projection units into a combined optical field representing the full projection image frame to be provided / projected onto the user's eye. Here, the terms optical field and combined optical field are used to specify the intensity profile of the light measured across the optical path of the image projected towards the eye, and optionally the color content. The light beams forming the combined optical field are transmitted from the combining unit to the optical relay, which directs the optical field towards the user's eye.

[0020] More specifically, in some embodiments, the optical relay is configured to relay the optical field such that it is projected directly onto the retina. Examples of such configurations and methods of optical modules including such relays that are configured and operable to project an image directly onto the retina of the eye and can be incorporated into the optical module of the present invention are described, for example, in PCT International Publication No. 2015 / 132775 and IL Patent Application No. 241033, both of which are assigned to the assignee of the present application and incorporated herein by reference.

[0021] In this regard, it should be understood that the term "direct projection" as used hereinafter relates to the projection of an optical field in which the propagating optical field converges onto an image plane on the user's retina. For example, the optical module and / or its optical relay may be configured such that the light beam of the optical field reaches the eye lens so that they are substantially collimated, and / or is focused onto the retina by the eye lens itself. Alternatively or additionally, such direct projection may be achieved by projecting the optical field towards the retina such that its cross-sectional diameter is substantially (e.g., more than twice) smaller than the pupil entrance of the eye (thereby obtaining a high depth of field of the image projection on the retina).

[0022] In some embodiments, the optical module comprises an orbit module (e.g., a movable or rotatable optical deflector showing a line-of-sight tracking optical deflector and / or a pupil position optical deflector as described in IL Patent Application No. 241033) configured and operable to adjust the optical path of image projection according to the line of sight (LOS) of the user's eye. For this purpose, the system can utilize and / or include a line-of-sight tracking unit configured to detect changes in the line of sight and / or the direction of the line of sight of the user's eye, provide corresponding data to the orbit module to change the orientation of the general optical path, and determine the deflection of the optical path provided by the orbit module. Thus, the image (optical field) may be projected by the system along a general optical path that changes according to changes in the direction of the line of sight (LOS) of the eye and / or changes in the pupil / eye position relative to the eyeball projection system / eye position relative to the eye projection system. For this reason, the orbit module may be configured to change the general optical path of light propagation along the optical module according to the orientation of the user's eye relative to the eyeball projection system (e.g., according to the orientation of the optical axis / line of sight (LOS) of the eye). Examples of such optical systems including optical relays, and eye-tracking optical deflectors (e.g., pupil position deflectors and line-of-sight direction deflectors) that can be used such that the position of the eye and its line-of-sight direction are projected onto the retina of the eye, change with respect to the eye projection system and are described, for example, in IL Patent Application No. 241033, which is assigned to the assignee of the present patent application and incorporated herein by reference.

[0023] For this purpose, by using the orbital module, the eye tracking unit, and the optical relay unit, the optical path of the optical module can be changed such that an optical field combined with two or more image portions is transmitted along a general optical path toward the pupil of the user. The projected light field can be directed to reach the position of the pupil from various angular orientations so that when the user changes the line of sight direction and / or when the relative displacement between the eye projection system and the eye changes, the user's eye forms a composite image at an appropriate position on the retina. As described above, the optical field is configured such that an image portion generated by the first projection unit forms a part of the image on a selected portion of the retina (i.e., the fovea), and an image portion generated by one or more second projection units forms a part of the image on other regions of the retina (the parafovea). Further, the position of the generated image portion on the retina can remain fixed even if the user changes the line of sight direction.

[0024] Accordingly, the optical relay (also referred to herein as the relay unit) is generally configured to generate an image on the retina of the user such that an image portion provided by the first projection unit is generated on the foveal region of the retina, and an image portion provided by the other projection units is generated on the parafoveal region of the retina, which is in the peripheral part of the retina.

[0025] It should be noted that the first and second image projection units may generally have different characteristics. For example, in order to project different fields of view, the image projection units may be configured and operable to output light beams / beams that span different angular ranges towards the optical module. Also, they may be configured to output images having different angular resolutions and / or different color depths. For example, the first image projection unit may be adapted to provide an RGB image (image portion) having a high angular resolution and a high color depth, and the second image projection unit may be configured to provide an image portion having a lower color depth, in some cases a single-color RGB image portion, and / or a lower angular resolution. The change in color depth may be such that the first projection unit provides an image with a color depth of, for example, 32 bits or 24 bits, and one or more second projection units provide an image with a color depth of 16 bits or 8 bits.

[0026] For this reason, in some cases, the first and second image projection units may be configured based on different technologies. For example, the first image projection unit may be configured as a scanned image projection in which the output image is generated by scanning (e.g., rastering) a light beam over the angular range in which the image is output, thereby generating a first optical field that encodes the generated image (image portion) and modulating the intensity of the light beam and, in some cases, the color content for output towards the optical module. Using scanned-based image projection can be advantageous with respect to the output and intensity of non-scanned-based (e.g., SLM-based) projection units. The second image projection unit modulates the intensity and possible color content of a plurality of pixels projected thereby simultaneously, either as the scanned image projection system described above or as an area image projection system that utilizes one or more spatial light modulators (SLMs, liquid crystal arrays and / or micromirror arrays). Examples of the configuration and operation of image projection units that use raster scanning and / or spatial light modulation to form an image are generally known in the art of image projection, and the principles of their configuration and operation need not be described in detail herein.

[0027] According to the present invention, it should be noted that the first and second image projection units are configured and operable to each output two first and second complementary image portions (optical fields) that spatially complement each other to form a projection of a continuous image on the retinal surface. For this purpose, the first image projection unit is adapted to project an image covering an angular / transverse range around the general optical axis of the optical module such that when directed towards the retina, it falls within the foveal region of its center. The second image projection system is configured and operable to cover a wider angular / transverse field extending around the general optical axis of the optical module, while optionally scanning / covering an annular portion (or more generally a frame or donut-shaped region) around the general optical axis of the optical module, such that the image portion generated thereby falls at least within the peripheral portion of the retina when directed towards the retina.

[0028] In this regard, the first and second image projection units are configured to generate image portions that spatially complement each other (e.g., they overlap or have a common boundary) in order to enable the optical module to appropriately combine the resulting optical fields (image portions). The resulting combined optical field corresponds to a foveal image portion in its central region (on the image plane) and a parafoveal image portion in its peripheral portion (on the image plane), both of which together provide a spatially continuous image with a substantially smooth transition between the image portions. To achieve this, the first and second image projections are arranged in the eye projection system such that the image portions output and combined by the combining unit propagate spatially along the optical path of the optical module.

[0029] In some embodiments of the present invention, the second image projection unit is configured and operable such that the lateral / angular range of the second (e.g., annular) image portion (optical field) output thereby and propagating along the optical path spatially overlaps with the first (e.g., central) image portion (optical field) output by the first projection unit and propagating along the optical path. For this purpose, several overlaps between the first image portion and the second image portion along at least the periphery (annular boundary) of the first image portion are used to provide a smooth and seamless transition between the high-quality first image portion and the low-quality second image portion.

[0030] According to this technique of the present invention, the rendering process is reduced by allocating the computing power required to generate a high-resolution image for the central visual field corresponding to the area fixed by the user. The periphery of the image and the periphery of the user's visual field can be rendered and projected at a lower resolution. This is because the parafoveal portion of the projected image is at the periphery of the user's attention, and the parafoveal region of the user's eye (generally referred to as the retina in this document) with a lower density of photoreceptor cells, a lower data space density, and a lower resolution captures it.

[0031] The image directed at the user's eye is generally rendered according to the orientation of the eye, and the transmission of the image / optical field is adjusted by the eye-tracking unit, so that the user can perceive a large visual field (without actually having an image boundary) and experience a complete virtual reality (or augmented reality), providing a sense of presence to the user.

[0032] Thus, according to a broad aspect of the present invention, a system for use in retinal image projection is provided, which at least first and second image projection units configured and operable to project at least first and second image portions respectively, and An eyeball projection optical module optically coupled to the at least first and second image projection units, combining the projection optical paths of the at least first and second image projection units along a common optical path, such that the light beams of the first and second image projection units respectively associated with the projection of the first and second image portions project a combined image including the first and second image portions onto the retina and are configured and operable to propagate towards the user's eye.

[0033] According to some embodiments, the first and second image projection units and the eyeball projection optical module are configured and operable such that the first image portion projected by the first image projection unit is directed towards a first central region on the retina of the user's eye, and the second image portion projected by the second image projection unit is directed towards a second annular region at the periphery of the retina.

[0034] In some embodiments, the second image projection unit may be configured to project the second image portion over an angular range larger than the angular range of the first image portion projected by the first image projection unit.

[0035] In some embodiments, the first image projection unit projects the first image portion onto a first central region of the retina so as to cover the foveal region of the retina, and the second region covers at least a part of the parafoveal region of the retina surrounding the foveal region.

[0036] The first and second projection units are further configured and operable to project an image portion with relatively high image projection quality onto the foveal region of the retina and to achieve the projection of an image portion with relatively low image projection quality onto the peripheral region of the retina. The image projection quality may be related to at least one of the angular resolution and color depth of the image projection.

[0037] According to some embodiments, at least one of the first and second image projection units may be a scanning type image projection unit configured and operable to project an image by scanning an image-encoded light beam onto the retina.

[0038] According to some embodiments, the system further comprises a control unit associated with an eye tracking module configured and operable to detect a change in the fixation direction of the eye, the eye projection optical module comprising an orbit module configured and operable to adjust a general optical path of the image projection towards the eye, the control unit being adapted to operate the orbit module in accordance with the detected change in the fixation direction.

[0039] The eye tracking module is configured and operable to detect a change in the lateral position of the pupil of the eye with respect to the system, and the control unit is adapted to operate the orbit module in accordance with the detected change in the lateral position of the pupil.

[0040] The control unit may be configured and operable to operate the orbit module so as to compensate for the detected change and thereby maintain the projected combined image at a fixed position on the retina.

[0041] According to some embodiments, the eye projection optical module is configured to direct input light into the user's eye, through the pupil and towards the retina, such that a cross-section of the light field (e.g., full width at half maximum, or 25% intensity) is smaller than the user's pupil. This provides an eyebox having a smaller diameter with respect to the user's pupil. The eye projection optical module changes at least one of the position and angle of the eyebox in accordance with data on the fixation position of the user's pupil received from the eye tracking module, thereby aligning the exit pupil with the optical axis of the user's eye.

[0042] According to yet another embodiment, the system comprises a control unit configured and operable to obtain image data representing the content of the combined image to be projected onto the user's eye and segment the image data into at least said first and second image portions, wherein said first and second image portions are complementary image portions that can be projected onto the central and peripheral regions of the retina by said first and second image projection units, thereby projecting the combined image onto the retina.

[0043] The optical projection module comprises an optical coupling element configured to combine the image projections of the first and second image projection units such that a first optical field generated by the first image projection unit and associated with the projection of the first image portion propagates along a central region of a plane perpendicular to the optical axis of the optical projection module, and a second optical field generated by the second projection unit propagates in a peripheral region of the plane with respect to the central region.

[0044] According to some embodiments, the system is configured and operable such that the first optical field propagating along the central region is projected towards the eye so as to cover a central portion of the eye's field of view, thereby providing an image projection onto the foveal region of the retina, and the second optical field propagating along the periphery of the optical path covers an annular region of the field of view, thereby providing an image projection onto the parafoveal region of the retina.

[0045] Said first and second optical fields are projected with high and low image projection qualities respectively, and the second projection unit is configured to provide image projection in a donut-shaped field of view, thereby providing image projection in the parafoveal region.

[0046] Additionally or alternatively, the first and second optical fields may overlap in a boundary region between the central region and the peripheral region, thereby providing a projection of an overlapping portion of the first and second image portions in the boundary region. The first and second image portions may be aligned such that the overlapping portions projected by the first and second image projection units correspond to similar image content.

[0047] According to some embodiments, each of the at least first and second projection units is configured to provide output light corresponding to an image projected within a projection angle range α max and the optical projection module is configured to relay the output light to the user's eye such that the images projected by the first and second projection units enter the user's pupil at respective angle ranges α 1 in and α 2 in and α 2 in > α 1 in . α 1 in may correspond to an angular range of 3°. α 2 in may correspond to an angular range greater than 20°.

[0048] According to some embodiments, the system may be configured for use with a head-mounted display unit.

[0049] According to some embodiments, the system may be configured to provide a virtual reality or augmented reality experience.

[0050] In some embodiments, the eye projection optical module may be configured to direct the images projected by the first and second projection units towards the user's eye while shielding ambient environmental light.

[0051] In some embodiments, the eye projection optical module may be configured to direct the images projected by the first and second projection units towards the user's eyes while allowing transmission of ambient environmental light.

[0052] To better understand the subject matter disclosed herein and to illustrate how it can be implemented in practice, the following embodiments will be described as non-limiting examples with reference to the accompanying drawings.

Brief Description of the Drawings

[0053]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0054] As described above, in the art, a novel configuration of an eyeball projection system is required. Referring to FIGS. 1A and 1B together, an eyeball projection system 100 and a method 250 for projecting an image onto a user's eye according to some embodiments of the present invention are shown. The eyeball projection system 100 is configured to generate an image and project the generated image onto the user's eye 10. For simplicity, only a single eye is shown in this figure, but in general, it should be understood that the image projection system 100 may be configured to project an image onto both eyes of the user and provide a three-dimensional experience by simultaneously providing a specific difference between the right-eye image and the left-eye image.

[0055] The eyeball projection system 100 includes at least first and second image projection units / modules 130, 140 (hereinafter also referred to as projection units), and an optical module 120 that guides light corresponding to the images projected by these projection units to the user's eye 10 to form an image on the user's retina 12. The system also generally includes or is connectable to at least one control unit 200. The control unit 200 typically renders image data and transfers it to be projected from the first and second projection units 130, 140.

[0056] To this end, rendering the image data projected by two or more (first and second) projection units 130, 140 may include performing the operations of the method 250 described herein. The control unit 200 is configured and operable to perform an operation 251 for receiving image data indicative of the content of a "projected image frame" (also referred to herein as a combined image) to be projected onto the user's eye. The image data can include, for example, the content and layout of the image to be projected onto the retina (where the content is information of one or more images to be projected onto the retina simultaneously, and the layout includes information regarding the arrangement / layout of the projection of these one or more images). The layout data can include, for example, lateral positioning data indicating the lateral position of the image on the retina (e.g., data regarding an anchor point in the image representing the intersection between the eye's LOS and the image plane). The control unit 200 may be adapted to perform any operation 252 for processing the image data to determine / generate a "projected image frame" indicative of the combined optical field (image content) to be projected onto the retina and arranging them within the frame according to the layout data. In any operation 253, the control unit performs an alignment operation to align the "projected image frame" with respect to the optical axis (LOS) of the eye. In other words, the alignment / alignment data indicating the aligned position of the "projected image frame" is with respect to the eye's LOS (e.g., the alignment / alignment data indicates the point intersected by the eye's LOS axis in the "projected image frame").

[0057] In operation 254, the control unit 200 divides the "projected image frame" into two or more segments (image portions) projected by two or more (first and second) image projection units 130, 140. At least one of the image projection units, for example 130, is adapted to project an image (image portion) onto the central (fovea) region of the retina. At least one other image projection unit, for example 140, is adapted to project an image (image portion) onto the peripheral (parafovea) region of the retina. Thus, in operation 254, the control unit 200 uses the alignment / alignment data obtained in 253 to divide the "projected image frame" into at least two image portions that are projected by the first and second image projection units 130, 140 onto the fovea region and parafovea of the retina, respectively. In this regard, in order to perform such a division, the control unit may utilize projection unit configuration data indicating projection parameters such as the numerical aperture (NA) of the first and second image projection units 130, 140 (i.e., data regarding the region of the retina covered by each projection unit and their angular ranges). Thereby, the control unit 200 can appropriately segment and divide the "projected image frame" between the image projection units 130 and 140.

[0058] At 255, the control unit 200 performs the rendering of the first and second image portions respectively projected by the image projection units 130, 140. The control unit 200 can render the corresponding first and second image portions respectively using the projection unit configuration data indicating the projection parameters such as the angular resolution and color depth provided by the image projection units 130, 140. In this regard, as shown above, the first image projection unit 130 configured to project an image onto the foveal region of the retina projects an image onto the retina with a higher angular resolution (higher DPI) and / or an improved color depth than the angular resolution (DPI) and / or color depth provided by the second image projection unit 140 that projects an image onto the parafoveal region of the retina. Then, at operation 256, the control unit provides the rendering data indicating the first and second image portions to the first and second image projection units 130 and 140 and projects them thereby. In this regard, the eyeball projection system 100 according to the present invention utilizes the characteristics of the anatomical structure of the human eye. Refer to FIG. 2 showing the anatomical structure of the human eye. Since the structure of the human eye is generally known, it will not be described in detail here, but the retina (12 in FIG. 1) is a photosensitive region that collects light and generates data to be sent to the brain. The retina includes a plurality of photosensitive cells that are sensitive to the intensity of light (black and white vision) and wavelength (color vision). More specifically, the retina includes rod cells (rods) sensitive to the luminous intensity (intensity of light) and cone cells (cones) sensitive to the chroma (color or wavelength). The central region of the retina contains cone cells (wavelength-sensitive cells) at a large density and is known as the fovea (denoted as 14 in FIG. 1). The fovea is responsible for providing a detailed image of what is located at the center of the visual field or what is located at the center of attention. Generally, the foveal region provides a higher spatial frequency or higher resolution, and in some cases a higher color perception ability, while the parafoveal region provides a low-resolution image perception (provides a blurred display of the scene periphery to the brain), and in some cases has a low color perception ability but is more sensitive to movement and tilt within the input light field.

[0059] Accordingly, the image projection units 130, 140 are configured and operable for projecting complementary portions of a combined optical field (the "projected image frame") onto the retina. The first image projection unit 130 is configured and operable to project a first image portion directed to the foveal region of the retina with high image projection quality (i.e., render / project the first image portion to have high angular resolution and / or high color depth). The second image projection unit is configured to project a second image portion (which is directed to the parafoveal region of the retina with lower image projection quality (i.e., lower angular resolution and / or lower color depth compared to the first image portion)).

[0060] For example, the image projection unit 130 is configured and operable to project a portion of the projected image frame with a high angular resolution of about or below 4 arc-minute per pixel 2 ). The image projection unit 140 is configured and operable to project a portion of the projected image frame with a low angular resolution of about or above 10 arc-minute per pixel 2 ). In some embodiments, the image projection unit 130 is configured to project each image portion with RGB color content (e.g., having a color depth of at least 8 bits (256 colors) or more (e.g., 24 bits)). The image projection unit 140 used to project an image to the periphery of the retina has a lower color depth (e.g., a color depth of 4 bits (16 colors)) and / or has little or no color information (e.g., a grayscale image).

[0061] Therefore, according to some embodiments of the present invention, the image projection unit 130 can be configured with a scanning image projection configuration (projecting an image by scanning (raster scanning) a light beam time-modulated with image information onto a projection surface (i.e., the foveal region of the retina)). With such a scanning image projection configuration of the image projection unit 130, high-quality projection is achieved with a compact size of the image projection unit 130. The image projection unit 140 may be configured with a scanning image projection configuration and / or may be configured by any of aerial image projection techniques that project each image portion onto the parafoveal region of the retina, for example, using a spatial light modulator.

[0062] The optical module 120 is configured to combine at least the image portions projected by the first and second projection units 130, 140 and guide the corresponding light rays to simultaneously form an image projected onto the user's eye 10. Further, the optical module is configured to direct the images generated by different projection units to different regions of the user's retina 12 (e.g., the fovea and parafoveal regions).

[0063] In this regard, according to some embodiments of the present invention, in the segmentation operation 254 of the method 250, the control unit 200 divides the "projected image frame" into two (or more) segments (a first and a second image portion), which have some overlap along the boundary region therebetween. Thus, in such embodiments, the first and second image projection units 130, 140 are configured and operable to project the first and second image portions onto the retina so as to overlap at the boundary therebetween. Thus, at the boundary, similar image information is projected by the first and second image projection units 130, 140 overlapping with high and low image projection qualities. The optical module 120 is configured to combine the image portions generated by the first projection unit 130 and the second projection unit 140 such that the overlap between the first and second image portions is maintained. Further, the optical module may be configured and / or operable to direct the image portions projected such that the boundary between the image portions substantially corresponds to the anatomical boundary of the fovea in the user's retina. The system may include setting parameters regarding the fovea image or the relative size / angle spread regarding the boundary position for the user's selection, or may be fixed to conform to the anatomical structure of a number of users. The overlap between the image portions is typically provided to achieve a smooth transition recognized between the higher quality of the image projected onto the fovea region of the retina and the lower quality of the image portion projected onto the parafovea region thereof, and / or to compensate for inaccuracies and anatomical diversities among users.

[0064] The control unit 200 also provides appropriate commands to the optical module 120 in response to eye tracking data regarding the orientation and / or position of the eye 10 (e.g., obtained from an eye tracking module as disclosed in IL patent application No. 241033), and may change the general path of image projection in order to correct the optical path of image projection according to the movement of the eye 10. For example, the optical module 120 may comprise an orbital module (e.g., 124 as shown in FIG. 5) that may include an adjustable line-of-sight beam deflector and / or an adjustable pupil position beam deflector (configured and operable as described in IL patent application No. 241033). The control unit 200 is configured and operable to adjust the position of one or both of these deflectors to change the general propagation path of image projection according to the direction of gaze of the eye (direction of LOS), and / or relative lateral displacement and / or relative angular orientation between the optical axis of the eye and the output optical axis of the optical module 120, so as to maintain a substantially fixed relative orientation and / or displacement therebetween. In fact, when a fixed relative orientation and displacement are maintained between the optical axis of the eye and the output optical axis of the optical module 120, the images / image portions from the first projection unit 130 and the second projection unit 140 are projected onto fixed positions on the retina.

[0065] Alternatively or additionally, in some embodiments, the control unit 200 configures the first and second projection units 130, 140 to shift and / or distort the projected image, thereby compensating for some / slight change / shift in the relative orientation and / or displacement between the eye and the optical axis of the optical module 120 by shifting / distorting in a plurality of directions to cancel out the change in the relative orientation / displacement of the projected optical field. For example, the use of such a technique to compensate for small eye movements is illustrated in more detail below with reference to FIG. 6.

[0066] As such, the eye projection system according to the present invention is generally configured to provide image projection with enhanced resolution in the foveal region of the retina while providing image projection with a relatively low (e.g., normal) resolution to the parafoveal region surrounding the fovea. Thereby, the system can actually utilize a high-resolution image and reduce the complexity of image rendering / processing for the high-resolution image while providing the high-resolution image to the region of the user's eye that requires it.

[0067] FIG. 3 shows a two-part image generated by the eye projection system of the present invention. The overall image includes two image parts (usually at least two, and the peripheral image part may be composed of several partial images generated by a plurality of projection units), which includes a parafoveal / retinal image part 1400 that usually provides peripheral image data surrounding the center of attention, and a foveal image part 1300 that is the main part of the image data and corresponds to the center of the user's attention. The foveal image part 1300 can typically have a higher resolution than the parafoveal image part 1400. The actual number of pixels in the foveal part 1300 and the parafoveal part 1400 may be the same or higher. The difference in image resolution is typically provided due to the difference in the area (field of view) covered by each image part. In particular, the foveal image part can be projected so as to generally cover the actual area of the retina or a slightly larger area that is significantly smaller than the surrounding area of the retina. Note that the image parts shown in FIG. 3 illustrate a circular field of view. However, generally, the field of view may be rectangular, elliptical, or any other arbitrary shape. The foveal region 1300 of the projected image is preferably circular or elliptical so as to cover the field of view of the fovea, and thus the sharp observation ability of this region of the eye is optimized. Also, FIG. 3 illustrates the angular range of the image parts of the fovea 1300 and the parafovea 1400 in the user's pupil. Typically, the angular range of the foveal image part is α 1 in and ranges from 3° to 10°, preferably about 5°. Further, the angular range of the parafoveal image part of the pupil input can be greater than 20° and is typically about 120° to 180°.

[0068] Referring to FIGS. 4 and 5, two configurations of the eyeball projection system 100 are shown, illustrating more specific configurations of the optical module 120 according to two exemplary embodiments of the present invention. As shown in FIG. 4, the first projection unit 130 and the second projection unit 140 are respectively associated with the corresponding initial relay modules 122a and 122b. In the example of FIG. 5, the relay modules are combined into a single relay module 122 including two (generally at least two) input lenses L1a and L1b and a single output lens L2. As shown in both examples of FIGS. 4 and 5, the optical system 120 preferably comprises a coupling module (M or M1 and M2), a first relay module 122 and a second relay module 126, and a tracking / trajectory module 124. In this connection, the first relay module including a plurality of separate relay modules as in FIG. 4 or the combined relay module as in FIG. 5 is configured to integrate the image projections generated by the first projection unit 130 and the second projection unit 140 such that each projection unit forms an image portion (i.e., an optical field) in a corresponding region along a cross-section perpendicular to the general propagation direction of the projection light. Further, FIG. 4 shows the output angle range α max of the first projection unit 130 and the second projection unit 140. As shown, the first projection unit 130 and the second projection unit 140 may or may not provide the same output angle range. The optical system 120 is configured to adjust the angle range of each projection unit as described above with reference to FIG. 3.

[0069] Referring to FIG. 4, each of the first projection unit 130 and the second projection unit 140 outputs light indicating an image or image stream marked in the figure by the maximum light rays R1a and R1b for the first projection unit 130, and R2a and R2b for the second projection unit 140. The output light from the first projection unit 130 is transmitted to the input lens of the relay module 122a and relayed to the orbit module 124. Specifically, light rays are output from the projection unit such that different pixels, or different points on the projected image, are associated with the propagation of different light angles. Thus, the maximum light rays R1a and R1b correspond to two extreme points on the projected image. The first lens L1a of the relay unit 122a refracts the light and directs it towards the second lens L2a, which refocuses the input light on the orbit module 124. At the output of the relay unit 122a, one or more beam combiners M1 and M2 are arranged as illustrated in the figure. The beam combiners M1, M2 are configured to combine the light projected from the first projection unit into the optical path of the light projected from the second projection unit 140. Similarly, the relay unit 122b typically includes a first lens L2a and a second lens L2b and is configured to enable the projection of light from the second projection unit 140 in a substantially similar manner. The exemplary light rays R2a and R2b indicate the extreme light rays of the projection unit 140. In general, the relay units 122a and 122b are composed of appropriately selected different optical powers of lenses, and the beam combiners M1 and M2 are arranged such that the image projected by the first projection unit 130 occupies a small area at the center of the image projection area surrounded by the portion of the image projected by the second projection unit 140, as illustrated in FIG. 3. Further, both relay units 122a, 122b and beam combiners M1, M2 are configured to form a common image plane by integrating the image portions (e.g., on the orbit unit 124). This is to ensure a common focus for the user's eyes.

[0070] Although not specifically described here, the relay unit 122a (and other relay units such as 122b, 126, etc.) may have additional lenses, but is shown as two lens units for simplicity. Note that the optical parameters of the relay unit are selected to provide a proper imaging having a desired resolution and sharpness as generally known, and / or can be determined by standard optical design tools.

[0071] The projection images generated by the first and second projection units 130, 140 are directed towards the orbit module 124. The orbit module 124 is configured to change its orientation so as to direct the light incident on a general optical path determined according to the tracking of eye movement, and can include, for example, one or more movable light deflectors / mirrors (e.g., the above-described line-of-sight tracking beam deflector and / or pupil position polarizer). The orbit module 124 and the line-of-sight tracking technology can be of any known configuration, and as described above, an exemplary configuration is described in IL Patent Application No. 241033 assigned to the assignee of the present application.

[0072] As described above, FIG. 5 shows an additional configuration of the first relay module 122 configured to combine the projection images from the first and second projection units 130, 140 within the relay module. The relay module 122 uses a common second lens L2, but separate first lenses L1a and L1b for the first lens 130 and the second lens 140. As shown, the output from the second projection unit 140 is relayed to the orbit module 124 via lenses L1b and L2. The positions and optical powers of lenses L1b and L2 are selected to provide an angular distribution of the projection light (e.g., the maximum rays R2a, R2b) and to provide a desired angular resolution for the user's peripheral vision. The light output of the first projection unit 130, exemplified by the maximum rays R1a and R1b, is collected by an input lens L1a that converts the divergent light into a set of parallel light beams propagating towards the beam combiner M. The beam combiner M may use a single surface (i.e., a reflective surface) or multiple surfaces as described above, or may be configured as a partial reflective surface (i.e., a beam splitter type), and propagates the output light of the first projection unit 130 to be positioned at the cross-sectional center of the light output from the second projection unit 140. Generally, the beam combiner M is configured to block the transmission of light from the second projection unit 140 within the central region of the field of view. However, in some configurations, the beam combiner M may be configured to partially transmit the light passing through it, and thus allow at least a portion of the light generated by the second projection unit 140 to pass through the center of the field of view. In some further embodiments, the beam combiner M can block the central region and transmit at its periphery to achieve a smooth transition in image projection between the images generated by the first projection unit 130 and the second projection unit 140. The combined projection light is further collected by the second lens L2 and oriented / focused onto the orbit module 124.

[0073] In this regard, it should be noted that when using beam combining technology, that is, using one, two, or more beam combiners as shown in FIGS. 4 and 5, the image projection by the first projection unit 130 (foveal image) and the image projection by the second projection unit 140 (parafoveal image) overlap to some extent. For this purpose, one or more beam combiners can be configured as a beam splitting surface that reflects 50% of the light and transmits 50%, and / or configured to have a high transmittance (or reflectance) at the peripheral portion of the surface and a high reflectance (or transmittance) at the central portion of the surface. In this way, the transition between the foveal image and the parafoveal image becomes relatively smooth. It should be noted that the image processing unit (GPU) may typically be configured to render different image portions so as to provide a smooth transition as described above. For example, the GPU can be configured to render the image while adjusting the image brightness at the boundary of the image portion to avoid a sharp gradient generated in the image synthesis.

[0074] Generally, according to the present invention described with reference to FIGS. 1, 4, and 5, the first and second projection units 130, 140 may be any type of projection unit, and are preferably configured as scanning laser projection units. Generally, a scanning laser type projection unit can provide greater efficiency with respect to light intensity as well as the resolution of the projected image. Typically, the first and second projection units 130, 140 have similar specifications, but provide projections of different image data sent for the control unit (200 in FIG. 1) or the image processing unit (GPU). The optical module is configured to combine the image projections of the first and second projection units (130, 140) as generally illustrated in FIG. 3. However, the image data provided to the second projection unit 140 may show the entire image including the central (fovea) region, or may include image data corresponding to a donut-shaped image (that is, a peripheral image having a hole region where the image projected by the first projection unit 130 is combined).

[0075] As described above, the first and second projection units (130 and 140) may preferably be scanning laser projection units. In such a projection unit, a raster light deflector (a movable mirror using MEMS) is configured to scan a laser beam within a certain angular scanning range (angular projection range) α max inside. The optical module 120 synthesizes the light of at least the first and second projection units, and at the user's pupil, the light generated by the first projection unit has an angular range α 1 in and the light generated by the second projection unit has α 1 in a larger α 2 in so as to direct it. Effectively, the difference in the light propagation angle at the user's pupil corresponds to different points within the visual field. This is because the angular resolution of the light projection generally corresponds to the resolution of the perceived image. Based on the anatomical structure of the human eye, the inventors have found that it is preferable to set the input angular range α 1 in of the light projection by the first projection unit within a range of about 3°. In some configurations, the optical module 120 and the relay module 126 are configured to provide an angular range of about α 1 in = 5° to reliably cover the foveal region within the retina. The angular range α 1 in is preferably determined according to the image resolution provided by the first projection unit 130 such that the angular resolution at the user's input pupil exceeds 2 arcminutes per pixel and preferably exceeds 1 arcminute per pixel. In contrast to the projection by the first projection unit 130, the light projection by the second projection unit 140 is generally configured to provide a meaningful image at the periphery of the visual field. Therefore, the angular range α 2 inPreferably greater than 20°, and in some configurations greater than 70°, it provides a wide field of view for the user's image projection and provides a sense of presence within the projected image. The second projection unit 140 may provide a similar number of different angular points such that the angular resolution decreases as the angular range increases.

[0076] When using a scanning laser projection unit, the laser beam generally includes light beams from three or more laser units that emit three or more primary colors (e.g., red, green, and blue), and varies the intensity of each color according to the scanning direction to provide imaging of the desired image data. The optical module 120 is configured to relay the light output from the first and second projection units so as to direct the projected light towards the user's eyes. Generally, the optical unit, more specifically the relay module 126, is configured to direct the input light towards the user's eyes such that the cross-section of the light at the user's pupil (i.e., the eyebox) has a small diameter with respect to the user's pupil. More specifically, the cross-section diameter of the light (e.g., full-width at half-maximum measurement or standard deviation measurement) is small with respect to the pupil diameter under strong illumination conditions. This is because the orbital module 124 reflects the general optical path in order to change the position and angle of the eyebox (the exit pupil of the system) according to the detected line of sight direction (LOS) and / or the position of the pupil (e.g., due to eye / LOS movement with respect to the pupil projection system 100). The output intensity of the projection unit may be scanning laser-based or non-laser or non-scanning, and in some embodiments is a spatial light modulator type image projection unit (e.g., LCD-based), preferably low enough or attenuated to avoid damage to the user and preferably avoid causing discomfort.

[0077] In this regard, it should be noted that the direct projection technique used in the optical module 120 according to the present invention projects an image onto the retina such that the input light field propagates onto the image plane on the retina. This is generally achieved regardless of the user's eye focal length / configuration (which is generally controlled based on the actual or virtual distance to the object of interest), since the size of the eyebox, or the cross-section of the light field at the pupil, is generally smaller than the pupil diameter. Thereby, an image projection with an improved depth of focus on the retina is obtained. Thus, the image is projected such that it is substantially in focus on the retina in substantially any focal state of the eye lens. For example, the image can be projected with a significant depth of focus while in focus on the retina when the eye lens is in any focal state within a wide focal length range from 4 meters to ∞.

[0078] Generally, according to some embodiments of the present invention, the eye projection system illustrated in FIGS. 4 and 5 projects an image onto the user's eye using an optical relay. In this regard, the technology of the present invention combines the projected images of the projection units (e.g., the first and second projection units), and the combined light field passes through the orbital module 124, tracks the eye movement, and is sent to the eye through the relay module 126. Thus, the optical module 120 can be configured to optimize the projection with respect to the eye orientation, illumination conditions, image characteristics, user preferences, etc. This is because the projection of different image portions by the projection units is combined to direct the image portions to the corresponding regions of the user's retina. As described above, in some embodiments of the present invention, the first projection unit provides an image projection directed to the foveal region of the user's eye, and the second projection unit provides a peripheral image directed to the retina around the fovea. The projected images are synthesized using one or more beam combiners and a first relay module. The latter is typically configured to adjust the spread of the projected image such that the pixel density in the "foveal" image projected by the first projection unit is greater than the pixel density of the surrounding "retinal" image projected by the second projection unit. Generally, the foveal image is projected at a resolution corresponding to 480p, 720p, 1080p or higher, and is projected onto an angular portion of approximately 3° - 5° of the field of view in each direction. The parafoveal / retinal image is projected with substantially the same number of pixels, but the projected image is relayed to the user's eye to occupy a predetermined portion of the user's field of view with a low projection intensity while leaving a central region corresponding to the foveal image as shown in FIG. 3, thereby enabling the projection of the foveal image by the first projection unit 130.

[0079] In this way, the configuration of the optical module enables adjustment of the exit pupil and the general optical path according to eye tracking and image characteristics. It should also be noted that by providing a high-resolution image directed towards the fovea along with lower-resolution peripheral image data, the system can optimize the experience while reducing the computational complexity. Further, to compensate for small eye movements, a GPU (Graphic Processing Unit) can be associated with the eye projection system and configured to render image data corresponding to an area slightly larger than the actual projected image data. Thus, there is rendered image data, which is sent directly to the projection unit based precisely on the eye position at the time of projection. This is illustrated in FIG. 6 as the fovea 1300 and retina 1400 rendering regions. More specifically, while the image data corresponding to regions 1300 and 1400 is projected onto the user's eye, the GPU processes the image data corresponding to the following frame. The GPU generates data corresponding to regions 1310 and 1410, which are larger than regions 1300 and 1400. Regions 1310 and 1410 are referred to in this document as shoulder image data and include image data that is approximately outside the field of view defined by image portions 1300 and 1400. In the newly processed image, the data is sent to the projection units (130 and 140), and the control unit (200) uses eye tracking technology to indicate the exact position of the user's pupil onto which the processed image is projected. This technology enables image changes that compensate for small eye movements by providing the already rendered shoulder image data. In this context, in a uniform resolution projection, a high resolution (i.e., below 4 arc-minute per pixel 2To provide (( )), it is necessary to generate image data having a very large number of pixels (such a hemispherical image with such a spatial resolution requires about 30 megapixels). The technology of the present invention enables reducing the image resolution to the region of the eye with low sensitivity while providing an image projection having a desired high perceived angular resolution. Therefore, the foveal image uses a high pixel density that provides an angular resolution of less than 4 minutes of arc per pixel, while the parafoveal image provides a lower angular resolution (for example, about 10 minutes of arc per pixel). As a result, the control unit and its GPU can generate image data corresponding to a low-resolution image with about 5 megapixels for the foveal image and 5 megapixels for the parafoveal image, a total of about 10 megapixels of rendered image data.

[0080] Thus, the present invention provides a system for projecting an image onto a user's eye. This system is configured to provide a user with a desired high-resolution image while reducing the complexity of image rendering and the data transfer from the processing / rendering unit to the projection unit. The system is generally configured to generate a composite image projection based on two or more image portions directed to corresponding portions of the user's retina and is configured to optimally utilize the local sensitivities of different regions of the retina. Those skilled in the art will readily understand that various modifications and changes can be applied to the above-described embodiments of the present invention without departing from the scope of the present invention defined in the appended claims.

Claims

Claim 1 In a system for directly projecting an image onto the retina of a user's eye, the system comprising: (a) an image projection device configured and operable to project an optical field having an intensity profile representing an image frame onto the retina with a spatially varying image projection quality across the retina, wherein the image projection device is an eye projection optical module having an eye projection optical module that defines an optical path along which light propagates towards an exit pupil of the eye projection optical module, the eye projection optical module having configuration data defining one or more projection parameters for projecting the image frame in the form of at least two image portions of the image frame in which at least one of one or more projection parameters including at least angular resolution differ from each other, providing at least different angular resolutions in the projection of the at least two image portions in different regions of the retina of the eye, the eye projection optical module being configured and operable to generate the optical field of the image frame formed by the optical fields of the at least two image portions at the exit pupil, the optical field having a cross-section smaller than an entrance pupil of the user's eye and having spatially varying projection parameters, an image projection device; (b) a processor, receiving image data indicative of an intensity profile of at least one of the image frames, utilizing the configuration data of the image projection device to process the image data to segment the projected image frame into at least two complementary segmented image portions according to the spatially varying image projection quality, and rendering the at least two segmented image portions according to the intensity profile across the frame, a processor configured to generate corresponding projection data for the image projection device; comprising The image projection device includes an orbital module, and the orbital module is disposed in the optical path such that, at the exit pupil of the eye projection optical module, the optical field of the image frame formed by the optical fields of the at least two image portions passes through the orbital module. The orbital module is configured and operable to change the direction of the optical path according to the direction of the user's line of sight (LOS) and the lateral position of the pupil, thereby maintaining the projection of the image frame at a fixed position on the retina. A system characterized by this is provided.

2. In the system according to claim 1, at least two of the image portions include at least one foveal image portion that will be received in the foveal region of the user's retina, and at least one parafoveal image portion that will be received in the parafoveal region of the user's retina. The image frame is at least segmented into a segmented foveal image portion showing the foveal image portion and a segmented parafoveal image portion showing the parafoveal image portion. A system characterized by this is provided.

3. The system according to claim 1 or 2 further includes an eye tracking module configured and operable to detect a change in the direction of the user's line of sight (LOS). A system characterized by this is provided.

4. In the system according to claim 3, which cites claim 2, the processor is configured to determine the at least two image portions according to alignment / alignment data indicating the position where the image frame is disposed with respect to the user's line of sight (LOS). A system characterized by this is provided.

5. The system according to claim 4 includes an eye tracking module configured and operable to detect a change in the direction of the user's line of sight (LOS), and by providing data indicating the change in the user's line of sight (LOS), it is possible to determine the alignment / alignment data. A system characterized by this is provided.

6. In the system according to any one of claims 2, 4, or 5, at least one of the divided foveal image portion and the divided parafoveal image portion corresponds to a large image area with respect to one of the foveal image portion and the parafoveal image portion to be projected, thereby rendering the segmented image portion with shoulder image data. A system characterized by that.

7. In the system according to claim 6, the processor is configured and operable to generate corresponding projection data using the shoulder image data and project the foveal image portion and the parafoveal image portion while compensating for small changes in the direction of the line of sight (LOS) of the user's eye. A system characterized by that.

8. In the system according to any one of claims 2, 4, 5, or 6, the foveal image portion and the parafoveal image portion have an overlap in a boundary region therebetween, so that different projection parameters for projecting the foveal image portion and the parafoveal image portion onto the retina are provided. A system characterized by providing a smooth transition in the boundary region.

9. In the system according to claim 6 or 7, the processor aligns the foveal image portion and the parafoveal image portion and provides the shoulder image data regarding the overlapping portions of the foveal image portion and the parafoveal image portion corresponding to similar image contents. A system characterized by that.

10. In the system according to any one of claims 1 to 9, the image projection device includes at least two image projection units configured to project the at least two image portions. A system characterized by that.

11. In the system according to claim 10, the eye projection optical module is optically connected to at least two of the image projection units, combines the projection optical paths of the image projection units along the optical path, and the light beams from the two image projection units are guided along the optical path toward the user's eye. A system characterized by being configured and operable as such.

12. In the system according to claim 11, the eye projection optical module is Two initial relay modules respectively associated with the first and second image projection units among the at least two image projection units, the two initial relay modules for relaying the light of the at least two image portions A beam combiner configured and operable to combine the projection optical paths of the at least two image portions from the at least first and second image projection units, along the optical path, the light beams of the at least two image portions are directed to propagate towards the user's eye so as to project the optical field of the image frame including the at least two image portions onto the retina. A system comprising:

13. In the system according to claim 11, the foveal image portion of the image frame projected with one projection parameter of at least two of the image projection units is directed to the central region of the image projection, and the parafoveal image portion of the image frame projected with the other projection parameter of at least two of the image projection units is directed to another annular region surrounding the central region of the image projection. A system characterized in that at least two of the image projection units and the eyeball projection optical module are configured and operable.

14. In the system according to any one of claims 2, 4, 5, 6, or 7, the foveal image portion and the parafoveal image portion are: the angular range of the parafoveal image portion is larger than the angular range of the foveal image portion; the foveal image portion has a higher image projection quality than the parafoveal image portion; and the foveal image portion has a higher color depth than the parafoveal image portion. A system characterized by having at least one of the features.

15. In the system according to any one of claims 1 to 14, the system characterized in that the processor has a graphics processing unit (GPU).

Images