Method for projecting image content onto a user's retina, image processing device for implementing the method, and optical system for a virtual retinal display - Patent Application 20070122999

Abstract

The present invention relates to a method for projecting image content onto a user's retina (22a) using an optical system (68a), where first image data (12a) is captured using an image processing device (10a). Then, at least a first subset (17a) of the image data (12a) is blanked, whereby an active second subset (19a) of the image data is generated, whereby the first subset (17a) and the second subset (19a) form a total amount of image data. Then, a projector unit (16a) is controlled using the image data (12a) such that at any time only one of the eyeboxes (A, A1 or B, B1) with the same active image data generated in a common imaging path (28a, 30a) is located within a first pupil area of ​​the user. The first pupil area surrounds the pupil midpoint. At least a portion of the eyebox (A, A1, B, B1) is disposed with at least a portion of the active image data within a second pupil region of the user, the second pupil region surrounding the pupil midpoint, the second pupil region being disposed within the first pupil region.

Description

[Technical Field]

[0001] The present invention relates to a method for projecting image content onto a user's retina using an optical system, and further to an image processing device for implementing the method, and an optical system for a virtual retinal display (retinal scanning display) comprising the image processing device. [Background technology]

[0002] From US Patent Application Publication No. 2016 / 377865 a method is already known for projecting image content onto a user's retina using an optical system. The distance of the pupil to the deflection unit integrated in the spectacle lens can change, for example due to misalignment of the glasses on the nose, so that the pupil is no longer located in the exit pupil plane of the optical system. Outside the exit pupil plane, the beam bundle resulting from the deflection of the light beam to the eye by the deflection unit is significantly divergent, which can cause double images for the user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2016 / 377865 Summary of the Invention [Problem to be solved by the invention]

[0004] Based on this problem, the object of the present invention is to develop a method for projecting image content onto a user's retina, which is able to avoid double images almost independently of the distance of the pupil to the deflection unit. [Means for solving the problem]

[0005] To solve this problem, a method is proposed for projecting image content onto a user's retina using an optical system according to claim 1. Furthermore, an image processing device according to claim 19 and an optical system for a virtual retinal display (retinal scanning display) according to claim 20 are proposed.

[0006] An optical system implementing a method for projecting image content onto a user's retina includes at least one image source delivering the image content in the form of image data. The image data is particularly configured as color image data, e.g., RGB image data. In particular, the image data may be configured as a still image or a moving image, e.g., a video. The optical system further includes a projector unit including a time-modulatable light source for generating at least one light beam and a controllable deflection device for the at least one light beam for scanning projection of the image content. The projector unit is particularly designed to emit the image content from the image data in the form of a scanned and / or rasterized light beam. The deflection device is particularly configured as a MEMS mirror (micromirror actuator) and controls the deflection of at least one light beam from the light source of the projector unit. The optical system further includes a deflection unit capable of projecting the image content and designed to deflect the projected image content toward the user's eye. The deflection unit here particularly includes an arrangement of optical elements, e.g., diffractive, reflective, refractive, and / or holographic optical elements. However, here, the deflection unit preferably always includes at least one holographic optical element. The deflection unit is provided, in particular, to deflect only a portion of the intensity of the projected image content toward the user's eyes. At least a further portion of the intensity of the projected image content passes through the deflection unit. The deflection unit appears essentially transparent to the user, at least from a perpendicular viewing direction. In particular, the deflection unit constitutes a projection area. Furthermore, the optical system includes an optical segmentation element arranged between the projector unit and the deflection unit, the optical segmentation element configured to project the image content onto at least one projection area of ​​the deflection unit via at least two different imaging paths. Here, at least each individual imaging path is individually controllable. In particular, at a first time, the complete image content is selectively projected onto at least one projection area of ​​the deflection unit via one of the at least two different imaging paths.In particular, at a first time point, a first portion of image data is projected onto at least one projection area of ​​the deflection unit via a first path of at least two different imaging paths, and at a second time point after the first time point, a second portion of image data is projected onto at least one projection area of ​​the deflection unit via a second path of the at least two different imaging paths. In particular, at the first time point, simultaneously, the first portion of image data is projected onto at least one projection area of ​​the deflection unit via a first path of the at least two different imaging paths, and the second portion of image data is projected onto at least one projection area of ​​the deflection unit via a second path of the at least two different imaging paths. The optical segmentation element can be configured as a spatially segmented optical element, in particular, which is provided for spatial segmentation of the image data. The optical segmentation element can also be configured as a temporally segmented optical element. This advantageously allows for good spatial resolution of the imaging. The optical system further includes an optical replication component, which is arranged in at least one projection area of ​​the deflection unit and is designed to replicate the projected image content and deflect it toward the user's eyes in a spatially offset manner, thereby generating multiple eyeboxes with spatially offset image content. Such eyeboxes are particularly arranged in the user's pupil plane and are configured as intersections of the light beam bundles deflected by the deflection unit at the pupil plane. A typical pupil plane may deviate from a perfect plane, for example, due to rotational movement of the eyeball. In particular, the pupil plane extends approximately parallel to the surface of the smart glasses' spectacle lens, particularly the surface of the portion of the smart glasses' spectacle lens that reflects the light beam. In particular, the light beam passes through the eyebox, and the light beam as a whole can convey the complete image content. The position of the pupil plane depends on the position of the pupil. Therefore, the pupil plane is particularly located between the deflection unit and the exit pupil plane.The exit pupil plane is particularly configured as a plane of the optical system, preferably smart glasses, essentially parallel to the pupil plane, where the pupil of a user of the optical system is approximately ideally located when the optical system is in use. In this plane, the light beam bundle is optimally focused. If the pupil plane is located at the exit pupil plane, the exit pupil on the exit pupil plane is also the eyebox. The exit pupil as the eyebox represents the optimally focused light beam bundle. Alternatively, the pupil plane can be located behind the exit pupil plane relative to the deflection unit. An optical duplication component is particularly to be understood as a component of an optical system including an optical element that generates spatially displaced optical duplication of the projected image content. In particular, the optical duplication component constitutes at least a part of the deflection unit. In particular, the optical duplication component is provided to duplicate all image content projected via the individual imaging paths of the optical segmentation element. In particular, the optical duplication component is provided to generate a number of eyeboxes corresponding to a multiple (e.g., double or triple) of the number of segmentations performed by the optical segmentation element.

[0007] In a method for projecting image content onto a user's retina using the optical system, image data is first captured using an image processing device. Subsequently, at least a first subset of the image data is blanked, thereby generating a second active subset of the image data. Here, the first and second subsets constitute the entire amount of image data. Blanking the image data specifically refers to deactivating a subset of the image data. Specifically, blanking the image data specifically refers to masking a subset of the image data. Here, pixels representing image data are specifically deactivated or blanked. Specifically, the subset has an amount ranging from 0 to 100% of the entire amount of image data. Then, the projector unit is controlled using the specifically blanked and active subsets of image data generated in a common imaging path so that only one eyebox with the same active image data is always positioned within a first pupil region of the user. Here, the first pupil region surrounds the pupil midpoint. At least a portion of the eyebox is positioned within a second pupil region of the user with at least a portion of the active image data. The second pupil region surrounds the pupil midpoint and is located within the first pupil region. This provides the advantage that the user does not experience double images, regardless of the distance of the pupil relative to the deflection unit within the two pupil regions. Such double images would be caused by simultaneously displaying the same image data in the eyebox of the common imaging path within the first pupil region. Furthermore, it is also ensured that at least a portion of the image data is always located within the smaller second pupil region.

[0008] Preferably, in a further method step, at least a third subset of the image data is blanked, thus generating an active fourth subset of the image data. Here, the third and fourth subsets form the entire amount of image data. The second and fourth subsets are at least partially different from each other. The distinction between the subsets does not particularly imply the number of image data in the subsets. Instead, the individual image data have positions, particularly pixel positions, assigned in the image plane by the image content. The subsets differ, particularly in these positions of the image data. Again, the subsets have an amount within the range of 0 to 100% of the entire amount of image data. In particular, with regard to this further method step, particularly in the first substep, at least one copy, particularly a complete copy, of the image data is generated using an image processing device, and there are at least two complete instances of the image data. In a second substep, partial blanking is performed in at least two complete instances of the image data, with a blanked first subset of the image data occurring in the first instance of the image data and a blanked third subset of the image data occurring in the second instance of the image data. Correspondingly, an active second subset of the image data occurs in the first instance of the image data and an active fourth subset of the image data occurs in the second instance of the image data. Alternatively, the two substeps can be combined. Alternatively, both substeps can be performed by logic operations without further temporary storage of the image data, for example during electronic, preferably digital, transmission of the image data within the image processing device, for example in one pipeline structure or several parallel pipeline structures.

[0009] Preferably, the image data allows the optical segmentation element to project the image content onto at least one projection area of ​​the deflection unit via at least four different imaging paths. In particular, different subsets of the image data allow the optical segmentation element to project at least a portion of each of the image content onto at least one projection area of ​​the deflection unit via at least four different imaging paths. This projection via the different imaging paths can occur simultaneously or, alternatively, sequentially. Thus, the at least four different imaging paths also allow for four different eyeboxes, which can be optionally increased using optical replication components. To this end, preferably, in a further method step, at least a fifth subset of the image data is blanked, thereby generating an active sixth subset of the image data. Here, the fifth and sixth subsets form the entire amount of image data. Furthermore, at least a seventh subset of the image data is blanked, thereby generating an active eighth subset of the image data. Here, the seventh and eighth subsets form the entire amount of image data. The second, fourth, sixth, and eighth subsets are at least partially different from one another. In particular, with respect to these further method steps, two further, particularly complete copies of the image data are generated using an image processing device. To generate the fifth through eighth subsets of image data, a two-step procedure is conceivable. In the first substep, at least two further complete copies of the image data are generated, for example in an image memory, resulting in a total of four instances of image data. In the second substep, partial blanking is performed on the instances of image data, resulting in blanked first, third, fifth, and seventh subsets of image data in the first, second, third, and fourth instances of image data. Thus, active second, fourth, sixth, and eighth subsets of image data are generated in the first, second, third, and fourth instances of image data. Alternatively, the two substeps can be combined.Alternatively, both sub-steps can also be performed without further temporary storage of the image data, for example by logical operations during electronic, preferably digital, transmission of the image data within the image processing device, for example in one pipeline structure or several parallel pipeline structures.

[0010] Preferably, all active subsets of image data together form at least the complete image content, thereby ensuring that the complete image content is shown to the user. Preferably, all active subsets of image data together form more than the complete image content, thereby making the overall method more robust and ensuring visibility of the complete image content even when the pupil position is not precisely known. The active subsets of image data preferably partially overlap, thereby multiplexing parts of the image content to the user in different eyeboxes. Preferably, the brightness of the active subsets of image data within the overlapping region of the active subsets of image data is adapted so that the brightness distribution of the projected image content is essentially uniform. In this regard, in particular, the brightness of the image data, in particular all active subsets of pixels, within the overlapping region is dimmed by a first dimming factor of the same magnitude. Alternatively, in particular, only one of the active subsets of image data within the overlapping region is dimmed by a second dimming factor. In particular, the dimming factor is generated using a brightness gradient. In particular, a luminance gradient is set such that within the overlap region, the luminance of the active subset of image data decreases as the distance to the blanked subset of image data decreases. In particular, information regarding whether a pixel belongs to the active or blanked subset of image data, along with per-pixel information regarding dimming factors in the overlap region, can be represented in a common luminance mask, in particular an alpha channel that is added to the red, green, and blue channels of the image information.

[0011] Preferably, the first pupil region has the largest possible pupil diameter of the user, wherein the largest possible pupil diameter represents the largest possible user pupil diameter, in particular adjusted by eye adaptation at the lower end of the ambient luminance range underlying the application.

[0012] Preferably, the second pupil region has the smallest possible pupil diameter of the user, wherein the smallest possible pupil diameter represents the smallest possible user pupil diameter, in particular as adjusted by eye adaptation at the upper end of the ambient luminance range underlying the application.

[0013] Preferably, at least one blanked subset of image data is selected, particularly using the image processing device, so that at least one generated active subset of image data includes at least image data whose image content in the plurality of eyeboxes spatially offset from one another is closest to the pupil midpoint, particularly in the pupil plane. In other words, at least one generated active subset of image data includes at least image data whose image content in the plurality of eyeboxes spatially offset from one another has the shortest distance to the pupil midpoint, particularly in the pupil plane. Thus, it is ensured that the generated active subset of image data is within the first pupil region and essentially also within the second pupil region.

[0014] Preferably, at least two of the blanked subsets are essentially the same size depending on the user's pupil position relative to the generated eyebox, particularly the pupil midpoint position. This means that there is at least one pupil position where at least two of the blanked subsets are essentially the same size. In this regard, it is contemplated that, depending on the pupil position, the second, fourth, sixth, and eighth subsets each represent one-quarter of the total amount of image data, preferably in conjunction with at least four different imaging paths. In other words, the image content is divided into four equal parts at the image plane. For the two different imaging paths, each active subset of image data represents one-half of the total amount of image data. This arrangement of equal-sized subsets occurs particularly when at least equal portions of the eyeboxes generated by the different imaging paths are located within the second pupil region. Alternatively, the blanked subsets of image data are preferably configured to be different sizes. This arrangement occurs particularly when differently sized portions of the eyeboxes produced by the different imaging paths lie within the second pupil region.

[0015] Preferably, the optical segmentation element and the optical replication component are used to generate a plurality of eyeboxes arranged essentially in a raster pattern. "Raster" particularly refers to a regular pattern distributed on a surface. In particular, the eyeboxes are arranged so that they do not overlap one another. In particular, various geometric arrangement patterns for the arrangement of the eyeboxes (eyebox pattern) in the pupil plane of the optical system are conceivable. Preferably, the eyeboxes are arranged essentially in a rectangular shape. In particular, the eyeboxes are arranged essentially in a square shape. In particular, as an alternative to this, the eyeboxes are arranged essentially in a parallelogram shape. In particular, the eyeboxes are arranged essentially in a rhombic or oblique shape. In particular, the eyeboxes are arranged essentially in a rhombic shape, with two interior angles essentially having an angle of 60°.

[0016] Preferably, in a further method step, the image data is predistorted using an image processing device so that distortions of the image content are at least partially compensated for via at least two imaging paths. This advantageously achieves a particularly large, effective overall eyebox, which in particular simultaneously has as large a field of view as possible and advantageously does not produce double images. Preferably, the predistortion and blanking of the image data are performed simultaneously. In particular, as an alternative to this, the predistortion and blanking of the image data are performed sequentially in time. In this connection, in particular, the blanking of the image data is performed temporally before the predistortion of the image data. In particular, the image processing device is further configured to rotate, offset, scale, etc. the image data from the image source. In particular, the image processing device is configured to assign position corrections, in particular individual position corrections for the red, green, and blue channels, to each individual pixel of the image data according to a mathematical rule, for example, a lookup table or an equation, in particular a polynomial, which can take calibration data into account.

[0017] Preferably, an eye tracker is used to further capture the user's pupil position relative to the generated eyebox, and in particular, the eye tracker is used to capture the position of the pupil midpoint relative to the generated eyebox, so that the user's changing pupil position and the resulting necessary changes in the blanked and active subsets of image data can be automatically taken into account in this manner.

[0018] A further subject of the present invention is an image processing device configured to implement the aforementioned method for projecting image content onto a user's retina. In this regard, the image processing device is configured to capture image data and blank at least a first subset of the image data. An active second subset of the image data is thus generated, the first subset and the second subset forming a total amount of image data. Furthermore, the image processing device is configured to control the projector unit using the blanked and active subsets of image data generated by the common imaging path so that only one eyebox with the same active image data is always located within a first pupil region of the user. Here, the first pupil region surrounds the pupil midpoint. Furthermore, at least a portion of the eyebox is located within a second pupil region of the user with at least a portion of the active image data. Here, the second pupil region surrounds the pupil midpoint. The second pupil region is located within the first pupil region.

[0019] A further subject of the present invention is an optical system for a virtual retinal display (retinal scanning display) having the aforementioned image source delivering image content in the form of image data. The optical system further comprises the aforementioned image processing device configured to also implement the aforementioned method for projecting image content onto a user's retina. The optical system further comprises the aforementioned projector unit having a time-modulatable light source for generating at least one light beam and a controllable deflection device for the at least one light beam for scanning projection of the image content. The optical system further comprises the aforementioned deflection unit capable of projecting the image content and designed to deflect the projected image content toward the user's eye. The optical system further comprises the aforementioned optical segmentation element arranged between the projector unit and the deflection unit, the optical segmentation element configured to project the image content onto at least one projection area of ​​the deflection unit via at least two different imaging paths, at least each imaging path being individually controllable. Furthermore, the optical system comprises an optical replication component, also as described above, arranged in at least one projection area of ​​the deflection unit and designed to replicate and spatially offset the projected image content towards the user's eyes, thereby generating a plurality of eyeboxes with image content arranged spatially offset from one another.

[0020] Preferably, the optical system includes smart glasses having a spectacle frame and spectacle lenses, wherein the at least one projector unit and the at least one optical segmentation element are arranged in the spectacle frame, and the at least one deflection unit is arranged in at least one region of the spectacle lenses together with the at least one optical replication component, and in particular, the deflection unit with the at least one optical replication component is integrated into the at least one spectacle lens. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 illustrates an optical system for a virtual retinal display (retinal scanning display). [Figure 2] FIG. 1 illustrates an optical system including smart glasses. [Figure 3] FIG. 1 illustrates a method for projecting image content onto a user's retina using an optical system. [Figure 4a] FIG. 10 illustrates various distances from the pupil to the deflection unit. [Figure 4b] FIG. 1 shows the eyebox at the pupil plane when the eye is at the exit pupil plane of the optical system. [Figure 4c] FIG. 1 illustrates the eyebox at the pupil plane when the eye is in front of or behind the exit pupil plane of the optical system. [Figure 5a] FIG. 5a shows one position of the pupil midpoint and the resulting various active portions of the square arrangement of the eyebox when the eye is in front of the exit pupil plane of the optical system. [Figure 5b] FIG. 5b shows a relevant subset of the image data. [Figure 6a] FIG. 6a shows one position of the pupil midpoint and the resulting various active portions of the square arrangement of the eyebox when the eye is in front of the exit pupil plane of the optical system. [Figure 6b] FIG. 6b shows a relevant subset of the image data. [Figure 7a] FIG. 7a shows one position of the pupil midpoint and the resulting various active portions of the square arrangement of the eyebox when the eye is in front of the exit pupil plane of the optical system. [Figure 7b] FIG. 7b shows a relevant subset of the image data. [Figure 8a] FIG. 8a shows one position of the pupil midpoint and the resulting various active portions of the diamond-shaped arrangement of the eyebox when the eye is in front of the exit pupil plane of the optical system. [Figure 8b]FIG. 8b shows a relevant subset of the image data. [Figure 9a] FIG. 9a shows one position of the pupil midpoint and the resulting various active portions of the diamond-shaped arrangement of the eyebox when the eye is in front of the exit pupil plane of the optical system. [Figure 9b] FIG. 9b shows a relevant subset of the image data. [Figure 10a] FIG. 10a shows one position of the pupil midpoint and the resulting various active portions of the diamond-shaped arrangement of the eyebox when the eye is in front of the exit pupil plane of the optical system. [Figure 10b] FIG. 10b shows a relevant subset of the image data. [Figure 11] FIG. 10 is a diagram illustrating predistortion of image data. DETAILED DESCRIPTION OF THE INVENTION

[0022] FIG. 1 shows a schematic diagram of an optical system 68a. The optical system 68a includes an image source (not shown here). The image source delivers image content in the form of image data 12a. The image source is, in particular, an integral part of the smart glasses 66a shown in FIG. 2. Alternatively, the image source can be configured as or as part of an external device 146a. The optical system 68a includes an image processing device 10a. The image data 12a can form, for example, a still image or a video feed. Furthermore, the optical system 68a includes an image processing device 10a configured to capture the image data 12a and blank at least a first subset 17a of the image data, thereby generating an active second subset 19a of the image data 14a. The first subset and the second subset form the total amount of image data. Furthermore, the image processing device is configured to control the projector unit 16a of the optical system 68a using the specific blanked and active subsets of image data so that only one of the eyeboxes A, A' or B, B', which have the same active subset 19a of image data 14a generated by the common imaging path 28a or 30a, is located within a first pupil region (not shown) of the user at any one time. The first pupil region surrounds the pupil midpoint (not shown), and at least a portion of the eyeboxes A, A', B, B' are located with at least a portion of the active image data within a second pupil region (not shown) of the user. Here, the second pupil region surrounds the pupil midpoint and is located within the first pupil region. The image processing device 10a can be partially integrated with the computing unit 78a. 1, the image processing device 10a generates four copies 14a of the image data, each containing the complete image content, and the copied image data is then partially blanked by the image processing device.In this case, image processing device 10a is designed to generate a matrix-like arrangement of copied and blanked subsets of image data and output them to projector unit 16a of optical system 68a.

[0023] The optical system 68a includes a projector unit 16a. The projector unit 16a receives the copied image data 14a from the image processing device 10a. The projector unit 16a is configured as a laser projector unit. The projector unit 16a is designed to emit the image data 14a in the form of a light beam 18a. The light beam 18a is configured as a scanning laser beam. The scanning laser beam generates an image of all the images 98a, 100a of the copied image data 14a each time it passes through the scanning area of ​​the projector unit 16a. The projector unit 16a includes a projector control unit 80a. The projector unit 16a includes a time-modulatable light source 132a. The time-modulatable light source 132a is designed to generate the light beam 18a. The projector control unit 80a is provided to control or adjust the generation and / or modulation of the light beam 18a by the light source 132a. In the illustrated exemplary embodiment, the light source 132a includes three (amplitude-modulatable) laser diodes 82a, 84a, and 86a. The first laser diode 82a generates a red laser beam. The second laser diode 84a generates a green laser beam. The third laser diode 86a generates a blue laser beam. The projector unit 16a includes a beam combining and / or beam shaping unit 88a. The beam combining and / or beam shaping unit 88a is designed to combine, in particular mix, the different color laser beams from the laser diodes 82a, 84a, and 86a to generate a color image. The beam combining and / or beam shaping unit 88a is designed to shape the light beam 18a, in particular a laser beam, emerging from the projector unit 16a. Details regarding the configuration of the beam combining and / or beam shaping unit 88a are assumed to be known from the prior art. The projector unit 16a includes a beam divergence adaptation unit 90a.The beam divergence adapting unit 90a is provided to adapt the beam divergence of the light beam 18a, particularly the laser beam, exiting the projector unit 16a, preferably to a path length of each currently emitted light beam 18a, which depends in particular on the arrangement of the optical elements of the optical system 68a. The beam divergence of the light beam 18a, particularly the laser beam, exiting the projector unit 16a is preferably adapted so that, after passing through the optical elements of the optical system 68a, a sufficiently small and sharp laser spot is generated at the position where the beam strikes the retina 22a of the user's eye 24a in the virtual retinal display, and further so that the beam divergence is at least essentially constant at the pupil plane 54a of the optical system 68a in front of the user's eye 24a throughout the entire imaging of the image data 12a generated by the light beam 18a, particularly the laser beam. Details regarding the configuration of the beam divergence adapting unit 90a, for example using lenses with fixed and / or variable focal lengths, are assumed to be known from the prior art. The projector unit 16a includes at least one controllable deflection device 92a. The controllable deflection device 92a is configured as a MEMS mirror. The MEMS mirror is part of a micromirror actuator (not shown). The controllable deflection device 92a is designed to deflect a laser beam under control to generate a raster image. Details regarding the configuration of micromirror actuators are assumed to be known from the prior art. The projector control unit 80a is designed to control or adjust the movement of the controllable deflection device 92a (see arrow 94a). The controllable deflection device 92a periodically sends its current position signal back to the projector control unit 80a (see arrow 96a).

[0024] The optical system 68a includes a deflection unit 20a. Image content can be projected onto the deflection unit 20a. The deflection unit 20a is designed to deflect the projected image content toward the user's eye 24a. The deflection unit 20a forms a projection area 34a. A light beam 18a that strikes the deflection unit 20a within the projection area 34a is at least partially deflected / projected toward the user's eye 24a. The deflection unit 20a is designed to influence (refract, scatter, and / or reflect) the light beam 18a so that at least a portion of the light beam 18a, preferably at least one image 98a, 100a generated from the image data 12a, is imaged onto the pupil plane 54a of the optical system 68a, particularly onto the retina 22a of the user's eye 24a. The optical system 68a is designed to form multiple eyeboxes A, A', B, B' using various optical elements. The optical system 68a is designed to influence the light beam 18a using various optical elements so that the generated eyeboxes A, A', B, and B' are generated and spaced apart from one another. The optical system 68a forms a pupil plane 54a. The eyeboxes A, A', B, and B' are all located to the left and right and / or above and below one another within the pupil plane 54a. In the illustrated case, the pupil plane 54a is located at the position of the exit pupil plane, which is formed as a plane in space provided with respect to the optimal position of the user's eye 24a (within the smart glasses 66a), particularly with respect to the position of the entrance pupil of the user's eye 24a (within the smart glasses 66a). The pupil plane 54a is preferably flat, but deviates from a perfect plane by a small curvature. The pupil plane 54a can be approximately considered / referred to as the pupil plane. The pupil plane 54a is located in front of the spectacle lenses 70a, 72a of the smart glasses 66a in the user's line of sight and extends at least essentially parallel to the lens surfaces of the spectacle lenses 70a, 72a, where it should be particularly understood that the term "essentially parallel" also includes deviations of up to 20° from a perfect plane (keywords: curvature angle and forward tilt angle of the spectacle lenses 70a, 72a).

[0025] The optical system 68a shown by way of example in FIG. 1 is designed to generate a spatial image segmentation of the copied and blanked image data 14a. In the spatial image segmentation, the image data 14a is divided into spatially separated (and possibly modulated) images of the image content / image data 12a, where each segment contains exactly one (in this case, partial) image of the image content / image data 12a. The optical system 68a includes at least one optical segmentation element 32a to generate the spatial segmentation of the copied and blanked subsets of the image data 14a. The optical segmentation element 32a is disposed between the projector unit 16a, specifically the deflection device 92a of the projector unit 16a, and the deflection unit 20a. Using the optical segmentation element 32a, the image content can be projected onto at least one projection area 34a of the deflection unit 20a via different imaging paths 28a, 30a. In the exemplary embodiment of FIG. 1, the optical segmentation element 32a is configured as a segmented lens, in particular a segmentation lens. Alternatively, the optical segmentation element 32a can be configured as a segmentation mirror (not shown), a segmentation optical grating (not shown), a volume hologram (not shown), or a beam splitter (not shown). The optical segmentation element 32a includes several individual segments 36a, 38a, in particular individual lenses. Each of the individual segments 36a, 38a projects a respective one of the images 98a, 100a. This results in a unique virtual deflection device (virtual MEMS mirror) 102a, 104a for each image 98a, 100a, which is arranged separately from the further virtual deflection devices (virtual MEMS mirrors) 102a, 104a and the real deflection device 92a. In particular, the virtual deflection devices (virtual MEMS mirrors) 102a and 104a can be configured as point sources (in theory), but in general, the virtual deflection devices (virtual MEMS mirrors) 102a and 104a do not constitute point sources, but rather non-point sources.Thereby, each image 98a, 100a is illuminated onto the projection area 34a of the deflection unit 20a via a different imaging path 28a, 30a, in particular from a different angle and a different distance.

[0026] The optical system 68a shown in FIG. 1 is designed to generate image replicas solely through its optical elements. The optical system 68a includes an optical replica component 150a. The optical replica component 150a is located in the projection area 34a of the deflection unit 20a. The optical replica component 150a is designed to replicate and spatially offset the projected image content toward the user's eyes 24a, generating multiple eyeboxes A, A', B, B' with the image content, spatially offset from one another. The optical replica component 150a is at least partially reflective and at least partially transmissive to generate the image replicas. The optical replica component 150a includes partially reflective and partially transmissive layers 106a and 108a. The layers 106a and 108a of the optical replica component 150a have different optical functions, particularly different deflection angles. The layers 106a, 108a of the optical replication component 150a are configured as deflecting and / or focusing holographic optical elements (HOEs). The entire eyeboxes A, A', B, B' are generated by a combination of image segmentation by the optical segmentation element 32a and image replication by the optical replication component 150a. The optical replication component 150a is incorporated into one of the eyeglass lenses 72a of the smart glasses 66a. The optical replication component 150a is positioned within the field of view of the smart glasses 66a.

[0027] In the exemplary embodiment shown in FIG. 1, the optical replication component 150a is realized as a layer structure having two holographic functionalized layers 106a, 108a. The optical replication component 150a includes two holographic functionalized layers 106a, 108a that completely overlap in the laterally direction, and these layers are arranged in a layered manner. Here, the layers 106a, 108a are arranged planarly and continuously. The optical replication component 150a is realized as a layer structure having at least two layers 106a, 108a arranged one above the other with different holographic functions, thereby generating multiple eyeboxes A, A', B, B' that are spatially offset from each other. Here, a portion of each light beam 18a is deflected by the first layer 106a, and the remainder of the light beam 18a passes through the first layer 106a. A further portion of the component of the light beam 18a that passes through the first layer 106a is deflected by the second layer 108a, and the remainder of the light beam 18a passes through the second layer 108a and the eyeglass lens 72a incorporating the optical replication component 150a.

[0028] The optical system 68a includes an eye tracker 62a. The eye tracker 62a is integrated into one of the eyeglass temples 74a, 76a (see FIG. 2). Alternative configurations of the eye tracker 62a are contemplated. The eye tracker 62a is configured to determine the position of the user's pupil, in particular the position of the pupil midpoint, relative to the generated eyeboxes A, A', B, B'.

[0029] The image processing device 10a is configured to adapt at least a first blanked subset 17a of the image data 14a in response to the pupil position captured using the eye tracker 62a, so that at any one time only one of the eye boxes A, A' or B, B' with identical active image data 19a generated in the common imaging path 28a or 30a is positioned within the user's first pupil region, and at least some of the eye boxes A, A', B, B' are positioned with at least some of the active image data 19a within the user's second pupil region.

[0030] The optical system 68a includes an electronic control or adjustment unit 26a. The control or adjustment unit 26a may be partially integrated with the computing unit 78a. The control or adjustment unit 26a shown in FIG. 1 is provided for controlling the image processing device 10a. The control or adjustment unit 26a is designed to control the image processing device 10a based on measurement data from the eye tracker 62a. The control or adjustment unit 26a receives measurement data regarding pupil position from the eye tracker 62a (see arrow 110a). Based on the data from the eye tracker 62a, the control or adjustment unit 26a generates control or adjustment commands for controlling the image processing device 10a. For example, these commands may be provided to increase or decrease the blanked subset 17a of the image data 14a.

[0031] FIG. 2 shows a schematic diagram of an optical system 68a including a pair of smart glasses 66a. The smart glasses 66a include spectacle lenses 70a and 72a. The spectacle lenses 70a and 72a are primarily transparent. The smart glasses 66a include a spectacle frame 144a having spectacle temples 74a and 76a. The smart glasses 66a constitute a part of the optical system 68a. In the case shown in FIG. 2, the optical system 68a includes an external device 146a. The external device 146a is configured as, for example, a smartphone. The external device 146a establishes a data communication connection 148a with the smart glasses 66a. Alternatively, the smart glasses 66a may completely configure the optical system 68a. The optical system 68a is provided for configuring a virtual retinal display. In the example shown in FIG. 2, the smart glasses 66a include a computing unit 78a. The computing unit 78a is integrated into one of the spectacle temples 74a and 76a. Alternative arrangements of the computing unit 78a in the smart glasses 66a, for example in the eyeglass lens edges, are also conceivable. By "computing unit 78a" is meant in particular a processor, a memory unit, and / or a controller with operating, control, and / or calculation programs stored in the memory unit. The computing unit 78a is provided for operating the smart glasses 66a, in particular the individual components of the smart glasses 66a.

[0032] In the illustration of Figure 2, for example, the projector unit 16a and the optical segmentation element 32a are arranged in the eyeglass frame 144a, and the deflection unit 20a with the replication component 150a is arranged within the eyeglass lens 72a, in particular is integrated into at least the eyeglass lens 72a, but it is also possible that, alternatively, at least the image source is arranged in the external device 146a together with the image processing device 10a, and the image data 14a is transmitted from the external device 146a to the projector unit 16a of the smart glasses 66a.

[0033] FIG. 3 illustrates in flowchart form a method for projecting image content onto a user's retina using an optical system. The optical system here is configured, inter alia, according to FIG. 1 above. In this method, in method step 200, image data is captured using an image processing device. In subsequent method step 205, at least a first subset of the image data is blanked, thereby generating an active second subset of the image data. Here, the first and second subsets form a total amount of image data. In subsequent method step 245, the projector unit is controlled so that at any one time, only one eyebox with identical active image data, generated in a common imaging path using the particular blanked and active subsets of image data, is positioned within the first pupil region of the user. Here, the first pupil region surrounds the pupil midpoint. In particular, the first pupil region has the largest possible pupil diameter of the user. At least a portion of the eyebox is positioned with at least a portion of the active image data within the second pupil region of the user. Here, the second pupil region surrounds the pupil midpoint. In particular, the second pupil region has the smallest possible pupil diameter of the user. The second pupil region is located within the first pupil region. Then, the method ends.

[0034] Method step 205 is followed by optional method step 210, in which at least a third subset of the image data is blanked, thus generating an active fourth subset of the image data, where the third and fourth subsets together make up the entire amount of image data, and the second and fourth subsets are at least partially different from each other.

[0035] Further optionally, the image data, in particular different subsets of the image data, can be projected using the optical segmentation element, in particular each at least a portion of the image content, onto at least one projection area of ​​the deflection unit via at least four different imaging paths. In optional method step 215 following method step 210, at least a fifth subset of the image data is blanked, thus generating an active sixth subset of the image data. The fifth and sixth subsets form the entire amount of image data. Furthermore, in this connection, in the following method step 220, at least a seventh subset of the image data is blanked, thus generating an active eighth subset of the image data. The seventh and eighth subsets form the entire amount of image data. The second, fourth, sixth, and eighth subsets are at least partially different from each other. Optionally, the second, fourth, sixth, and eighth subsets each correspond, in particular, to one-fourth of the entire amount of original image data.

[0036] Further optionally, all active subsets of image data are at least complete image content. Further optionally, the active subsets of image data at least partially overlap. In this regard, method step 220 is followed by optional method step 225, in which the brightness of the active image data within the overlapping regions of the active subsets of image data is adapted so that the distribution of brightness of the projected image content is essentially uniform.

[0037] Further optionally, at least one blanked subset of image data is selected so that at least one active subset of the generated image data includes at least image data having image content that is closest to the pupil midpoint, particularly at the pupil plane, in a plurality of eyeboxes that are spatially offset from one another.

[0038] Further optionally, the blanked subsets of image data are essentially the same size. Alternatively, the blanked subsets of image data are different sizes.

[0039] Optionally, the optical segmentation element and the optical replication component are used to generate a plurality of eyeboxes arranged essentially in a raster. The eyeboxes, particularly each of four eyeboxes, are arranged essentially in a rectangular shape, particularly in a square shape. Alternatively, the eyeboxes, particularly each of four eyeboxes, are arranged essentially in a parallelogram shape, particularly in a diamond shape. In particular, the eyeboxes are arranged essentially in a diamond shape, with two interior angles essentially at 60°.

[0040] Method step 225 is followed in a further optional method step 230 in which the image data is pre-distorted using an image processing device such that distortions in the image content are at least partially compensated for via at least two imaging paths. In particular, the pre-distortion and blanking of the image data are performed simultaneously. Alternatively, the pre-distortion and blanking of the image data are performed sequentially in time.

[0041] Method step 230 is followed in a further optional method step 235 in which the position of the user's pupil, in particular the position of the pupil midpoint, relative to the further generated eyebox is captured using an eye tracker. In a subsequent optional method step 240, blanking of at least a first subset of the image data is performed depending on the captured pupil midpoint.

[0042] 4a shows a schematic diagram of the influence of the distance of the pupil planes 252 and 256 relative to the deflection unit (not shown here) on the diameters 253 and 255 of the light bundles belonging to the eyebox D'. Here, the exit pupil plane as the pupil plane 256 is at the optimum distance, since at this distance the light beam deflected by the deflection unit is focused to the smallest possible diameter 255 of the exit pupil as the eyebox D'. FIG. 4b shows a schematic diagram of the exit pupils as the eyeboxes A and D' that result at the exit pupil plane as the pupil plane 256 for four different imaging paths and three replications. In this embodiment, the eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''', respectively, are arranged in a rectangular raster. At this distance of the pupil plane 256 relative to the deflection unit, only one of the eyeboxes A, A', A'', A''', or B, B', B'', B''', or C, C', C'', C''', or D, D', D'', D''', with identical active image data generated by a common imaging path, is located within the user's first pupil region 261, where the first pupil region 261 surrounds the pupil midpoint 264. Furthermore, at least a portion of the eyeboxes A''', B'', C', and D with at least a portion of their active image data is located within the user's second pupil region 262, which surrounds the pupil midpoint 399. The second pupil region 262 is located within the first pupil region 261, where the first pupil region 261 represents the user's largest possible pupil diameter. The second pupil region 262 represents the smallest possible pupil diameter of the user, so that the user does not experience double images and the image is displayed to the user even with the smallest possible pupil diameter.

[0043] In contrast, in FIG. 4c, the diameter of the light beam at pupil plane 252 is increased. In this embodiment, eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''' are also arranged in a rectangular raster. In this case, it can be seen that parts of eyeboxes A'' and A''', B and B'', C and C', and D and D'', which belong to the common imaging path, are located within first pupil region 261. Therefore, at this point, the user will see double images, which is undesirable.

[0044] 5-10 illustrate a solution to the problem by way of example. FIG. 5a shows a rectangular arrangement of eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', C''', and D''', similar to the arrangement of the eyeboxes in FIG. 4c. The pupil midpoint is located in the center, specifically the center, of eyeboxes A''', B'', C', and D in FIG. 5a. In contrast to the projected image data in FIG. 4c, in this case, parts of eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''', are deactivated to prevent double images for the user. To this end, in this exemplary embodiment, a first subset 281 of image data 330 is blanked, thereby generating a second subset 280 of active image data. The first subset 281 and the second subset 280 make up the entire amount of image data 330. Furthermore, the third subset 285 of image data is blanked, thereby generating an active fourth subset 285 of image data. The third subset 285 and the fourth subset 284 make up the entire amount of image data 330. Furthermore, the fifth subset 282 of image data 330 is blanked, thereby generating an active sixth subset 283 of image data. The fifth subset 282 and the sixth subset 283 make up the entire amount of image data. Furthermore, the seventh subset 287 of image data is blanked, thereby generating an active eighth subset 286 of image data. The seventh subset 287 and the eighth subset 286 make up the entire amount of image data 330. The second subset 280, the fourth subset 284, the sixth subset 283, and the eighth subset 286 are distinct from one another. This distinction does not refer to the number of image data in each subset, but rather to the location, specifically the pixel location, of each image data on the image plane as assigned by the image content. In this exemplary embodiment, a total of three, specifically complete copies, of the image data 330 are generated to generate the distinct subsets.In this exemplary embodiment, blanked subsets 281, 282, 285, and 287 of image data are essentially the same size, each representing one-quarter of the total amount of image data 330. All active image data or active subsets 280, 283, 284, and 285 represent the complete image content. The different subsets 280, 281, 282, 283, 284, 285, 286, and 287 of image data 330 allow the optical segmentation element to project each portion of the image content via four different imaging paths onto at least one projection area of ​​the deflection unit. Thus, eyeboxes A, A', A'', and A''' have subsets 280 and 281, eyeboxes B, B', B'', and B''', subsets 284 and 285, eyeboxes C, C', C'', and C''', subsets 282 and 282, and eyeboxes D, D', D'', and D''', subsets 286 and 286. Correspondingly, eyeboxes A, A', A'', and A''', have active sub-regions 270 of image data and blanked sub-regions 271 of image data. Correspondingly, eyeboxes B, B', B'', and B''', have active sub-regions 272 of image data and blanked sub-regions 273 of image data. Correspondingly, eyeboxes C, C', C'', and C''', have active sub-regions 274 of image data and blanked sub-regions 275 of image data. Correspondingly, eyeboxes D, D', D'', and D''' have active sub-regions 277 of image data and blanked sub-regions 276 of image data. Selection of subsets 280, 281, 282, 283, 284, 285, 286, and 287 allows only one of eyeboxes A, A', A'', A''', or B, B', B'', B''', or C, C', C'', C''', or D, D', D'', D''', with identical active image data generated by a common imaging path to be located within first pupil region 261 at any one time. Thus, double images to the user are prevented.Additionally, eyeboxes A''', B'', C', and D having subregions 270, 272, 274, and 277 are located within second pupil region 262. Blanked subsets 281, 282, 285, and 287 are selected such that subsets 280, 283, 284, and 286 contain image data having image content that is closest to pupil midpoint 399 in multiple eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''', that are spatially offset from one another.

[0045] 6a and 6b show different positions of the pupil midpoint 399 for the generated eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''', compared to FIGS. 5a and 5b. In this case, the first subset 291 of image data 330 is blanked such that the second active subset 290 of image data comprises half of the image data 330, specifically the top half. Furthermore, the third blanked subset 294 of image data 330 and the seventh blanked subset 295 of image data 330 each correspond to the entire amount of image data 330. Furthermore, the fifth subset 292 of image data 330 is blanked such that the sixth active subset 291 of image data comprises half of the image data 330, specifically the bottom half. Thus, only subregions 305 and 309 of eyeboxes A, A', A'', and A''', and C, C', C'', and C''', have active image data. Subregions 306 and 308 of eyeboxes A, A', A'', and A''', and C, C', C'', and C''', are deactivated. Furthermore, eyeboxes B, B', B'', and B''', and D, D', D'', and D''', are completely deactivated.

[0046] 7a and 7b show different positions of the pupil midpoint 399 for the generated eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''', which differ from the previous figures. In this case, the pupil midpoint 399 is located centrally, specifically centrally, within eyebox C'. In this case, the sixth subset 301 corresponds to the entire amount of image data. The first subset 300, the third subset 302, and the seventh subset 303 also correspond to the entire amount of image data. Thus, only eyeboxes C, C', C'', and C''' are fully activated. In contrast, eyeboxes A, A', A'', A''', B, B', B'', B''', D, D', D'', and D''', are fully deactivated.

[0047] 8-10 illustrate a parallelogram, specifically a diamond, arrangement of the eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D'', which differs from FIGS. 5-7. In accordance with the arrangement of the pupil midpoint 399 for the eyeboxes A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D'', in FIG. 8a, the first subset 381 of image data 330 in FIG. 8b is blanked such that the generated active second subset 281 of image data has half the total amount of image data, specifically the left half. Furthermore, the third subset 385 of image data 330 is blanked such that the generated active fourth subset 386 of image data has one-quarter the total amount of image data 330, specifically the top right. Similarly, seventh subset 383 of image data 330 is blanked so that the resulting active eighth subset 384 of image data comprises one-quarter of the total amount of image data 330, specifically the bottom right. Fifth subset 382 of image data comprises the entire amount of image data 330. Thus, only subregions 350, 353, and 356 of eyeboxes A, A', A'', A''', B, B', B'', B''', D, D', D'', and D''', are activated. Conversely, further subregions 351, 352, and 355 of eyeboxes A, A', A'', A''', B, B', B'', B''', D, D', D'', and D''', as well as complete eyeboxes C, C', C'', and C''', are deactivated.

[0048] 9a and 9b show different positions of the pupil midpoint 399 for the generated exit pupils A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''', which differ from the previous figures. In this case, the pupil midpoint 399 has the same shortest distance to the eyeboxes B'' and D. In this case, both the first subset 387 and the fifth subset 388 correspond to the entire amount of image data 330. The third subset 389 of image data 330 is selected such that an active fourth subset 390 of image data corresponds to half of the image data 330, specifically the upper half. The seventh subset 391 of image data 330 is selected such that an active eighth subset 392 of image data corresponds to half of the image data 330, specifically the lower half. Thus, only sub-regions 361 and 364 of eyeboxes B, B', B'', B''', D, D', D'', and D''' are activated. Conversely, further sub-regions 360 and 362 of eyeboxes B, B', B'', B''', D, D', D'', and D''' and the complete eyeboxes A, A', A'', A''', C, C', C'', and C''' are deactivated.

[0049] 10a and 10b show different positions of the pupil midpoint 399 for the generated exit pupils A, B, C, D, A', B', C', D', A'', B'', C'', D'', A''', B''', C''', and D''', which differ from the previous figures. In this case, the pupil midpoint 399 is located centrally, specifically centrally, within the eyebox A'''. In this case, the second subset 393 corresponds to the entire amount of image data. The third subset 394, the fifth subset 395, and the seventh subset 396 also correspond to the entire amount of image data. Therefore, only the eyeboxes A, A', A'', and A''' are fully activated. Meanwhile, the eyeboxes B, B', B'', B''', C, C', C'', C''', D, D', D'', and D''' are fully deactivated.

[0050] 11 shows an optional method step for predistorting image data using an image processing device, where distortions in the image content are at least partially compensated for via four different imaging paths. Image data 400 captured using an image processing device (not shown) is then completely copied and predistorted three times. The predistorted image data 401a, 401b, 401c, and 401d are then used to control a projector unit (not shown) and then imaged onto the pupil plane. An undistorted image 402 is shown to the user.

Claims

1. 1. A method for projecting image content onto a user's retina using an optical system (68a), the optical system comprising: an image source providing image content in the form of image data (12a, 330, 400); an image processing device (10a) for said image data (12a, 330, 400); a projector unit (16a) comprising a time-modulatable light source (132a) for generating at least one light beam (18a) and a controllable deflection device (92a) for said at least one light beam (18a) for scanning projection of said image content; a deflection unit (20a) capable of projecting said image content and designed to direct said projected image content towards a user's eye (24a); an optical segmentation element (32a) arranged between the projector unit (16a) and the deflection unit (20a), the optical segmentation element (32a) being designed to project the image content onto at least one projection area (34a) of the deflection unit (20a) via at least two different imaging paths (28a, 30a), at least some of the imaging paths (28a, 30a) being individually controllable; an optical replication component (150a) arranged in the at least one projection area (34a) of the deflection unit (20a) and designed to replicate and spatially offset the projected image content towards the user's eyes (24a), whereby a plurality of eyeboxes (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''') with the image content, spatially offset from one another, are generated; The method comprises at least capturing (200) the image data (12a, 330, 400) using the image processing device (10a); a method step of blanking (205) at least a first subset (17a, 281, 291, 300, 381, 387) of said image data (12a, 330, 400), thereby generating an active second subset (19a, 280, 290, 380, 393) of said image data (12a, 300, 4000), said first subset (17a, 281, 291, 300, 381, 387) and said second subset (19a, 280, 290, 380, 393) comprising the entire amount of said image data (12a, 330); controlling (245) the projector unit (16a) using the blanked image data and the active image data so that only one eyebox (A, A', A'', A''' or B, B', B'', B''' or C, C', C'', C''' or D, D', D'', D''') with the same active image data generated by the common imaging path (28a, 30a) is always located within a first pupil region (261) of the user, a pupil region (261) surrounding a pupil midpoint (399), and at least a portion of the eyebox (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''') being located with at least a portion of the active image data within a second pupil region (262) of the user, the second pupil region (262) surrounding the pupil midpoint (399), and the second pupil region (262) being located within the first pupil region (261); A method comprising:

2. 2. The method of claim 1, further comprising blanking at least one third subset of the image data, thereby generating an active fourth subset of the image data, wherein the third subset and the fourth subset represent a total amount of the image data, and the second subset and the fourth subset are at least partially different from each other.

3. 2. The method according to claim 1, wherein the image data (12a, 330, 400), in particular different subsets (280, 281, 282, 283, 284, 285, 286, 287, 290, 291, 293, 294, 295, 300, 301, 302, 303, 380, 381, 382, ​​383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396) of the image data (12a, 330, 400), enable, by means of the optical segmentation element (32a), in particular at least a respective portion of the image content to be projected onto at least one projection area (34a) of the deflection unit (20a) via at least four different imaging paths (28a, 30a).

4. blanking (215) at least a fifth subset (282, 292, 382, ​​388, 395) of the image data (12a, 330, 400), thereby generating an active sixth subset (283, 293, 301) of the image data (12a, 330, 400), the fifth subset (282, 292, 382, ​​388, 395) and the sixth subset (283, 293, 301) comprising the entire amount of the image data (12a, 330, 400); blanking (220) at least a seventh subset (287, 295, 303, 383, 391, 396) of said image data (12a, 330, 400), thereby generating an active eighth subset (286, 384, 392) of said image data (12a, 330, 400), and blanking (220) said seventh subset (287, 295, 303, 383, 391, 396) and said an eighth subset (286, 384, 392) being the entire amount of the image data (12a, 330, 400), and the second subset (19a, 280, 290, 380, 393), the fourth subset (284, 386, 389), the sixth subset (283, 293, 301), and the eighth subset (286, 384, 392) being at least partially different from one another; 3. The method of claim 2, further comprising:

5. 2. The method of claim 1, wherein all active subsets (280, 290, 380, 393, 284, 386, 389, 283, 293, 301, 286, 384, 392) of the image data (12a, 330, 400) together represent at least the complete image content.

6. 6. The method of claim 5, wherein all active subsets (280, 290, 380, 393, 284, 386, 389, 283, 293, 301, 286, 384, 392) of the image data (12a, 330, 400) together constitute the complete image content.

7. 7. The method of claim 6, wherein the active subsets (280, 290, 380, 393, 284, 386, 389, 283, 293, 301, 286, 384, 392) of the image data (12a, 330, 400) at least partially overlap.

8. 8. The method of claim 7, further comprising adjusting (225) the brightness of the active subset (280, 290, 380, 393, 284, 386, 389, 283, 293, 301, 286, 384, 392) of the image data (12a, 330, 400) in overlapping regions of the active subset (280, 290, 380, 393, 284, 386, 389, 283, 293, 301, 286, 384, 392) of the image data (12a, 330, 400), such that the adjustment results in a substantially uniform brightness distribution of the projected image content.

9. 2. The method of claim 1, wherein the first pupil area (261) has a maximum possible pupil diameter of the user.

10. The method of claim 1 , wherein the second pupil area (262) has a smallest possible pupil diameter of the user.

11. the at least one generated active subset (280, 290, 380, 393, 284, 386, 389, 283, 293) of the image data (12a, 330, 400) in which image content in the plurality of eyeboxes (A, A, A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''') spatially offset from one another is closest to the pupil midpoint (399) in the pupil plane (252, 256), in particular; 2. The method of claim 1, wherein the at least one blanked subset (281, 282, 285, 287, 291, 292, 294, 295, 300, 302, 303, 381, 382, ​​383, 385, 387, 388, 390, 391, 394, 395, 396) of the image data (12a, 300, 400) is selected such that the blanked subset (281, 282, 285, 287, 291, 292, 294, 295, 300, 302, 303, 381, 382, ​​383, 385, 387, 388, 390, 391, 394, 395, 396) of the image data (12a, 300, 400) includes at least the image data (12a, 300, 400).

12. 3. The method of claim 2, wherein at least two of the blanked subsets (281, 282, 285, 287, 291, 292, 294, 295, 300, 302, 303, 381, 382, ​​383, 385, 387, 388, 390, 391, 394, 395, 396) of the image data (12a, 330, 400) are substantially the same size depending on the position of the pupil midpoint (399) of the user relative to the generated eyebox (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''').

13. 5. The method of claim 4, wherein the second subset (19a, 280, 290, 380, 393), the fourth subset (284, 386, 389), the sixth subset (283, 293, 301), and the eighth subset (286, 384, 392) each represent a quarter of the total amount of the image data (12a, 330, 400) depending on the position of the pupil midpoint (399) of the user.

14. 2. The method of claim 1, wherein the optical segmentation element (32a) and the optical replication component (150a) are used to generate the plurality of eyeboxes (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''') arranged substantially in a raster.

15. 15. The method according to claim 14, characterized in that the eyeboxes (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''') are arranged in a substantially rectangular, in particular square, configuration or in a substantially parallelogram, in particular rhombus, in particular with two interior angles of substantially 60°.

16. 2. The method of claim 1, further comprising predistorting (235) the image data (12a, 330, 400) using the image processing device (10a) so that distortions in the image content are at least partially compensated for via the at least two imaging paths (28a, 30a).

17. 17. The method of claim 16, wherein the predistortion (235) and blanking (205, 210, 215, 220) of the image data (12a, 330, 400) is performed simultaneously.

18. 2. The method of claim 1, further comprising using an eye tracker (62a) to detect (235) the position of the pupil midpoint (399) of the user relative to the generated eyebox (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''').

19. An image processing device (10a) designed to implement the method according to claim 1, comprising: capturing image data (12a, 330, 400); blanking at least a first subset (17a, 281, 291, 300, 381, 387) of said image data (12a, 330, 400), thereby generating an active second subset (19a, 280, 290, 380, 393) of said image data, said first subset (17a, 281, 291, 300, 381, 387) and said second subset (19a, 280, 290, 380, 393) comprising the entire amount of said image data (12a, 330, 400); and controlling a projector unit (16a) using the blanked image data and the active image data (12a, 300, 400) in particular so that only one eyebox (A, A', A'', A''' or B, B', B'', B''' or C, C', C'', C''' or D, D', D'', D''') with the same active image data generated in a common imaging path (28a, 30a) is always located within a first pupil area (261) of the user, a pupil region (261) of the user (A) surrounds the pupil midpoint (399), and at least a portion of the eyebox (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''') is located with at least a portion of the active image data within a second pupil region (262) of the user, the second pupil region (262) surrounding the pupil midpoint (399), and the second pupil region (262) being located within the first pupil region (261); An image processing device (10a) designed to:

20. An optical system (68a) for a virtual retinal scanning display, comprising: an image source providing image content in the form of image data; an image processing device (10a) for image data (12a, 330, 400) according to claim 19; a projector unit (16a) comprising a time-modulatable light source (132a) for generating at least one light beam (18a) and a controllable deflection device (92) for said at least one light beam (18a) for scanning projection of said image content; a deflection unit (20a) capable of projecting the image content and directing the projected image content towards a user's eye (24a); an optical segmentation element (32a) arranged between the projector unit (16a) and the deflection unit (20a), the optical segmentation element (32a) being designed to project the image content onto at least one projection area (34a) of the deflection unit (20a) via at least two different imaging paths (28a, 30a), at least some of the imaging paths (28a, 30a) being individually controllable; an optical replication component (150a) arranged in the at least one projection area (34a) of the deflection unit (20a) and designed to replicate and spatially offset the projected image content towards the user's eyes (24a), whereby a plurality of eyeboxes (A, A', A'', A''', B, B', B'', B''', C, C', C'', C''', D, D', D'', D''') with the image content, spatially offset from one another, are generated; an optical system (68a) having at least

21. 21. The optical system (68a) according to claim 20, comprising smart glasses (66a) having a spectacle frame (144a) and spectacle lenses (70a, 72a), characterized in that the at least one projector unit (16a) and the at least one optical segmentation element (32a) are arranged in the spectacle frame (144a), and the at least one deflection unit (20a) having the at least one optical replication component (150a) is arranged in the area of ​​the at least one spectacle lens (70a, 72a).

Images