Projection device for multi-view projection of images and information

Abstract

The invention relates to a projection device for projecting images and information, comprising at least two projectors (1 i ) for multi-view projection from at least two light beams (2 i ), a waveguide (3) into which the at least two light beams (2 i ) can be coupled and which has at least as many decoupling elements (31 i ) as projectors (1 i ), which have a first sub-area and a second sub-area, and at least two pupil expanders (4 i ) arranged between the projectors (1 i ) and the waveguide (3) and dividing each light beam (2 i ) into a first partial light beam (2 i1 ) and at least one second partial light beam (2 i2 ), wherein the pupil expanders (4 i ) deflect each first partial light beam (2 i1 ) into the first sub-area and each second partial light beam (2 i2 ) or further partial light beam (2 ii ) into the second sub-area or a further sub-area of each of the decoupling elements (31 i ).

Description

Technical Field

[0001] The present invention relates to a projection device for multi-field projection of images and information, which is particularly suitable for augmented reality applications.

[0002] Augmented Reality (AR) refers to computer-assisted enhancement of reality perception targeting at least one human sensory modality. However, AR is often understood as simply representing the visual presentation of information, that is, supplementing images or videos with computer-generated additional information and / or virtual objects through overlay or superimposition. Specifically, the visual presentation or projection of images, user interfaces, or information (such as directions, weather information, or news) constitutes a common application of AR and is increasingly being used in so-called AR glasses, which can present images, user interfaces, or information directly on the lenses of eyeglasses or the user's retina.

[0003] Microscanners (also known as microelectromechanical systems, or MEMS for short) can be used to project images or text information. A light beam generated and subsequently shaped by a light source is deflected onto the microscanner, which is, for example, arranged in the temple of AR glasses. The microscanner can then scan the light beam, thereby generating an image on the field of view. Such imaging systems with microscanners require relatively few optical components, thus enabling miniaturized and cost-effective projectors. Background Technology

[0004] For example, a microscanner is described in DE 10 2021 116 151 B3. The MEMS scanner disclosed therein can simultaneously rotate and oscillate around two resonant oscillation axes to achieve nonlinear Lissajous projection in the field of view by deflecting the beam incident on the deflection element during the oscillation.

[0005] Due to the oscillations, the field of view (FOV) is scanned at a high frequency, similar to a Lissajous figure. Compared to traditional raster scanning methods that periodically scan the FOV from top to bottom at the highest resolution, this method can process hundreds of local images simultaneously and achieve smoother motion rendering. Furthermore, artifacts in the 3D perception of fast-moving objects are significantly reduced.

[0006] In projection devices, the beam emitted by the projector is coupled into a waveguide and guided within it by total internal reflection. The field of view of the projection device, from which the user perceives the image, is limited by the critical angle of total internal reflection. To expand the field of view, waveguide materials with high refractive indices can be used, as the range of total internal reflection angles increases with the refractive index of the waveguide material. However, due to the high cost of these materials, using materials with very high refractive indices is not suitable for mass production. Summary of the Invention

[0007] Therefore, the object of the present invention is to provide a projection device for projecting images and information, which significantly expands the field of view and can be manufactured cost-effectively. Furthermore, the object of the present invention is to provide augmented reality glasses that include a projection device with a significantly expanded field of view.

[0008] According to the present invention, the objective is achieved by a projection device for projecting images and information, the projection device comprising: at least two projectors, each projector adapted to scan at least one beam within a defined field of view; a waveguide arranged such that at least two beams can be coupled into the waveguide, and the waveguide having at least the same number of decoupling elements as the projectors for the defined field of view, wherein each decoupling element is illuminated by at least one assigned projector and has a first sub-region and at least one second sub-region; at least two pupil multipliers arranged between the projectors and the waveguide and configured as amplitude beam splitter surfaces to reflect each beam at least proportionally multiple times, or configured as diffraction structures to diffract each beam proportionally multiple times, such that each beam is split into a first portion beam and at least one second portion beam, wherein the pupil multipliers are configured to deflect each first portion beam into the first sub-region and deflect each second or other portion beam into a second sub-region or another sub-region of each decoupling element.

[0009] In the field of augmented reality, the term "field of view" (FOV) refers to the visible area overlaid on the user's physical environment by an AR application or device. FOV is typically measured in degrees and can be specified horizontally, vertically, or diagonally. A larger FOV allows virtual elements to be more fully integrated into the physical world, creating a more immersive AR experience. To optimize the user experience and minimize the boundary between the real and virtual worlds, it is generally necessary to provide the largest possible FOV.

[0010] For the purposes of this invention, multi-field projection is understood as a technique in which an optical system, specifically a projection device, simultaneously projects one or more images onto multiple regions. Multi-field projection can significantly expand the field of view. Furthermore, the beam can be guided using a pupil multiplier and decoupling elements, ensuring that almost no beam falls into the field of view that does not coincide with the eyepiece or the user's field of view. Therefore, during projection, there is almost no beam loss or a very small loss ratio, and the projection efficiency of the projection device can be improved.

[0011] One advantage of using multiple projectors and multiple decoupled windows is that the total field of view generated by all the field of view angles of the individual projectors is greater on the viewer's side than the individual field of view angles of the projectors.

[0012] If only one projector is used, the total field of view equals the field of view of the projector. Therefore, it is advantageous for each projector to have the largest possible field of view. While this is technically feasible, in this case, the waveguide must advance over the entire field of view of the projector. If the light is guided in the waveguide by total internal reflection (common in cost-effective waveguides), the refractive index of the waveguide must be very high. This significantly limits the degrees of freedom for other properties of the projection device, as it results in a lack of flexibility in both the waveguide material and the arrangement of the projector and waveguide.

[0013] Therefore, using multiple projectors and multiple decoupled windows is advantageous: this allows for greater flexibility in material selection and arrangement of the optical components of the projection device. For example, this can reduce the weight of the projection device and / or improve its compactness, and allow for the use of more cost-effective projectors and waveguides.

[0014] The projection angle ranges of partial beam priority scanning overlap, so there are no visible boundaries between the areas scanned by partial beams.

[0015] The waveguide is preferably plate-shaped and made of a material through which the beam of light emitted from the projector can propagate. The beam preferably propagates within the waveguide via total internal reflection at its outer edge. For this purpose, the beam should preferably be coupled into the waveguide at an angle greater than the total internal reflection angle. The total internal reflection angle depends on the material constituting the waveguide. Preferably, the waveguide is made of a material with a refractive index greater than 1.8. More preferably, the waveguide is made of a material with a refractive index greater than 2.0. The waveguide can consist of multiple layers or only one layer. Advantageously, the waveguide is a plate made of glass or plastic, such as polymethyl methacrylate (PMMA), polycarbonate (PC), polystyrene (PS), or polyethylene terephthalate (PETG). More preferably, the waveguide is a lens for augmented reality glasses or a windshield of a car. The waveguide can also be introduced into any other medium.

[0016] The waveguide has multiple decoupling elements. These decoupling elements are preferably optical elements, each designed as a decoupling lattice. Alternatively, the decoupling elements are also preferably prisms or mirrors incorporated into the waveguide. Furthermore, the decoupling elements can be adapted to perform beam shaping on the corresponding beam.

[0017] The decoupling elements, pupil multiplier, and projector are adapted and arranged so that each portion of the beam is scanned after leaving the waveguide at a specific projection angle.

[0018] The waveguide may also preferably have multiple coupling elements. The arrangement of the coupling elements is substantially the same as that of the decoupling lattice and is suitable for coupling the beam emitted by the projector into the waveguide.

[0019] A pupil multiplier can increase the exit pupil size of a projection device. A pupil multiplier is a beam splitter or beam splitter array that divides a light beam into multiple preferentially parallel partial beams; or it is formed by a diffraction structure of a diffracted beam. The diffraction structure can also be constructed as a holographic optical element.

[0020] Advantageously, the decoupling element, pupil multiplier, and projector are adapted and arranged such that the projection angles of the decoupling element (at which a portion of the beam is coupled out from the decoupling element) are superimposed on each other within a predetermined eyebox region (hereinafter referred to as the "eyebox"). This reduces the proportion of beam typically lost and improves projection efficiency. The area surrounding the eyebox is asymmetrically filled or illuminated. This means that not every point in the area surrounding the eyebox is illuminated by the same number of light sources or projectors.

[0021] The angle at which a portion of the beam is coupled out from the decoupling element is called the projection angle. The angle at which the projector emits the beam is called the field of view. Therefore, each projection angle is generated by the field of view and the arrangement and design of the pupil multiplier, waveguide, and decoupling element.

[0022] The "eyebox" refers to the area that receives light from all projection angles of a partial beam of light. It is there that an eye, or pupil, can see the entire image. When the eye moves laterally out of the eyebox (unlike the true exit pupil in a microscope, for example), a portion of the image's field of view it sees shrinks. If a portion of the image's field of view can still be seen around the eyebox, but the area that, by definition, no longer belongs to the eyebox shrinks, the optical efficiency of the projection device can be improved, thus potentially extending battery life.

[0023] Preferredly, some beams are coupled out in a manner that is not perpendicular to the waveguide surface, and their projection angles are different from each other.

[0024] Preferably, each projector includes at least one microscanner. The microscanner can be configured, in particular, as a microelectromechanical system (MEMS), and / or adapted to realize nonlinear Lissajous projection in the observation field of view. The microscanner is configured to scan a beam in the observation field of view, thereby generating an image on the observation field of view. Scanning at least one beam along a Lissajous figure allows for the simultaneous processing of hundreds of local images and enables smoother motion rendering. Furthermore, artifacts in the user's 3D perception of fast-moving objects are significantly reduced.

[0025] Preferably, the number of pupil multipliers is the same as the number of projectors. Also preferably, alternatively or additionally, the number of decoupling elements is the same as the number of projectors. If exactly one pupil multiplier and one decoupling element are allocated to each projector, each pupil multiplier and each decoupling element can be customized according to the associated projector. Therefore, the pupil multipliers and decoupling elements can be customized, for example, according to the spectral range and / or field of view of each projector, thereby reducing the requirements for pupil multipliers and decoupling elements while still achieving better image quality. It is also possible to have more or fewer projectors and / or more or fewer decoupling elements than projectors.

[0026] If the projection device includes a microscanner, it can advantageously also include a sensor and a control unit. The sensor senses the gaze direction, and the control unit drives the microscanner such that portions of the beam are scanned only at projection angles corresponding to the user's gaze direction. Therefore, the projection angles are updated based on the gaze direction, and areas outside the user's field of view are not illuminated. This saves power and data transmission bandwidth.

[0027] For the purposes of this invention, gaze direction refers to the orientation of the user's head and eyes.

[0028] The projection device preferably includes a control unit and an evaluation unit, wherein the evaluation unit classifies the information to be presented into a first information group and at least one second information group, and the control unit drives the projector such that the information already allocated to the first information group can be presented with increased intensity and / or increased contrast. This can save optical power for the information already allocated to the second information group.

[0029] The objective is further achieved by augmented reality glasses, which include a projection device for multi-field projection of images and information according to any of the described embodiments. Attached Figure Description

[0030] The invention will now be described in more detail with reference to the accompanying drawings and exemplary embodiments. Each index i represents an arbitrary natural number. In the drawings: Figure 1 The diagram shows a top view of a first embodiment of a projection device for multi-field projection of images and information, the projection device having three projectors, three pupil multipliers, and three decoupling elements. Figure 2 The diagram shows a top view of a second embodiment of a projection device for multi-field projection of images and information, the projection device having four projectors, four pupil multipliers, and four decoupling elements. Figure 3The diagram shows a top view of a third embodiment of a projection device for multi-field projection of images and information, the projection device having four projectors, two pupil multipliers, and four decoupling elements. Figure 4A A side view of the waveguide and decoupling element of a projection device according to the prior art is shown. Figure 4B A side view of the waveguide and decoupling elements of a fourth embodiment of a projection device for multi-field projection of images and information is shown. The projection device has three decoupling elements. Figure 5 A side view of the waveguide and decoupling elements of a fourth embodiment of a projection device for multi-field projection of images and information is shown. The projection device has five decoupling elements, and the intensity of the corresponding coupled portions of the beam within the projection angle range is shown. Figure 6 The image shows a view of augmented reality glasses, which include a projection device for multi-field projection of images and information. Detailed Implementation

[0031] Figure 1 The first embodiment of a projection apparatus for multi-field projection of images and information is depicted. The first embodiment of the projection apparatus includes three projectors 11, 12, and 13, each adapted to scan a beam 2 within the field of view. i The projection device further includes a waveguide 3, which is arranged such that three light beams 21, 22, and 23 are coupled into the waveguide 3. The projection device has three decoupling elements 31. i Each decoupling element 31 i By the projector 1 assigned to it i Illumination is provided, and the area comprises three sub-regions (indicated by dashed lines). Projector 1... i Three pupil multipliers 4 are arranged between waveguide 3 and waveguide 3. i , so that each beam 2 i Divided into three beams 2 i1 2 i2 and 2 i3 .

[0032] Each pupil multiplier 4 i Designed so that each first part of the beam 2 i1 Deflected by the first pupil multiplier 41 to the first sub-region, each second portion beam 2 i2 The light is deflected by the pupil multiplier 42 to the second sub-region, and each third portion of the beam 2 i3 The deflection is transferred from the third pupil multiplier 43 to the decoupling element 31. i The third subregion of one of them.

[0033] In the first embodiment, the waveguide 3 has the shape of an eyeglass lens for AR glasses. Projectors 11, 12 and 13, decoupling elements 311, 312 and 313, and pupil multipliers 41, 42 and 43 are each attached to or in the waveguide 3.

[0034] Figure 2 A second embodiment of a projection device for multi-field projection of images and information is shown. The second embodiment has four projectors 11-14, four pupil multipliers 41-44, and four decoupling elements 311-314. Each projector 1 i Each is equipped with a pupil multiplier 4 i Each pupil multiplier has 4 i The characteristics and arrangement of the projector can be determined according to the corresponding projector 1. i Customization. If projector 1 i This is particularly advantageous when the emitted light has different wavelength ranges. In this case, it can be achieved by multiplying each pupil by 4... i With corresponding projector 1 i The wavelength range is matched to minimize imaging errors.

[0035] According to Figure 2 The second embodiment and Figure 3 In the design depicted, each projector 1 i It includes at least one microscanner, which is configured as a microelectromechanical system (MEMS). The microscanner is depicted as a circular or disk-shaped structure. Each microscanner is adapted to realize nonlinear Lissajous projection in the observation field of view. Furthermore, a projector 1... i Includes a light source (not depicted) and components for beam guiding and shaping (also not depicted).

[0036] The projection device also includes a sensor 5 and a control unit 6, wherein the sensor 5 is used to sense the gaze direction. The control unit 6 can drive a micro-scanner so that (partial) beam 2 i Each scan is performed using only the field of view or projection angle corresponding to the direction of gaze.

[0037] Furthermore, a second embodiment of the projection device includes an evaluation unit 7. The evaluation unit 7 is capable of classifying the information to be presented into a first information group and at least one second information group. The control unit 6 can drive the projector 1. i This causes the information already assigned to the first information group to be presented with increased intensity and / or increased contrast.

[0038] Figure 3A third embodiment of a projection device for multi-field projection of images and information is shown. This embodiment has four projectors 11-14, two pupil multipliers 41 and 42, and four decoupling elements 311-314. Each pair of projectors 11-14... i One pupil multiplier 4 is allocated i Because there are only two pupil multipliers 4 i And 42, thus reducing the amount of assembly and adjustment work compared to the second embodiment. In return, the pupil multiplier 4 i The requirements have increased.

[0039] In the prior art, all decoupling elements 31 are typically... i and projector 1 i Arranged such that each decoupling element 31 i They all exhibit the same decoupling behavior. Figure 4A The waveguide 3 and decoupling element 31 of the projection device according to the prior art are shown. i Example. Partial beam 2 ii With surface O, which is substantially perpendicular to waveguide 3 (thick arrows indicate each decoupling element 31) i The beam is coupled out in the manner of (average decoupling direction), therefore part of the beam 2 ii At least a significant portion remains unused, especially since edge rays (depicted in dashed lines) do not contribute to imaging or projection.

[0040] Figure 4B The waveguide 3 and decoupling element 31 are depicted according to a fourth embodiment of a projection device for multi-field projection of images and information. i The projection device has three decoupling elements 311-313.

[0041] In the fourth embodiment, decoupling element 31 i 4 pupil multipliers i and projector 1 i After adjustment and setup, the projector 1 i The projection angle β i The beams overlap within the predetermined eyebox region E, with some beams 2 ii From the decoupling element 31 at the projection angle i Coupled out.

[0042] In addition, part of the beam 2 ii It is coupled out in a manner that is not perpendicular to the surface O of waveguide 3, and part of the beam 2 ii The projection angle β i They are different from each other. If possible, partial beam 2. ii From decoupling element 31 i The coupling points out towards eye box E.

[0043] Figure 5 Waveguide 3 and decoupling element 31 are shown in a fourth embodiment of a projection device for multi-field projection of images and information. i The projection device has five decoupling elements 311-315. Figure 5 Also depicting the corresponding coupled portion of the beam 2 ii At projection angle β i Inner strength I i It also depicts (in thicker lines) various intensities I. i Total intensity I produced total The total intensity for each point is the sum of the individual intensities I. i The sum of .

[0044] Waveguide 3, especially diffractive waveguides, often suffers from (multicolor) projection uniformity issues because it frequently transmits light across different wavelength ranges. This leads to imaging errors and reduces image quality. Figure 5 The diagram schematically illustrates how so-called field-of-view stitching can improve image quality during projection. Field-of-view stitching refers to the method of seamlessly connecting multiple partial images or parts of the observation field of view to form an observation field of view or a complete image. The goal of field-of-view stitching is to obtain a larger observation field of view without sacrificing detail.

[0045] Figure 6 An augmented reality (AR) glasses display is shown, comprising a projection device for multi-field projection of images and information. Each lens of the AR glasses includes a waveguide 3, which has four decoupling elements 311-314. (Partial beam 2) ii Through decoupling element 31 i It extends toward the user's eyes, that is, toward the temples of the AR glasses. List of reference numerals 1 i Projector 2 i beam 2 ii Partial beam 3 Waveguide 31 i Decoupling elements 4 i pupil multiplier 5 sensors 6 Control Unit 7. Evaluation Unit β i Projection angle I i strength I total Total strength E Eye Box (Area) O surface.

Claims

1. A projection device for projecting images and information, comprising: - At least two projectors (1 i Each projector is adapted to scan at least one beam (2) within a defined field of view. i ), to produce by the at least two projectors (1 i At least two beams scanned (2) i The multi-field projection formed by ) - Waveguide (3), the waveguide being arranged such that the at least two beams (2 i ) can be coupled into the waveguide (3), and the waveguide has at least a connection with the projector (1) i The same number of decoupling elements (31) i ) for the defined field of view, where each decoupling element (31 i ) by at least one projector assigned to it (1 i The irradiation has a first sub-region and at least one second sub-region. - At least two pupil multipliers (4 i The at least two pupil multipliers are arranged in the projector (1) i Between the waveguide (3) and the waveguide (2), and configured as an amplitude beam splitter surface to proportionally reflect and transmit each beam (2) multiple times. i ), or be constructed as a diffraction structure to diffract each beam multiple times in proportion (2 i ), such that each beam (2 i ) is divided into the first part of the beam (2) i1 ) and at least one second beam (2 i2 ), - The pupil multiplier (4) mentioned above i ) is configured to distribute each first portion beam (2 i1 ) deflected to the first sub-region, and each second portion beam (2 i2 ) or other partial beams (2 ii ) deflected to the decoupling element (31) i The second sub-region or another sub-region of one of the ).

2. The projection device according to claim 1, wherein the decoupling element (31) i ), the pupil multiplier (4) i ) and the projector (1 i After adjustment and arrangement, the decoupling element (31) is made so that... i The projection angle (β) of ) i The partial beams (2) overlap each other in the predetermined eyebox region (E). ii ) at the projection angle from the decoupling element (31) i ) Coupled out.

3. The projection device according to claim 2, wherein the portion of the light beam (2) ii The portion of the beam (2) is coupled out substantially at an angle to the surface (O) of the waveguide (3), and the portion of the beam (2) is coupled out substantially at an angle to the surface (O) of the waveguide (3). i The projection angle (β) of ) i They are different from each other.

4. The projection apparatus according to any one of claims 1 to 3, wherein each projector (1 i ) comprises at least one microscanner, which is configured as a microelectromechanical system (MEMS).

5. The projection apparatus of claim 4, wherein each microscanner is adapted to perform nonlinear Lissajous projection in the field of view.

6. The projection apparatus according to any of claims 1 to 5, wherein the number of pupil multipliers (4 i ) is the same as the number of projectors (1 i ).

7. The projection apparatus according to any one of claims 1 to 6, wherein the number of decoupling elements (31 i ) is the same as the number of projectors (1 i ).

8. The projection device according to any one of claims 4 to 7, wherein the projection device comprises a sensor (5) and a control unit (6), the sensor being used to sense the gaze direction, and the control unit (6) driving the micro-scanner such that the partial beam (2) ii Each only uses the projection angle (β) corresponding to the gaze direction. i (Scan) 9. The projection device according to any one of claims 1 to 8, wherein the projection device comprises a control unit (6) and an evaluation unit (7), wherein the evaluation unit (7) can classify the information to be presented into a first information group and at least one second information group, and the control unit (6) can drive the projector (1 i This allows the information already assigned to the first information group to be presented with increased intensity and / or increased contrast.

10. The projection device according to any one of claims 1 to 9, wherein the waveguide (3) is made of plastic.

11. An augmented reality glasses, the augmented reality glasses comprising a projection device for multi-field projection of images and information according to any one of claims 1 to 10.

Images