Display device with reduced power consumption

By adopting a combination of lighting devices, spatial light modulation devices, optical systems and control devices in an augmented reality display, the problem of taking into account both high frame rate and low power consumption in the prior art is solved, and the problem of taking into account both high frame rate and low power consumption is realized. It has low power consumption, high brightness and compact and lightweight structure, suitable for mobile display devices.

CN114174893BActive Publication Date: 2024-11-29SEEREAL TECHNOLOGIES SA
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202080051725.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-05-14
Publication Date
2024-11-29
Estimated Expiration
2040-05-14

AI Technical Summary

Technical Problem

The existing spatial light modulators are difficult to balance between achieving high frame rates and low power consumption, especially in mobile display devices such as head-mounted displays, where power consumption problems are particularly prominent, and the prior art is difficult to meet the high-level of augmented reality displays. Brightness and low power consumption provide a compact and lightweight construction.

Method used

Using a display device including an illumination device, a spatial light modulation device, an optical system and a control device, the image of the spatial light modulation device is directed to a defined position in the user's field of view through a deflection device, and light is emitted and virtual information is generated only when needed. Reduce data transmission and power consumption, and use spatial light modulators of small pixels combined with efficient optical design to achieve low power consumption and high brightness.

Benefits of technology

It realizes a compact and lightweight structure with low power consumption and high brightness in augmented reality display, significantly reducing data transmission power consumption and ensuring that users can efficiently display virtual information in mobile environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114174893B_ABST
    Figure CN114174893B_ABST
Patent Text Reader

Abstract

The present invention relates to a display device, comprising an illumination device for emitting light, a spatial light modulation device for modulating incident light, an optical system and a control device. The optical system is used to generate at least one image of the spatial light modulation device as a segment, wherein the optical system also includes a deflection device for guiding the image of the spatial light modulation device to a defined position in a user's field of view. The control device is connected to the illumination device and the deflection device and is designed to switch the illumination device based on the control of the deflection device.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The present invention relates to a display device for displaying two-dimensional and / or three-dimensional information (such as objects or scenes), which provides low power consumption. The present invention preferably relates to an augmented reality (AR) display device or display. For example, this includes head-mounted displays and head-up displays. However, it should be noted that the present invention should not be interpreted as being limited to such displays.

[0002] Furthermore, the invention relates to a method for displaying two-dimensional and / or three-dimensional information, by means of which the power consumption of a display device can be reduced.

[0003] So-called spatial light modulators, which modulate the incident light according to the information required and to be displayed, are usually used in display devices for representing two-dimensional and / or three-dimensional scenes. In this context, different types of spatial light modulators, also referred to as SLMs, are known. For example, one spatial light modulator that can be used for this purpose is the LCoS-SLM (liquid crystal on silicon spatial light modulator), which reflects the incident light instead of transmitting it. The range of an LCoS-SLM is usually very small, for example having a diagonal of less than 20 mm, but can have a large number of pixels, for example 4000×2000 pixels. LCoS-SLMs exist as a commercial product for modulating the amplitude and phase of light. The advantage of an LCoS-SLM is obviously its reflectivity. However, there are also some minor disadvantages, such as its speed, which is limited by the reaction time of the liquid crystals in the SLM. In some cases, LCoS-SLMs can achieve frame rates of, for example, 60 Hz, or even several hundred Hz (for example 180 Hz or 240 Hz). However, frame rates greater than 500 Hz cannot be achieved.

[0004] Another type of SLM is the MEMS-SLM (micro-electromechanical system spatial light modulator), whose main advantage over the LCoS-SLM is its higher speed. Another advantage is that it is possible to set the modulation value, phase value or amplitude value in a more stable manner and is less prone to errors than the LCoS-SLM. For example, in the case of the MEMS-SLM, the modulation values ​​of adjacent pixels can be set independently of each other in an improved manner, while in the case of the LCoS-SLM, the modulation values ​​of adjacent pixels can undesirably influence each other. On the other hand, commercially available amplitude-modulated MEMS-SLMs are currently limited to binary types, and phase-modulated MEMS-SLMs, for example for use in the field of lithography, are not currently found on the display market. In addition, such MEMS-SLMs have a relatively small number of pixels, for example less than a VGA (640×480 pixels).

[0005] Generally speaking, the effort and cost of manufacturing a new spatial light modulator (SLM) increases with the number of pixels. In addition, the power consumption or current consumption in the display device also plays an important role. In this regard, the current consumption of the spatial light modulator must also be taken into account. A large part of the current consumption of the spatial light modulator can be traced back to the data transmission to the pixels of the spatial light modulator. However, the power consumption also depends on the length of the data lines from the edge of the spatial light modulator to the individual pixels. A spatial light modulator with a smaller volume and a lower average number of pixels has shorter data lines and therefore usually consumes less power. Therefore, compared with a spatial light modulator with a large number of pixels but a lower frame rate, a spatial light modulator with a lower number of pixels will be more energy-efficient even if it is operated at a high frame rate or frame frequency if the same number of pixels are written per unit time.

[0006] Power consumption is a particularly important factor for mobile display devices, such as head mounted displays that are attached to the head of a viewer or user and cannot be connected to a power system via a cable.

[0007] In this case, a display device using a spatial light modulator with a small number of pixels would be considered advantageous. On the other hand, for a head mounted display (HMD), a large number of pixels is required to produce a large field of view (FoV) with good resolution. For a flat two-dimensional (2D) image display, a typical value is 60 pixels / degree field of view, since this value (60 pixels / degree field of view) corresponds to the resolution of the human eye. However, for a holographic representation of a three-dimensional (3D) scene, a higher number of pixels per degree field of view is required.

[0008] For example, a 60 degree × 30 degree field of view requires 3600 pixels × 1800 pixels to generate a flat two-dimensional image, but more pixels (e.g., 15,000 pixels × 7,500 pixels) are required to generate a holographic image. LCoS-SLMs with a resolution of approximately 4000 pixels × 2000 pixels already exist as commercial products. However, these products still have significant disadvantages. Higher resolutions are typically achieved by using smaller pixels, such as pixels with a size of 3 to 5 microns, which increases the sensitivity of the modulation value to errors in the LCoS-SLM, such as the undesirable influence of adjacent pixels on the modulation value. However, if larger pixels are used, such as pixels with a size of 8 to 10 microns, the size and weight of the LCoS-SLM is obtained at a resolution of 4000 × 2000 pixels, which is not conducive to the installation volume and total weight of the head-mounted display. This size will also have an adverse effect on the production cost of the LCoS-SLM.

[0009] For example, the applicant's patent documents (e.g., patent documents WO 2018 / 146326A1, WO 2019 / 012028A1, WO 20118 / 211074A1, WO 2019 / 076963A) have disclosed display devices in the form of head-mounted displays, each display device including a light guide device, a coupling device and a decoupling device, and an additional optical element arranged in the light path before the light is coupled to the light guide. The head-mounted display does not include a light guide, but it is known to arrange a focusing device, such as a lens element and / or a curved reflector element, for example from patent documents US 2010 / 0097671 or US 2013 / 0222384.

[0010] Patent document US 2013 / 0222384 describes segmented multiple images of a spatial light modulator. In this case, a large field of view is generated using an imaging spatial light modulator by generating individual segments in a time sequence, wherein each imaging spatial light modulator is performed at a different position in the field of view. In one embodiment of patent document US 2013 / 0222384, the segments are generated using an arrangement of two mirrors that rotate to the same degree.

[0011] While the entire field of view, or at least a large part of it, is usually filled with content or information in a display device (such as a TV, notebook or tablet or other VR (virtual reality) head-mounted display), the situation is different for AR (augmented reality) display devices or AR displays. Such augmented reality displays, also known as mixed reality displays, allow people to look through a transparent or translucent system to observe their physical environment, and also see images of virtual objects (such as text, graphics, videos, etc.), for example, which appear as part of the physical environment by superposition. Therefore, the display or representation or superposition of additional information to a person's natural perception or surrounding environment is called augmented reality (AR). In this case, this additional displayed information (for example, more precisely set as a speed display, temperature display, signs, warnings or as an auxiliary function, navigation system function, radio function or store display) is overlaid in the person's field of view, and the person's behavior or operating behavior is not adversely affected. Therefore, for AR displays, it is important that users can still observe their physical environment in addition to the virtually generated objects. Therefore, only a small part of the physical environment can be hidden by the content of the object, which is displayed to the user through the AR display. However, compared to applications for display devices such as TVs or VR-HMDs, AR displays require greater brightness even in bright sunlight, because users should see information or virtual objects at approximately the same brightness level as in the physical environment. In addition, AR displays as mobile devices, such as AR head-mounted displays (AR-HMDs), also require low power consumption.

[0012] The object of the present invention is therefore to provide a device and a method which allow three-dimensional representation of information. Furthermore, the device should be compact, low in weight and energy efficient.

[0013] According to the invention, this object is achieved by a device having the features of claim 1 .

[0014] According to the present invention, a display device is provided, which is specifically implemented as an AR (augmented reality) display device or an AR display. Preferably, the AR display is implemented as an AR head-mounted display or an AR head-up display. The device includes an illumination device, a spatial light modulation device, an optical system and a control device. The illumination device is used to emit, for example, substantially coherent light. The spatial light modulation device is used to modulate the light emitted by the illumination device and may include at least one spatial light modulator. The optical system is correspondingly used to generate at least one image of the spatial light modulation device as a fragment, and the optical system, in addition to including at least one imaging element, also includes a deflection device for guiding the image of the spatial light modulation device to a defined position in the user's field of view. The control device is connected to the illumination device and the deflection device, and is implemented to control or switch the illumination device based on the control of the deflection device.

[0015] This purpose will be explained in more detail with respect to AR displays (particularly AR head mounted displays), but as previously mentioned should not be construed as being limited to only this type of display.

[0016] An AR display is a display device in which virtual information or objects are superimposed on the physical environment of a person or user using the AR display, so that the user can receive additional information that is being or may be used while observing his physical environment. For example, when sightseeing, information related to points of interest or navigation suggestions can be displayed to a user or device of a display device according to the present invention, and then superimposed on the physical environment of the user's field of view. In order to obtain the superposition of virtual information on real information in the user's field of view, the optical system of the display device according to the present invention includes a deflection device, through which the image of the generated spatial light modulation device is guided by the optical system to a defined position or location in the user's field of view, so as to superimpose the required information or required object on the user's physical environment and display it to the user. To this end, the control device of the display device according to the present invention is connected to the lighting device and the deflection device so that the lighting device and the deflection device can be appropriately controlled. The lighting device is controlled or switched by the control device according to the control of the deflection device. This means controlling the deflection device of the optical system and setting the position in the field of view where the virtual information or virtual object should be displayed or represented. Once the desired position in the field of view is reached, the lighting device is controlled by the control device so that the lighting device emits light accordingly, the light is incident on the spatial light modulation device, and an image is generated using the optical system. Thus, this image of the spatial light modulation device is guided as a segment to a defined position in the user's field of view and overlaid on the physical environment in the field of view so that the user can observe the information represented thereby.

[0017] In this way, a display device for displaying two-dimensional and / or three-dimensional information can be created, which includes a small number of components and is therefore compact and has a low weight. In addition, the display device according to the present invention is able to represent the required information to the user in an energy-saving manner, because when the control device controls the lighting device to emit light accordingly, only the data for generating the virtual information to be represented is transmitted or transferred to the spatial light modulation device or generated by the spatial light modulation device itself. This means that the calculation of the virtual information to be displayed is only implemented for the area in the field of view where the virtual information is to be represented or displayed using the display device according to the present invention, such as the calculation and addition of sub-holograms of the overall hologram for displaying holographic three-dimensional information or objects, or another type of image processing, such as blurring of objects away from the user's focus of the display of the stereoscopic scene and its display. Data is not calculated in other areas of the field of view where virtual information is not intended to be displayed, or in another embodiment, data is not calculated and the data is not transmitted to the spatial light modulation device. In this way, the power consumption of data transmission can be significantly reduced.

[0018] Therefore, the display device according to the present invention can be advantageously implemented as an augmented reality display that combines a physical environment and represents or displays virtual information (such as two-dimensional and / or three-dimensional objects). In this case, at least one image of a spatial light modulator as a fragment generated to contain virtual information only occupies a portion of the field of view, such as 2% to 30%, or only 5% to 20% in addition. This means that only very little virtual information is contained in the field of view. In other words, the image of the spatial light modulator represented as a fragment or the multiple images of the spatial light modulator represented as multiple fragments do not completely fill the field of view or only constitute a portion of the field of view, so that the gap or area that fills the real information in the field of view or allows the user to observe its physical environment exists between the represented virtual information, and the represented virtual information each constitutes a complete virtual information. The image of the spatial light modulator as a fragment or the multiple images of the spatial light modulator as a fragment can form virtual information together, which is then separated from another virtual information represented in the field of view by the gap, and the observer can perceive the physical environment through the gap.

[0019] The virtual information to be represented can be generated in a holographic or stereoscopic manner. In addition, the virtual information can be displayed as a two-dimensional or three-dimensional representation. A combination of a two-dimensional representation and a three-dimensional representation is also possible. According to the present invention, the term "virtual information" should not be understood as referring only to completely generated virtual information, such as an object or a scene, but should also represent only part of the virtual information, such as a part of an object or a part of a scene.

[0020] Further advantageous configurations and developments of the invention emerge from the further dependent claims.

[0021] In a particularly advantageous configuration of the present invention, an optical system can be set up for generating at least two images of a spatial light modulation device and for generating a virtual visible area according to the number of images of the spatial light modulation device, wherein at least two images of the spatial light modulation device as fragments are presented in the field of view.

[0022] Advantageously, at least two images of the spatial light modulation device as segments in the field of view can be combined with one another and / or can partially overlap or can be separated from one another by a gap.

[0023] The image of the spatial light modulation device is preferably generated in a time-continuous manner in the field of view.

[0024] Generating at least two images of the spatial light modulation device and their representation in the field of view of a user of the display device according to the invention forms a segmented representation of virtual information in the field of view. By combining multiple images of the spatial light modulation device as segments, a large field of view or a large viewing angle can be created. For example, a certain number of segments, such as more than 10 segments, more than 30 segments or more than 50 segments, can be used to achieve the representation of virtual information in the field of view of the user.

[0025] The number of images of the spatial light modulator as segments in each frame can advantageously be set differently between a minimum value (e.g. one image as a segment) and a maximum value (e.g. 10 to 50 images as segments), and the positions of the images of the spatial light modulator as segments in each frame in the field of view can be set differently.

[0026] In this case, the determination of the number and position of the images of the spatial light modulation device as segments in the field of view depends on the physical environment of the user. This means that the number and position of the images of the spatial light modulation device can be set according to the real environment. Therefore, the number and position of the displayed images of the spatial light modulation device as segments are variable and adjustable according to the virtual information required in the field of view.

[0027] For this purpose, a user or observer observes the represented two-dimensional and / or three-dimensional information or the represented two-dimensional and / or three-dimensional object via a virtual visible area on an observer plane.

[0028] This means that when generating the individual images of the spatial light modulation device as segments, virtual visible areas are generated in the observer plane, wherein all generated virtual visible areas should appear at the same position in the observer plane and should overlap one another for the observer's eyes.

[0029] According to the invention, the situation should be interpreted as that during the holographic generation and representation of the virtual information encoded on the spatial light modulation device along the encoding direction of the hologram, there is a virtual observer window as a virtual visible area, and in the case of a stereoscopic representation of the virtual information in the field of view, there is an optimal visual area also called a "sweet spot" as the virtual visible area. Depending on the representation of the virtual information, the virtual observer window and the sweet spot therefore each or together form a virtual visible area in the observer plane, where the user (specifically the user's eyes) is located to observe the generated information.

[0030] Thus, for example, one or more segments of the three-dimensional information to be represented can be generated and represented holographically, which should be located in the viewing direction of the user's eye and therefore incident on the retina in the center of the fovea of ​​the eye. However, one or more segments of the same or another three-dimensional information to be represented can be generated and represented stereoscopically, which segments are not located in the viewing direction of the user's eye, which are incident on the retina of the eye but not in the center of the fovea.

[0031] In order to substantially or completely avoid potential convergence-accommodation conflicts, the individual segments used to display virtual information in the field of view should be generated purely holographically, as holographic approaches can achieve a more realistic depth representation of the reconstructed information or objects compared to a stereoscopic representation of the virtual information.

[0032] Furthermore, the at least one image of the spatial light modulation device may be provided as an image representation of the entire spatial light modulation device or as an image representation of only a part of the spatial light modulation device.

[0033] In general, the image of the spatial light modulation device forms a fragment that is superimposed on the user's physical environment or real field of view and is therefore part of the user's field of view. The fragment in the field of view can be formed by an image representation of the entire spatial light modulation device (i.e. the total area of ​​the spatial light modulation device), the fragment being generated by the display device according to the present invention and the fragment containing virtual information, so that all pixels of the spatial light modulation device contribute to the generation of the fragment. Optionally, the generated fragment can also be created by an image representation of a part or portion of the spatial light modulation device, i.e. not all pixels of the spatial light modulation device are used for the generation of the fragment.

[0034] Therefore, each generated fragment only covers or hides a small area in the field of view of the user of the device according to the present invention. For example, a single fragment may cover an area of ​​only about 3°×3° or about 5°×3° or about 7°×7° of the entire field of view, but it is not intended to limit the present invention to these numerical data. For example, in this case, the entire field of view may span an area of ​​about 40°×20° or about 60°×30° or 60°×60°, and these numerical data are also not intended to be interpreted as limiting. This also means that the number of fragments containing virtual information and superimposed on the physical environment is less than the number of fragments required for the entire field of view constructed by the fragments or the number of fragments used to generate the entire field of view using the fragments. For example, if the entire field of view contains an area of ​​about 60°×60° and the size of a single fragment is about 5°×5°, then theoretically 12×12 fragments (i.e. 144 fragments) will be required to generate the entire field of view. However, if only about 15% of the field of view needs to be filled with virtual information (e.g. using two-dimensional and / or three-dimensional objects), then depending on the size of the segments, selecting only about 25 to 30 segments is sufficient to display the required information in the field of view. Thus, more time can be saved and higher energy efficiency can be achieved.

[0035] In an advantageous configuration of the invention, it can be provided that the deflection device comprises at least one movably mounted scanning mirror element or at least one grating element.

[0036] Preferably, the device for deflecting and directing light according to the invention comprises at least one scanning mirror element. Due to being mounted in a movable manner, the at least one scanning mirror element can be moved or rotated and can direct the generated image of the spatial light modulation device as a segment to a defined position in the user's field of view. In this way, multiple images of the spatial light modulation device can be generated and these images can be directed to defined positions in the field of view. Commercial scanning mirrors can be used as scanning mirror elements.

[0037] In another configuration of the present invention, the deflection device may include at least one grating element, such as a switchable grating element or a polarization selective grating element (such as a polarization grating combined with a polarization switch). For example, the deflection device may include a stacked structure of grating elements with different grating periods, so that a value from 2 to 2 to the power of 2 (2 N ) at different deflection angles.

[0038] In another configuration of the present invention, the deflection device may also include a combination of at least one scanning mirror element and at least one grating element (e.g., a volume grating). In this case, at least one grating element or volume grating has an angle selectivity. If the scanning mirror element is arranged so that the light is incident on the grating element or volume grating within the angle selection range of the grating element, the light is further deflected by the grating element or volume grating. Therefore, the deflection angle of the scanning mirror element is amplified by the grating element or volume grating. If the scanning mirror element is arranged so that the light is incident on the grating element or volume grating outside the angle selection range of the grating element, the light is not deflected by the grating element or volume grating. The scanning mirror element can be used to select one of a plurality of grating elements or volume gratings, whose angle selectivity and deflection angle have different settings, and the selected grating element or volume grating then further deflects the light.

[0039] However, the present invention should not be construed as being limited to a certain type of deflection device.

[0040] In order to superimpose the virtual information generated by the device according to the invention on the real information in the field of view, it can be advantageous for the optical system to include at least one combiner.

[0041] Thus, at least one combiner combines information from the user's physical environment and information generated by the device according to the invention in the user's field of view, so that the user's eyes can see and observe both items of information in the field of view.

[0042] In one configuration of the invention, at least one combiner may be a partially reflective mirror element that reflects light, which at least partially emits reflected light from the light modulation device in the light path in the direction of the user's eyes and at least partially transmits ambient light. In the case of a head-up display, at least one combiner may be, for example, a windshield of a vehicle, such as a vehicle.

[0043] In another configuration of the invention, at least one combiner can also be a light guide, which decouples the light emitted from the light modulation device in the direction of the user's eyes in terms of the optical path and at least partially transmits ambient light.

[0044] Advantageously, the deflection device can be arranged between the spatial light modulation device and the combiner or between the illumination device and the spatial light modulation device.

[0045] For example, the deflection device and therefore at least one scanning mirror element thereof can be arranged in the Fourier plane of the spatial light modulation device. Furthermore, this Fourier plane is imaged by means of an optical system to an observer plane, wherein the eyes of the user are located in the observer plane, in which a virtual visible area (i.e. a virtual observer window or sweet spot) is generated, through which the user must gaze in order to be able to observe the virtual information represented in the field of view. If the at least one scanning mirror element of the deflection device is in motion, while the position of the virtual visible area in the observer plane remains stationary, the generated image of the spatial light modulation device as a segment moves to a defined position in the field of view.

[0046] In a further advantageous configuration of the invention, it can be provided that the deflection device comprises two scanning mirror elements which can be rotated synchronously with one another.

[0047] It is also possible to use a combination of two scanning mirror elements to guide the image of the spatial light modulator as a fragment to a defined position in the field of view. These scanning mirror elements can be rotated or moved in a synchronized manner with each other. Due to the synchronized movement of the scanning mirror elements with each other, it is also possible to obtain an image of the spatial light modulator as a fragment or a movement of the image plane of the spatial light modulator to a defined position in the field of view without a change in the position of the virtual visible area in the observer plane. The two scanning mirror elements can then be arranged in the device according to the invention so that, for example, one scanning mirror element is arranged upstream of the Fourier plane of the spatial light modulator in the direction of the light and the other scanning mirror element is arranged downstream of the Fourier plane in the direction of the light.

[0048] Likewise, in another configuration of the present invention, it can also be arranged that the deflection device includes at least two grating elements, which are switched in a synchronized manner with each other. Due to the synchronized switching of the two grating elements, it is also possible to obtain a movement of the image of the spatial light modulation device as a segment or the image plane of the spatial light modulation device to a defined position in the field of view, but the position of the virtual visible area in the observer plane does not change. For example, one grating element is arranged upstream of the Fourier plane of the spatial light modulation device in the direction of light, and the other grating element is arranged downstream of the Fourier plane in the direction of light.

[0049] Preferably, at least one combiner may comprise at least one focusing element or at least one focusing function.

[0050] At least one combiner may comprise at least one focusing element in order to direct or arrange the virtual information to be represented in a depth region at a desired depth in the field of view. In this case, the focusing element is preferably implemented in such a way that it does not impair or affect the perception of the physical environment in the field of view. For example, the focusing element may be implemented as a grating element with a limited acceptance angle, preferably a volume grating with a limited acceptance angle. In this case, the acceptance angle matches the angle of incidence of the light carrying the information and does not match the range of angles of incidence of the light from the physical environment incident on the grating element. As a result, the light from the physical environment incident on the grating element is not affected by the grating element and passes through the grating element unimpaired.

[0051] For example, the combiner may be implemented as a partially reflective mirror or light guide, with focusing elements, such as grating elements, being arranged or connected to the combiner surface.

[0052] As an alternative thereto, at least one combiner itself has a focusing function by having a curved or at least partially curved embodiment. With at least one combiner, focusing of the light or the image of the spatial light modulation device as a segment at a defined z position (along the z direction or along the optical axis direction of the optical system) can be achieved by a curved embodiment of the combiner. In other words, at least one combiner can be at least partially curved. For example, if at least one combiner is implemented as a partial reflector element, the reflector surface can be curved or arched, for example in the form of a concave mirror, and thus has a focusing function in the configuration of the invention. For example, it is also possible to combine a curved surface with an additional grating element on this surface of at least one combiner.

[0053] Thus, at least one combiner can be implemented as an eyeglass or as a windshield. In this case, it can have a flat or planar embodiment and include a focusing element. However, at least one combiner can also have an at least partially curved embodiment and thus itself serve as a focusing element or additionally be combined with a focusing element.

[0054] In an advantageous configuration of the invention, it can also be provided that a continuous movement of the at least one scanning mirror element or a stepwise movement of the at least one scanning mirror element in fixed defined increments is provided in the deflection device.

[0055] Thus, at least one scanning mirror element may be moved continuously or stepwise in order to direct the image of the spatial light modulation device as a segment to a desired position in the field of view.

[0056] For example, the stepwise movement of at least one scanning mirror element can be implemented in such a way that the scanning mirror element moves a defined angle and then stops at a defined fixed scanning mirror element position at this point, so that the control device of the device according to the invention is controlled accordingly, so that the illumination device emits light, thereby generating an image of the spatial light modulator as a segment, and guiding the segment to a position in the field of view by tracking the scanning mirror. This means that the control device controls the illumination device only in the stopped state of the scanning mirror element, so that the light emitted by the illumination device is modulated with the required information by the spatial light modulator, and an image of the spatial light modulator is generated by the optical system, which is then guided to a specified position in the field of view by the scanning mirror element. After the image of the spatial light modulator is generated and guided to the required position in the field of view, the scanning mirror element is moved by the control device by another defined angle, wherein the movement of the scanning mirror element is stopped again, so that another image of the spatial light modulator can be generated and guided to another set position in the field of view. This start-stop movement of the scanning mirror element is feasible at high speeds. Suitable mirror elements for this purpose are known. For this purpose, the illumination device of the device according to the invention should include at least one light source that can be operated in a pulsed manner. Then, the illumination device or at least one light source is activated only when the scanning mirror element is in a stopped state. If the scanning mirror element is in motion, the illumination device or the light source is in a turned-off state.

[0057] As an alternative to the stepwise movement of at least one scanning mirror element, a continuous movement of at least one scanning mirror element can also be provided. However, the continuous movement of at least one scanning mirror element also results in a continuous displacement of the image generated by the spatial light modulator. However, this is not necessary. In order to counteract this continuous movement of the image of the spatial light modulator after its generation, in the case of continuous movement of at least one scanning mirror element, it can be advantageously arranged that at least one scanning mirror element is combined with a compensating mirror element, and the compensating mirror element moves synchronously with at least one scanning mirror element, that is, in the case of the same degree of movement of the two mirror elements, the image of the spatial light modulator is generated at a fixed position, and in the case of opposite movement of the two mirror elements, the image of the spatial light modulator can be shifted in the field of view. Due to the provision of a compensating mirror element with a controllable implementation, the image of the spatial light modulator can be kept in the same required position during image generation, and thus the image is guided to a defined position in the field of view. The compensation control element can also be controlled by the control device. The two movements, namely the movement of at least one scanning mirror element and the movement of the compensating mirror element synchronized therewith, can be combined with each other so as to guide the image of the spatial light modulator as a segment to a desired position in the field of view. For example, at least one scanning mirror element can be combined with the compensating mirror element so that during the continuous movement of the scanning mirror element, the scanning mirror element reaches the position where the image of the spatial light modulator should be generated, but now due to the continuous movement of the scanning mirror element, this image of the spatial light modulator is no longer guided to the required defined position. In this case, the compensating mirror element is controlled so that the compensating mirror element performs a movement synchronized with the movement of the scanning mirror element. As a result, the generated image of the spatial light modulator as a segment is displaced or moved in a direction opposite to the movement direction of the scanning mirror element so that due to this compensating movement of the compensating mirror element, it can be displaced and guided to the required defined position in the field of view. The image of the spatial light modulator generated subsequently is guided to its defined position in the field of view in the same way. However, as long as the lighting device or at least one light source is in the on state, the synchronous movement of the compensating mirror element is maintained by controlling the control device. In other words, the same degree of movement of the scanning mirror element and the compensating mirror element can be set as long as the illumination device is in the switched-on state. When the illumination device or at least one light source is in the switched-off state, the compensating mirror element can be moved to its initial state.

[0058] Advantageously, it can be provided that, in order to generate at least two images of the spatial light modulation device as segments in the field of view within a frame, a continuous movement of at least one scanning mirror element with different predetermined speeds or a stepwise movement of at least one scanning mirror element with different adaptive increments can be provided.

[0059] When generating two or more images of a spatial light modulator as segments, the guiding of the segments to the required defined positions in the field of view by at least one scanning mirror element can be performed at different speeds, in the case of continuous movement or in the case of stepwise movement with different adaptive increments. In this case, the speed or increment of movement of at least one scanning mirror element depends on the required position of the image of the spatial light modulator as segments to be generated in the user's field of view. For example, if virtual information is represented or displayed in the left area of ​​the field of view, and likewise in the right area of ​​the field of view of the user of the device according to the invention, it is necessary to define a normal speed to guide and display the image of the spatial light modulator as segments in the left area of ​​the field of view, but for the right area of ​​the field of view, a greater speed of at least one scanning mirror element is correspondingly required to guide and display the image of the spatial light modulator as segments, because the two virtual information items are far apart in the field of view, but the user wants to observe these information items as simultaneously as possible. In order to ensure this, the scanning mirror element needs to be operated at a higher speed so that the image of the spatial light modulator as segments in the right area of ​​the field of view can be displayed as simultaneously as possible with the image of the spatial light modulator in the left area of ​​the field of view. However, if it is only necessary to display the virtual information in the left region of the field of view and the location of the virtual information is close, the at least one scanning mirror element can be operated at a low speed between two or more representations of the virtual information. In other words, it can be advantageously provided that the speed or increment of the movement of the at least one scanning mirror element can be adjusted to the defined position of the individual images of the spatial light modulation device as segments in the field of view.

[0060] Due to possible aberrations in the optical system, the size and shape of the generated segments may vary with position in the field of view. To correct for variations in the size and shape of the segments in the field of view, the speed or increment of movement of at least one scanning mirror element may also be changed accordingly.

[0061] In addition, in another configuration of the present invention, it can be arranged that the size and / or shape of at least one image of the spatial light modulation device as a segment in continuous frames is variable, or the size and / or shape of at least two images of the spatial light modulation device as segments having a defined position in the field of view within a frame or in continuous frames are variable.

[0062] In particular, a change in the size and / or shape of at least one image of a spatial light modulation device as a segment in consecutive frames is advantageous in the case where the size and / or shape of the represented scene or object also changes in consecutive frames and should be represented by a fixed number of segments.

[0063] For example, if the size of an object changes so that it can be represented using a single segment in one frame, but it is slightly larger than the segment in the next frame, it may be more advantageous to adjust the size and / or shape of at least one image of the spatial light modulator as a segment rather than to represent the object with two segments of fixed size. The change of the size or shape of at least one image of the spatial light modulator as a segment with a defined position in the field of view within a frame can be used for the same purpose, but it can also be used if a higher resolution is required in some areas of the field of view than in other areas. For example, segments with a small range and fine resolution are generated in the central area of ​​the field of view, while segments with a larger range and coarser resolution are generated in the edge areas of the field of view.

[0064] However, for example, a change in the size or shape of at least one image of a spatial light modulation device as a segment having a defined position in the field of view within a frame can also be used to simplify the optical system. This is acceptable in the case of a simple optical system, when generating at least one image of a spatial light modulation device as a segment, a change in magnification according to a change in position in the field of view or a change in optical distortion affecting the image shape can be generated.

[0065] In a particularly advantageous configuration of the invention, at least one combiner can be embodied as a partially reflective mirror element or as a light guide.

[0066] At least one combiner can be implemented as a partially reflective mirror element. For example, such a partially reflective mirror element can be a windshield or a spectacle lens in a vehicle.

[0067] Furthermore, at least one combiner can be implemented as a light guide, wherein light propagates in the light guide by total internal reflection. The light guide as a combiner is part of an optical system, which is also used to generate an image of a spatial light modulation device as a segment. In this case, light from the user's physical environment can pass through the light guide in an unimpeded manner, so that at least one image of a spatial light modulation device as a segment carrying virtual information generated by the device according to the invention is superimposed on the user's physical environment in the field of view.

[0068] Advantageously, in the configuration according to the invention it can be provided that the deflection device is embodied as a switchable coupling element for coupling light into the combiner in the form of a light guide and / or as a decoupling element for decoupling light from the combiner in the form of a light guide.

[0069] In the configuration of the device according to the present invention, the combiner is implemented as a light guide, and the deflection device for guiding the image of the spatial light modulation device to a defined position in the field of view can be implemented as a switchable coupling element and / or additionally as a switchable decoupling element. For example, a switchable decoupling element can be used, which decouples the light from the light guide at different positions of the light guide, thereby generating an image of the spatial light modulation device as a fragment located at different positions of the field of view. For example, a light guide as described in patent document WO2018 / 146326A1 can be used as the light guide, and the patent disclosure is intended to be fully incorporated herein. The light propagates in the light guide by reflecting on the boundary surface of the light guide, and after a predetermined number of reflections of the light on the boundary surface of the light guide, the light is decoupled from the light guide. The optical decoupling device can be designed to be controllable, wherein the optical decoupling device is controlled in the following manner, that is, the light is decoupled after a predetermined number of reflections in the control state of the optical decoupling device, and the light continues to propagate in the light guide in another control state of the optical decoupling device. In addition, patent document WO2018 / 146326A1 describes a display device, specifically a near-eye display device, which includes an illumination device having at least one light source, at least one spatial light modulator, an optical system and such a light guide device. For image representation or for individual fragments of multiple images of the spatial light modulator, decoupling of light from different pixels of the spatial light modulator after entering the light guide can be provided after all pixels each perform the same number of reflections on the boundary surface of the light guide. For different fragments of multiple images, the number of reflections of light on the boundary surface of the light guide used to generate one fragment may be different from the number of reflections of light on the boundary surface of the light guide used to generate another fragment. For different fragments of multiple images, the number of reflections of light on the boundary surface of the light guide may be equal, and for these fragments, the coupling position of light with the light guide may be different. In order to move the coupling position of light with the light guide, a light deflection device may be arranged upstream of the light guide in the direction of the light.

[0070] In this case, the light deflecting device for displacing the light coupling position and / or the decoupling element for decoupling light from the light guide is controlled only if a position in the field of view connected to the decoupling position requires an image of the spatial light modulation device as a segment.

[0071] In a further configuration of the invention, in which at least one combiner is likewise embodied as a light guide, the deflection device can have the at least one scanning mirror element already mentioned.

[0072] The optical system and the light guide are implemented in the following manner, that is, the light beams emitted from the individual pixels of the spatial light modulation device are incident and coupled into the light guide at overall different angles relative to the surface of the light guide, thereby defining a coupling angle spectrum, and the light beams propagating in the light guide can be decoupled from the light guide at overall different angles relative to the virtual visible area, thereby defining a decoupling angle spectrum.

[0073] In particular, in a preferred configuration of the present invention, the light guide can be implemented in the manner disclosed in patent document WO2019 / 012028A1, the content of which is intended to be incorporated herein in its entirety. Therein is described a light guide that increases the decoupling angular spectrum of light compared to the coupling angular spectrum of light. However, in another configuration of the present invention, the decoupling angular spectrum of light and the coupling angular spectrum of light can also be of the same size.

[0074] Preferably, at least one scanning mirror element can be arranged in the optical path between the spatial light modulation device and the light guide acting as a combiner. In this way, for example, the angle at which the light is coupled into the light guide can be changed by moving or rotating the scanning mirror element, so that the propagation angle of the light in the light guide is also changed. The light guide acting as a combiner can include passive or active decoupling elements. By means of these decoupling elements, the light can be decoupled from the light guide at different positions and / or at different decoupling angles, respectively, so that the light is guided to a defined position in the field of view.

[0075] For example, instead of using a spatial light modulation device with a large number of pixels (e.g. more than 1000 pixels in one direction), a spatial light modulation device with a relatively small number of pixels (e.g. less than 1000 pixels in one direction) may be used in conjunction with a light guide as a combiner and at least one scanning mirror element. In devices according to the prior art, a spatial light modulation device with a large number of pixels will produce a specific coupling angle spectrum into the light guide, wherein the field of view obtained by the device will be proportional to the coupling angle spectrum.

[0076] According to the invention, preferably a spatial light modulator with a small number of pixels, for example several hundred pixels in one direction, is used. The spatial light modulator generates a small coupling angular spectrum for each segment generated in the field of view. At least one scanning mirror element is implemented so that for each segment containing virtual information to be generated, the central angle of coupling of its light with the light guide is different. In this way, the same light guide can be used to generate images of the spatial light modulator as segments located at different positions in the field of view, in which virtual information is displayed to the user. In this case, each generated segment has a field of view itself, which is proportional to the small coupling angular spectrum, and the combination of multiple segments, when considered as a whole, correspondingly produces a large angular spectrum.

[0077] It is not intended to limit the present invention to a certain type of spatial light modulation device. Various types or combinations of more than one spatial light modulation device may also be used in the present invention. Preferably, the spatial light modulation device may include an LCoS-SLM or a MEMS-SLM.

[0078] LCoS-SLM is a spatial light modulation device with a large number of pixels but a relatively low frame rate. In contrast, MEMS-SLM has only a small number of pixels but a relatively high frame rate.

[0079] For a spatial light modulator with a large number of pixels, the spatial light modulator can be subdivided into virtual areas or parts. In order to generate an image of the spatial light modulator as a fragment containing virtual information for the user, only the virtual areas or parts of the spatial light modulator are illuminated, rather than the entire spatial light modulator. Each pixel of the spatial light modulator that contributes to the representation of virtual information, or in another embodiment, all pixels are assigned to at least one virtual area or part. The virtual areas or parts may also overlap on the spatial light modulator. In the case of overlapping virtual areas or parts, a pixel of the spatial light modulator may also be assigned to more than one virtual area or part. For example, the spatial light modulator may be an LCoS-SLM with a pixel count of 4000×2000 pixels. By subdividing the LCoS-SLM into virtual areas or parts, images of these areas of the LCoS-SLM as fragments with a size of 400×400 pixels can be generated by only illuminating these virtual areas or parts respectively.

[0080] For each image, data is written to all pixels of the spatial light modulator, or precisely only to those pixels that should contribute to the representation of virtual information. In this case, writing the data can be achieved, for example, by line scanning on the spatial light modulator. In order to generate an image of the spatial light modulator as a segment in the field of view, virtual areas or parts of the spatial light modulator (e.g., LCoS) are continuously illuminated. To this end, a deflection device including at least one scanning mirror element can be arranged in the optical path between the illumination device and the spatial light modulator, and the light can be guided to the respective virtual areas or parts of the spatial light modulator by means of a control device through a defined movement of at least one scanning mirror element. To this end, the workflow for illuminating the virtual areas or parts of the spatial light modulator can be matched with the workflow for writing data to the pixels of the spatial light modulator. The areas or parts of the spatial light modulator can have the same size or have different sizes.

[0081] In the case of continuous movement of at least one scanning mirror element, the virtual areas or parts of the spatial light modulation device can also have a relatively large overlap. For example, virtual areas or parts arranged adjacent to each other can have an overlap of only one pixel or an overlap of only a few pixels. When the spatial light modulation device is scanned, the lighting device is only in the on state when it is in the pixel or virtual area representing virtual information in the user's field of view. If the pixels or virtual areas on the spatial light modulation device that do not contribute to the representation of virtual information are scanned, the lighting device is in the off state.

[0082] However, it is also possible to use spatial light modulation devices, such as MEMS-SLMs, which only have a relatively small number of pixels. In this case, the number of pixels can be lower than a typical resolution of a display device or display, for example less than 640×480 pixels (VGA), for example 200×200 pixels or 300×200 pixels or else 400×400 pixels, wherein the number of pixels of the spatial light modulation device of the device according to the invention should not be interpreted as being limited to these disclosed numbers of pixels.

[0083] For spatial light modulators with a relatively small number of pixels, the deflection device can preferably be arranged in the optical path between the spatial light modulator and at least one combiner. In this configuration of the present invention, it can be preferably arranged that the image of the spatial light modulator as a fragment corresponds to the image of the entire area of ​​the spatial light modulator. Therefore, all generated fragments preferably have the same size. The images of the spatial light modulator as fragments are represented or displayed in a chronological order in the user's field of view. This effect is achieved by writing information, such as an encoded hologram, into the spatial light modulator and imaging the spatial light modulator by an optical system, wherein the corresponding images of the spatial light modulator as fragments are each guided to different defined positions in the field of view by means of a deflection device (for example, by means of at least one scanning reflector element).

[0084] In a particular configuration of the invention, it can be provided that the additional information is represented stereoscopically in the field of view of a user of the device according to the invention. This means that a flat or planar two-dimensional image is generated and displayed, wherein images of the represented information are generated for the left eye and the right eye of the user. In turn, this means that one display device according to the invention is provided for generating an image for the left eye and one display device according to the invention is provided for generating an image for the right eye of the user. Two display devices according to the invention can be combined with each other, for example in the form of a pair of glasses.

[0085] It is also possible to provide a display device according to the invention for both eyes, for example using an illumination device and a spatial light modulation device, in which an optical system is arranged to deflect light to the left eye and to deflect light to the right eye of the user by time division or space division multiplexing. For example, in this case, it is also possible to assign a separate combiner to each eye of the user, such as a separate light guide for the left eye and a separate light guide for the right eye, wherein the light from the spatial light modulation device is coupled into one or the other light guide in a time-sequential manner by a switching element. In other applications, such as head-up displays, it is also possible to provide a common combiner for both eyes of the user; in this case, for example, the combiner can be a windshield of a vehicle or a vehicle.

[0086] In a further advantageous configuration of the invention, it can be provided that the optical system comprises a zoom system which is able to set the distance between at least one image of the spatial light modulation device as a segment in the field of view and the user.

[0087] The device according to the invention may have a zoom configuration. This means that the virtual information generated in the field of view can be adjusted in depth along the optical axis of the optical system of the device. Depth or depth position is understood as the distance of the virtual information (i.e. the generated image of the spatial light modulation device as a segment) from the observer plane where the user's eyes are located. Therefore, the zoom system of the optical system can be used to set the distance of the segment with virtual information according to the desired depth position in the field of view.

[0088] Preferably, the zoom system may include at least one grating element with a controllable grating period or a combination of active and passive imaging elements. For example, the grating element may be a liquid crystal grating element. Furthermore, the zoom system may include a combination of an adjustable grating element and a passive imaging element such as a lens element. The zoom system may also include a switchable grating element, such as a switchable polarization grating, or a passive grating combined with a polarization switch.

[0089] The zoom system is preferably arranged near or in the Fourier plane of the spatial light modulation device, but other arrangements are also possible. The Fourier plane of the spatial light modulation device is formed in the optical path, for example between the spatial light modulation device and at least one combiner.

[0090] If the zoom system comprises at least one grating element with a controllable grating period, it can be provided according to the invention that the at least one grating element with a controllable grating period has a prism function and / or a phase function for correcting aberrations caused by the optical system.

[0091] In addition to the lens function, the at least one grating element may also have other functions for correcting aberrations, such as a prism function or a phase function. These functions may be written into the at least one grating element.

[0092] In a particularly advantageous configuration of the invention, a gaze tracking system can be provided for detecting the viewing direction of the user.

[0093] Providing a gaze tracking system allows determining what the user is looking at in the field of view at a particular time or representing virtual information in the field of view, or determining which part or object of real information is currently of interest to the user and is therefore being focused on by the user. Thus, the depth position of one or more objects of interest in the z direction that are actively observed or focused on by the user is determined. For the real information, the depth position of the real information (i.e., the distance from the user's eyes) can be detected by an additional sensor. Virtual information or content displayed in the environment of the real information focused in the field of view should be represented at the same depth as the depth of the real information focused on by the user, virtual information related to the real information focused in the field of view, or other important virtual information (e.g., warnings). Using a zoom system, the depth position of the image of the spatial light modulation device as a segment can be moved in the z direction to the depth position at which the user is currently actively focusing. Virtual information within the field of view that the user is not focusing on or looking at can be influenced by a software system so that the information is represented in a manner that is, for example, slightly out of focus or slightly blurred or distorted, or can be chosen not to be displayed at all.

[0094] Furthermore, in addition to or as an alternative to the gaze tracking system, a detection device may be provided for determining the region of the field of view in which the virtual information should be represented.

[0095] In the user's field of view, the detection means determines in which region of the field of view one or more virtual information items should be generated, displayed or represented.

[0096] The illumination device of the display device according to the present invention may include at least one light source controlled in a pulsed manner.

[0097] Furthermore, the object of the invention is also achieved by a method having the features of claim 28 .

[0098] The method according to the invention has the following characteristics:

[0099] - controlling a control device connected to an illumination device for emitting light and a deflection device of an optical system of a display device, operating the illumination device according to the control of the deflection device so as to direct at least one image of the spatial light modulation device as a segment to a defined position in a field of view of a user,

[0100] - directing the light to a spatial light modulation device and generating at least one image of the spatial light modulation device as a segment by an optical system,

[0101] - directing the image of the spatial light modulation device as a segment to a defined position in the field of view of the user by means of a deflection device, and

[0102] -Represent virtual information as fragments in the user's field of view.

[0103] The method according to the invention facilitates energy-efficient representation of information in the user's field of view, since information is represented and displayed to the user only when required.

[0104] Preferably, the detection device can be used to determine, for each frame of a sequence of images to be represented by the display device, which parts or areas of the field of view should be filled or displayed with virtual information (e.g. two-dimensional and / or three-dimensional objects or scenes), and which parts or areas of the field of view should not contain virtual information.

[0105] In this case, the optical system can be used to generate at least two images of the spatial light modulation device and a virtual visible area corresponding to the number of images of the spatial light modulation device, wherein at least two images of the spatial light modulation device are formed as fragments in the user's field of view, and the at least two images are preferably combined with each other or overlap with each other or separated by a gap.

[0106] Advantageously, the at least one combiner of the spatial light modulation device can superimpose real information in the field of view on virtual information generated additionally in the field of view by displaying at least one image of the spatial light modulation device as a segment.

[0107] At least one image of the light modulator as a fragment can be generated according to the required position in the field of view. At least one image of the spatial light modulator can be dynamically generated for each frame, so that the generated image of the spatial light modulator as a fragment depends on the position of the virtual information represented by the respective frame in the field of view. In this case, the image of the spatial light modulator as a fragment can be generated in such a way that, for a single frame or a single image, the virtual information in the form of an object is located within the minimum number of images of the spatial light modulator as a fragment. That is, in order to represent virtual information in the form of navigation suggestions, for example, only three images of the spatial light modulator as fragments are required. However, in order to represent virtual information of different objects, for example, seven images of the spatial light modulator as fragments may be required. Therefore, the number of images of the spatial light modulator to be generated required to represent virtual information in the form of an object also depends on the size of the object to be displayed. For example, for virtual information in the form of an object whose size is smaller than the image range of the spatial light modulator, the image of the spatial light modulator can be generated in such a way that the entire object is generated and represented in the field of view by only one image of the spatial light modulator as a fragment. To this end, for example, the center of the object can coincide with the center of the image of the spatial light modulator as a fragment.

[0108] In a configuration of the present invention, the field of view can be subdivided into grid domains, wherein each frame is checked to determine in which grid domain of the field of view virtual information should be represented, wherein the spatial light modulator and at least one scanning mirror element of the deflection device are controlled in the following manner, i.e., so that for each frame, an image of the spatial light modulator is generated as a fragment only in the grid domain where the virtual information should be represented and the image is guided to a defined position in the field of view.

[0109] The field of view can be implemented in the form of a grid arrangement with multiple grid domains. This grid arrangement can be fixedly defined and therefore the same for each frame. However, the number of segments generated as an image of a spatial light modulator representing virtual information is less than the total number of segments required to generate the entire field of view. In this configuration of the present invention, the time for generating each image of the spatial light modulator as a segment is 1 / M of the total image time, where M is the number of segments containing virtual information. In the example of the field of view of 60°×60° mentioned above, 144 segments must be generated to generate the entire field of view, and the number of segments of the image of the spatial light modulator containing virtual information will be M=30. Accordingly, this will allow the spatial light modulator to be controlled at a lower frame rate. However, this will require the optical system to have a more flexible configuration. For example, the image of the spatial light modulator as a segment can or must be generated at different positions with gaps or distances in the field of view. In this case, the size of the gap or distance may be different from frame to frame. Therefore, it depends on the position of each segment in the field of view so that the segments of the left field of view area and the segments of the right field of view area can be generated in a relatively large interval. To this end, for example, in order to connect gaps of different sizes, the scanning mirror elements of the deflection device must be moved at different speeds. The number M of segments containing virtual information can also vary from frame to frame.

[0110] In an optional configuration of the present invention, the field of view can be subdivided into grid domains, each grid domain being scanned continuously by at least one scanning mirror element of the deflection device, the grid domain of the field of view in which virtual information should be represented is checked for each frame, and an image containing the virtual information of the spatial light modulation device as a segment is generated and is distributed by the optical system only to the individual grid domains that should also represent virtual information.

[0111] In this optional embodiment, the field of view can also be implemented in the form of a grid arrangement with multiple grid domains. This grid arrangement can be fixedly defined and is therefore the same for each frame. The number of fragments generated as an image of a spatial light modulator representing virtual information is also less than the total number of fragments required to generate the entire field of view. However, in this case, all grid domains of the grid arrangement are scanned continuously, however, only if virtual information should be represented in the corresponding grid domain currently to be scanned, an image of the spatial light modulator is generated by controlling the illumination device, the spatial light modulator and the deflection device. Individual images of the spatial light modulator as fragments are generated in a time sequence, in which virtual information should be displayed. In this configuration of the present invention, the time used to generate the individual images of the spatial light modulator as fragments is 1 / N of the total image time, where N is the total number of fragments.

[0112] Therefore, for the grid domains that should not represent virtual information, the lighting device is not controlled and therefore no image of the spatial light modulation device is generated. Therefore, no data is transmitted to the spatial light modulation device either, and the data transmission is reduced. If the lighting device is still activated, it is also possible to globally reset to a level in all pixels assigned to the grid domains that should not display virtual information, at which level the liquid crystal layer of the spatial light modulation device is controlled to return the liquid crystal to the initial state. Another option is to convert these pixels to an undefined state.

[0113] At least one scanning mirror element of the deflection device can be moved stepwise, either continuously or in defined increments, to direct at least one image of the spatial light modulation device as a segment to a defined position in the field of view.

[0114] In the case of a stepwise movement of the at least one scanning mirror element, it can be provided that, when the at least one scanning mirror element is in a holding state or a stopped state after a defined increment, the illumination device is respectively activated and the spatial light modulation device is illuminated to generate an image of the spatial light modulation device, so that the generated image of the spatial light modulation device as a segment is directed to a defined position in the field of view. When the at least one scanning mirror element is in a moving state, the illumination device is deactivated.

[0115] Therefore, the lighting device is controlled in conjunction with the control of the deflection device (specifically at least one scanning mirror element), and the lighting device is turned on and off according to whether the at least one scanning mirror element is in a stopped state or a moving state. In this case, both the lighting device and the deflection device are controlled by the control device.

[0116] In case of a continuous movement of the at least one scanning mirror element, the compensating mirror element can be combined with the at least one scanning mirror element, wherein the compensating mirror element and the at least one scanning mirror element perform a synchronous movement, preferably an identical movement, when the lighting device is in the switched-on state.

[0117] The synchronous movement of the compensating mirror element for the movement of the at least one scanning mirror element is only performed when the lighting device is in the switched-on state or activated. In this configuration of the invention according to the invention, the lighting device is also connected to the deflection device, wherein the deflection device comprises a compensating mirror element in addition to the at least one scanning mirror element. The compensating mirror element may also not be a component of the deflection device. If this is the case, the lighting device is not only connected to the deflection device, but also to the compensating mirror element. Accordingly, the control device controls the lighting device and the deflection device and optionally controls the compensating mirror element in the case where the compensating mirror element is not a component of the deflection device.

[0118] At least one scanning mirror element and at least one compensating mirror element can also be implemented to be movable in two dimensions, preferably horizontally and vertically. Due to the movement of the scanning mirror element and the compensating mirror element both in the horizontal direction and in the vertical direction, the generated image of the spatial light modulator as a segment is guided to a defined horizontal and vertical position in the field of view. By a movement in one direction (e.g. horizontally), but an opposite movement in a direction perpendicular thereto (e.g. vertically), the position of the image of the spatial light modulator as a segment is moved in one direction (e.g. vertically), but remains unchanged in a direction perpendicular thereto (e.g. horizontally).

[0119] The scanning mirror element may be moved back and forth continuously between a minimum setting and a maximum setting, such as continuously moving from left to right and then continuously moving from right to left while slowly moving from top to bottom. The scanning mirror element may be moved back to its initial position after the frame ends.

[0120] However, the two-dimensional continuous movement of the at least one scanning mirror element can also be implemented, for example, in the form of a Lissajous picture, so that after a frame the initial state of the scanning mirror element is restored.

[0121] There can be a calibration which is used to determine which settings of the scanning mirror element and of the optional compensating mirror element correspond to that position of the image of the spatial light modulation device as a segment in the field of view. For example, in the case of a stepwise movement of the scanning mirror element, if the scanning mirror element is equipped with a stepper motor and is controlled, a certain number of steps of the stepper motor can be assigned to a position in the field of view.

[0122] In the case of continuous movement of the scanning mirror element, the position in the field of view is assigned by means of the movement speed and the time interval using a calibration device.

[0123] For example, the calibration data may be stored in a look-up table and the look-up table may be used by the control means to control the scanning mirror element.

[0124] Preferably, at least one combiner can be implemented as a light guide, wherein the spatial light modulation device is illuminated by the illumination device and the light modulated by the spatial light modulation device is guided to the deflection device, which deflects the light onto the combiner, and the combiner is implemented as a light guide, wherein the light is coupled into the combiner and propagates in the combiner, wherein the light propagating in the combiner is decoupled according to a desired defined position in the field of view, and at least one image of the spatial light modulation device as a segment is guided to the defined position.

[0125] In a particularly advantageous configuration of the invention, at least one image of the spatial light modulation device as a segment can be moved in the z direction along the optical axis of the optical system to a depth position in the field of view accepted by the user by means of a zoom system.

[0126] Moving the image of the spatial light modulation device as a segment to the depth position at which the user of the device according to the present invention focuses or the depth position at which the user is gazing can be implemented very accurately by the zoom system, especially for those segments arranged in the user's viewing direction or near the user's viewing direction. The segments represented and displayed in the field of view away from the user's viewing direction can be arranged at any fixed defined depth or at the same depth as the depth of the segments located in the user's viewing direction. However, these segments arranged at a fixed defined depth or at the same depth as the depth of the segments in the user's viewing direction have lower precision, that is, have some tolerance. This may be relevant when the same setting parameters of the zoom system are used for all generated images of the spatial light modulation device as segments. In this case, when the spatial light modulation device is imaged for different images of the spatial light modulation device as segments, the depth position of the image of the spatial light modulation device may vary depending on the aberrations of the optical system (e.g. field curvature). By setting the correct depth position of the image of the spatial light modulator as a segment in the viewing direction according to the user's gaze depth determined accordingly by means of a gaze tracking device and allowing some tolerance for the depth position of the image of the spatial light modulator as a segment away from the user's gaze direction, a zoom system can be advantageously used which has a lower frame rate than that required by the deflection device.

[0127] In one configuration of the present invention, the device according to the present invention can be implemented as a stereoscopic display device or a variable focus stereoscopic display device, in which an amplitude modulation spatial light modulation device that writes two-dimensional amplitude data is used.

[0128] In another embodiment of the stereoscopic display device or the zoom stereoscopic display device, the spatial light modulator can be implemented as a complex-valued spatial light modulator. For example, this can be a phase-modulating spatial light modulator combined with a beam combiner. In this case, the two-dimensional information to be represented is also written to the spatial light modulator via amplitude data. The ability of the spatial light modulator to modulate the phase of the light can then be used to write a phase function, for example for aberration correction.

[0129] At least one grating element with a lower frequency (e.g., 50 Hz-200 Hz) and a controllable and adjustable grating period can be used in combination with a spatial light modulation device that operates at a higher frequency and is implemented as a MEMS-SLM, and can be combined with a static optical element for aberration correction. In this way, the static optical element can perform aberration correction on the entire display device. At least one grating element with a controllable and adjustable grating period can have the same lens function for all images of the spatial light modulation device as segments, so that the depth position of all segments moves, but can also provide aberration correction for the limited depth position of the segment. In this case, the aberration correction for all segments is the same. The fast MEMS-SLM can also perform separate aberration correction on each image of the spatial light modulation device as a segment, because the two-dimensional image information and phase function in each segment can be updated for aberration correction.

[0130] A scattering device or diffuser may be provided in the stereoscopic display device. For example, the scattering device may be arranged near the spatial light modulator or in the intermediate image plane of the spatial light modulator. The scattering device may be used to extend the area of ​​the sweet spot so that a large virtual visible area may be created on the observer plane.

[0131] In another configuration of the present invention, it can be arranged that virtual information is generated and represented in a holographic manner in the field of view of the user. To this end, the spatial light modulation device can be implemented as an amplitude modulation, phase modulation or complex valued (amplitude and phase) spatial light modulation device, in which the holographic data or the encoded hologram is written. Preferably, the spatial light modulation device has a complex valued embodiment, for example, a phase modulation spatial light modulation device combined with a beam combiner. In another configuration, it can be implemented as a phase modulation spatial light modulation device, in which the calculated hologram is iteratively written, for example using the Gerberg-Saxton method.

[0132] Holographic display devices generally do not require a zoom system, since the three-dimensional information to be represented is already generated with complete depth information by the hologram encoded in the spatial light modulation device. However, in some cases, it is advantageous to provide a gaze tracking device and / or a zoom system in the holographic display device, so that the depth position of the image of the spatial light modulation device is moved or placed in a defined position. For example, by appropriate selection of the depth position of the image of the spatial light modulation device, the complexity of the hologram calculation can be reduced.

[0133] In the configuration of the holographic display device, at least one grating element with a controllable and adjustable grating period can also be provided, which grating element does not change the position of the image of the spatial light modulation device as a segment, but is provided for aberration correction.

[0134] In a manner similar to a zoom system, the holographic display device again allows the aberration correction to be combined with static optical elements or to be performed using at least one grating element with a controllable and adjustable grating period, wherein the grating period can be different for each frame and depends on the information represented in the respective frame. In addition, the aberration correction can be performed directly in the spatial light modulation device so that the correction of the aberration is already taken into account and included when calculating the hologram. The hologram encoded in the spatial light modulation device can be different for each image to be generated by the spatial light modulation device as a segment carrying virtual information.

[0135] If a grating element with a controllable and adjustable grating period can be operated at a sufficiently high frequency, the grating element can also perform different aberration corrections for each image of the spatial light modulation device as a segment.

[0136] Static aberration correction can also be performed if the light emitted from the spatial light modulation device is incident on at least one combiner at a larger tilt angle, within which the spatial light modulation device is tilted relative to the optical system of the display device according to the invention during imaging by the spatial light modulation device.

[0137] There are now various possibilities for advantageously configuring the teaching of the invention and / or for combining the exemplary embodiments or configurations described with one another. For this purpose, reference is firstly made to the patent claims that are subordinate to the optional independent claims and secondly to the following description of preferred exemplary embodiments of the invention with the aid of the drawings, wherein preferred configurations of the teaching are also generally described. The invention is explained in principle with the aid of the described exemplary embodiments, but is not intended to be restricted to these.

[0138] In the figure:

[0139] Figure 1 : shows a rough representation of an AR display device, in the form of a pair of glasses, in which only the user's field of view is displayed;

[0140] Figure 2 : A rough representation of a display device according to the present invention is shown in a top view;

[0141] Figure 3 : A rough representation of another embodiment of a display device according to the present invention is shown in a top view;

[0142] Figure 4 :shows the Figure 1 A coarse representation of the user's field of view subdivided into a grid domain;

[0143] Figure 5 : shows a rough representation of an image of a spatial light modulation device as segments in a field of view, the segments each containing virtual information of a user;

[0144] Figure 6 : shows another rough representation of the image of the spatial light modulation device as segments in the field of view, each segment containing virtual information of the user;

[0145] Figure 7 : shows a rough representation of a deflection device according to the present invention in various control states, wherein the deflection device is arranged in a display device according to the present invention;

[0146] Figure 8 : shows a rough representation of coupling light into a light guide when using a spatial light modulation device with a large number of pixels according to the prior art; and

[0147] Fig. 9 : shows a rough representation of a display device according to the invention, which comprises a combiner implemented as a light guide and which is used to generate at least two images of the spatial light modulator as fragments in the field of view when a spatial light modulator with relatively few pixels is used.

[0148] It should be briefly mentioned that in the figures identical elements / parts / components also have the same reference numerals.

[0149] Figure 1 A display device according to the invention is shown, which is implemented here as an augmented reality (AR) display. In this case, the AR display device is implemented in the form of a pair of glasses, so that the display device is implemented as an AR head-mounted display, showing with a view what a user B of the AR glasses can observe through the glasses in his field of view S. For example, for the sake of clarity, user B is represented only by two arms holding the handlebars of a bicycle. The display device in the form of a pair of AR glasses is fixed to the head of user B. Therefore, user B looks through the AR glasses and observes his natural or physical environment R through them. Therefore, Figure 1 Only user B’s field of view S is shown. In their field of view, user B sees Figure 1 , a street scene with buildings, roads, and traffic, which the user sees with both eyes. The shape of the illustrated lenses should not be related to any specific type of AR glasses, but only serves as an example of how a pair of AR glasses can be configured. Therefore, it goes without saying that other shapes of AR glasses are also possible. In addition, this means that although the display device shown in the figure of this example is embodied in the form of a pair of glasses, other applications are also possible, such as as a head-up display.

[0150] Furthermore, virtual information items C1, C2 and C3 are represented or displayed in the field of view S of the user B by means of a display device, said virtual information being superimposed on the physical environment R and being able to be displayed to the user B in addition to the physical environment R. The virtual information displayed in the field of view S are traffic signs C1, information about shops C2 located in buildings, and arrows C3 used as navigation tools to indicate the direction of the street. Thus, the represented virtual information items C1, C2 and C3 only fill a small part of the field of view S, which means that only a small part of the field of view S is formed by the virtual information. The main field of view S of the user is formed by the content of the physical environment R.

[0151] Figure 2 A possible configuration of a display device is described. This configuration can be used for both an AR head-mounted display and a head-up display. In the following, the display device is implemented as an AR head-mounted display to establish a connection with Figure 1 contact.

[0152] The display device comprises an illumination device 10, which may have at least one light source, wherein three light sources according to the primary colors RGB (red, green, and blue) may be provided for the color representation of the virtual information. A spatial light modulation device 11, hereinafter referred to as SLM, is arranged after the illumination device 10 in the direction of the light so that the illumination device illuminates the SLM. The SLM 11 is implemented as an SLM with a relatively small number of pixels, for example less than 1000 pixels in one direction. After the SLM 11 in the direction of the light are a deflection device 12 and a combiner 13, both of which are components of the optical system of the display device. In this case, the deflection device 12 comprises a scanning mirror element 12-1, which is arranged in a movable manner and can be moved or rotated around its rotation axis. The deflection device 12 having the scanning mirror element 12-1 is arranged in the optical path between the SLM 11 and the combiner. The scanning mirror element 12-1 can perform continuous movement or other stepwise movements with fixed defined increments, in which way the incident light can be deflected in a certain direction. According to Figure 1The combiner 13 implemented as an eyeglass is used to superimpose the virtual information generated by the display device on the information of the physical environment in the user's field of view. The combiner 13 is implemented in such a way that light from the physical environment can pass through the combiner unhindered (i.e. unaffected by the combiner). The combiner 13 can have a flat or planar embodiment or a curved embodiment. Figure 2 As shown, the optical system can include further imaging elements, for example an imaging element 14 which is implemented as a passive lens element in this case.

[0153] Furthermore, the display device comprises a control device 15, which is connected to the illumination device 10 and the deflection device 12. Thus, the illumination device 10 can be controlled according to the control of the deflection device 12 (especially in the case of the scanning mirror element 12-1) and can be switched accordingly, i.e. into an on state and an off state. The control device 15 can also be connected to the SLM 11. However, the SLM 11 can also be operated by its own control device to write data.

[0154] The following describes the process of generating virtual information (such as Figure 1The general process when the virtual information C1 in the image is displayed. The virtual information should be generated in a holographic manner, but the generation of stereoscopic information is naturally also feasible. If the lighting device 10 controlled by the control device 15 has entered the corresponding on state, the lighting device 10 emits light that is substantially coherent enough and incident on the SLM 11, wherein the virtual information data is transmitted or transferred to the SLM 11. The light emitted by the lighting device 10 and incident on the SLM 11 is represented here by an arrow. The light modulated by the SLM 11 with the virtual information to be represented passes through the imaging element 14, thereby generating an image of the SLM 11 on the scanning mirror element 12-1 of the deflection device 12. The deflection device 12 is arranged in the Fourier plane of the SLM 11. Before the lighting device 10 is controlled, the scanning mirror element 12-1 is controlled by the control device 15 so that it has moved to the position required to represent the virtual information at a defined position of the field of view S of the user B, who is represented here by the eye. In order to determine the position to which the scanning mirror element 12-1 must be moved in order for the virtual information in the field of view S to also represent the correct position, a detection device 16 is used before the virtual information is generated to determine which area of ​​the field of view S should be filled with virtual information (e.g. a two-dimensional or three-dimensional object or scene) for each frame and which area of ​​the field of view S does not contain virtual information but only information from the physical environment of the user B. The image of the SLM 11 as a segment S1 is now directed by the scanning mirror element 12-1 to the direction of the combiner 13, which superimposes the image of the SLM 11 as a segment S1 on the physical environment. Furthermore, the image of the SLM 11 as a segment S1 is imaged by the combiner 13 in the observer plane 17 in order to generate a virtual visible area 18 in the observer plane 17. The virtual visible area 18 can be a virtual observer window in the case of a holographic display device or a sweet spot in the case of a stereoscopic display device. In this way, the virtual information is represented and displayed at a defined position in the field of view S. In order to be able to observe the virtual information in the field of view S, the user B must position his eyes in the observer plane 17 and look through the virtual visible area 18 .

[0155] The same described process can be performed in order to represent further virtual information in the field of view S of the user B. Thus, images of the SLM 11 as segments S2 and S3 can be generated and directed to the required and defined positions in the field of view S by means of the scanning mirror element 12-1, these segments S2 and S3 being superimposed on the physical environment by means of the combiner 13 and being represented and displayed in the field of view S for the user B. The images of the SLM 11 as segments S1, S2 and S3 are generated and represented and displayed in the field of view S in a time sequence. However, this is carried out at a sufficiently high frequency that the eye of the user B cannot recognize this continuous generation of segments S1, S2 and S3 with the naked eye and therefore considers that the generation occurs simultaneously.

[0156] The same process is performed in subsequent frames: first, it is detected where in the field of view S the virtual information is displayed, then the virtual information is generated as segments in a time sequence and displayed in the field of view in a manner superimposed on the physical environment of user B.

[0157] In this way, a large field of view can be created even if only a few fragments containing the desired virtual information are generated.

[0158] The combiner 13 may also include a fixed focus (non-variable focus) element, such as a raster element.

[0159] Figure 2 The display device in further comprises a zoom system 19. The zoom system 19 allows the distance between the image of the SLM 11 as a segment in the field of view and the user B to change, i.e. the depth of the image of the SLM 11 as a segment in the field of view S can be set. The zoom system 19 is preferably arranged in the Fourier plane region of the SLM 11, i.e. in the Fourier plane of the SLM 11 or at least near the Fourier plane of the SLM 11, and can comprise, for example, at least one grating element with an adjustable grating period, in which a lens function is written. Preferably, the depth of the image of the SLM 11 as a segment can be set in conjunction with the detection of the viewing direction of the user B. The gaze tracking unit 20 determines the viewing direction of the user B and the depth position in the field of view where the user B is focused or the depth at which the user B is focused. Then, the generated image of the SLM 11 as a segment can be moved by the zoom system 19 to the depth position related to the user B at which the user B is concerned or focused at this time.

[0160] In a holographic display device, a zoom system is not absolutely necessary, since the virtual information is already represented holographically at its required depth. However, it is still useful to use a zoom system, for example, in order to correct aberrations of the optical system due to the movement of the image of the SLM 11 as a segment in depth or in the z-direction. The zoom system 19 may include at least one grating element with an adjustable grating period, which grating element has, for example, a prismatic function or a phase function.

[0161] In a stereoscopic display device such a zoom system 19 is preferably used in order to shift the depth of the image of the SLM as a segment and / or in order to correct aberrations of the optical system.

[0162] Figure 3 It represents another display device, which can also be implemented as an AR display device or an AR display and can be used as an AR head-mounted display and an AR head-up display.

[0163] The display device includes an illumination device 30, an SLM 31, a deflection device 32, a combiner 33 and imaging elements, of which only the imaging element 34 is shown here. The deflection device 32, the combiner 33 and the imaging element 34 are components of the optical system of the display device. In this structure of the display device, the deflection device 32, which also includes a scanning mirror element 32-1, is arranged after the illumination device 30. The imaging element 34, the SLM 31 and the combiner 33 are arranged downstream of the deflection device 32 in the direction of light. Therefore, this means that in this case the deflection element is arranged between the illumination device 30 and the SLM 31. In this case, the SLM 31 is implemented as an SLM with a relatively large number of pixels, for example, an SLM with more than 1000 pixels in one direction. The scanning mirror element 32-1 of the deflection device 32 has a movable arrangement, which can be represented by a dotted line, and can therefore be moved or rotated around its rotation axis. In this case, the scanning mirror element 32-1 can also perform a continuous movement or else a stepwise movement in fixed defined increments, with which the incident light can be deflected in a specific direction. The combiner 33 (which can also be implemented as a spectacle lens in this embodiment, but should not be interpreted as being limited) is used to superimpose the virtual information generated by the display device on the information in the physical environment of the field of view S of the user B. The combiner 33 is also implemented in the following way, that is, the light from the physical environment can pass through the combiner 33 unimpeded, that is, it is not affected by the combiner 33. The combiner 33 can have a flat or planar or curved embodiment.

[0164] In addition, the display device includes a control device 35, which is connected to the lighting device 30 and the deflection device 32. As a result, the present exemplary embodiment also provides a lighting device 30 that can be controlled based on the control of the deflection device 32 (especially in the case of the scanning reflector element 32-1), and can be switched accordingly, that is, into an on state and an off state. The control device 35 can also be connected to the SLM 31. However, the SLM 31 can also be operated by its own control device to write data. In addition, the display device can also include a gaze tracking device 39, which determines the viewing direction of the user B and the depth position in the field of view where the user B is focused or the depth of focus. If necessary, the generated image of the SLM 31 as a segment can be moved by the zoom system to the depth position related to the user B that the user B is focusing on or gazing at at this time.

[0165] Combine the following Figure 3 The display device depicted generates virtual information (eg Figure 1 The general process of virtual information C1) shown in the figure. In this case, the virtual information should also be generated in a holographic manner, and naturally it can also be the generation of three-dimensional information.

[0166] If the illumination device 30 controlled by the control device 35 enters a corresponding on state, the illumination device 30 emits light which is substantially sufficiently coherent and which is incident on the deflection device 32, in particular the scanning mirror element 32-1. Now, in this case, the deflection device 32 is arranged upstream of the SLM 31 in the direction of the light. Before the illumination device 30 is controlled, the scanning mirror element 32-1 is controlled by the control device 35 so that it has been moved to the position required for representing the required virtual information at a defined position of the field of view S of the user B, who is represented here by the eye. In order to also determine the position to which the scanning mirror element 32-1 must be moved so that the virtual information is represented at the correct position of the field of view S in this configuration of the display device, the detection device 36 is used before the virtual information is generated to determine which area of ​​the field of view S should be filled with virtual information (for example a two-dimensional or three-dimensional object or scene) for each frame and which area of ​​the field of view S does not contain virtual information but only contains information from the physical environment of the user B. The light emitted by the illumination device 30 and incident on the scanning mirror element 32-1 is also indicated here by an arrow. The light L1 reflected by the scanning mirror element 32-1 of the deflection device 32 and directed to a defined position in the field of view S is incident on the imaging element 34 which collimates the light L1. This collimated light L1 now impinges on the SLM 31, in the process only a part of the SLM 31 is illuminated. Figure 3, only the left part of the SLM 31 is illuminated by the light L1, wherein this part may be the entire left area of ​​the SLM 31, or only a part of the left area of ​​the SLM 31. This illustration should be regarded as purely exemplary. Furthermore, virtual information data is transmitted or transferred to the corresponding part of the SLM 31, namely the left part in this case. Therefore, the information of the virtual information in the field of view S to be represented by the light L1 is only located on the illuminated part of this SLM 31. The light incident on this part of the SLM 31 is modulated by the information to be represented and then incident on the combiner 33 as a segment S1. The combiner 33 now acts as an imaging element to generate an image of the SLM 31, in this case an image of a part of the SLM 31, and also superimposes this image of the SLM 31 as a segment S1 on the physical environment of the user B. The image of the SLM 31 as a segment S1 is imaged into the observer plane 37, thereby forming a virtual visible area 38. The virtual visible area 38 may be a virtual observer window in the case of a holographic display device or a sweet spot in the case of a stereoscopic display device. In this way, virtual information is represented and displayed at a defined position in the field of view S. In order to be able to observe the virtual information in the field of view S, the user B must position their eyes in the observer plane 37 and observe through the virtual visible area 38.

[0167] The same described process can be performed in order to represent further virtual information in the field of view S of the user B. Thus, by means of different positions of the scanning mirror element 32-1 of the deflection device 32, light beams L2, L3 of different directions corresponding to the required positions of the virtual information in the field of view S can be generated, for example, said light beams then impinge on different parts of the SLM 31. Thus, images of the SLM 31 as segments S2 and S3 are generated and directed to the required and defined positions in the field of view S and represented and displayed in the field of view S for the user B. Images of the SLM 11 as segments S1, S2 and S3 are generated in a time sequence and represented and displayed in the field of view S. However, this operation is carried out at a very high frequency so that the eyes of the user B cannot recognize this continuous generation of segments S1, S2 and S3 with the naked eye, and therefore perceive this generation as occurring simultaneously.

[0168] The same process is performed in subsequent frames: first, it is detected where in the field of view S the virtual information is displayed, and then the virtual information is generated in a fragmented manner in a time sequence and displayed in the field of view in a manner superimposed on the physical environment of user B.

[0169] In this way, a large field of view can be created even if only a few fragments containing the desired virtual information are generated.

[0170] The combiner 33 may also include a focus element, such as a grating element.

[0171] like Figure 3 The display device shown may also include a zoom system. In this case, the Figure 2 The zoom system 19 shown in FIG. 1 is used to implement the zoom system, so it should also be applied according to Figure 3 display device.

[0172] according to Figure 2 and Figure 3 The display device can be used according to Figures 4 to 7 and Fig. 9 The following embodiments and configurations, the specific steps of the method for generating virtual information are described in these figures.

[0173] Figure 4 Shows Figure 1 As shown in the AR glasses, user B wears the AR glasses on his head to obtain additional virtual information, which can be presented and displayed in the user's physical environment in the field of view.

[0174] Obviously, the field of view S of user B is subdivided into individual grid domains RF, which are arranged as a grid type or form a grid arrangement. In this case, the grid domains RF all have the same shape and size. In this exemplary embodiment, they have a square embodiment. Of course, the grid domains RF can also have different shapes and sizes. In addition, the size and shape of the grid domains can change as the field of view S changes.

[0175] Now, in order to generate virtual information items C1, C2 and C3 using a display device, superimpose said virtual information items on the physical environment of the field of view and display said information items to user B, the field of view S having a grid field RF is scanned, and a detection device is used to determine where in the field of view the virtual information items C1, C2 and C3 useful to user B should be represented or displayed. In other words, the grid fields RF of the field of view S where the virtual information items C1, C2 and C3 need to be represented are checked and determined. This is because only these grid fields RF of the field of view S need to be filled with appropriate virtual information, which is superimposed on the presented real information in the grid fields RF. The scanning of the field of view can be achieved in the grid fields by scanning the grid fields row by row or column by column.

[0176] Virtual information (e.g. Figure 1The embodiment of the generation and representation of virtual information C1) is as follows. For the purpose of explaining this process, a line-by-line scanning of the field of view is assumed, wherein this process can naturally also be performed column-by-column. For each frame, each grid field RF of the field of view S is tracked line by line by the scanning mirror element of the deflection device moving step by step in defined increments and the image of the SLM as a segment is assigned only to the grid field that is also to represent the virtual information. With regard to the representation of the virtual information C1, this now means that the grid field RF1 in the field of view is tracked by the scanning mirror element, wherein the fact that the virtual information should be represented in this grid field RF1 is known, because a scan of the entire field of view was previously performed and it was determined that the virtual information should be represented in this grid field. Based on this determination, the moving scanning mirror element is put into a stopped state by the control device, so that the lighting device is also controlled by the control device and enters the on state, thereby generating an image of the SLM as a segment, and assigning the image of the SLM as a segment to the grid field RF1 in combination with the SLM and the combiner and at least one imaging element of the optical system. As a result, a part of the virtual information C1 is displayed. The control device controls the scanning mirror element and the lighting device again, so that the scanning mirror element is in the on state and the lighting device is in the off state. The scanning mirror element now continues to move in defined increments to achieve tracking of the grid domain RF2, which has also been determined to contribute to the representation of the virtual information C1. The control device now controls the scanning mirror element and the lighting device accordingly again, so that the scanning mirror element enters a stopped state and the lighting device enters an on state. Therefore, an image of the SLM as a fragment corresponding to the portion of the virtual information to be represented is generated by the SLM, the combiner and at least one imaging element of the optical system, and the image is assigned to the grid domain RF2 so that the corresponding virtual information is displayed in the grid domain RF2. According to the above-disclosed method, the scanning mirror element is used to track the subsequent two grid domains RF3 and RF4, and each generates an image of the SLM as a fragment carrying the virtual information. The two images of the SLM as fragments are then assigned to the two grid domains RF3 and RF4, as shown in FIG. Figure 5 As shown. The scanning mirror element is then moved stepwise in defined increments along the row above the grid and is moved to a stop state and an on state respectively when scanning further grid domains. Since the further grid domains RF5 to RF15 of the row are determined not to need to provide or display virtual information, the control device will not control the lighting device, so the lighting device remains in the off state and no image of the SLM as a segment is generated for these grid domains RF5 to RF15. Therefore, the second row of the grid is tracked by the scanning mirror element, wherein, as shown Figure 5As shown, no SLM image as a segment is generated for the first grid domain RF16 because virtual information should not be displayed in the grid domain. For the following grid domains RF17 to RF26, the grid domains are tracked accordingly by using the scanning mirror element in the manner described above, and appropriate images of the SLM as segments are generated, and the segments are assigned to the corresponding grid domains so that the virtual information items are displayed in the grid domains. Similarly, each grid domain of the grid is tracked in turn by the scanning mirror element, and images of the SLM as segments are generated for the other grid domains RF35 and RF48 to RF51 and RF63, RF64, and the images are assigned to the relevant grid domains for display. Therefore, the grid domains RF1, RF2, RF3, RF4, RF17, RF18 and RF19 contribute to the representation of virtual information C1. The grid domains RF20 to RF26 and RF35 contribute to the representation of virtual information C2, and the grid domains RF48 to RF51 and RF63, RF64 contribute to the representation of virtual information C3. In this case, the above procedure is performed for each frame. The generation and representation of individual images of the SLM as segments is performed in a chronological order.

[0177] A method for stepwise movement of the scanning mirror element is described. However, continuous movement of the scanning mirror element is also possible. Figure 7 This is described in detail in .

[0178] Furthermore, the procedure may also be slightly modified. In this case, the field of view is initially subdivided in a grid style into grid domains RF, which are subsequently scanned, and a detection device is used to determine where in the field of view the virtual information items C1, C2 and C3 that are useful to user B should be represented or displayed. That is, the grid domains RF of the field of view S where the virtual information items C1, C2 and C3 need to be represented are checked and determined. This is because only these grid domains RF of the field of view S need to be filled with appropriate virtual information, which is superimposed on the presented real information in this grid domain RF. In this process, the scanning of the field of view can also be achieved in the field of view row by row or column by column. However, unlike Figure 5 Unlike the above-described exemplary embodiments of the invention, all grid fields RF of the grid are not tracked sequentially in a row-by-row or column-by-column manner by the scanning mirror element, but only those grid fields RF which represent and display the virtual information items C1, C2, C3 or contribute to the representation of the virtual information C1, C2, C3 are tracked. This is because only for these grid fields RF the control device must respectively control the scanning mirror element and the illumination device in order to be able to generate and represent an image of the SLM as a segment. This process is more efficient in generating and representing virtual information.

[0179] Figure 6Another exemplary embodiment related to the process of generating and representing virtual information items is shown. In this case, as described in accordance with Figure 4 and Figure 5 As explained in the embodiment of FIG. 1 , the images of the SLM as segments for representing the virtual information items are not implemented on a grid or a fixed grid, but can be freely selected in the field of view of the user B of the display device. In this way, the images for representing the virtual information items can be reduced. Figure 4 and Figure 5 The number of images of the SLM as fragments of virtual information C1, C2 and C3 or virtual information with the same content. This means that Figure 6 In the method, a smaller number of SLM images as fragments are used to generate virtual information items C1, C2 and C3. Specifically, only 17 SLM images as fragments are used instead of using Figure 5 21 SLM images as fragments in . Therefore, for each frame, images of the SLM as fragments for the corresponding virtual information C1, C2 or C3 are generated only at those positions in the field of view where this information is also needed. To this end, the images of the SLM as fragments are generated in the following way, that is, if the range of the object is larger than the entire image of the SLM, the entire image of the SLM is set as the virtual information of the object, or if the range of the object is smaller than the image of the SLM, the virtual information of the object is fully provided in the image of the SLM. For example, the center of the object can coincide with the center of the image of the SLM. This can achieve that only the minimum image of the SLM as fragment is used to represent the required virtual information.

[0180] For example, the generation and representation of virtual information C1 in the field of view is described below. The detection device is used to determine where in the field of view the virtual information C1 should be represented and displayed. Therefore, the number of images of the SLM as segments must be determined, and the information C1 to be represented is to be generated and represented with as few images as possible or even with only one image of the SLM as a segment. If the appropriate number and required positions of the corresponding images of the SLM as segments are determined in the field of view S, the scanning mirror element of the deflection device is moved by the control device to the relevant position of the image of the SLM as a segment in the field of view S and then kept in a stopped state. The data of the virtual information represented in this segment is transmitted to the SLM or generated by the SLM itself and encoded on the latter, preferably already encoded during the movement of the scanning mirror element and / or while maintaining the stopped state of the scanning mirror element. After completing the transmission of the data of the segment to the SLM or generating it by the SLM and while maintaining the stopped state of the scanning mirror element, the control device then controls the lighting device so that the lighting device switches to the on state, whereby the data of the virtual information to be represented is represented in the following manner, i.e., the virtual information as overall information or partial information is completely placed into the image of the SLM, so that only a smaller number of images of the SLM as segments are required to represent the virtual information in the field of view S. The lighting device now illuminates the SLM so that the SLM modulates light according to the virtual information and generates an image of the SLM as a segment BS1 together with the combiner and the optical system, and directs the image at a determined position in the field of view S by means of the scanning mirror element. Then, in order to represent the virtual information C1, another image of the SLM as a segment BS2 is generated and represented in the same manner. This process is implemented until the virtual information C1 is completely displayed in the field of view S of the user. Therefore, in this exemplary embodiment, the generation and representation of the image of the SLM as a segment is also implemented in chronological order. From Figure 6 As can be clearly seen in Figure 2, the individual images of the SLM as segments can also be overlapped to represent Figure 1 The generation and representation of the virtual information items C2 and C3 are implemented in the same manner as the generation and representation of the virtual information C1.

[0181] Therefore, directly after the generation and representation of the virtual information C1, further virtual information items (eg Figure 1In order to track the virtual information C2 or C3 of the image of the SLM as a segment BS6, the scanning mirror element can be moved at a higher speed from the position of the image of the SLM as a segment BS6 to the position of the image of the SLM as a segment BS7, which has not yet been generated, for example, for the virtual information C2, so as to more quickly pass through the gap between the two segments BS6 and BS7. Here, the movement speed of the scanning mirror element should be higher than the speed when tracking the corresponding position of the SLM image as a segment for the virtual information C1. Then, the tracking of the individual positions of the image of the SLM as a segment for the virtual information C2 in the field of view S, which have not yet been generated and represented, is implemented at a lower movement speed of the scanning mirror element (similar to the speed when representing the virtual information C1). The generation and representation of the individual images of the SLM as a segment for the virtual information C2 are the same as the generation and representation of the virtual information C1. The same process can also be used to represent the virtual information C3. Here too, the switch from the last generated image of the SLM as segment BS13 for the virtual information C2 to the not yet generated image of the SLM as segment BS14 for the virtual information C3 is implemented at a higher movement speed of the scanning mirror element in order to traverse the large gap more quickly. This sequence of tracking multiple positions and generating images of the SLM as segments in the field of view is only an example. Of course, a different sequence is also possible. For example, after the last image of the SLM as segment BS13, an not yet generated image as segment BS17 can also be generated and represented, because the gap between the two segments BS13 and BS17 is not as large as the gap between the segments BS13 and BS14, so that this position of the segment BS 17 can be tracked more quickly.

[0182] like Figure 6 As shown, images of the SLM as segments are freely represented in the field of view, can overlap each other and can also have different shapes and / or sizes.

[0183] Figures 4 to 6 Used to describe the generation and presentation of virtual information by means of stepwise movement of scanned mirror elements with defined increments. Figures 4 to 6 The stepwise movement of the scanning mirror element in these exemplary embodiments can also be replaced by a continuous movement of the scanning mirror element and in this way represent the image of the SLM as segments in the field of view.

[0184] Figure 7 Such a deflection device is described which provides a continuous movement of the scanning mirror element. Figure 7Figures a), b) and c) in the figure represent the generation of two images of the SLM as segments. The deflection device 50 comprises a scanning mirror element 51 and a compensating mirror element 52. The scanning mirror element 51 is the element of the deflection device 50 that performs continuous movement. The compensating mirror element 52 is also mounted in a movable manner. The scanning mirror element 51 and the compensating mirror element 52 are arranged at an angle to each other, such as Figure 7 As shown. Figure 7 In Figure (a) of FIG. 5 , this angle is approximately 90 degrees. Both the scanning mirror element 51 and the compensating mirror element 52 are controlled by a control device 53 which also controls the illumination device (not shown) and optionally the SLM.

[0185] Figure 7 Figure a) in Figure 1 represents the generation of a first image of the SLM as segment BS1. To this end, when generating an image of the SLM as segment BS1 at a predetermined position in the field of view of the user, the two mirror elements 51 and 52 form a defined angle relative to each other, wherein the compensating mirror element 52 remains stationary, i.e. does not move. In contrast, the scanning mirror element 51 continues to move continuously. Therefore, the light L incident on the deflection device 50 is initially incident on the scanning mirror element 51 and is reflected by it in a direction towards the compensating mirror element 52 according to the direction in which the light is aligned with the scanning mirror element 51. The light incident on the compensating mirror element 52 is also reflected by the compensating mirror element 52 according to the direction in which the light is aligned with the compensating mirror element 52, then incident on the combiner and then directed to the corresponding position in the field of view of the user. For example, when a first image of the SLM as a fragment is generated at a specific position in the field of view, this embodiment of the deflection device 50 may appear so that at the first control moment when the control device 53 continuously moves the scanning mirror element 51 and the lighting device has also been controlled and illuminated by the control device 53, an image of the SLM as a fragment for virtual information is generated and represented in the field of view at the first position of the scanning mirror element.

[0186] Figure 7FIG. b) shows that the scanning mirror element 51 moves or rotates along the arrow shown in the figure from the dotted line position to another position to be indicated by the solid line under the control of the control device 53. In the process, the compensating mirror element 52 rotates together in the same direction and with the same absolute value as the scanning mirror element 51, so that when the control device 53 of the display device controls the lighting device and turns it on, the SLM is illuminated and an image of the SLM as a segment is generated and represented at the same field of view position as in FIG. a). Therefore, the image of the SLM as a segment BS1 is displayed at the same position in the field of view. In addition, this means that, in the case of continuous movement of the scanning mirror element 51, an image of the SLM as a segment can be generated and displayed at the same position as in FIG. a). Therefore, the compensating mirror element 52 compensates for the movement of the scanning mirror element 51. The dotted lines and arrows should represent the light beams incident on the two mirror elements 51 and 52 according to FIG. a), wherein the solid lines and arrows are intended to illustrate the incident light beams offset therefrom.

[0187] exist Figure 7 c) of the drawing, the scanning mirror element 51 is continuously moved by controlling the scanning mirror element 51 using the control device 53. However, the compensating mirror element 52 has been rotated or moved in the opposite direction by the control device 53, so that it has now been moved from the dashed line position to the solid line position. In this way, an image of the SLM as segment BS2 is generated and represented at a position in the field of view that is different from the position of the image of the SLM as segment BS1.

[0188] The advantage of this arrangement of scanning mirror elements and compensating mirror elements in the deflection device is that the movement speed of a continuously scanning or moving mirror element tends to be faster than a mirror element which is moved stepwise from point to point and then stopped. Figure 7 In an exemplary embodiment of the embodiment of the invention, the image of the SLM as segment BS1 can be displayed at all times between the states of the mirror elements 51 and 52 shown in figures a) and b) (even when the mirror elements 51 and / or 52 are moved or rotated). Figure 7 When the compensating mirror element 52 is moved to its initial state, no image of the SLM as a segment is generated until the new desired position of the compensating mirror element 52 is reached. When the compensating mirror element 52 is moved to its initial state, the lighting device is deactivated or in the switched-off state. Only when the compensating mirror element 52 reaches its new desired position is the lighting device switched back on by the control device 53. Then, the mirror elements 51 and 52 or only one of the two mirrors can continue to move and another or additional image of the SLM as a segment can be generated and displayed in the field of view for the user.

[0189] Figure 8The overall generation of information in combination with light coupled into a light guide according to the prior art is shown. In this case, an SLM 60 is used, which has a relatively large number of pixels and thus has an HD (High Definition) TV resolution or higher. Light of a coupling angular spectrum 63 modulated and emitted by the SLM 60 is generated by an imaging device 62 (a lens element arranged in the optical path between the SLM 60 and the light guide 61), so that the light beams emitted from the individual pixels of the SLM 60 are generally incident on the light guide 61 at different angles relative to the surface of the light guide 61. This coupling angular spectrum 63 is incident on the light guide 61 and coupled into the light guide 61 by a reflective mirror 64. The reflective mirror 64 is fixedly arranged in the light guide 61 at a fixed angle. The light beam incident on the reflective mirror 64 is reflected by the reflective mirror 64 and propagates in the light guide 61 by total internal reflection. By means of a decoupling device 65 with corresponding decoupling elements (eg decoupling grating elements), light can be decoupled from the light guide 61 in the direction of the eyes of the user B, so that a decoupling angular spectrum 66 can be determined. The image of the SLM 60 is imaged in an observer plane 67 in order to generate a virtual visible area 68 there.

[0190] For example, further options for the propagation of light in a light guide and the decoupling of light from the light guide are described in patent document WO2019 / 012028A1, and the disclosure of patent document WO2019 / 012028A1 is intended to be incorporated herein in its entirety. The disclosure describes a light guide that is implemented to achieve decoupling after a fixed number of reflections in the light guide, and the decoupling angular spectrum is increased compared to the coupling angular spectrum. The decoupled light propagates to the visible region, and the decoupling angular spectrum corresponds to the field of view. However, the propagation and decoupling of light should not be limited to these options.

[0191] As a response to this, Fig. 9 An image of an SLM as a segment generated by a display device according to the invention is shown. Fig. 9 The display device can also be specifically implemented as an AR display device, such as an AR head-mounted display or an AR head-up display. Figures 2 to 7 Like a display device, the display device can also generate virtual information in a holographic or stereoscopic manner and display the information in the user's field of view.

[0192] Figure a) shows the display device when generating a first image of the SLM as a fragment, Fig. 9 Figure b) shows the generation of a second image of the SLM as a segment. Fig. 9 The display device comprises an illumination device 70, an SLM 71, a combiner 72, a deflection device 73 and at least one imaging element 74 of an optical system. Figure 8In comparison, the SLM 71 here has only a relatively small number of pixels, for example less than 1000 pixels in one direction. In order to deflect the incident light, the deflection device 73 here comprises a scanning mirror element 73-1, which is mounted in a movable and thus rotatable manner. The scanning mirror element 73-1 is arranged near the light coupling surface of the combiner 72, so that coupling can be achieved with high precision. In this case, the optical system of the display device should be represented by an imaging element 74, a combiner 72 and a deflection device 73, wherein naturally a plurality of imaging elements or other optical elements can also be provided. The imaging element 74 is arranged in the optical path between the SLM 71 and the deflection device 73. In this exemplary embodiment, the combiner 72 is implemented as a light guide that can have a flat or planar or other curved embodiment. In addition, a control device 75 is provided, which is connected to the lighting device 70 and the deflection device 73 (specifically the scanning mirror element 73-1). In addition, the control device 75 can also be connected to the SLM 71, wherein the SLM 71 itself can also be operated by its own control device.

[0193] If the lighting device 70 is now in the on state due to the control of the control device 75, the lighting device 70 emits light to the SLM 71, the light is modulated by the SLM 71 and incident on the deflection device 73 using the imaging element 74. Fig. 9 It can be clearly seen from the two figures a) and b) that the coupling angular spectrum of the light generated by the imaging element 74 is smaller than Figure 8 The coupling angular spectrum of the light is then coupled into the combiner 72 implemented as a light guide, for example, by a coupling device, which may include, for example, a mirror element or other at least one grating element, such as a volume grating. The decoupling of the light from the combiner 72 may be achieved by a decoupling device 77, which may include at least one decoupling element, such as a decoupling grating element of a volume grating, so that the decoupled light is guided to the observer plane 78 in the direction of the user B and forms a virtual visible area 79 in the plane, and the user B can then observe the virtual information generated in the field of view through the virtual visible area 79.

[0194] exist Fig. 9In figure a), a first image of the SLM 71 as a fragment is generated and represented in the field of view, which image contains virtual information. To this end, it is determined by the detection device where in the field of view the virtual information should be represented. If this is known, the control device 75 controls the scanning mirror element 73-1 to move the latter to a defined position, which is required to represent at least some of the virtual information. In addition, the control device 75 sets the lighting device 70 to the on state so that the SLM 71 is illuminated by light, wherein data representing the virtual information is transmitted to the SLM 71. The light incident on the SLM 71 is modulated by the SLM 71 according to the virtual information and is incident on the imaging element 74, so that an image of the SLM 71 is generated on the scanning mirror element 73-1, which is arranged in the Fourier plane of the SLM 71. The light 76 of the angular spectrum generated by the imaging of the SLM 71 by the imaging element 74 is coupled into the combiner 72 implemented as a light guide at a first central angle α through the scanning mirror element 73-1, and propagates forward in the combiner 72 by total internal reflection. If the light propagating in the combiner 72 is incident on the decoupling device 77 at a defined angle, the light is decoupled from the combiner 72, and the image of the SLM 71 as a segment is guided to the observer plane 78, forming a virtual visible area 79 in the observer plane 78. In this way, a first image of the SLM 71 as a segment in the form of virtual information is generated and represented at a defined position in the field of view.

[0195] exist Fig. 9 In FIG. b), a second image of the SLM 71 as a fragment is generated and represented at a second defined position in the field of view of user B, the second defined position being different from the first defined position. Obviously, for this purpose, the scanning mirror element 73-1 is moved or rotated by the control device 75 to another position, i.e., along the arrow from the position indicated by the dotted line to the position indicated by the solid line. This changes the center angle of the light beam coupled into the combiner 72. Then, the light 76 of the angular spectrum generated by the imaging of the SLM 71 is coupled into the combiner 72 through the scanning mirror element 73-1 at a different center angle, i.e., the center angle β set by the scanning mirror element 73-1 is rotated by a defined angle. However, the light propagating in the combiner 72 at an angle different from that in FIG. a) can be decoupled in the same manner as described in FIG. a). In this way, a second image of the SLM 71 is generated and represented in the field of view at a position different from the first image of the SLM 71 as a fragment, so as to display additional virtual information.

[0196] However, for clarity, Fig. 9In the figure, only two images of the SLM 71 as segments are shown and illustrated, both of which are generated in chronological order. However, further images of the SLM 71 as segments (e.g. 5 or 10 images of the SLM 71 as segments, or even more images if necessary) can be generated and represented in the field of view in the same way. For this purpose, the scanning mirror element 73-1 is rotated by the control device 75 by further defined increments, so that the images of the SLM 71 as segments are represented at respectively defined positions in the field of view.

[0197] Assume that the scanning mirror element 73-1 moves step by step to explain Fig. 9 Of course, Fig. 9 The deflection device 73 can also be arranged according to Figure 7 is implemented, thus providing a continuous movement of the scanning mirror element.

[0198] When the general Fig. 9 and Figure 8 When compared, it is clear that the overall angular spectrum of the light obtained by the combination of the angular spectrum of the first image of the image of the SLM 71 as a segment and the second image of the SLM 71 as a segment and coupled to the combiner 72 is different from the angular spectrum according to Figure 8 The angular spectrum of the light coupled into the light guide 61 is the same. Fig. 9 It also generates Figure 8 The same size field of view as in the example above.

[0199] The present invention should not be construed as being limited to the exemplary embodiments described herein. In addition, other embodiments or exemplary embodiments may also be implemented. Finally, the following fact should be specifically mentioned, that is, the exemplary embodiments described above are only used to describe the teachings described, and are not intended to be construed as being limited to the exemplary embodiments described.

Claims

1. A display device for displaying two-dimensional and / or three-dimensional information to a user, comprising: - lighting devices for emitting light, - a spatial light modulation device for modulating incident light, - an optical system comprising a deflection device, and - a control device connected to the lighting device and the deflection device, An image of a spatial light modulator as a segment is generated by the optical system, the image of the spatial light modulator is guided by the deflection device to a defined position in a user's field of view, and once the deflection device is adjusted to generate the image of the spatial light modulator at the defined position, the control device switches the lighting device according to the control of the deflection device so that the lighting device correspondingly emits light incident on the spatial light modulator, and generates the image of the spatial light modulator at the defined position through the optical system.

2. A display device according to claim 1, wherein the optical system is configured to generate at least two images of the spatial light modulation device and generate a virtual visible area according to the number of the images of the spatial light modulation device, wherein the at least two images of the spatial light modulation device exist as fragments in the field of view.

3. The display device according to claim 2, wherein the at least two images of the spatial light modulation device as segments in the field of view are combined with each other and / or partially overlap each other or are separated from each other by a gap.

4. A display device according to claim 1, wherein the number of the images of the spatial light modulation device as segments can be set differently between a minimum value and a maximum value in each frame, and the positions of the images of the spatial light modulation device as segments in the field of view can be set differently in each frame.

5. The display device of claim 4, wherein the determination of the number and the positions of the images of the spatial light modulation device as segments in the field of view depends on the physical environment of a user. 6 . The display device according to claim 1 , wherein the image of the spatial light modulation device is an imaging of the entire spatial light modulation device or an imaging of only a portion of the spatial light modulation device.

7. The display device according to claim 1, wherein the deflection device comprises at least one scanning mirror element and / or at least one grating element which is movably mounted.

8. The display device of claim 1, wherein the optical system comprises at least one combiner for superimposing virtual information onto real information in the field of view.

9. The display device according to claim 8, wherein the deflection device is arranged between the spatial light modulation device and the combiner or between the illumination device and the spatial light modulation device.

10. The display device according to claim 1, wherein the deflection device is arranged between the illumination device and the spatial light modulation device.

11. The display device according to claim 1, wherein the deflection device comprises two scanning mirror elements that can rotate in a synchronized manner with each other.

12. The display device of claim 8, wherein the at least one combiner comprises at least one focusing element or at least one focusing function.

13. The display device of claim 12, wherein the at least one focusing element is implemented as a grating element. The display device of claim 13 , wherein the grating element is a volume grating. The display device according to claim 13 , wherein the grating element is a grating element having a limited acceptance angle.

16. The display device of claim 8 or 12, wherein the at least one combiner is at least partially curved.

17. A display device according to claim 7, wherein a continuous movement of at least one scanning mirror element or a stepwise movement of the at least one scanning mirror element in fixed defined increments is provided in the deflection device.

18. The display device according to claim 17, wherein in the case where the at least one scanning mirror element continuously moves, the at least one scanning mirror element is combined with a compensation mirror element, and the compensation mirror element implements a movement synchronized with the movement of the at least one scanning mirror element, and in the case where the movements of the two mirror elements are the same, the image of the spatial light modulator is generated at a fixed position, and in the case where the movements of the two mirror elements are opposite, the image of the spatial light modulator moves in the field of view.

19. A display device according to claim 18, wherein the same movement of the scanning mirror element and the compensating mirror element is provided as long as the illumination device is in an on state.

20. A display device according to claim 7, wherein the continuous movement of the at least one scanning mirror element with different predetermined speeds within a frame or the stepwise movement of the at least one scanning mirror element with different adaptive increments is configured to generate at least two images of the spatial light modulation device as fragments in the field of view.

21. A display device according to claim 20, wherein the speed or increment of movement of the at least one scanning mirror element is adjusted to be suitable for the defined positions of the respective images of the spatial light modulation device as segments in the field of view.

22. A display device according to claim 1, wherein the size and / or shape of the image of the spatial light modulator as a fragment varies within consecutive frames, or the size and / or shape of at least two images of the spatial light modulator as fragments at the defined positions in the field of view vary within a frame or in consecutive frames.

23. The display device of claim 8, wherein the at least one combiner is implemented as a partially reflective mirror element or a light guide.

24. The display device according to claim 23, wherein the deflection device is embodied as a switchable coupling element for coupling light into a combiner embodied as a light guide and / or as a decoupling element for decoupling light from the combiner embodied as a light guide.

25. The display device of claim 1, wherein the optical system comprises a zoom system capable of setting a distance between at least one image of the spatial light modulation device as a segment in the field of view and the user.

26. The display device according to claim 25, wherein the zoom system comprises at least one grating element with a controllable grating period or a combination of an active imaging element and a passive imaging element.

27. The display device according to claim 26, wherein the at least one grating element having a controllable grating period has a prism function and / or a phase function for correcting aberrations.

28. The display device according to claim 1 further comprises a gaze tracking system for detecting a viewing direction of the user and / or a detection device for determining in which area of ​​the field of view virtual information should be represented.

29. The display device according to claim 1, wherein the illumination device comprises at least one light source which can be controlled in a pulsed manner.

30. The display device of claim 1, wherein the display device is implemented as an augmented reality display that combines a physical environment and represented virtual information.

31. A method for displaying two-dimensional and / or three-dimensional information to a user, the method comprising the steps of: - controlling a control device which is connected to a deflection device of an optical system of an illumination device and a display device for emitting light, wherein the deflection device of the optical system is adjusted to generate at least one image of the spatial light modulation device at a defined position in the user's field of view, -The lighting device is switched according to the control of the deflection device, thereby guiding the at least one image of the spatial light modulator as a segment to the defined position in the user's field of view, so that the lighting device emits light incident on the spatial light modulator accordingly, and generates the image of the spatial light modulator at the defined position through the optical system, thereby representing virtual information in the user's field of view with the segment.

32. A method according to claim 31, wherein the optical system generates at least two images of the spatial light modulation device and a virtual visible area according to the number of images of the spatial light modulation device, wherein the at least two images of the spatial light modulation device are formed as fragments in the user's field of view.

33. A method according to claim 31 or 32, wherein at least one combiner of the optical system causes virtual information generated separately in the field of view to be superimposed on real information in the field of view by displaying at least one image of the spatial light modulation device as a fragment.

34. A method according to claim 31 or 32, wherein at least one image of the light modulating device is generated as a segment according to a required position in the field of view.

35. A method according to claim 31 or 32, wherein the field of view is subdivided into grid domains, wherein a check is performed for each frame to determine in which grid domain of the field of view the virtual information should be represented, wherein the spatial light modulator and at least one scanning mirror element of the deflection device are controlled to generate in each frame an image of the spatial light modulator as a fragment only in the grid domain where the virtual information is to be represented, and to guide the image to a defined position in the field of view.

36. A method according to claim 31 or 32, wherein the field of view is subdivided into grid domains, wherein each of the grid domains is scanned continuously by at least one scanning mirror element of the deflection device, wherein a check is performed for each frame to determine in which grid domain of the field of view the virtual information should be represented, and an image containing the virtual information is generated as a segment of the spatial light modulation device, and the optical system assigns the image only to the individual grid domains in which the virtual information should also be represented.

37. The method of claim 35, wherein at least one scanning mirror element is moved continuously or stepwise in defined increments to direct at least one image of the spatial light modulation device as a segment to a defined position in the field of view.

38. A method according to claim 37, wherein in the case of stepwise movement of the at least one scanning mirror element, when the at least one scanning mirror element is in a holding state after a defined increment, the lighting device is activated respectively and the spatial light modulator is illuminated to generate an image of the spatial light modulator, so that the generated image of the spatial light modulator as a segment is guided to a defined position in the field of view, wherein when the at least one scanning mirror element is in a moving state, the lighting device is deactivated.

39. A method according to claim 37, wherein a compensation mirror element is combined with the at least one scanning mirror element in the case where the at least one scanning mirror element moves continuously, wherein when the lighting device is in an on state, the compensation mirror element performs a movement synchronized with the movement of the at least one scanning mirror element.

40. A method according to claim 33, wherein the spatial light modulation device is illuminated by the illumination device and the light modulated by the spatial light modulation device is guided to the deflection device, the deflection device deflects the light incident on the combiner, the combiner is implemented as a light guide, wherein light is coupled into the combiner and propagates in the combiner, wherein the light propagating in the combiner is decoupled according to a desired defined position in the field of view, and the at least one image of the spatial light modulation device as a fragment is guided to the defined position.

41. The method of claim 31, wherein at least one image of the spatial light modulation device as a segment is moved by a zoom system in the z direction along the optical axis of the optical system to a depth position in the field of view accepted by a user.

Citation Information

Patent Citations

  • Head-Mounted Display Device for Generating Reconstructions of Three-Dimensional Representations

    US20100097671A1

  • Display device for enlarging the field of view

    WO2019012028A1

  • Display device and method for producing a large field of vision

    WO2019076963A1

  • Virtual image display apparatus

    CN103837985A

  • A display for two-dimensional and / or three-dimensional images

    CN109154432A