Transparent display and imaging system with lensless imaging capability

Abstract

A transparent display (1) with lensless imaging capability comprises: display pixels (11) configured to generate a display image; a light emitter (12) configured to illuminate the scene or object (2) with electromagnetic radiation outside the visible range; a light sensitive element (21) configured to capture electromagnetic radiation received from a scene or object (2) and to generate a light signal depending on the captured electromagnetic radiation; and an optical modulator (31) configured to transmit the electromagnetic radiation in the visible region and to modulate the electromagnetic radiation in the illumination wavelength range. The display is substantially transparent in the visible region.

Description

[0001] The present disclosure relates to a transparent display with lens-free imaging capability, an imaging system including such a display, and a method of manufacturing a transparent display with lens-free imaging capability.

[0002] A transparent display or see-through display is an electronic display that allows a user to view content displayed on the screen of the display while still being able to see scenes or objects behind the screen from the user's perspective. Such displays are commonly used in applications such as head-up displays, particularly in the automotive field, and in augmented reality systems such as smart glasses. In general, transparent displays embed pixels of the active matrix of the display in a distributed manner in the field of view so that visible light can pass through the gaps between the pixels so that the display appears transparent to the user.

[0003] The above-mentioned applications of head-up displays and augmented reality devices additionally require some form of imaging, since typically information is to be displayed on a screen that is related to an object or scene in the field of view of the user behind the display. For example, a head-up display indicates speed limits, information detected from road signs, or navigation instructions. To this end, state-of-the-art approaches typically rely on projection systems that are bulky and inherently do not allow for interactivity. On the other hand, embedded cameras do not allow for the flat form factor that is typically desired, and furthermore introduce notches and dark spots on the display since they are opaque.

[0004] Therefore, an object to be achieved is to provide a transparent display with lens-free imaging capabilities, which overcomes the limitations of existing solutions. Further objects are to provide a transparent imaging system comprising such a transparent display, and a method of manufacturing such a transparent display.

[0005] These objects are achieved with the subject-matter of the independent claims. Further developments and embodiments are described in the dependent claims.

[0006] The present disclosure overcomes the above limitations of modern devices by embedding a transparent camera within a transparent display. Specifically, the improved concept is based on the idea of ​​using one or more transparent substrates and materials whose transparency spans at least the visible range of the electromagnetic spectrum. On these substrates, display pixels, illuminators, image sensors, and lensless optics are embedded in a distributed manner, ensuring both display and imaging capabilities while keeping the arrangement appearing transparent to the user.

[0007] In one embodiment, a transparent display with lensless imaging capability includes a plurality of display pixels arranged on a display substrate. The plurality of display pixels are configured to generate a display image in the visible domain of the electromagnetic spectrum. The transparent display also includes: a plurality of light emitters configured to illuminate a scene or object with electromagnetic radiation in an illumination wavelength range outside the visible domain; and a plurality of photosensitive elements configured to capture electromagnetic radiation in the illumination wavelength range received from the scene or object and generate a light signal depending on the captured electromagnetic radiation. The transparent display also includes an optical modulator arranged on the incident side of the plurality of photosensitive elements and configured to transmit electromagnetic radiation in the visible domain and modulate electromagnetic radiation in the illumination wavelength range. Wherein, the display is substantially transparent in the visible domain.

[0008] Display pixels are formed from visible light emitters configured to emit light in the visible domain of the electromagnetic spectrum, i.e., optical wavelengths ranging from 400 to 750 nm. For example, display pixels are formed from RGB light emitting diodes and arranged in a two-dimensional matrix to form a screen for displaying content. Among them, the light emitters of the pixels can be controlled to output specific corresponding colors, or each pixel is formed by, for example, red, green and blue sub-pixels. The pixels can be connected to a controller that controls the image formed by the pixels to be displayed on the screen. The display pixels are arranged on a transparent substrate (e.g., a glass or film substrate) or on a transparent foil. Here and throughout the disclosure, the term "transparent" refers to the visible part of the electromagnetic spectrum, i.e., light at wavelengths to which the human eye is sensitive.

[0009] The illumination light emitter is configured to emit light at an optical illumination wavelength outside the visible spectrum, such as light in the NIR domain within the range of 750nm and 1.4μm or light in the SWIR domain within the range of 1.4μm and 3μm. The illumination light emitter can be arranged on the same substrate as the visible light emitters forming the display. For example, the display is formed from a pixel matrix, wherein each pixel includes sub-pixels, which include the aforementioned visible sub-pixels formed from red, green and blue sub-pixels, and illumination sub-pixels formed from light emitters in the NIR or SWIR domain. Alternatively, only some pixels may include illumination light emitters, such as pixels along the periphery of the display. Alternatively, the illumination light emitters can be arranged independently of the display pixels, such as forming a frame around the display or arranged on another transparent substrate.

[0010] The photosensitive element is configured to capture electromagnetic radiation emitted by the light emitter and reflected away from the scene or object. Therefore, the photosensitive element has sensitivity within the illumination wavelength range. For example, if the illumination light emitter emits light at an optical wavelength in the NIR domain (e.g., at 840 or 930 nm), the photosensitive element can be a silicon-based photodiode. Each photosensitive element also generates a light signal depending on the electromagnetic radiation captured by the corresponding element. The photosensitive element forms an image sensor and can be arranged on the same substrate as the visible light emitter and / or the illumination light emitter. For example, the display is formed from a pixel matrix, wherein each pixel includes a sub-pixel, which includes the aforementioned visible sub-pixel formed from red, green and blue sub-pixels, and an optional illumination sub-pixel and a photosensitive element (e.g., a photodiode). Alternatively, only some pixels may include photosensitive elements, such as pixels arranged in the central portion of the display. Alternatively, the photosensitive element can be arranged independently of the display pixel and the illumination emitter, such as between pixels, or they can be arranged on another transparent substrate.

[0011] The optical modulator is arranged between the object or scene and the image sensor formed from the photosensitive element. Unlike conventional imaging using an optical lens, in which each pixel of the image sensor captures light from a different area of ​​the object or scene to be imaged, the use of an optical modulator and thus lensless imaging leads to the encoding of object or scene information, for example, by having a different system response for each pixel of the image sensor. Among them, each pixel of the image sensor can be understood as an effective weighted sum of the system response function captured from each point of the object or scene. The image can be reconstructed using a computational inverse algorithm for the optical signal. The optical modulator can be illumination modulated, mask modulated, or implemented as a programmable modulator system. The optical modulator can be arranged on a separate transparent substrate that is arranged adjacent to the substrate including the photosensitive element. Specifically, the optical modulator is configured to transmit visible light, that is, the optical modulator does not encode optical wavelengths in the visible domain, and encodes light under the illumination wavelength range, such as light in the NIR or SWIR domain.

[0012] The aforementioned elements are arranged so that the display is substantially transparent in the visible domain. This means that a user viewing the display can still discern objects or scenes arranged behind the display from the user's perspective. If 70% of the optical power in the visible domain is transmitted through an object, the object appears substantially transparent. Therefore, display pixels, illumination light emitters and photosensitive elements (which are themselves usually not transparent) can be arranged in a distributed manner, i.e. with gaps in between, so as to ensure that a total of at least 70% of the visible light is transmitted through the display.

[0013] In one embodiment, the illumination wavelength range is in the near infrared (NIR) domain. For example, the illumination wavelength range is in the range of about 840 and / or 930 nm. This wavelength range is invisible to the human eye, so that illumination of a scene or object does not affect its appearance to a user. Furthermore, within this wavelength range, well-established technologies for light emitters and light sensitive elements can be employed for efficient and cost-effective manufacture and operation of the display. For example, the light sensitive element is a silicon photodiode, which shows sensitivity in the visible domain as well as in the NIR domain up to about 1100 nm. In order to limit the sensitivity of the light sensitive element to the illumination wavelength range, the light sensitive element may include, for example, a filter element for blocking visible light.

[0014] In one embodiment, the light emitter is an OLED, a microLED or a vertical cavity surface emitting laser (VCSEL). The light emitter used to illuminate the scene or object can be selected from various existing technologies, such as an organic LED or a microLED. For example, the illumination light emitter is of the same type as the visible light emitter forming the display, so that they can be easily manufactured on the same substrate following the same or similar manufacturing process. Alternatively, the illumination light emitters can be of different types, for example, the display pixels are formed from OLEDs and the illumination light emitters are formed from microLEDs, or vice versa. In this way, they can be manufactured separately on different substrates, for example, in order to adapt the display technology and the illumination to specific applications. In addition, with respect to the requirements of having coherent light or having a higher range of illumination, the illumination light emitters can be formed from VCSELs, also arranged on the same substrate as the display pixels or on a different substrate.

[0015] A microLED (light emitting diode, LED for short) is a semiconductor light emitting diode with particularly small dimensions. For example, a growth substrate for the epitaxial growth of the semiconductor layer sequence of the microLED is removed from the microLED. In other words, the microLED does not include the growth substrate. For example, the thickness or height of the microLED in the growth direction of the semiconductor layer sequence is between 1, 5 μm and 10 μm.

[0016] For example, the light emission surface of the microLED can be rectangular or can have a different shape. In particular, in a plan view of the layers of the semiconductor layer sequence, each lateral extension of the light emission surface is at most 100 μm or at most 70 μm. For example, if the microLED has a rectangular shape, the side length of the microLED - in particular in a plan view of the layers of the semiconductor layer sequence - is at most 70 μm or at most 50 μm.

[0017] For example, the microLED is disposed on a wafer having a detachable holding structure so that the microLED can be detached from the wafer without damage. The microLED may also be referred to as μLED, μ-LED, uLED, u-LED or micro light emitting diode.

[0018] In one embodiment, multiple light emitters are arranged on a display substrate. Arranging the illumination light emitters next to the display pixels on a common substrate results in a compact system by minimizing the layers of the final display. For example, each or some display pixels include visible sub-pixels, such as formed by RGB LEDs, and illumination light emitters, such as NIR LEDs. Alternatively, the illumination light emitters can be arranged in the periphery of the display pixels, for example, forming a frame around the display, depending on the intensity required, for example, by a specific application. In particular in embodiments where the display pixels and the illumination pixels are formed following similar processes, for example both elements are semiconductor devices such as micro-LEDs or VCSELs, arranging the display pixels and the illumination light emitters on a common substrate can be achieved in a straightforward manner.

[0019] In one embodiment, a plurality of photosensitive elements are arranged on the display substrate. Arranging the photosensitive elements next to the display pixels on a common substrate also leads to further minimizing the overall size of the display. For example, according to a combination of the aforementioned arrangement examples, the display pixels, the photosensitive elements and the illumination light emitters are arranged on a common substrate. Thus, some or all display pixels may include photosensitive elements and / or illumination light emitters.

[0020] In one embodiment, the display further comprises a detection substrate, wherein a plurality of photosensitive elements are arranged on the detection substrate. In particular in embodiments where the photosensitive elements are of different technologies than the display pixels, arrangements on different substrates may be necessary for manufacturing purposes. For example, the photosensitive elements are organic photodiodes, while the display pixels and the illumination light emitters are semiconductor-based, such as micro-LEDs and VCSELs. Alternatively, the illumination light emitters may be arranged next to the photosensitive elements on the detection substrate.

[0021] In one embodiment, a plurality of display pixels form an OLED display, a micro-LED display, or a liquid crystal display (LCD). The display elements of the transparent display with capture capability can be formed from existing technologies already employed in transparent displays, such as OLED or LCD. Micro-LEDs in particular, due to their small footprint, can be further used to increase the transparency of the display. Likewise, the illumination light emitters can be formed accordingly based on different technologies, depending on the requirements of the application and which application is best suited.

[0022] In one embodiment, the photosensitive element is a silicon-based photodiode or an organic photodetector or photodiode (OPD). Like display pixels, the photosensitive element can be manufactured according to different technologies and thus adapted to specific applications in terms of, for example, sensitivity, power consumption and size. Among them, organic photodiodes are characterized by lower power consumption compared to their semiconductor counterparts. Therefore, the photosensitive element can be a silicon photodiode for target sensitivity in the NIR range.

[0023] In one embodiment, the display further comprises an optical substrate, wherein the optical modulator is arranged on the optical substrate. The optical modulator is arranged in close proximity to the imaging sensor portion. Thus, the optical substrate can be arranged in close proximity to the substrate comprising the photosensitive element. Alternatively, a spacer or even a display substrate can be arranged between the detection substrate and the optical substrate. For example, the transparent display comprises exactly two substrates, wherein the first substrate comprises the display pixels, the photosensitive element and the illumination light emitter, and the second substrate comprises the optical modulator.

[0024] In one embodiment, the optical modulator is an active matrix based on one of liquid crystal, optical switch, and other types of spatial and digital light processors. The transparent display in these embodiments may also include a controller for controlling the elements of the active matrix. Dynamic and programmable modulators (also called spatial light modulators) can be used as a substitute for fixed masks. For example, the active matrix is ​​based on liquid crystal technology, which can be used for programmable amplitude modulation. For programmable phase modulation, LC devices on silicon can be used in the active matrix. Other examples include vanadium oxide transistors, which act as optical switches in the NIR region while being transmissive at visible frequencies. Programmable modulators have the following advantages: they can easily and quickly change the mask pattern so that multiple images can be captured in a continuous manner, each with different optical encodings. In addition, in embodiments with a relatively small number of photosensitive elements, programmable modulators can facilitate obtaining sufficient measurements for reconstruction. In addition, in the case of a sensor array of photosensitive elements, the ability to change the modulation mode between acquisitions provides additional degrees of freedom, which can further improve the reconstruction performance or the resolution of the reconstructed image. However, the active matrix requires additional components (such as the aforementioned controller) and is characterized by an additional power consumption channel.

[0025] In one embodiment, the optical modulator is a passive matrix based on an amplitude mask, a phase mask, and one of a plurality of diffraction elements. As an alternative to an active matrix, an amplitude mask, a diffraction mask, a random reflective surface, and a modified microlens array can be used to passively perform optical modulation. Among them, the amplitude modulator includes transparent and shielded areas arranged in a fixed spatial pattern on a 2D mask. On the other hand, the phase modulator changes the relative optical path length or effective phase delay in the 2D pattern. Since the attenuation of the incoming light is non-existent or negligible, the phase masks can be more optically efficient, and they can gather light to create clearer and higher contrast patterns on the image sensor, thus also improving the image reconstruction performance.

[0026] In one embodiment, the optical modulator is implemented by a plurality of pinholes distributed in space. The amplitude mask allows incident light to pass through, blocks incident light, or attenuates incident light. One of the simplest architectures of a binary amplitude mask is a pinhole distribution. Amplitude masks generally have the advantage of being easier to make over a wide wavelength range. For example, outside the visible range, materials that can block light are easier and cheaper to find than materials that can refract light. In particular, materials that are transmissive in the visible range and opaque in the NIR or SWIR range can easily form an optical modulator mask. For example, dye-based polymers, such as Fujifilm SIR850W, SIR850N, or SIR940, can be used to form an amplitude mask. These polymers show narrow absorption bands in the infrared band between 850 and 940 nm while maintaining high transparency in the visible range. Their thickness ranges from 0.8 to 1.36 μm. Another example of an optical modulator suitable for lensless imaging is a diffuser. In addition to amplitude modulation, the interaction of light with the pinhole can be based on the modulation of the phase or polarization of the incoming light. For this purpose, phase-shifting materials can be used in the illumination wavelength range as long as they are transparent in the visible range.

[0027] In one embodiment, the optical modulator forms a coded aperture mask, which is characterized in particular by a uniform redundant array (URA) or an optimized random pattern (ORA). For the purpose of computational imaging, the optical modulator of the lensless system can be a regular pattern, such as a uniform redundant array (URA) and a random (or pseudo-random) M sequence. In lensless imaging, the coded aperture or coded aperture mask is a grid, grating or other pattern of material that is opaque to electromagnetic radiation of various wavelengths (i.e., the irradiation wavelength range). By blocking radiation in a known pattern, a coded shadow is projected on a plane. The properties of the original radiation source can then be mathematically reconstructed from the shadow using an algorithm that includes knowledge of the mask pattern. Examples include Fresnel zone plates, optimized random patterns, URAs, and hexagonal or modified URAs. The concept of coded aperture imaging itself is a well-known concept, particularly in X-ray and gamma-ray imaging systems, where the rays cannot be focused using lenses or mirrors designed for visible light.

[0028] In one embodiment, substantial transparency of the display in the visible domain is achieved by a distributed arrangement of display pixels and photosensitive elements with spaces therebetween, such that a substantial amount of electromagnetic radiation in the visible domain incident on the display is transmitted. As described above, the optical elements may be arranged such that at least 70% of the light intensity in the visible domain is transmitted through the display. Thus, the display pixels, photosensitive elements and illumination light emitters are spaced apart from one another such that the spaces therebetween allow visible light to be transmitted. Wherein the substrate used is also transparent at visible wavelengths. Furthermore, the optical modulator affects light at the illumination wavelength range, but does not affect light at visible frequencies.

[0029] In addition, a transparent imaging system is provided, comprising a transparent display according to one of the aforementioned embodiments of the transparent display, and a processing unit coupled to the display and configured to reconstruct an image by applying an algorithm to a light signal. The transparent imaging system may be an integrated system, wherein the processing unit is embedded within the display. Alternatively, the processing unit may be an external component coupled to an active element of the transparent display. The processing unit may further be used to synchronize the illumination of a scene or object by activating and deactivating the emission of an illumination light emitter with the active detection (i.e., integration time) of a photosensitive element. For reconstruction, the processing unit is configured to apply an algorithm to the light signal, wherein the algorithm takes into account properties of the optical modulator, such as the layout or pattern of the coded aperture mask. Depending on the application, the algorithm may be used to: reconstruct an image of a scene, such as via deconvolution or a deep neural network; or perform a specific recognition task, such as facial identification; or perform feature extraction from a raw sensor image.

[0030] In one embodiment, the processing unit is further configured to control the generation of a display image based on the reconstructed image. For example, the information to be displayed on the transparent display may depend on information identified in the captured and reconstructed image. To this end, the processing unit may be configured to: identify an object in the reconstructed image, such as a road sign; and display content related to the object on the display, such as driving or walking instructions.

[0031] In addition, an electronic device is provided, comprising a transparent display or a transparent imaging system according to one of the aforementioned embodiments. The electronic device may be a wearable device such as smart glasses. Alternatively, the transparent display or transparent imaging system may be employed in a transparent object, such as a windshield of a vehicle, to form a head-up display with imaging capabilities that are independent of projection. In particular, transparent displays and transparent imaging systems are used in applications where it is desired to display an image while keeping the line of sight behind the display unobstructed from the user's perspective. In such a device, for example, the display may be used as an augmented reality display, or for displaying data detected in an image sensor of the display and a user's field of view, such as a speed limit identified from a road sign.

[0032] In addition, a method for manufacturing a transparent display with lensless imaging capability is provided. The method includes: arranging a plurality of display pixels on a display substrate, the plurality of display pixels being configured to generate a display image in the visible domain of the electromagnetic spectrum; and providing a plurality of light emitters, the plurality of light emitters being configured to illuminate a scene or object with electromagnetic radiation in an illumination wavelength range outside the visible domain. The method also includes: providing a plurality of photosensitive elements, the plurality of photosensitive elements being configured to capture electromagnetic radiation in the illumination wavelength range received from a scene or object, and generating a light signal depending on the captured electromagnetic radiation; and providing an optical modulator, the optical modulator being arranged on the incident side of the plurality of photosensitive elements and being configured to transmit electromagnetic radiation in the visible domain and modulate electromagnetic radiation in the illumination wavelength range. Wherein, the display is substantially transparent in the visible domain.

[0033] Further embodiments of the method will be apparent to readers skilled in the art from the aforementioned embodiments of the transparent display, the imaging system, and the electronic device, and vice versa.

[0034] The following description of the drawings may further illustrate and explain various aspects of the transparent display, the imaging system, and the method of manufacturing the transparent display. Components and parts of the transparent display that function the same or have the same effect are indicated by the same reference numerals. Identical or substantially identical components and parts may be described only with respect to the drawing in which they first appear. Their description need not be repeated in subsequent drawings.

[0035] In the attached picture:

[0036] Figure 1 An exemplary embodiment of a transparent imaging system including a transparent display according to an improved concept is shown;

[0037] Figure 2 and Figure 3 Aspects of a first exemplary embodiment of a transparent display having lens-free imaging capabilities are shown;

[0038] Figure 4 and Figure 5 Aspects of a second exemplary embodiment of a transparent display having lens-free imaging capabilities are shown;

[0039] Figure 6 A third exemplary embodiment of a transparent display having lens-free imaging capability is shown; and

[0040] Figure 7 and Figure 8 Embodiments of an electronic device and a vehicle windshield including an imaging system are shown.

[0041] Figure 1An exemplary embodiment of a transparent imaging system 100 including a transparent display 1 according to the improved concept is shown in panel (a). The imaging system 100 also includes a processing unit 101, which is electrically coupled to the transparent display 1 (e.g., an integrated circuit of the transparent display 1). The processing unit 101 is configured to control the illumination of the scene or object 2 by means of the illumination light emitter 12, and to control the capture by means of the image sensor formed by the photosensitive elements 21. For example, the processing unit 101 synchronizes the illumination and imaging processes. The processing unit 101 is also configured to reconstruct an image from the light signal received from each photosensitive element 21 by means of an application algorithm. Depending on the application, the algorithm can be used to reconstruct an image of the scene via deconvolution or a deep neural network, or to perform a specific recognition task (e.g., face identification), or to perform feature extraction from the original sensor image. The processing unit 101 can also be configured to control the image displayed on the transparent display 1 by controlling the emission of the display pixels 11. For example, the displayed image depends on the reconstructed image or the features recognized in the reconstructed image.

[0042] Figure 1 An exploded view of an exemplary embodiment of a transparent display 1 according to the improved concept is shown in panel (b). The transparent display 1 includes a display substrate 10, on which display pixels 11 and illumination light emitters 12 are arranged. The display substrate 10 is transparent in the visible range of the electromagnetic spectrum (e.g., at optical wavelengths between 400 and 750 nm). For example, the display substrate 10 is a foil, a film, a silicon dioxide or a glass substrate. The display pixels 11 are arranged in a two-dimensional matrix arrangement so as to form a display for displaying images in the visible domain. In addition, in this embodiment, the illumination light emitters 12 are arranged in the periphery of the display pixels 11 and are configured to illuminate the scene or object 2 with light outside the visible domain. For example, the illumination light emitters 12 are configured to emit light in the NIR or SWIR domain. For example, the display pixels 11 and the illumination light emitters 12 can be formed from liquid crystal pixels, micro-LEDs, OLEDs or VCSELs. Therein, the optical elements forming the display pixels 11 and the illumination light emitters 12 are arranged in a distributed manner with gaps between each other, so that the display substrate 10 together with the pixels 11 and the emitters 12 appear transparent to the user.

[0043] The transparent display 1 also includes a detection substrate 20, on which photosensitive elements 21 are arranged. Like the display pixels 11, the photosensitive elements can be arranged in a two-dimensional matrix arrangement so as to form an image sensor for capturing images. Among them, the matrix arrangement of the photosensitive elements 21 can have a size similar or equal to that of the matrix arrangement of the display pixels 11, or smaller as shown in this exemplary embodiment, where the photosensitive area is limited to the central part of the entire surface. For example, the photosensitive element 21 is a semiconductor photodiode, such as a silicon-based photodiode particularly used for imaging in the NIR region, or an organic photodiode (OPD). The photosensitive element on its incident side may include a coating or a filter element for blocking visible light. For example, the photosensitive element is sensitive to a small wavelength range that includes the illumination wavelength range of the light emitter 12, such as the NIR or SWIR domain. Like the display substrate 10, the detection substrate 20 is transparent in the visible range of the electromagnetic spectrum and can be formed from a foil, a film, a silicon dioxide or a glass substrate.

[0044] The transparent display 1 also includes an optical substrate 30, on which an optical modulator 31 is arranged. The optical modulator 31 implements an optical encoder for lensless imaging. For example, the optical modulator 31 is a passive modulator and is implemented by an amplitude mask, a phase mask, or a plurality of diffraction elements. Examples include arrangements of pinholes, diffusers, Fresnel zone plates, and coded aperture masks, characterized by, for example, a uniform redundant array or an optimized random pattern. The optical modulator 31 is engineered to manipulate only light in or near an illumination range (e.g., a NIR or SWIR domain), however, the optical modulator 31 is transparent to light in the visible domain.

[0045] The layers formed from the display substrate 10, the detection substrate 20 and the optical substrate 30 are arranged in a stacked manner, optionally with spacers between them, to form a final transparent display with lensless imaging capabilities, as depicted in panel (a). The transparent display 1 may include additional transparent layers, such as a transparent circuit substrate including active and passive circuits for operating the display 1.

[0046] Figure 2A display substrate 10 of a first exemplary embodiment of a transparent display 1 with lensless imaging capability is shown. Therein, the display substrate 10 includes the aforementioned matrix arrangement of display pixels 11 formed by sub-pixels 11a, 11b, 11c, which are used to emit light in different sub-domains of the visible spectrum. For example, the sub-pixels 11a, 11b, 11c are formed from red, green and blue emitting diodes, respectively, where the diodes can be, for example, micro-LEDs. In this embodiment, each pixel 11 also includes an irradiation light emitter 12 (e.g., a micro-LED emitting NIR), and a photosensitive element 21 (e.g., a silicon-based micro-photodiode). Alternatively, the sub-pixels 11a, 11b, 11c, the irradiation light emitter 12 and the photosensitive element 21 are organic diodes. In addition, instead of each pixel 11 of the display 1 including a light emitter 12 and a photosensitive element 21, only some pixels may include a photosensitive element 21, such as pixels 11 arranged in the central part of the display 1, and only some pixels 11 may include an irradiation light emitter 12, such as pixels 12 located in the periphery of the display 1.

[0047] Figure 3 1 shows an exploded view of a first exemplary embodiment of a transparent display 1 comprising a display substrate 10 with active optical elements and an optical substrate 30 with an optical modulator 31. Figure 1 Compared to the embodiment of FIG. 1 , in this embodiment no dedicated detection substrate 20 is needed. Therefore, these embodiments provide a compact solution, especially in the case where the display pixels 11, the light emitters 12 and the light sensitive elements 21 are based on the same technology (i.e., for example, all elements are semiconductor-based or organic).

[0048] Figure 4 A display substrate 10 of a second exemplary embodiment of a transparent display 1 with lensless imaging capability is shown. Therein, the display substrate 10 includes the aforementioned matrix arrangement of display pixels 11 formed by sub-pixels 11a, 11b, 11c, which are used to emit light in different sub-domains of the visible spectrum. For example, the sub-pixels 11a, 11b, 11c are formed from red, green and blue emitting diodes, respectively, where the diodes can be, for example, micro-LEDs. In this embodiment, each pixel 11 also includes an irradiation light emitter 12, such as a micro-LED that emits NIR. Alternatively, the sub-pixels 11a, 11b, 11c and the irradiation light emitter 12 are organic diodes. In addition, instead of each pixel 11 of the display 1 including a light emitter 12, only some pixels 11 can include an irradiation light emitter 12, such as pixels 12 located in the periphery of the display 1.

[0049] In this second embodiment, the photosensitive elements 21 are arranged on a separate detection substrate 20. This may be beneficial if the display pixels 11 and the light emitters 12 are based on a different technology than the photosensitive elements 21. For example, the elements on the display substrate are semiconductor-based, while the photosensitive elements are organic, or vice versa, such as due to manufacturing reasons, a combined arrangement is not possible. However, the advantage of choosing different technologies may be adaptation to specific applications, since organic elements are known to have lower power consumption compared to semiconductor-based devices at the expense of sensitivity and / or efficiency. The matrix arrangement of the photosensitive elements 21 may correspond to the matrix arrangement of the display pixels 11. However, different arrangements in terms of the number of rows and columns and / or spacing may also be chosen. In a further alternative embodiment, the illumination light emitters 12 may be arranged next to the photosensitive elements 21 on the detection substrate 20.

[0050] Figure 5 An exploded view of a second exemplary embodiment of a transparent display 1 is shown, similar to Figure 1 In the embodiment of the present invention, the transparent display 1 includes a display substrate 10 having an emitting optical element, a detection substrate 20 including a photosensitive element 21, and an optical substrate 30 having an optical modulator 31.

[0051] Figure 6 A third exemplary embodiment of a transparent display 1 having lens-free imaging capability is shown. Figure 2 Compared to the embodiment of , in this embodiment, the optical modulator 31 is an active element, i.e., an active matrix. Therefore, the display 1 also includes a controller 32 electrically coupled to the optical modulator 31 for controlling the elements of the active matrix. The active matrix may include liquid crystals, optical switches, and / or other types of spatial and digital light processors. For example, the optical modulator 31 is based on liquid crystal technology, which can be used for programmable amplitude modulation. For programmable phase modulation, LC devices on silicon can be used in the active matrix. Other examples include vanadium oxide transistors, which act as optical switches in the NIR region while being transmissive at visible frequencies.

[0052] The programmable optical modulator 31 has the advantage that the mask pattern can be easily and quickly changed so that multiple images can be captured in a consecutive manner, each image having a different optical encoding. In addition, in embodiments with a relatively small number of photosensitive elements, the programmable modulator 31 can facilitate obtaining sufficient measurements for reconstruction. Furthermore, in the case of a sensor array of photosensitive elements, the ability to change the modulation pattern between acquisitions provides an additional degree of freedom, which can further improve the reconstruction performance or the resolution of the reconstructed image. The controller 32 can be coupled to the processing unit 101 of the imaging system to synchronize with the emission of the illumination light emitter 12 and the detection of the photosensitive element 21, i.e., the exposure phase.

[0053] Figure 7 An embodiment of an electronic device 200 including an imaging system 100 (particularly a transparent display 1) according to an improved concept is shown. As shown in the enlarged portion of the image, similar to Figure 2 In a first embodiment, a first layer (e.g., display substrate 10) includes display pixels 11 formed, for example, from the aforementioned sub-pixels 11a, 11b, 11c, illumination light emitters 12, and photosensitive elements 21. Additionally or alternatively, light emitters 12 may be arranged in the periphery, as also shown in the figure. Among other things, the transparent display 1 and imaging system 100 enable applications that require an unobstructed view of a scene or object 2 while enhancing the displayed information.

[0054] Figure 8 An alternative application is illustrated in which a transparent display 1 is integrated into the windshield of a car for enhancing the scene or objects 2 visible to the driver and / or passengers, while maintaining the absolutely necessary unobstructed view of the scene in front of the car.

[0055] Embodiments of the transparent display 1, transparent imaging system 100, and method of manufacturing a transparent display disclosed herein have been discussed for the purpose of familiarizing the reader with the novel aspects of the ideas. Although preferred embodiments have been shown and described, those skilled in the art may make changes, modifications, equivalents, and substitutions to the disclosed concepts without unnecessarily departing from the scope of the claims.

[0056] It will be appreciated that the present disclosure is not limited to the disclosed embodiments and is not limited to what has been particularly shown and described above. On the contrary, the features described in separate dependent claims or in the specification may be advantageously combined. In addition, the scope of the present disclosure includes those variations and modifications that will be clear to a person skilled in the art and that fall within the scope of the appended claims.

[0057] The term "comprising", as far as it is used in the claims or in the description, does not exclude other elements or steps of the corresponding features or processes. Where the terms "a" or "an" are used in conjunction with a feature, they do not exclude a plurality of such features. Furthermore, any reference signs in the claims shall not be construed as limiting the scope.

[0058] This patent application claims the priority of German patent application DE 102022113231.5, the disclosure content of which is hereby incorporated by reference.

[0059] Reference numerals

[0060] 1 Display

[0061] 2 Scene or object

[0062] 10Display substrate

[0063] 11 display pixels

[0064] 11a, 11b, 11c sub-pixels

[0065] 12 Illuminating light emitter

[0066] 20 Detection substrate

[0067] 21 Photosensitive element

[0068] 30 Optical substrate

[0069] 31 Optical modulator

[0070] 32 Controllers

[0071] 100 Imaging System

[0072] 101 Processing Unit

[0073] 200 electronic equipment

Claims

1. A transparent display (1) with lens-free imaging capability, comprising: - a plurality of display pixels (11) arranged on the display substrate (10), the plurality of display pixels (11) being configured to generate a display image in the visible domain of the electromagnetic spectrum; - a plurality of light emitters (12) arranged on the display substrate (10) and configured to illuminate the scene or object (2) with electromagnetic radiation in an illumination wavelength range outside the visible domain; - a plurality of photosensitive elements (21) configured to capture electromagnetic radiation received from a scene or object (2) within an illumination wavelength range and to generate optical signals dependent on the captured electromagnetic radiation; as well as - an optical modulator (31) arranged on the incident side of the plurality of photosensitive elements (21) and configured to transmit electromagnetic radiation in the visible domain and to modulate electromagnetic radiation in the illumination wavelength range; and - wherein the display is substantially transparent in the visible domain.

2. The transparent display (1) according to claim 1, wherein the illumination wavelength range is in the near infrared (NIR) domain.

3. The transparent display (1) according to claim 1 or 2, wherein the light emitter (12) is an OLED, a microLED, or a vertical cavity surface emitting laser (VCSEL).

4. The transparent display (1) according to one of claims 1 to 3, wherein the plurality of light sensitive elements (21) are arranged on a display substrate (10).

5. The transparent display (1) according to one of claims 1 to 3, further comprising a detection substrate (20), wherein the plurality of photosensitive elements (21) are arranged on the detection substrate (20).

6. The transparent display (1) according to one of claims 1 to 5, wherein the plurality of display pixels (11) forms an OLED display, a micro-LED display, or a liquid crystal display (LCD).

7. The transparent display (1) according to one of claims 1 to 6, wherein the light-sensitive element (21) is a silicon-based photodiode or an organic photodetector OPD.

8. The transparent display (1) according to one of claims 1 to 7, further comprising an optical substrate (30), wherein the optical modulator (31) is arranged on the optical substrate (30).

9. The transparent display (1) according to one of claims 1 to 8, wherein the optical modulator (31) is an active matrix based on one of liquid crystals, optical switches, digital light processors and spatial light processors.

10. The transparent display (1) according to one of claims 1 to 8, wherein the optical modulator (31) is a passive matrix based on one of an amplitude mask, a phase mask and a plurality of diffractive elements.

11. The transparent display (1) according to one of claims 1 to 10, wherein the optical modulator (31) is realized by a plurality of spatially distributed pinholes.

12. The transparent display (1) according to one of claims 1 to 11, wherein the optical modulator (31) is formed from a dye-based polymer.

13. The transparent display (1) according to one of claims 1 to 12, wherein the optical modulator (31) forms a coded aperture mask, characterized in particular in that Uniform Redundant Array URA or Optimized Random Pattern ORA.

14. A transparent display (1) according to one of claims 1 to 13, wherein the substantial transparency of the display (1) in the visible domain is achieved by a distributed arrangement of display pixels (11) and photosensitive elements (21) with gaps therebetween, such that a substantial amount of electromagnetic radiation in the visible domain incident on the display (1) is transmitted.

15. A transparent imaging system (100), comprising: - A transparent display (1) according to one of claims 1 to 14; and - a processing unit (101) coupled to the display (1) and configured to reconstruct the image by applying an algorithm to the light signal.

16. The transparent imaging system (100) according to claim 15, wherein the processing unit (101) is further configured to control generation of a display image based on the reconstructed image.

17. An electronic device (200) comprising the transparent display (1) according to one of claims 1 to 14 or the transparent imaging system (100) according to claim 15 or 16.

18. A method of manufacturing a transparent display (1) having lens-free imaging capability, the method comprising: - arranging a plurality of display pixels (11) on the display substrate (10), the plurality of display pixels (11) being configured to generate a display image in the visible domain of the electromagnetic spectrum; - providing a plurality of light emitters (12) configured to illuminate a scene or object (2) with electromagnetic radiation in an illumination wavelength range outside the visible domain, wherein the plurality of light emitters (12) are arranged on the display substrate (10); - providing a plurality of photosensitive elements (21) configured to capture electromagnetic radiation received from a scene or object (2) within an illumination wavelength range and to generate optical signals dependent on the captured electromagnetic radiation; as well as - providing an optical modulator (31) arranged on the incident side of the plurality of photosensitive elements (21) and configured to transmit electromagnetic radiation in the visible domain and to modulate electromagnetic radiation in the illumination wavelength range; and - wherein the display (1) is substantially transparent in the visible domain.

Images