Display device for ambient light sensing and methods of operating and manufacturing thereof

Abstract

A display device (1) is provided that comprises a substrate (10) comprising a main surface (12) and a plurality of emitters (20) arranged in pixels (2) on the main surface (12) of the substrate (10), wherein each emitter (20) comprises a light emitting diode, LED, that is configured to emit light during operation. It further comprises a sensor (30) in at least one pixel (2), the sensor (30) being configured to sense light during operation. It further comprises a filter (70) arranged on or above the sensor (30), the filter (70) being configured to transmit light of a predetermined wavelength range to the sensor (30). Methods of operating and manufacturing such a display device (1) are also provided. The emitters and sensors can be implemented as micro-LEDs.

Description

[0001] Description

[0002] DISPLAY DEVICE FOR AMBIENT LIGHT SENSING AND METHODS OF

[0003] OPERATING AND MANUFACTURING THEREOF

[0004] A display device, a method of operating a display device and a method of manufacturing a display device are specified.

[0005] Displays, in particular micro-LED displays, have many metal layers to fan out the driver signal to the emitters. They are therefore not very transparent, so that behind display sensing technologies cannot be used as in e.g. OLED displays. An ambient light sensor, ALS, can be arranged outside the display, e.g. at a notch of the display.

[0006] At least one object of particular embodiments is to provide in-display sensing.

[0007] This object is achieved by the subject-matter and the methods according to the independent claims. Advantageous embodiments and developments of the subject-matter and the method are characterized in the dependent claims, and are also disclosed by the following description and the drawings.

[0008] According to at least one embodiment, a display device is provided .

[0009] The display device comprises a display. For example, the display device is or is integrated in a touch screen, a smartwatch, a smartphone, a tablet, a dashboard, an automotive dashboard. According to at least one embodiment , the display device comprises a substrate comprising a main surface .

[0010] The substrate may also be called carrier or backplane . For example , the substrate is implemented as silicon backplane or as flex-plane . The substrate can be or can comprise a foil , for instance . Thus , the substrate can be rigid or flexible . The substrate has a main plane of extension . Here and in the following, lateral directions refer to directions that run parallel to the main plane of extension . The vertical direction refers to a direction that runs perpendicular to the main plane of extension . The main surface of the substrate is parallel to the main plane of extension of the substrate .

[0011] According to at least one embodiment , the display device comprises a plurality of emitters arranged in pixels on the main surface of the substrate .

[0012] This can mean that the emitters are arranged in groups . The pixels may form rectangular or square regions on the main surface of the substrate , wherein said regions are adj acent to each other . The pixels form the display of the display device . Each pixel can comprise one or more emitters . Further, each pixel can comprise wirings to electrically connect the emitters . The pixels and / or the emitters in the pixels can be operated individually or in groups . That the emitters are arranged on the main surface of the substrate can mean that they are mounted on the main surface . This can mean that they are not integrated or embedded in the substrate . For example , they are attached to the substrate by means of a trans fer process , in particular a mass trans fer process, by which a plurality of emitters are attached to the substrate simultaneously.

[0013] According to at least one embodiment, each emitter comprises a light emitting diode, LED, that is configured to emit light during operation.

[0014] Here and in the following, light may refer to electromagnetic radiation in general. Each emitter is for example configured to emit electromagnetic radiation in a main emission direction during intended operation. The main emission direction is preferably oriented perpendicular to the main surface of the substrate. In particular, each emitter is configured to emit electromagnetic radiation having a main wavelength in the spectral region visible to the human eye, i.e. between 380 nm and 750 nm. Different emitters may emit light in different wavelength ranges. Each pixel may comprise one or more emitters, thus one or more LEDs. Each pixel may comprise at least three LEDs (e.g. red, green, blue) for covering the visible wavelength range. Each pixel may also comprise more than three LEDs to provide a certain degree of redundancy and / or additional functionality.

[0015] According to at least one embodiment, the display device comprises a sensor in at least one pixel, the sensor being configured to sense light during operation.

[0016] In other words, the display device comprises at least one sensor. In other words, at least one pixel of the plurality of pixels in the display device comprises a sensor. This can mean that said pixel comprises only the sensor or the sensor in addition to one or more emitters. In particular, said pixel comprises the sensor and at least three emitters to cover the visible wavelength range. That the sensor is configured to sense light during operation can mean that the sensor is configured to sense ambient light, but not display light, i.e. light emitted by the emitters. The sensor may be sensitive to a particular wavelength range in the visible spectrum. The sensor can have a viewing angle. In particular, the viewing angle includes the vertical direction, such that the sensor is a "top-looking" sensor. The field of illumination of the emitters and the field of view of the sensor may at least overlap. The display device may comprise more than one sensor that are distributed in respective pixels of the display device. It is also possible that more than one sensor is arranged in a single pixel. In case of two or more sensors in the display device the sensors can be sensitive to light in different wavelength ranges.

[0017] According to at least one embodiment, the display device comprises a filter. The filter is arranged on or above the sensor. The filter is configured to transmit light of a predetermined wavelength range to the sensor.

[0018] The filter can be referred to as wavelength filter. The filter is arranged in the vertical direction on or above the sensor. If the display device comprises more than one sensor, each sensor can be provided with a respective filter. In this case, the filters can be different, such that light of different wavelength ranges is transmitted to the sensor assigned to the respective filter. The filter is configured to filter light, in particular ambient light, before it hits the sensor. This can mean that only a portion of ambient light, in particular the portion corresponding to the predetermined wavelength range, reaches the sensor. Light of wavelengths outside the predetermined wavelength range is prevented from reaching the sensor .

[0019] According to at least one embodiment , a display device comprises a substrate comprising a main surface . It further comprises a plurality of emitters arranged in pixels on the main surface of the substrate , wherein each emitter comprises a light emitting diode , LED, that is configured to emit light during operation . It further comprises a sensor in at least one pixel , the sensor being configured to sense light during operation . It further comprises a filter arranged on or above the sensor, the filter being configured to transmit light of a predetermined wavelength range to the sensor .

[0020] The display device described here is based on the following considerations , among others .

[0021] Displays , in particular micro-LED displays , have many metal layers to fan out the driver signal to the emitters . They are therefore not very transparent and behind display sensing technologies cannot be used as in e . g . OLED displays . It is therefore beneficial to use in-display sensing technology, where the sensor is embedded and integrated into the display such that no display notch is needed . In other words , integrating the sensor into the display removes any notches from the display without the need to enable behind display sensing .

[0022] The sensor can be embedded into the display . The sensor can be used for wavelength dependent ambient light sensing, ALS . I f several pixels comprise a respective sensor, the display can be partially or entirely populated with the additional sensors . Thus , a distributed ambient light sensor is formed . While a localized ambient light sensor does not work when it is covered, a distributed (delocalized) in-display sensor has redundancies and can also work when the display (e.g. of a smartphone) is partially covered.

[0023] Sensing can be synchronized to the pulse width modulation (PWM) of the emitter LEDs to prevent backscattered display light from being sensed. This allows to distinguish between ambient light and display light.

[0024] In-display wavelength dependent sensing is provided. Possible applications are auto white balance, AWB, for camera, AWB for display, spectral sensing and / or ambient light sensing, ALS . In particular, it is possible to manage and calibrate display color appearance (auto white balance) locally and directly.

[0025] According to at least one embodiment, each emitter is formed by a micro-LED.

[0026] As a broad definition, a micro-LED could be seen as any light emitting diode (LED) - generally not a laser - with a particularly small size. As a rule - and this is a very important criterion in addition to size - a growth substrate is removed from micro-LEDs, so that typical heights of such micro-LEDs are in the range of 0.5 pm to 10 pm, for example. In principle, a micro-LED does not necessarily have to have a rectangular radiation emission surface. Generally, for example, an LED could have a radiation emission surface in which, in plan view of the layers of the layer stack, any lateral extent of the radiation emission surface is less than or equal to 100 pm or less than or equal to 70 pm.

[0027] For example, in the case of rectangular micro-LEDs, an edge length - especially in plan view of the layers of the layer stack - smaller than or equal to 70 qm or smaller than or equal to 50 qm is often cited as a criterion . Mostly, such micro-LEDs are provided on wafers with - for the qLED nondestructive^ - detachable holding structures . At present , micro-LEDs are mainly used in displays . The micro-LEDs form pixels or subpixels and emit light of a defined color . Small pixel si ze and a high density with close distances make micro-LEDs suitable , among others , for small monolithic displays for AR applications , especially data glasses . In addition, other applications are being developed, in particular regarding the use in data communication or pixelated lighting applications . Di f ferent ways of spelling micro-LED, e . g . qLED, q-LED, uLED, u-LED or micro light emitting diode can be found in the relevant literature .

[0028] According to at least one embodiment , the pixels are formed as RGB-pixels , in which a first emitter configured to emit red light , a second emitter configured to emit green light , and a third emitter configured to emit blue light are arranged .

[0029] In particular, each emitter is intended to emit radiation of a di f ferent main wavelength . A main wavelength is to be understood as a wavelength in an emission spectrum where the intensity reaches a global maximum . For example , a first emitter is intended to emit radiation of a red color, a second emitter is intended to emit radiation of a green color, and a third emitter is intended to emit radiation of a blue color in its intended operation . Such a red, green, blue ( short : RGB ) arrangement is especially suitable for a pixel of a display device . Red light may refer to light in a wavelength range from 640 nm to 780 nm, for example . Green light may refer to light in a wavelength range from 490 nm to 570 nm, for example. Blue light may refer to light in a wavelength range from 430 nm to 490 nm. Thus, the entire visible spectrum is covered by additive color mixing of the three components R, G and B. As mentioned above, the emitters may be formed by respective LEDs, in particular micro-LEDs. Depending on the light color the LEDs can comprise different semiconductor materials, as for example gallium arsenide (GaAs) , gallium arsenide phosphide (GaAsP) , gallium phosphide (GaP) , aluminum indium gallium phosphide (AlInGaP) , indium gallium nitrogen (InGaN) and / or gallium nitride (GaN) . In a possible implementation, each pixel is formed as an RGB-pixel comprising a red light emitter, a green light emitter and a blue light emitter. The pixel can also comprise more than one emitter for each color, for example to provide a certain degree of redundancy.

[0030] According to at least one embodiment, the sensor is formed as a photodiode, wherein the photodiode is embedded in the substrate .

[0031] This can mean that the photodiode, PD, is integrated in the substrate and / or formed in the substrate. Thus, the photodiode is an embedded photodiode. In particular, if the substrate comprises silicon, the photodiode can be a silicon photodiode, but other material are also possible. For efficiency reasons, the silicon photodiode can have a larger size in lateral directions and can be thicker than the emitters. Silicon photodiodes are suitable to detect visible light. The substrate on which the emitters are arranged thus has additional sensing capabilities. The emitters can be arranged on regions of the substrate outside the photodiode ( s ) . Alternatively, the photodiode is arranged on the main surface of the substrate next to the emitters .

[0032] Such a photodiode can form a separate detector chip . In particular, the photodiode can be a micro-PD . This can mean that the photodiode has a similar si ze as each of the emitters , in particular i f they are formed as micro-LEDs . It is also possible that the photodiode has a larger si ze for ef ficiency reasons . I f the photodiode is arranged on the substrate , its thickness can preferably be similar or equal to the thickness of the emitters . That the photodiode is arranged on the substrate can mean that it is mounted on the substrate , for example by means of a (mass- ) trans fer process . That the photodiode is arranged next to the emitters can mean that the photodiode and the emitters are arranged in a common pixel .

[0033] According to at least one embodiment , the sensor is implemented as an LED, in particular micro-LED that is arranged on the main surface of the substrate next to the emitters and configured to be biased for sensing a photocurrent .

[0034] That the sensor is implemented as (micro- ) LED can mean that the sensor and the emitters have a same or similar structure . In particular, the sensor and the emitters may be formed by a common semiconductor layer stack . Designed as an LED, the sensor can emit light when appropriately biased, in particular i f the sensor is forward biased . However, i f the sensor is zero biased or reverse biased, it is configured to sense a photocurrent depending on light hitting the sensor . It has been found that the external quantum ef ficiency, EQE , of red, green and blue light emitting micro-LEDs in reverse or zero bias reach values above 20% depending on the wavelength to be detected . Hereby, red, green / blue light emitting micro-LEDs used in combination as sensors span the entire visible spectrum . In other words , the EQE curves of the blue or green and the red micro-LED match together to span the visible wavelength range . Thus , additional red or blue / green micro-LEDs can be placed in some or all pixels for sensing . As mentioned above , the pixels may comprise additional (micro- ) LEDs for redundancy reasons anyway . These additional emitters can be used as sensors , thus reducing complexity of the manufacturing process . The (micro- ) LEDs can be operated and biased depending on whether they are used as emitters or as sensors . Dynamic use as emitter and sensor is also possible . This means that an emitter can be used as sensor by dynamically changing its electrical bias , e . g . by integrating a multiplexer in the driving circuit . Biasing LEDs for sensing a photocurrent represents a simple and cost-ef ficient solution to provide sensing capabilities of a display device . By using reverse or zero biases micro-LEDs additional components other than display components are not required .

[0035] According to at least one embodiment , the emitters and the sensor are manufactured according to a common manufacturing process .

[0036] This can mean that at least some process steps for manufacturing the emitters are also used for manufacturing the sensor . This is in particular true i f both the emitters and the sensor are formed as (micro- ) LEDs . Thus , the emitters and the sensor can be fabricated on a common carrier substrate . They can simultaneously be trans ferred to the substrate of the display device . Therefore , fabrication of the display device is facilitated and costs are reduced . It is also possible to trans fer all display LEDs with double density for redundancy to avoid broken LED repair . LEDs that are not used for emitting light but are functional can be used for sensing . This would mean that the sensor has a coverage of less than 100% but could still have a satis factory functionality .

[0037] According to at least one embodiment , the display device comprises a plurality of sensors being configured to sense light during operation, wherein the sensors are distributed over the display in respective pixels of the display device , and wherein each sensor is provided with a respective filter configured to transmit light of a predetermined wavelength range to the sensor .

[0038] All features disclosed for the sensor are also disclosed for the plurality of sensors . The above mentioned sensor can be one of the plurality of sensors . Correspondingly, all features disclosed for the filter are also disclosed for the respective filter assigned to each of the sensors . In a possible implementation, each pixel comprises at least one sensor . However, it is also possible that only a subset of pixels comprises at least one sensor . For example , each second or each third pixel comprises a sensor . Preferably, the sensors are evenly distributed . The sensors can be the same or di f ferent . That the sensors are di f ferent can mean that their sensitivities for particular wavelengths di f fer from each other . However, at least some of the sensors can have same or similar sensitivities for particular wavelength ranges . In addition, the respective filters assigned to each sensor can be the same or di f ferent . This can mean that the filters transmit light in a same or in a di f ferent wavelength range. In particular, at least some of the filters are the same. In particular, at least some of the filters are different. The plurality of sensors arranged in respective pixels form a distributed spectral sensor that has more redundancies than a local sensor (e.g. at a notch of the display or a dedicated area behind the display) . Further, it is less sensitive to disturbances, e.g. covering the sensor by hairs, fingers, dirt or dust. Integrating the spectral sensor into the display removes any notches from the display without the need to be behind the display like e.g. behind OLED sensing. As the sensors are distributed over the display, the distributed spectral sensor has a 2D spatial resolution .

[0039] According to at least one embodiment, different spectral channels are formed by the plurality of sensors and filters, wherein each spectral channel is sensitive for a respective wavelength range.

[0040] Thus, the display device comprises a plurality of spectral channels. A particular spectral channel is formed by sensorfilter pairs that are configured for sensing light in a particular wavelength range. Combinations of different sensors and filters lead to a plurality of different spectral channels and therefore to a distributed spectral sensor of the display device.

[0041] According to at least one embodiment, at least one spectral channel is placed over the display with a density different from a density of at least one other spectral channel.

[0042] The individual channels can be placed with different density to tune or equalize the overall spectral response of the channels. For example, to compensate for a first channel having lower sensing efficiency than a second channel, more sensor-filter pairs forming the first channel can be placed in the display than sensor-filter pairs forming the second channel. In other words, the difference in EQE on the different color channels can be made up by placing more sensors of low sensitivity channels and less of high sensitivity channels.

[0043] In addition or alternatively, sensors of at least one spectral channel are driven with an amplification gain different from an amplification gain of sensors of at least one other spectral channel. The channels, in particular the respective sensors, can be driven with different amplification gains to tune or equalize the overall spectral response of the channels.

[0044] According to at least one embodiment, pixels of a first group comprise a respective sensor that is formed as blue or green micro-LED and configured to be biased for sensing a photocurrent .

[0045] According to at least one embodiment, pixels of a second group comprise a respective sensor that is formed as red micro-LED and configured to be biased for sensing a photocurrent .

[0046] It is also possible that pixels comprise both blue / green and red micro-LEDs used as sensors. In other words, the display device comprises red, blue and / or green micro-LEDs used as sensors. Blue, green and red micro-LEDs refer to micro-LEDs that emit light in the blue, green and red wavelength range, respectively, if they are operated in forward bias. It has been found that the external quantum ef ficiency, EQE , of red, green and blue light emitting micro-LEDs in reverse or zero bias reach values above 20% depending on the wavelength to be detected . Blue and green micro-LEDs cover a wavelength range from about 350 nm to 500 nm, while red micro-LEDs cover a wavelength range from about 450 nm to 650 nm . Thus , the display device can use both red and blue / green micro-LEDs as sensors to cover the full visible spectrum .

[0047] According to at least one embodiment , the filter is implemented as interference filter or as color filter .

[0048] This also applies to the plurality of filters assigned to the plurality of sensors . Interference filters comprise several layers of materials with di f ferent refractive indices and thicknesses . The selection of these materials makes it possible to transmit or block certain wavelengths of light by using the interference of light waves . For example , the interference filter comprise a stack of dielectric and / or metallic layers . In case of a color filter, the filter may comprise glass , plastics or resin, by way of example , in order to selectively transmit speci fic wavelengths of light and block others .

[0049] According to at least one embodiment , the filter is arranged on the sensor .

[0050] This can mean that the filter is directly applied on the sensor such that it is in physical contact with a sensitive surface of the sensor . In this configuration, advantageously, proper filter function without side leakage is enabled . According to at least one embodiment, the display device further comprises a transparent cover covering the pixels.

[0051] The cover is transparent for visible light. For example, the cover comprises glass or a transparent plastic material. The cover can be arranged directly above the emitters and the sensor (s) . It is also possible that the cover has a distance to the emitters and the sensor (s) . The cover protects the emitter and the sensors (s) . Further, the cover may provide additional functionality, it may be formed as touch screen, for example.

[0052] According to at least one embodiment, the filter is arranged on the cover.

[0053] This can mean that the filter is attached to, mounted to or structured on the cover. The filter can be arranged on a side of the cover that faces the sensor or on a side that faces away from the sensor. It is also possible that the filter is integrally formed with the cover. Thus, it is possible to align the filter (s) with the respective sensor on or in the substrate. The sensor does not need to be further processed, e.g. by depositing and structuring filter layers on it. By placing the filter onto the cover no filter structuring on the display components itself is required.

[0054] According to at least one embodiment, the display device further comprises a control circuit.

[0055] For example, the control circuit is integrated in the substrate. It is however also possible that the control circuit is arranged on the main surface of the substrate next to the pixels or in the pixels. The control circuit can also be arranged on an opposite rear surface of the substrate and can be electrically connected to the pixels by through- substrate-vias, TSVs . Further, the control circuit can be arranged in or on a further substrate. The control circuit can comprise circuitry for driving and biasing the emitters. Thus, the control circuit can comprise driving circuitry for the emitters and can be electrically connected to at least some of the emitters in the pixels. The display device can comprise more than one control circuit. For example, each control circuit is assigned to a group of pixels, e.g. to a group of 16 x 16 pixels. The respective control circuit can drive the emitters and evaluate signals from sensors in said group of pixels.

[0056] In an embodiment, the control circuit is, based on the light sensed by the one or more sensors, configured to evaluate ambient light around the display. In particular, the color composition of the ambient light can be analyzed.

[0057] In addition or alternatively, the control circuit is, based on the light sensed by the one or more sensors, configured to evaluate flicker noise. Therefore, based on a sensed flicker frequency, the refresh rate of the display can be adjusted.

[0058] In addition or alternatively, the control circuit is, based on the light sensed by the one or more sensors, configured to evaluate color of objects close to the display. For example, light is scattered by nearby objects and strikes the sensor ( s ) .

[0059] In addition or alternatively, the control circuit is, based on the light sensed by the one or more sensors, configured to adjust a white balance of the display. White balance ensures that the display colors appear natural under di f ferent lighting conditions . It corrects the color temperature of light to ensure that white is perceived as a neutral hue . The control circuit may be configured to analyze the sensed ambient light to determine the dominant color temperature . Based on the detected color temperature , the control circuit may be configured to adj ust the emitter color channels ( e . g . red, green, blue ) .

[0060] In addition or alternatively, the control circuit is , based on the light sensed by the one or more sensors , configured to adj ust a color calibration of the display . Color calibration refers to the process of adj usting and aligning the colors produced by the display to ensure accurate and consistent color representation . This may be important because displays can vary in terms of color accuracy due to manufacturing di f ferences , aging, or other factors .

[0061] In addition or alternatively, the control circuit is , based on the light sensed by the one or more sensors , configured to dynamically adj ust color and brightness of emitted display light . This can in particular mean that a closed loop process is enabled, in which the display light is continually adj usted based on the sensed ambient light and the feedback from the system itsel f . Thus , the display device is allowed to sel f-regulate and maintain a desired state or performance .

[0062] Further, a method of operating a display device is provided . The method of operating can preferably be carried out with the display device described above . This means that all features disclosed for the display device are also disclosed for the method of operating the display device and vice versa . According to at least one embodiment , the method of operating the display device is carried out with a display device comprising a plurality of emitters arranged in pixels on a main surface of a substrate , a sensor in at least one pixel , and filter on or above the sensor .

[0063] According to at least one embodiment , the method comprises emitting light by the emitters , wherein each emitter comprises a light emitting diode , LED . In particular, light is emitted by the emitters in a main direction that runs perpendicular to the main surface of the substrate . Each emitter / LED has a field of illumination .

[0064] According to at least one embodiment , the method comprises filtering, by the filter, ambient light and transmitting light of a predetermined wavelength range to the sensor . In particular, the filter can be formed as wavelength filter . Ambient light refers to light that is not emitted by the emitters but that comes from other light sources outside the display device . Ambient light may comprise light in a plurality of wavelength ranges . The filter may filter the ambient light , such that only a portion of the originally wavelength spectrum is transmitted to the sensor .

[0065] According to at least one embodiment , the method further comprises sensing, by the sensor, light transmitted by the filter . Thus , the sensed light can be filtered ambient light . Preferably, the emitters and the sensor are arranged such that display light , i . e . light emitted by the emitters , is not sensed by the sensor . For example , light barriers are arranged between the emitters and the sensor and / or the sensor is outside the field of illumination of the emitters and / or the sensor is disabled when the emitters are on .

[0066] According to at least one embodiment , the method of operating a display device , wherein the display device comprises a plurality of emitters arranged in pixels on a main surface of a substrate , a sensor in at least one pixel , and a filter arranged on or above the sensor, comprises emitting light by the emitters , wherein each emitter comprises a light emitting diode , LED . It further comprises filtering, by the filter, ambient light and transmitting light of a predetermined wavelength range to the sensor . It further comprises sensing, by the sensor, light transmitted by the filter .

[0067] Thus , in-display sensing, in particular spectral sensing is enabled . The sensor can be embedded into the display . The sensor can be used for wavelength dependent ambient light sensing, ALS . I f several pixels comprise a respective sensor, the display can be partially or entirely populated with the additional sensors . Thus , a distributed ambient light sensor is formed . Possible applications are auto white balance , AWB, for camera, AWB for display, spectral sensing and / or ambient light sensing, ALS . In particular, it is possible to manage and calibrate display color appearance ( auto white balance ) locally and directly .

[0068] According to at least one embodiment , the method further comprises adj usting a brightness of the emitted light by pulse width modulation .

[0069] Pulse width modulation, PWM, for LEDs is a technique used to control the brightness of an LED by varying the duty cycle of a pulsating electrical signal . In PWM, the LED is rapidly turned on and off at a fixed frequency, and the average intensity of light perceived by the human eye depends on the ratio of time the LED is on (the on-state) to the time it is off (the off-state) during each cycle. PWM is an efficient and effective way to control LED brightness without changing the LED's forward current.

[0070] According to at least one embodiment, sensing light is performed when the emitters are in an off-state during the pulse width modulation.

[0071] Thus, sensing can be synchronized to the pulse width modulation of the LEDs. Thus, light emitted by emitters is prevented from being sensed by the sensor. This allows to distinguish between ambient light and display light. For ambient light sensing, flicker detection, white balance etc. the surrounding light can be measured and disturbing crosstalk from the display light can be avoided.

[0072] According to at least one embodiment, sensing light is performed continuously during the pulse width modulation and the sensed light is evaluated by filtering a detection signal in frequency space.

[0073] This can mean that light is sensed during the full PWM cycle, i.e. during the on- and the off-state of the LEDs. Thus, the detection signal contains information about ambient light and (backscattered) display light. By knowing the PWM frequency post-processing is possible to filter the display light channel out of the detection signal and to obtain the pure ambient light channel. According to at least one embodiment , the method further comprises , based on the sensed light , evaluating ambient light around the display . In addition or alternatively, the method further comprises , based on the sensed light , evaluating flicker noise . In addition or alternatively, the method further comprises , based on the sensed light , evaluating color of obj ects close to the display . In addition or alternatively, the method further comprises , based on the sensed light , adj usting a white balance of the display . In addition or alternatively, the method further comprises , based on the sensed light , adj usting a color calibration of the display . In addition or alternatively, the method further comprises , based on the sensed light , dynamically adj usting color and brightness of emitted display light .

[0074] Further, a method of manufacturing a display device is provided . All features disclosed for the display device are also disclosed for the method of manufacturing the display device and vice-versa .

[0075] The method of manufacturing the display device comprises providing a substrate . For example , the substrate is or is formed by a backplane or a flex-plane . For example , the substrate comprises silicon or a foil .

[0076] The method of manufacturing the display device further comprises mounting a plurality of emitters on a main surface of the substrate , wherein the emitters are arranged in pixels and each emitter comprises a light emitting diode , LED, configured to emit light during operation . In particular, the emitters are formed by micro-LEDs and mounting the emitters on the substrate comprises a mass trans fer of said emitters from a carrier substrate to the substrate of the display device . For example , an elastomer stamp is used for the mass trans fer .

[0077] The method of manufacturing the display device further comprises forming a sensor in at least one pixel , the sensor being configured to sense light during operation .

[0078] Forming the sensor may comprise embedding a photodiode in the substrate . This can mean, for example , that the substrate is semiconductor substrate and the photodiode is formed in the substrate by a semiconductor manufacturing process , e . g . a CMOS process . Thus , the photodiode is integrated in the substrate . Alternatively, forming the sensor may comprise mounting a photodiode on the main surface of the substrate next to the emitters . This can mean that the photodiode forms a single detector chip that is trans ferred onto the substrate and attached to it . For example , the photodiode is a microphotodiode and trans ferred to the substrate in a similar way as the emitters , in particular by means of a further mass trans fer . As a further alternative , forming the sensor may comprise mounting an LED on the main surface of the substrate next to the emitters , and biasing said LED for sensing a photocurrent . Said " sensor-LED" may have a similar or same structure as the "emitter-LEDs" . Thus , the sensor and the emitters may be formed by a common manufacturing process and they may be trans ferred simultaneously to the substrate . The sensor and the emitters may di f fer only in the way they are biased . For example , the sensor is a reverse biased or zero biased (micro- ) LED .

[0079] The method of manufacturing the display device further comprises forming a filter on or above the sensor, the filter being configured to transmit light of a predetermined wavelength range to the sensor . The filter can be formed by depositing one or more filter layers on the sensor, or on a transparent cover that covers the emitters and the sensor . The filter can be implemented as multi-layer interference filter or as color filter, by way of example . The filter may be referred to as wavelength filter .

[0080] Advantageously, a display device with in-display spectral sensing can be manufactured . In case of more than one sensor, a distributed ambient light sensor, ALS , is formed that has multiple redundancies compared to a single ALS in a notch of a display . It also works when the sensor is partially covered . Further, an in-display sensor can manage and calibrate the display color appearance ( auto white balance ) locally and directly . I f (micro- ) LEDs are used as sensors , fabrication can be facilitated, as both emitters and sensor can have the same structure and can be manufactured according to a common manufacturing process .

[0081] According to at least one embodiment , the method further comprises forming recesses on the main surface of the substrate , wherein in each recess a respective emitter or sensor is mounted .

[0082] In case of a silicon or glass substrate , forming recesses can comprise an etching process . Otherwise it is also possible , that recesses are formed by a molding process or a thermocompression, for example . The emitters and / or the sensor can be placed into the recesses by a pick-and-place process or, preferably, by a mass trans fer process , e . g . using an elastomer stamp . I f the emitters and the sensor are arranged in recesses of the substrate , a planari zed surface of the substrate can be provided . The following description of Figures may further illustrate and explain aspects of the display device , the operating method and the manufacturing method . Components and parts of the display device that are functionally identical or have an identical ef fect are denoted by identical reference symbols . Identical or ef fectively identical components and parts might be described only with respect to the Figures where they occur first . Their description is not necessarily repeated in successive Figures .

[0083] Figures 1 to 3 show display devices according to embodiments of the present invention .

[0084] Figure 4 shows the external quantum ef ficiency of micro-LEDs i f biased for sensing a photocurrent .

[0085] Figures 5 to 7 show display devices according to other embodiments of the present invention .

[0086] Figure 8 show transmission characteristics of di f ferent filters .

[0087] Figure 9 show the transmission characteristics of Figure 8 folded with the external quantum ef ficiencies according to Figure 4 .

[0088] Figure 10 shows a display device according to another embodiment of the present invention .

[0089] Figure 11 shows a method of operating a display device according to an embodiment . Figure 12 shows a method of manuf cturing a display device according to an embodiment .

[0090] Figure 1 shows a cross-sectional view of a display device 1 according to an embodiment . The display device 1 comprises a substrate 10 with a main surface 12 . A plurality of emitters 20 is arranged in pixels 2 on the main surface 12 of the substrate 10 , wherein each emitter 20 comprises a light emitting diode , LED, that is configured to emit light during operation . Further, a sensor 30 is integrated in at least one of the pixels 2 , the sensor 30 being configured to sense light during operation . Possibly, Figure 1 shows a crosssection of only one pixel 2 . The display device further comprises a filter 70 arranged on or above the sensor 30 , the filter 70 being configured to transmit light of a predetermined wavelength range to the sensor 30 .

[0091] For example , each emitter 20 is formed by a micro-LED . Further, the sensor 30 may be formed as a photodiode , wherein the photodiode is arranged on the main surface 12 of the substrate 10 next to the emitters 20 . Alternatively, the sensor 30 is implemented as an LED, in particular micro-LED, that is arranged on the main surface 12 of the substrate 10 next to the emitters 20 and configured to be biased for sensing a photocurrent . However, i f the sensor 30 is implemented as photodiode , it is also possible that the photodiode is embedded in the substrate 10 (not shown) . Further, the filter 70 can be implemented as interference filter or as color filter .

[0092] In the embodiment of Figure 1 , the display device 1 further comprises a transparent cover 60 . The cover 60 is arranged in a vertical direction above the main surface 12 of the substrate 10 , such that the emitters 20 and the sensor 30 are covered . The filter 70 is arranged on the cover 60 . As shown in Figure 1 , the filter can be arranged at a side of the cover 60 facing the sensor 30 .

[0093] In Figure 2 another embodiment of the display device 1 is shown . The embodiment of Figure 2 is di f ferent from the embodiment of Figure 1 in that the filter is arranged directly on the sensor 30 .

[0094] Figures 3a and 3b show di f ferent embodiments of a respective pixel 2 within the display device 1 in a top view . According to Figures 3a and 3b the pixel 2 is formed as RGB-pixel 2 , in which a first emitter 20- 1 configured to emit red light , a second emitter 20-2 configured to emit green light , and a third emitter 30-3 configured to emit blue light are arranged . The emitters 20 are formed as LEDs , in particular micro-LEDs . The emitters 20 are arranged along a diagonal of the pixel 2 . However, di f ferent arrangements are also possible . Further, the pixels 2 of Figures 3a and 3b comprise a respective sensor 30 arranged along the diagonal of the emitters 20 . The sensor 30- 1 of the pixel 2 according to Figure 3a can be formed by an LED, in particular micro-LED, that can emit blue or green light in forward bias mode . However, said LED is configured to be biased for sensing a photocurrent . That is , said LED is configured to be reverse biased or zero biased . It has been found that blue or green micro-LEDs can generate a photocurrent in reverse bias with an external quantum ef ficiency of about 20% or more in a wavelength range between 350 nm and 500 nm . The sensor 30-2 of the pixel 2 according to Figure 3b is formed by an LED, in particular micro-LED, that can emit red light in forward bias mode . However, said LED is configured to be biased for sensing a photocurrent . That is , said LED is configured to be reverse biased or zero biased . It has been found that red micro-LEDs can generate a photocurrent in reverse bias with an external quantum ef ficiency of about 20% or more in a wavelength range between 500 nm and 650 nm . Thus , by means of reverse biased or zero biased (micro- ) LEDs for red and blue / green light the visible wavelength range is covered .

[0095] This is illustrated in Figure 4 showing the external quantum ef ficiency of micro-LEDs as function of the wavelength X . It can be seen that the blue (B ) and green ( G) micro-LEDs cover a wavelength range from about 350 nm to 500 nm, while red (R) micro-LEDs cover the wavelength range from about 450 nm to 650 nm . Thus , the display device 1 can use both red and blue / green micro-LEDs as sensors 30 to cover the full visible spectrum .

[0096] The display device 1 may comprise di f ferent groups of pixels 2 , wherein pixels 2 of a first group comprise a blue / green LED used as sensor 30 , and wherein pixels 2 of a second group comprise a red LED used as sensor 30 . It is also possible that blue / green LEDs and red LEDs used as sensors 30 are formed in the same pixel 2 . As both the emitters 20 and sensors 30 can be implemented as (micro- ) LEDs , they can be manufactured according to a common manufacturing process .

[0097] For example , each pixel 2 or at least a subset of the plurality of pixels 2 comprises a respective sensor 30 . This means that the display device 1 can comprise a plurality of sensors 30 . Each sensor 30 can be provided with a respective filter 70 configured to transmit light of a predetermined wavelength range to the sensor 30 . Thus , the sensor area can be large and a distributed spectral sensor 30 can be formed . A distributed spectral sensor has more redundancies than a local sensor and is less sensitive to disturbances . Integrating the sensor 30 into the display removes any notches from the display without the need to arrange the sensor behind the display like e . g . behind OLED sensing . The display has not to be transparent .

[0098] Figures 5 to 7 show further embodiments of the display device 1 , in which recesses 80 are formed on the main surface 12 of the substrate 10 . In each recess 80 a respective emitter 20 or sensor 30 is mounted .

[0099] In Figure 5 it is illustrated that a metal coating 90 can be deposited on the sidewalls of the recess 80 and on the base of the recess 80 between the substrate 10 and the emitter 20 or the sensor 30 , respectively . The metal coating 90 can act as bottom electrode for the emitter 20 or the sensor 30 and as reflector . Further, a transparent filler 92 can be arranged on lateral sides of the emitter 20 / sensor 30 to fill the remaining space of the recess 80 . In addition, a transparent top electrode 94 is arranged on the main surface 12 of the substrate 10 , the transparent filler 92 and the emitter 20 / sensor 30 .

[0100] Figure 6 further shows the transparent cover 60 , on which a filter 70 is structured on a side facing the sensor 30 . As there is space between the filter 70 and the sensor 30 , light leakage from lateral sides is possible , as shown by an arrow .

[0101] Light leakage can be prevented, i f the filter 70 is patterned directly on top of the sensor 30 , as shown in Figure 7 . Multi-layer interference filters 70 or metal filters 70 or color filters 70 can be micro-patterned using standard lithography procedures. For example, patterning of the filters 70 can be done after the sensor transfer onto the display plane and after top contacting.

[0102] If the display device 1 comprises a plurality of sensors 30 and filters 70, different spectral channels are formed by the sensor-filter pairs. Each spectral channel is sensitive for a respective wavelength range. The spectral channels are distributed over the entire display with one or more channels in a pixel 2.

[0103] The filters 70 can be different, which means that they can exhibit different transmission characteristics. Figure 8 shows exemplary filter transmission curves for ten different filters Fl to F10 in dependence of the wavelength X. Figure 9 shows these filter transmission curves folded with the spectral quantum efficiency of red and blue micro-LEDs as shown in Figure 4. Filters Fl to F3 are placed on reverse or zero biased blue micro-LEDs and Filters F4 to F10 are placed on reverse or zero biased red micro-LEDs. It can be seen in Figure 9 that spectral responses to certain wavelength ranges may differ. To tune or equalize the overall spectral response of the spectral channel, the individual channels can be placed with different density or can be driven with different amplification gains.

[0104] Figure 10 shows another embodiment of the display device 1 in a top view. The display device 1 comprises a plurality of pixels 2 forming the display. The plurality of pixels 2 are arranged in an array on the main surface 12 of the substrate 10. The display device 1 further comprises a control circuit 40. Based on the light sensed by the one or more sensors 30, the control circuit 40 may be configured to evaluate ambient light around the display, and / or evaluate flicker noise , and / or evaluate color of obj ects close to the display, and / or adj ust a white balance of the display, and / or adj ust a color calibration of the display, and / or dynamically adj ust color and brightness of emitted display light . The control circuit 40 is electrically connected to the substrate 10 and the pixels 2 on the substrate 10 by means of electrical interconnections 45 . The electrical interconnections 45 can be formed as flex connection 45 . Further, the control circuit 40 and the substrate 10 can be arranged on a common carrier 15 . Alternatively (but not shown) , the control circuit can be arranged in or on the substrate 10 .

[0105] Figure 11 shows a possible operating method for the display device 1 . The method comprises emitting light by the emitters 20 , wherein each emitter 20 comprises an LED . In the shown example the emitters 20 comprise a first emitter 20- 1 configured to emit red light , a second emitter 20-2 configured to emit green light and a third emitter 20-3 configured to emit blue light . A brightness of the emitted light can be adj usted by pulse width modulation, PWM, as shown in Figure 11 . The brightness can be adj usted separately for each of the emitters 20 , in particular the red LED 20- 1 , the green LED 20-2 and the blue LED 20-3 . However, the on- states and of f-states of the LEDs may be synchroni zed, such that there is a common on-state phase and a common of f-state phase . In other words , a PWM frequency f may be the same for all emitters 20 , as shown in Figure 11 . To detect ambient light AL by means of the sensor 30 , the detection is locked to time frames TA, in which the display is not on . The sensor 30 may be disabled when the emitters 20 are on . The on-states of the display are given by the pulse width modulation frequency and duty cycle . Having a dedicated ambient light sensing window TA may refer to a PWM lock-in detection mode of the display device 1 . Alternatively, ambient light sensing is performed continuously during the pulse width modulation and the sensed light is evaluated by filtering a detection signal in frequency space . Thus , sensed ambient light can be distinguished from backscattered display light . Before being sensed, the ambient light is filtered by the filter 70 , such that only light of a predetermined wavelength range is transmitted to the sensor 30 .

[0106] With Figure 12 a manufacturing process of the display device is shown schematically . The method comprises the following steps that are not necessarily carried out in this order but can be carried out in this order .

[0107] In a first step S I , a substrate is provided . For example , the substrate is implemented as silicon backplane or as flexplane .

[0108] In an optional second step S2 , recesses 80 are formed on a main surface 12 of the substrate 10 . For example , the recesses are formed by etching the substrate 10 . It is also possible that the recesses are formed by molding, e . g . imprint molding, thermo-compression molding or inj ection molding .

[0109] In a third step S3 , a plurality of emitters 20 are mounted on the main surface 12 of the substrate 10 , wherein the emitters 20 are arranged in pixels 2 and each emitter 20 comprises a light emitting diode , LED, configured to emit light during operation . Mounting the emitters 20 can comprise a trans fer process , in particular mass trans fer process , by which a plurality of emitters 20 are attached to the substrate 10 simultaneously . For example , an elastomer stamp is used for the mass trans fer .

[0110] In a fourth step S4 , a sensor 30 is formed in at least one pixel 2 , the sensor 30 being configured to sense light during operation . Forming the sensor 30 may comprise embedding a photodiode in the substrate 10 , in particular i f the substrate 10 is a silicon substrate 10 . This can be done by a semiconductor process , e . g . a CMOS process . In this case , forming the sensor 30 can be performed before mounting the emitter 20 to the substrate 10 . Forming the sensor 30 may also comprise mounting a photodiode on the main surface 12 of the substrate 10 next to the emitters 20 . In this case the photodiode may be formed by a separate detector chip that is trans ferred and attached to the substrate 10 . Forming the sensor 30 may also comprise mounting an LED, in particular micro-LED, on the main surface 12 of the substrate 10 next to the emitters 20 , and biasing said LED for sensing a photocurrent . Said sensing (micro- ) LED can have the same structure as the emitter 20 . Thus , it can be formed in a same manufacturing process and it may be trans ferred to the substrate 10 in the same trans fer process as the emitters 20 . Biasing said LED for sensing a photocurrent can mean that this sensing LED is zero biased or reverse biased .

[0111] In a fi fth step S5 , a filter 70 is formed on or above the sensor 30 , the filter 70 being configured to transmit light of a predetermined wavelength range to the sensor 30 . Forming the filter 70 can comprise a depositing and structuring one or more filter layers directly on the sensor 30 or on a transparent cover 60 that covers the emitters 20 and the sensor 30 . The embodiments of the display device and the methods of operating and manufacturing the display device disclosed herein have been discussed for the purpose of familiari zing the reader with novel aspects of the idea . Although preferred embodiments have been shown and described, many changes , modi fications , equivalents and substitutions of the disclosed concepts may be made by one having skill in the art without unnecessarily departing from the scope of the claims .

[0112] It will be appreciated that the disclosure is not limited to the disclosed embodiments and to what has been particularly shown and described hereinabove . Rather, features recited in separate dependent claims or in the description may advantageously be combined . Furthermore , the scope of the disclosure includes those variations and modi fications , which will be apparent to those skilled in the art and fall within the scope of the appended claims .

[0113] The term " comprising" , insofar it was used in the claims or in the description, does not exclude other elements or steps of a corresponding feature or procedure . In case that the terms " a" or " an" were used in conj unction with features , they do not exclude a plurality of such features . Moreover, any reference signs in the claims should not be construed as limiting the scope .

[0114] This patent application claims the priority of German patent application 102023125788 . 9 , the disclosure content of which is hereby incorporated by reference . References

[0115] 1 display device

[0116] 2 pixel

[0117] 10 substrate

[0118] 12 main surface

[0119] 15 carrier

[0120] 20 emitter

[0121] 20- 1 first emitter

[0122] 20-2 second emitter

[0123] 20-3 third emitter

[0124] 30 sensor

[0125] 30- 1 sensor

[0126] 30-2 sensor

[0127] 40 control circuit

[0128] 45 electrical interconnection

[0129] 60 cover

[0130] 70 filter

[0131] 80 recess

[0132] 90 metal coating

[0133] 92 filler

[0134] 94 top electrode

[0135] AL ambient light

[0136] F1-F10 filter

[0137] S 1-S5 manufacturing steps

[0138] TA ambient light detection window

Claims

Claims1. Display device (1) , comprising:- a substrate (10) comprising a main surface (12) ,- a plurality of emitters (20) arranged in pixels (2) on the main surface (12) of the substrate (10) , wherein each emitter (20) comprises a light emitting diode, LED, that is configured to emit light during operation,- a sensor (30) in at least one pixel (2) , the sensor (30) being configured to sense light during operation,- a filter (70) arranged on or above the sensor (30) , the filter (70) being configured to transmit light of a predetermined wavelength range to the sensor (30) .

2. Display device (1) according to the preceding claims, wherein each emitter (20) is formed by a micro-LED.

3. Display device (1) according to one of the preceding claims, wherein the pixels (2) are formed as RGB-pixels (2) , in which a first emitter (20-1) configured to emit red light, a second emitter (20-2) configured to emit green light, and a third emitter (30-3) configured to emit blue light are arranged .

4. Display device (1) according to one of the preceding claims, wherein the sensor (30) is formed as a photodiode, wherein the photodiode is embedded in the substrate (10) or is arranged on the main surface (12) of the substrate (10) next to the emitters (20) .

5. Display device (1) according to one of claims 1 to 3, wherein the sensor (30) is implemented as an LED, in particular micro-LED, that is arranged on the main surfaceconfigured to be biased for sensing a photocurrent.

6. Display device (1) according to the preceding claim, wherein the emitters (20) and the sensor (30) are manufactured according to a common manufacturing process.

7. Display device (1) according to one of the preceding claims, comprising a plurality of sensors (30) being configured to sense light during operation, wherein the sensors (30) are distributed over the display in respective pixels (2) of the display device (1) , and wherein each sensor (30) is provided with a respective filter (70) configured to transmit light of a predetermined wavelength range to the sensor ( 30 ) .

8. Display device (1) according to the preceding claim, wherein different spectral channels are formed by the plurality of sensors (30) and filters (70) , wherein each spectral channel is sensitive for a respective wavelength range, and- wherein at least one spectral channel is placed over the display with a density different from a density of at least one other spectral channel, and / or- wherein sensors (30) of at least one spectral channel are driven with an amplification gain different from an amplification gain of sensors (30) of at least one other spectral channel.

9. Display device (1) according to one of the preceding claims, wherein pixels (2) of a first group comprise a respective sensor (30-1) that is formed as blue or green micro-LED and configured to be biased for sensing aphotocurrent, and / or wherein pixels (2) of a second group comprise a respective sensor (30-2) that is formed as red micro-LED and configured to be biased for sensing a photocurrent .

10. Display device (1) according to one of the preceding claims, wherein the filter (70) is implemented as interference filter or as color filter.

11. Display device (1) according to one of the preceding claims, wherein the filter (70) is arranged on the sensor (30) .

12. Display device (1) according to one of the preceding claims, further comprising a transparent cover (60) covering the pixels (2) , wherein the filter (70) is arranged on the cover ( 0 ) .

13. Display device (1) according to one of the preceding claims, further comprising a control circuit (40) that is, based on the light sensed by the one or more sensors (30) , configured to- evaluate ambient light around the display, and / or- evaluate flicker noise, and / or- evaluate color of objects close to the display, and / or- adjust a white balance of the display, and / or- adjust a color calibration of the display , and / or- dynamically adjust color and brightness of emitted display light.

14. Method of operating a display device (1) , wherein the display device (1) comprises a plurality of emitters (20)arranged in pixels (2) on a main surface (12) of a substrate (10) , a sensor (30) in at least one pixel (2) , and a filter (70) arranged on or above the sensor (30) , the method comprising :- emitting light by the emitters (20) , wherein each emitter (20) comprises a light emitting diode, LED,- filtering, by the filter (70) , ambient light and transmitting light of a predetermined wavelength range to the sensor ( 30 ) ,- sensing, by the sensor (30) , light transmitted by the filter (70) .

15. Method according to the preceding claim, further comprising adjusting a brightness of the emitted light by pulse width modulation.

16. Method according to the preceding claim, wherein sensing light is performed- when the emitters (20) are in an off-state during the pulse width modulation, or- continuously during the pulse width modulation and the sensed light is evaluated by filtering a detection signal in frequency space.

17. Method according to one of the preceding claims, further comprising, based on the sensed light,- evaluating ambient light around the display, and / or- evaluating flicker noise, and / or- evaluating color of objects close to the display, and / or- adjusting a white balance of the display, and / or- adjusting a color calibration of the display , and / ordynamically adjusting color and brightness of emitted display light.

18. Method of manufacturing a display device (1) , the method comprising :- providing a substrate (10) ,- mounting a plurality of emitters (20) on a main surface(12) of the substrate (10) , wherein the emitters (20) are arranged in pixels (2) and each emitter (20) comprises a light emitting diode, LED, configured to emit light during operation,- forming a sensor (30) in at least one pixel (2) , the sensor (30) being configured to sense light during operation,- forming a filter (70) on or above the sensor (30) , the filter (70) being configured to transmit light of a predetermined wavelength range to the sensor (30) .

19. Method according to the preceding claim, wherein forming the sensor (30) comprises- embedding a photodiode in the substrate (10) , or- mounting a photodiode on the main surface (12) of the substrate (10) next to the emitters (20) , or- mounting an LED on the main surface (12) of the substrate (10) next to the emitters (20) , and biasing said LED for sensing a photocurrent.

20. Method according to one of claims 18 to 19, further comprising forming recesses (80) on the main surface (12) of the substrate (10) , wherein in each recess (80) a respective emitter (20) or sensor (30) is mounted.

Images