A light-emitting display configured to have a through-display spectroscope

Abstract

The mobile computing device includes an emissive display panel used as a grating and spectrometer optics located behind / below the emissive display panel. The mobile computing device includes an emissive display panel and a spectrometer located below the emissive display panel. The emissive display panel includes a first periodic pattern of pixels including one or more LEDs and a second periodic pattern of circuit elements that control the pixels. The first and second periodic patterns are configured to diffract light received from outside the device that passes through the emissive display, the diffraction being wavelength-dependent. The spectrometer is configured to detect the intensity of different wavelength ranges of the diffracted light.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of U.S. Application No. 62 / 949,197, filed on December 17, 2019, and all of its disclosure is incorporated herein by reference.

[0002] Technical Field The present disclosure relates to flat - panel displays, and more particularly to displays used in mobile devices that enable spectroscopic sensing through the display.

Background Art

[0003] Background From at least the perspective of the user experience, it would be desirable to expand the display to cover more areas of a mobile device (e.g., a mobile phone, a tablet terminal, etc.). However, electro - optical devices (e.g., a front - facing camera, an optical sensor, a spectrometer, etc.) disposed on the surface including the display of the mobile device will compete for the space on this surface including the display of the device. Thus, in order to accommodate other sensors on the display surface of the device, the size of the light - emitting area of the display will have to be compromised.

Summary of the Invention

[0004] Summary The present disclosure describes a mobile computing device having a light - emitting display panel and a spectrometer located under the display panel. The light that the spectrometer detects and analyzes passes through the display panel and is diffracted by the elements of the display panel in a wavelength - dependent manner.

[0005] In an inclusive aspect, a mobile computing device includes a light-emitting display panel and a spectrometer disposed under the light-emitting display panel. The light-emitting display panel includes pixels of a first periodic pattern including one or more LEDs and circuit elements of a second periodic pattern that control the pixels. The first periodic pattern and the second periodic pattern are configured to diffract light received from outside the device that passes through the light-emitting display, and the diffraction is wavelength-dependent diffraction. The spectrometer is configured to detect the intensities of different wavelength ranges of the diffracted light.

[0006] Embodiments may include one or more of the following features. For example, the display panel may include an AMOLED (Active Matrix Organic Light Emitting Diode) display panel.

[0007] The circuit elements may be arranged to form openings through which light passes. The one or more LEDs may be configured to illuminate an object, which reflects the light received from outside the device, and the light is diffracted by the periodic pattern.

[0008] The spectrometer may include an optical sensor array, and the optical sensor array responds to the intensities of the diffracted light projected onto different portions of the optical sensor array.

[0009] The mobile computing device may include a memory configured to store the baseline sensitivity of the array.

[0010] Mobile computing Device may include a fiber Optic plate between the light-emitting display panel and the optical sensor array, and the fiber Optic plate is configured to project the diffracted light onto the optical sensor array.

[0011] The mobile computing device may further include a lens disposed between the light-emitting display panel and the spectrometer, and the lens is configured to project the diffracted light onto the spectrometer.

[0012] The mobile computing device may further include an opaque layer between the light-emitting display panel and the spectrometer, and the opaque layer includes an aperture configured to allow diffracted light to pass through the display panel to the spectrometer.

[0013] The mobile computing device may further include a processor configured to process a signal from the spectrometer, the signal being a signal based on the intensities of different wavelength ranges of the detected diffracted light, and identifying the product of the light-receiving source of the diffracted light as genuine or authentic.

[0014] In another aspect, the light-emitting display panel may include pixels of a first periodic pattern including one or more LEDs and circuit elements of a second periodic pattern for controlling the pixels, and the first periodic pattern and the second periodic pattern are configured to diffract light received from outside the device passing through the light-emitting display, the diffraction being wavelength-dependent diffraction, and the diffracted light can be detected by a spectrometer disposed under the light-emitting display panel.

[0015] Embodiments may include one or more of the following features. For example, the display panel may include an AMOLED (active matrix organic light emitting diode) display panel.

[0016] The display panel may include an AMOLED (active matrix organic light emitting diode) display panel.

[0017] The circuit elements may be arranged to form an opening through which light passes. The one or more LEDs may be configured to illuminate an object that reflects light received from outside the device, and the light is diffracted by the periodic pattern.

[0018] The light-emitting display panel may further include an opaque layer between the light-emitting display panel and the spectroscope, and the opaque layer includes an aperture configured to allow the diffracted light to pass through the display panel and the opaque layer.

Brief Description of the Drawings

[0019]

Figure 1A

Figure 1B

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0020] The components in the drawings are not necessarily drawn to scale and are not in relative sizes to each other. Throughout the multiple drawings, the same reference numerals refer to corresponding parts.

[0021] Detailed Description In the present disclosure, a flat panel display (i.e., a display panel) that can be used in a mobile device (e.g., a mobile phone, a tablet terminal, etc.) will be described. The front surface of the mobile device includes, in a region facing the front surface, a display that normally operates as a GUI (Graphic User Interface) and one or more optical devices that operate as sensors / emitters. The one or more optical devices can be configured for various functions including detection of illumination conditions (e.g., a light sensor), detection of proximity (e.g., an electromagnetic sensor), taking of images (e.g., a front camera), and / or provision of light (e.g., a flash) (but not limited to these).

[0022] Conventionally, the display and the optical device occupied separate regions on the front surface. For example, FIG. 1A is a diagram showing a mobile device 101 having a display 110 and a camera 111 that occupy different parts of the front surface. In addition to the camera 111, the mobile device 101 may include other optical elements that receive and detect optical signals, such as a spectroscope.

[0023] Using a spectrometer, the incident electromagnetic radiation can be analyzed to identify the constituent wavelengths of the incident electromagnetic radiation, as well as the irradiance and output at these different constituent wavelengths. The information provided by the spectrometer can be utilized in a variety of different ways. For example, spectral information can be used to identify the raw materials and components of a product based on the spectral signature of the product. In another example, if a product (e.g., pharmaceuticals, luxury consumer goods, etc.) has a known characteristic spectral signal that cannot be reproduced in counterfeits, spectral information can be used to determine the authenticity of this product. In another example, spectral information can be used to determine the state of an object. For example, the sugar content of a fruit can change as the fruit ripens, and if the sugar content of the fruit affects the electromagnetic spectrum detected from the fruit, a spectrometer can be used to determine the relative maturity of the fruit. Health-related information (e.g., blood oxygen saturation, body fat percentage, etc.) can also be detected by spectroscopic means.

[0024] Recent advancements in emissive display technologies (e.g., AMOLED (Active Matrix Organic Light Emitting Diode)) have made it easier to expand the emissive (i.e., active) area of a display towards the edge of a mobile device. By expanding the active area of the display towards the edge of the mobile device, the user will be able to obtain the benefits of a large display without the drawbacks of a large device. However, this will not leave enough space for a spectrometer and other optical devices outside the area of the emissive display on the front of the mobile device. In addition to this, adding a spectrometer function to a mobile device involves providing a grating for diffracting the incident light, which can be yet another component that has to be procured, inspected, and incorporated into the device.

[0025] The light-emitting display disclosed in this specification is configured to share the front surface of a mobile device with a spectroscope so that the active area of the display can extend to the edge without leaving a gap in the display for the spectroscope or leaving space around the display. Therefore, a spectroscope disposed under the display can receive electromagnetic radiation (e.g., light) through the display, and one or more portions of the display panel of the present disclosure covering the spectroscope can be configured such that the display itself operates as a grating to divide the received radiant energy into wavelengths for spectroscopic detection and analysis.

[0026] FIG. 1B is a diagram showing a mobile device 102 in which a display 112 is extended toward the edge. Different from a mobile device in which the display is excluded from the area secured for the optical device, the light-emitting (i.e., active) area of the display 112 extends across substantially the entire front surface. Thus, substantially the entire front surface of the mobile device 102 can be used to provide color images, black-and-white images, grayscale images, graphics, and / or text. In some embodiments, the display 112 may include one or more areas 120 behind (i.e., below) which a spectroscope is disposed.

[0027] The size, shape, and / or arrangement of the area 120 may be implemented in various ways. For example, the area 120 shown in FIG. 1B is circular (e.g., round) and is disposed away from the edge of the display 112. This need not be the case. For example, the shape of the area 120 can be square, and the area 120 can be disposed along the edge of the display 112.

[0028] Figure 2A shows a side cross-sectional view of a mobile device having a display 112 with regions 120A, 120B, and electromagnetic radiation can pass through regions 120A, 120B to an optical device, such as a camera or a spectrometer, located on the back. The mobile device may include a plurality of optical devices 140A, 140B. Each of the optical devices 140A, 140B is disposed behind a respective different region 120A, 120B. Figure 2B shows a side cross-sectional view of a mobile device having a display 112 with one region used by a plurality of optical devices 140A, 140B.

[0029] The optical devices 140A, 140B may transmit and / or receive electromagnetic radiation 125 through regions 120A, 120B, 120C. The present disclosure is generally applicable to any optical device configured to transmit or receive electromagnetic radiation (e.g., from the millimeter wave portion, visible portion, or infrared portion of the electromagnetic spectrum), but throughout the present disclosure, a specific embodiment of a spectrometer configured to receive visible light and / or infrared light will be considered.

[0030] In some embodiments, the regions 120A, 120B, 120C of the display 112 through which light passes to the sensors on the back may have the same pixel density and / or pixel arrangement as the rest of the display. In some embodiments, the regions 120A, 120B, 120C of the display 112 through which light passes to the sensors on the back may have a different pixel density and / or pixel arrangement from the rest of the display. For example, in some embodiments, the pixel resolution of the display area in the rest of the display may be higher than the pixel resolution of the regions 120A, 120B, 120C of the display that transmit light to the spectrometer. FIG. 3A is a diagram showing pixels and signal lines in a high-resolution portion of a light-emitting display, and FIG. 3B is a diagram showing pixels and signal lines in a low-resolution portion of a light-emitting display. In FIGS. 3A and 3B, the pixels of the display may each include a plurality of light-emitting elements (e.g., light-emitting diodes) that emit different colors, and all visible colors are created by the pixels depending on the amount of light mixed from the different elements. For example, in some embodiments, one pixel may include one red LED 302, one blue LED 304, and two green LEDs 306. In the low-resolution portion of the display, more light passes through the display than in the high-resolution portion. However, the light passing through the display may contact the signal lines 242 extending in the vertical (Y) direction and / or the horizontal (X) direction.

[0031] FIG. 4 shows a side cross-sectional view of a light-emitting display panel 200 suitable for use with the mobile device of FIG. 1B. In one embodiment, the display panel 200 may be an AMOLED display. The principles of the present disclosure can be applied to various other display technologies (e.g., LCD, LED), but throughout the present disclosure, embodiments of AMOLED displays are considered.

[0032] As shown in FIG. 4, the AMOLED display panel 200 includes a plurality of layers. These layers are disposed behind (i.e., below) the cover glass layer 210. The cover glass layer 210 may form the front face of the mobile device 102. In a realizable embodiment, the display panel 200 may include a polarizing film layer 215. Also, the display panel 200 may include a touch sensor layer 220 including touch sensor electrodes 222. The pixels 237 of the display are formed from separate elements of a cathode layer 230, an OLED emitter stack 235, and an anode layer 236. The elements of the anode layer 236 may be reflective elements such that light is directed from the anode layer 236 in the longitudinal (Z) direction. The elements of the anode layer 236 can be connected to a TFT (thin film transistor) structure 240 including a source, a gate, and a drain that can be controlled by an electrical signal transmitted through the signal line 242. The display panel 200 may further include a barrier layer 245 of SiNx or SiONx and a substrate layer 250 of PI (polyimide). A metal layer / film 410 for diffusing heat and shielding electricity is disposed under the display panel 200 to protect the display from local high temperature spots caused by heat generating elements included in the mobile device, such as a CPU, a GPU, etc., and electrical signal noise from electrical components directly below.

[0033] The layers of the display panel 200 may include transmissive circuit elements and non - transmissive circuit elements. For example, all of the TFT structure 240, the pixels 237, the signal lines 242, and / or the touch sensor electrodes 222 may block light propagating through the display panel 200. The light may be reflected or absorbed by non - transmissive (e.g., opaque) circuit elements. In addition, the circuit elements may define a range of voids (e.g., periodic slits) where light can interact. For example, the light may be diffracted by voids formed between adjacent circuit elements of the same layer. Also, although less effective than diffraction by elements of the same layer, the light may be diffracted by voids between circuit elements of different layers.

[0034] FIG. 5 is a side cross-sectional view of a part of the display panel 200 of FIG. 4 in which light is diffracted by circuit elements of the display. As shown in the figure, light 300A can pass between electrodes 222A and 222B and between pixels 237A and 237B without being significantly changed when the distances between adjacent touch sensor electrodes 222A and 222B and between adjacent pixels 237A and 237B are greater than the wavelength of the light. However, the light 300A can be diffracted by the gap formed between two adjacent signal lines 242A, 242B that are arranged close to each other with a narrow gap therebetween. The effect of the gap on the light can be determined by the size of the gap relative to the wavelengths of the light 300A, 300B. For example, the degree of diffraction of the light can vary depending on the relative size of the gap ("d") compared to the wavelength of the light ("λ"). When d >> λ, there is little diffraction, and when d << λ, little light passes through the gap. However, when d ~ λ, significant diffraction of the light can occur. Therefore, if the formed gap is a gap of appropriate size, the light may be diffracted by the gaps formed by other combinations of circuit elements. For example, if the relative sizes of the first gap formed between the signal line 242C and the TFT structure 240 and the second gap between the adjacent signal lines 242A and 242B compared to the wavelength of the light passing through these gaps are different, the light 300B diffracted by the first gap can be diffracted differently from the light 300A diffracted by the second gap.

[0035] When light is diffracted, it can be interpreted that the diffraction effectively changes the propagation direction of a part of the light so that the diffracted lights 320A, 320B are dispersed over the entire diffraction angle. Generally, the narrower the gap in the display, the larger the diffraction angle.

[0036] When the light passing through the display panel 200 is optically projected onto a sensor located under the display (for example, the sensor of a camera located under the display), diffraction of light by the sensor electrodes 222A, 222B, pixels 237A, 237B, signal lines 242A, 242B, 242C, and the TFT structure 240 can be an obstacle. However, by advantageously using the diffraction of light by the diffraction element of the display, spectral information about the light passing through the display panel 200 can be obtained.

[0037] For example, FIG. 6 is a schematic diagram of wavelength-dependent diffraction of light by the diffraction element of the light-emitting display panel 200. The incident broad-spectrum light 602 can interact with the diffraction element of the light-emitting display, for example, the touch sensor electrode 222, and different wavelengths of the broad-spectrum light can be diffracted by different amounts. The diffracted light 604 can be projected onto the photosensor array 608 (for example, by the lens 606). In the photosensor array 608, different ranges of wavelengths are resolved on different pixels of the array 608. In some embodiments, a fiber optic plate can be used instead of the lens 606, which enables reducing the distance between the diffraction element of the light-emitting display and the photosensor array, or providing a curved image plane. Using the intensity of each separately resolved wavelength range, the spectrogram of the incident light can be identified. A thermal diffusion layer 610 is disposed under the display panel 200 to protect the display from local high-temperature areas caused by heat-generating elements provided in mobile devices, such as the CPU, GPU, etc. The thermal diffusion layer 610 may contain a metal, may be optically opaque, and may have small openings made in the layer 610 to allow light to pass through the display and reach the photosensor array 608 located under the display panel 200.

[0038] The schematic diagram of FIG. 6 shows a simplified diffraction pattern due to the interaction of light rays with one end of the diffraction element. In reality, the incident light is diffracted by a plurality of diffraction elements with a more complex pattern provided in the light-emitting display 200. For example, referring again to FIG. 3B, when passing through a part of a display comprising a plurality of pixels each including a plurality of LEDs and a plurality of signal lines, the incident light is diffracted by the plurality of elements. However, due to the spatial periodicity of the diffraction elements of the light-emitting display, the display can function as a two-dimensional diffraction grating for the incident light. FIG. 7 is a diagram showing the measured point image distribution function of the light diffracted by the two-dimensional array of actual diffraction elements provided in the light-emitting display. Here, in the point image distribution function shown in FIG. 7, the characteristics of the spatial pattern of the diffraction elements of the two-dimensional pattern of the light-emitting display are reflected.

[0039] The optical sensor array 608 can be disposed under the light-emitting display panel 200 at a position and orientation corresponding to a part of the point image distribution function characterized by good color resolution in the diffraction pattern of light by the diffraction elements of the light-emitting display. Thereafter, using the signals on the sensor array, the spectrum of the incident light passing through the light-emitting display panel 200 and reaching the optical sensor array 608 can be specified.

[0040] FIG. 8 is a schematic diagram of a mobile device 800 having a light-emitting display 802. The light-emitting display 802 includes a diffraction element having a periodic pattern capable of diffracting light when incident light 804 passes through the display 802. The diffracted light 806 can pass through an aperture of an opaque layer 808 under the light-emitting display 802 and can be projected onto a photosensor array 812 (e.g., by a lens 810). The photosensor array 812 can measure the spectrum of the incident light 804. In some embodiments, a fiber optic plate can be used instead of the lens 810, thereby reducing the distance between the diffraction element of the light-emitting display and the photosensor array. Incident light 804 from a target object 814 can be received, and the target object 814 can be illuminated by light 816 provided by one or more LEDs 818 of the light-emitting display 802. The light 816 provided by the light-emitting display 802 can have spectral characteristics determined by the device 800. For example, light provided from an AMOLED light-emitting display 802 can exhibit a spectrum having a relatively high intensity near wavelengths corresponding to the bandgaps of the individual OLED emitters included in the display. In some embodiments, the photosensor array 812 under the display panel 802 can be calibrated for use with the light provided by the light-emitting display panel. For example, before measuring the spectrum received from an unknown object using the mobile device 800, the light from the light-emitting display 802 is provided to an object having a known spectral sensitivity, and the spectrum of the light received from the object in response to the provided light is measured to calibrate the photosensor array and determine the baseline sensitivity of the array 812. The baseline sensitivity of the array can be stored in the memory of the device 800. Thereafter, when light is provided from the display 802 to an unknown object, the spectrum of the light received by the array 812 in response thereto can be compared to the baseline to identify the spectrum from the unknown object. In some embodiments, after identifying the spectrum of a self-luminous object or ambient light using the mobile device 800, the spectral characteristics of the light 816 can be selected such that the irradiance distribution of the light 816 provided by one or more LEDs covers the range of the target wavelength.When the light 816 is added to the ambient light hitting the object 814 of interest, the object has a uniform irradiance distribution over the wavelength range of interest.

[0041] FIG. 9 is a schematic diagram of the mobile device 800 when used in the ambient light lighting mode. The ambient light can irradiate the object 814 of interest. Then, the light received from the object 814 of interest can pass through the aperture of the opaque layer 808 below the light-emitting display panel 802 and be projected onto the photosensor array 812 that can measure the spectrum of the incident light 804. Also, the photosensor array 812 can measure the spectrum of the ambient light diffracted by the diffraction element of the pattern of the light-emitting display 802 to analyze the spectral characteristics of the ambient light itself. Information about the ambient light spectrum can be used, for example, for applications such as dynamically changing the brightness of the light-emitting display according to the ambient light.

[0042] The display of the present disclosure has been presented in the context of a mobile device such as a tablet terminal or a smartphone. However, the disclosed principles and techniques can be more generally applied to any display where it is desirable to place a sensor behind the display. For example, a virtual agent home terminal, a television, or an ATM (automated teller machine) are a non-limiting series of other applications that utilize a camera placed behind the active area of the display. Furthermore, the motivation for placing a spectrometer behind the display is not limited to extending the display to the edge of the device. For example, there may be cases where it is desirable to place a spectrometer behind the display for aesthetic or concealment purposes.

[0043] This specification and / or the drawings disclose exemplary embodiments. The present disclosure is not limited to these exemplary embodiments. The use of the term "and / or" includes any and all combinations of one or more of the associated listed items. Unless otherwise specified, specific terms are used in a general descriptive sense and not for limitation. When used in this specification, spatially relative terms (e.g., in front of, behind, above, below, etc.) include different orientations of the device in use or operation in addition to the orientation shown in the figures. For example, the "front" of a mobile computing device may be the surface facing the user. In this case, the phrase "in front of" implicitly indicates being close to the user side. In addition to this, the "upper surface" of the display may be the surface facing the user. In this case, the phrase "below" implicitly indicates the inner back side of the mobile computing device.

[0044] As described in this specification, specific features of the above-described embodiments have been illustrated, but those skilled in the art will be able to come up with many improvements, replacements, changes, and equivalents. Therefore, it should be understood that the appended claims cover all of these improvements and changes included within the scope of the embodiments. These are presented as examples only and not limitations, and it should be understood that various changes can be made in form and detail. Any part of the apparatus and / or method described in this specification may be combined in any combination, except mutually exclusive combinations. The embodiments described in this specification may include various combinations and / or partial combinations of the functions, components, and / or features of the different embodiments described.

Claims

1. A device comprising: a light-emitting display panel, the light-emitting display panel comprising: pixels of a first periodic pattern including one or more LEDs, and circuit elements of a second periodic pattern for controlling the pixels, the first periodic pattern and the second periodic pattern being configured to diffract light passing through the light-emitting display panel that is received from outside the device and incident on the light-emitting display panel, the diffraction being wavelength-dependent diffraction, the device further comprising: an optical sensor array disposed under the light-emitting display panel and configured to receive light diffracted by at least one of the first periodic pattern or the second periodic pattern as a result of the light passing through the light-emitting display panel and to measure a spectrum of the diffracted light, the device being configured to analyze the diffracted light received by the optical sensor array to identify a constituent wavelength of the light received from outside the device and incident on the light-emitting display panel and an irradiance or output at the constituent wavelength.

2. The device according to claim 1, wherein the light-emitting display panel includes an AMOLED (active matrix organic light emitting diode) display panel.

3. The device according to claim 1 or claim 2, wherein the circuit elements are arranged to form openings through which light passes.

4. The device according to any one of claims 1 to 3, wherein the one or more LEDs are configured to illuminate an object, the object reflects the light received from outside the device, and the light is diffracted by the first periodic pattern and the second periodic pattern.

5. The device according to any one of claims 1 to 4, wherein the optical sensor array responds to an intensity of the diffracted light projected onto different portions of the optical sensor array.

6. The device according to any one of claims 1 to 5, further comprising a memory configured to store a baseline sensitivity of the optical sensor array.

7. The device according to any one of claims 1 to 6, further comprising a fiber optic plate between the light-emitting display panel and the optical sensor array, the fiber optic plate being configured to project the diffracted light onto the optical sensor array.

8. The device according to any one of claims 1 to 7, further comprising a lens disposed between the light-emitting display panel and the optical sensor array, the lens being configured to project the diffracted light onto the optical sensor array.

9. The device according to any one of claims 1 to 8, further comprising an opaque layer between the light-emitting display panel and the optical sensor array, the opaque layer including an aperture configured to allow the diffracted light to pass through from the light-emitting display panel to the optical sensor array.

10. The device according to any one of claims 1 to 9, further comprising a processor configured to process a signal from the optical sensor array, the signal being a signal based on a detected intensity of the constituent wavelength of the diffracted light, and identifying the product from which the diffracted light is received as genuine or authentic.

11. The device according to any one of claims 1 to 10, wherein the device is a mobile computing device.

12. The second periodic pattern includes a first gap and a second gap having different sizes, The device according to any one of claims 1 to 11, wherein the optical sensor array receives the light diffracted by the first gap and the second gap.

Images