A method and system for integrated on-display driving backplane spectral detection

Abstract

The present application relates to the intersection field of semiconductor display technology and computational spectroscopy, and particularly to a kind of integrated in display driving backboard's computational spectrum detection method and system, system includes display function layer, diffraction modulation layer, photosensitive cell array and computational reconstruction module sequentially arranged along the direction of light path, diffraction modulation layer multiplexing light-tight metal layer in thin film transistor array and diffraction microstructure is formed on it;Method includes inputting incident light signal into display panel, diffraction microstructure carries out Fraunhofer diffraction modulation to generate diffraction light signal, photosensitive cell array receives diffraction light signal and converts into electrical signal, and computational reconstruction module reconstructs spectral distribution function using regularization transformation algorithm.The present application directly utilizes inherent metal layer of display panel to construct diffraction structure, realizes in situ integration of zero additional thickness, can make display equipment obtain spectral perception ability under the premise of not changing appearance, not increasing thickness.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of semiconductor display technology and computational spectroscopy, and in particular to a computational spectral detection method and system that utilizes the metal layer of thin-film transistors inside a display panel to construct a diffraction structure. Background Technology

[0002] As a core tool for material identification and optical measurement, spectrometers have demonstrated enormous application potential in fields such as color calibration, ambient light analysis, and personal health monitoring. With the increasing integration and thinner designs of consumer electronics devices, integrating spectral analysis functions into the display panels of mobile phones, tablets, and wearable devices has become a significant direction for industry evolution.

[0003] However, traditional spectrometers (such as those based on gratings or prisms) are bulky and expensive, making it difficult to meet the urgent needs of mobile smart terminals for miniaturization and integration. In recent years, the rise of computational spectroscopy has provided a new path for the miniaturization of spectrometers. This technology uses micro- and nano-structures to spatially or spectrally encode incident light and reconstructs spectral information using mathematical algorithms.

[0004] Through the search and analysis of existing technologies, the current miniature spectrometer solutions mainly suffer from the following typical design flaws:

[0005] Spectral schemes based on externally stacked microstructures: These schemes achieve encoding by constructing an array of diffraction apertures of different sizes on a light-blocking layer of a transparent substrate. However, such schemes face significant difficulties in practical applications: First, the structure is an external, independent optical component that needs to be physically stacked and mounted above the photodetector. This not only increases the overall thickness of the device but also involves stringent alignment processes, making it highly susceptible to reconstruction algorithm failure due to assembly deviations. Second, because its opaque light-blocking layer is added separately, it cannot be integrated in situ with existing semiconductor display processes.

[0006] Schemes based on specific arrays and pre-calibration: These schemes rely on specific Fraunhofer diffraction modules and array distributions, typically requiring a complex pre-calibration process using narrowband filters. The drawbacks of these schemes are: extremely high precision requirements for micro / nano fabrication; any geometric deviations during fabrication (such as aperture scaling or deformation) will cause the pre-stored point spread function to deviate from the actual value, resulting in severe spectral distortion. Furthermore, the system complexity remains high due to the need for independent convex lenses and filter elements.

[0007] Cutting-edge improvements in computational algorithms in academia: Although existing research has solved the problem of "high tolerance for reconstruction" at the algorithm level, its hardware implementation is still limited to the aperture of independent metal thin films in a laboratory environment, and has not solved the problem of how to seamlessly embed this physical structure into commercial electronic devices (especially display panels with sensing requirements).

[0008] In summary, the existing technology has the following main problems:

[0009] Low integration and excessive thickness: Existing micro spectrometers are mostly regarded as independent sensor devices. When integrated into devices with displays such as mobile phones and tablets, they often need to avoid the display area or make holes on the side, which goes against the current trend of electronic devices pursuing "high screen ratio" and "thinness".

[0010] Manufacturing process incompatibility: Existing diffraction structures (such as photonic crystals and metasurfaces) often require high-cost processes such as electron beam lithography, which are difficult to be compatible with the large-area, low-cost thin-film transistor display panel process.

[0011] Environmental noise suppression is difficult: When attempting to integrate sensors into the display area, the strong light emitted by the display pixels themselves becomes serious background noise. Existing technologies lack a system-level solution that can both utilize the display backplane structure for light sensing and effectively eliminate display light interference.

[0012] Therefore, developing an integrated spectral sensing technology that can directly utilize the inherent hierarchical structure of the display panel, has high process tolerance, and does not require additional physical thickness is a technical challenge that urgently needs to be solved in the field of portable spectral detection. Summary of the Invention

[0013] In view of this, embodiments of the present invention provide a computational spectral detection method and system integrated into a display driver backplane. By using functional layer reuse technology, an optical modulation structure is directly constructed using the inherent metal driver layer inside the display panel, thereby achieving in-situ integration of spectral detection function and display function.

[0014] On one hand, embodiments of the present invention provide a computational spectral detection system integrated into a display driver backplane, comprising: a display panel, the display panel including, along the optical path direction, a display functional layer, a diffraction modulation layer, a photosensitive unit array, and a computational reconstruction module; the display functional layer comprising multiple pixel units for image display; the diffraction modulation layer multiplexing at least one opaque metal layer in a thin-film transistor array, at least one diffraction microstructure being formed on the opaque metal layer by photolithography, the diffraction microstructure being used for Fraunhofer diffraction modulation of the incident light signal; the photosensitive unit array being integrated on a thin-film transistor substrate and located below the diffraction modulation layer, for receiving diffracted light signals and converting them into electrical signals; the computational reconstruction module being electrically connected to the photosensitive unit array, for processing the electrical signals and reconstructing the spectral distribution function of the incident light.

[0015] Optionally, the diffraction modulation layer reuses at least one opaque metal layer in the thin-film transistor array, including: the diffraction modulation layer reuses a source / drain metal layer, which retains a micro-hole pattern in the non-display area corresponding to the pixel gap while forming the source / drain electrodes; or the diffraction modulation layer reuses a gate metal layer, which forms a diffraction slit in a specific area while forming the gate scan line; the diffraction microstructure is a through-hole or slit penetrating the opaque metal layer, the geometric size of the diffraction microstructure is on the same order of magnitude as the wavelength of the spectrum to be detected, and the diffraction microstructure has an arbitrary shape and is either single or sparsely distributed.

[0016] Optionally, the system further includes: a reflective layer disposed on the back side of the glass substrate for reflecting downwardly propagating diffracted light signals upwards; and a photosensitive unit array including photosensitive units integrated on the front side of the display panel for receiving diffracted light signals reflected by the reflective layer.

[0017] Optionally, the diffraction microstructure is disposed in the non-opening region between pixel units or in the light-shielding metal layer region above the thin-film transistor device. The feature size of the diffraction microstructure is smaller than the pixel pitch. The diffraction microstructure is a through hole or slit penetrating the opaque metal layer. The diffraction microstructure has an arbitrary shape and can be single or sparsely distributed.

[0018] On the other hand, embodiments of the present invention provide a computational spectral detection method integrated into a display driver backplane, implemented based on any of the systems described above, the method comprising the following steps:

[0019] The system determines the incident light signal to be detected and the display panel. The display panel includes, in sequence along the optical path, a display function layer, a diffraction modulation layer, a photosensitive unit array, and a computational reconstruction module. The display function layer contains multiple pixel units. The diffraction modulation layer reuses at least one opaque metal layer from the thin-film transistor array. At least one diffraction microstructure is formed on the opaque metal layer by photolithography. The photosensitive unit array is integrated on the thin-film transistor substrate and located below the diffraction modulation layer. The computational reconstruction module is electrically connected to the photosensitive unit array.

[0020] The incident light signal to be detected is input to the display panel. The incident light signal to be detected passes through the display functional layer in sequence and reaches the diffraction modulation layer. The diffraction microstructure performs Fraunhofer diffraction modulation on the incident light signal to be detected to generate a diffraction light signal. The diffraction light signal continues to propagate downward and reaches the photosensitive unit array.

[0021] The diffracted light signal is received by the photosensitive unit array, converted into an electrical signal, and transmitted to the calculation and reconstruction module.

[0022] The electrical signal is processed by the computational reconstruction module, and the spectral distribution function of the incident light is reconstructed using a regularization transformation algorithm to obtain the spectral detection result.

[0023] Optionally, the incident light signal to be detected sequentially passes through the display functional layer to reach the diffraction modulation layer, including:

[0024] The incident light signal to be detected enters from above the display panel, passes sequentially through the encapsulation layer, cathode layer, organic light-emitting layer, anode layer, planarization layer, and interlayer dielectric layer, and reaches the diffraction microstructure located on the metal layer of the thin-film transistor backplane; the diffraction microstructure performs Fraunhofer diffraction on the incident light signal to be detected, and the diffracted light signal continues to propagate downward through the gate insulating layer, buffer layer, and glass substrate, and is finally received by the photosensitive unit array located below the glass substrate.

[0025] Optionally, the method further includes:

[0026] A reflective film is deposited on the back of the glass substrate to form a reflective layer. The diffracted light signal is reflected upwards by the reflective layer after propagating to the lower surface of the glass substrate. The reflected diffracted light signal is received by a photosensitive unit integrated on the front of the display panel, thereby doubling the effective diffraction distance.

[0027] Optionally, the step of processing the electrical signal through the calculation and reconstruction module and reconstructing the spectral distribution function of the incident light using a regularization transform algorithm includes:

[0028] A reference diffraction pattern is acquired through a single quasi-monochromatic illumination. Based on the wavelength scaling characteristics of Fraunhofer diffraction, a point spread function matrix covering the entire wavelength range is generated through numerical affine transformation. The broadband diffraction intensity collected by the photosensitive unit array is vectorized into a measurement vector. A system of linear equations is established between the measurement vector and the spectral vector to be determined. The measurement matrix of the linear equations is composed of the point spread function matrix. The system of linear equations is solved using the Tikhonov regularization method. By introducing a regularization term to suppress noise amplification, the spectral vector is obtained. The spectral vector is then linearly interpolated and spectrally calibrated to obtain the spectral distribution function of the incident light.

[0029] Optionally, the method further includes: setting the diffraction microstructure in the non-opening region between pixel units or in the light-shielding metal layer region above the thin-film transistor device, wherein the feature size of the diffraction microstructure is smaller than the pixel pitch; in spectral acquisition mode, turning off the pixel units within a preset range around the diffraction microstructure through the display driving circuit to form a local dark field environment; dividing a dedicated spectral acquisition time period within the display frame cycle, pausing pixel emission or placing it in a reset state during the spectral acquisition time period, and simultaneously controlling the photosensitive unit array to perform sampling, thereby isolating display light interference in time.

[0030] Optionally, the method further includes: within a first display frame, controlling the pixel unit to display normally and acquiring a combined signal of ambient light and display light; within a second display frame, controlling the pixel units surrounding the diffraction microstructure to turn off or insert a black frame, acquiring only the ambient light signal; and performing differential processing on the combined signal acquired in the first display frame and the ambient light signal acquired in the second display frame to obtain a pure incident light signal component to be detected.

[0031] The embodiments of the present invention have the following beneficial effects:

[0032] This invention utilizes the inherent opaque metal layer in the backplane of the thin-film transistor in the display panel to construct diffraction microstructures, achieving in-situ integration with zero added thickness without the need for any additional optical components or physical films. By arranging the diffraction microstructures in the non-aperture area between pixels or in the inherent light-shielding area above the thin-film transistor devices, the coexistence of spectral acquisition and image display functions in the same physical space is achieved without sacrificing the display aperture ratio.

[0033] The diffraction microstructures of this invention are fabricated in the same layer and process as the electrodes of thin-film transistors. Only the microstructure pattern needs to be added during the mask design stage, without any additional photolithography, deposition, or etching processes. This allows for direct mass production on existing thin-film transistor production lines, significantly reducing manufacturing costs. The diffraction microstructures used in this invention do not require a precise periodic array; single or sparsely distributed microstructures are sufficient to meet spectral reconstruction requirements. Furthermore, the microstructures can be of arbitrary shapes, tolerating common photolithography deviations in thin-film transistor processes, thus significantly improving process tolerance.

[0034] This invention shuts down pixels surrounding the microstructure during the spectral acquisition period and inserts black frames for sampling, isolating display light interference in both time and space. Combined with double-sampling differential processing, it can eliminate most of the stray interference from the display light. For organic light-emitting diode display panels, the back-reflection optical path scheme proposed in this invention utilizes the inherent cathode layer of the panel or the back reflective layer of the glass substrate to allow diffracted light to propagate back and forth inside the panel, effectively doubling the diffraction distance and improving the spectral resolution by approximately double without increasing the thickness.

[0035] This invention employs a broadband point spread function mapping strategy, requiring only a single quasi-monochromatic light illumination to acquire a reference diffraction pattern. The full-band point spread function can then be generated through numerical transformation, significantly simplifying the calibration process. Furthermore, this invention utilizes the Tikhonov regularization algorithm to solve ill-conditioned linear equations, exhibiting high tolerance to microstructure geometric deviations and measurement noise. In conjunction with display driving timing, it effectively suppresses the impact of display light interference on spectral reconstruction, ensuring the reliability of detection while the display is operating. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly described below.

[0037] Figure 1 This is a schematic diagram of the spectrometer architecture embedded inside the display panel provided in an embodiment of the present invention;

[0038] Figure 2 This is a schematic flowchart of a computational spectral detection method integrated into a display driver backplane provided by an embodiment of the present invention;

[0039] Figure 3 This is a block diagram of the overall system architecture of the computational spectrometer and display panel integrated according to an embodiment of the present invention;

[0040] Figure 4 This is a partial cross-sectional structural diagram of a display panel pixel unit provided in an embodiment of the present invention;

[0041] Figure 5This is a top view of the diffraction microstructure and a spatial layout diagram of the pixel array provided in the embodiments of the present invention;

[0042] Figure 6 This is a schematic diagram of timing control for display and perception collaborative driving provided in an embodiment of the present invention;

[0043] Figure 7 This is a flowchart of the spectral reconstruction algorithm based on numerical affine transformation provided in an embodiment of the present invention. Detailed Implementation

[0044] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0045] refer to Figure 1 This invention provides a computational spectral detection system integrated into a display driver backplane, comprising: a display panel, the display panel including, along the optical path direction, a display functional layer, a diffraction modulation layer, a photosensitive unit array, and a computational reconstruction module; the display functional layer includes multiple pixel units for image display; the diffraction modulation layer reuses at least one opaque metal layer from a thin-film transistor array, and at least one diffraction microstructure is formed on the opaque metal layer by photolithography, the diffraction microstructure being used to perform Fraunhofer diffraction modulation on the incident light signal; the photosensitive unit array is integrated on a thin-film transistor substrate and located below the diffraction modulation layer, for receiving diffracted light signals and converting them into electrical signals; the computational reconstruction module is electrically connected to the photosensitive unit array, for processing the electrical signals and reconstructing the spectral distribution function of the incident light.

[0046] In some embodiments, the diffraction microstructure is disposed in the non-opening region between pixel units or in the light-shielding metal layer region above the thin-film transistor device. The feature size of the diffraction microstructure is smaller than the pixel pitch. The diffraction microstructure is a through-hole or slit penetrating the opaque metal layer. The diffraction microstructure has an arbitrary shape and is either single or sparsely distributed.

[0047] The functions of each part are summarized in Table 1 below:

[0048] Table 1. System Structure and Functions;

[0049]

[0050] Unlike existing technologies that stack diffraction devices as independent modules, this invention directly utilizes the inherent metal layer in the manufacturing process of the display panel backplane to construct the diffraction structure.

[0051] In some embodiments, the diffraction modulation layer multiplexes at least one opaque metal layer in the thin-film transistor array, including: the diffraction modulation layer multiplexes a source-drain metal layer, which retains a micro-hole pattern in the non-display area corresponding to the pixel gap while forming the source-drain electrodes; or the diffraction modulation layer multiplexes a gate metal layer, which forms a diffraction slit in a specific area while forming the gate scan line; the diffraction microstructure is a through-hole or slit penetrating the opaque metal layer, the geometric size of the diffraction microstructure is on the same order of magnitude as the wavelength of the spectrum to be detected, and the diffraction microstructure is of arbitrary shape and can be single or sparsely distributed.

[0052] The geometric dimensions of the diffraction microstructure are on the same order of magnitude as the wavelength of the spectrum to be measured (the aperture or slit width is preferably 1 μm to 10 μm). The microstructure can be of any shape (circular, square, cross-shaped, or irregular polygonal), and there is no need to form a periodic array—single or sparsely distributed microstructures can meet the requirements for spectral reconstruction. This characteristic significantly reduces the requirements for processing precision, enabling the present invention to be compatible with the large-area photolithography tolerances of mainstream thin-film transistor processes.

[0053] II. Optical path design and photosensitive unit integration;

[0054] 2.1 Optical path design;

[0055] The core principle of the optical path design in this invention is to utilize the inherent layered structure of the display panel as the light propagation medium, eliminating the need for additional optical components or physical space. Taking an organic light-emitting diode (OLED) display panel as an example, the typical layered structure of a modern display panel, from top to bottom, is as follows:

[0056] Table 2. Typical stack-up structure of organic light-emitting diode display panels;

[0057]

[0058] In this embodiment of the invention: layers 5 to 10 (from the planarization layer to the glass substrate) constitute a continuous set of transparent dielectric layers, with a total thickness ranging from hundreds of micrometers to millimeters, which is sufficient to meet the propagation distance required for Fraunhofer diffraction. Simultaneously, the opaque metal layers (layer 8 light-shielding metal layer, layer 2 cathode layer, and layer 4 anode layer) serve as a carrier for the diffraction microstructure and a natural aperture. Different optical path design schemes are adopted for specific display panels:

[0059] Optical path scheme 1: Forward transmission optical path (suitable for organic light-emitting diodes / liquid crystal displays);

[0060] In some embodiments, the incident light signal to be detected enters from above the display panel, passes sequentially through the encapsulation layer, cathode layer, organic light-emitting layer, anode layer, planarization layer, and interlayer dielectric layer, and reaches the diffraction microstructure located on the metal layer of the thin-film transistor backplane; the diffraction microstructure performs Fraunhofer diffraction on the incident light signal to be detected, and the diffracted light signal continues to propagate downward through the gate insulating layer, buffer layer, and glass substrate, and is finally received by the photosensitive unit array located below the glass substrate.

[0061] Referring to Table 2, the diffraction distance is on the order of 0.5-1.2 mm, which fully meets the propagation distance required for Fraunhofer diffraction. The advantages of this scheme are that it makes full use of the thickness of the glass substrate, resulting in a large diffraction distance, which is beneficial to improving spectral resolution. In addition, the photosensitive unit is located on the back of the substrate, allowing the use of a standard complementary metal-oxide-semiconductor image sensor, and the process is mature.

[0062] Optical path scheme two: Back-reflection optical path (suitable for organic light-emitting diodes);

[0063] In some embodiments, a reflective film is deposited on the back of the glass substrate to form a reflective layer. The diffracted light signal is reflected upwards by the reflective layer after propagating to the lower surface of the glass substrate. The effective diffraction distance is doubled by receiving the reflected diffracted light signal by a photosensitive unit integrated on the front of the display panel.

[0064] Back-reflective optical paths utilize the inherent metal reflective layer (usually a cathode layer or a separate reflective electrode layer) in the organic light-emitting diode display panel as the optical path folding element. After external incident light penetrates the transparent layer at the front of the panel, Fraunhofer diffraction occurs at the metal microstructure of the thin-film transistor backplane. The diffracted light continues to propagate downwards to the lower surface of the glass substrate, is reflected by the reflective layer, and then returns upwards, finally being received by the photosensitive unit integrated on the front of the panel.

[0065] The reflective layer can be achieved by depositing a reflective film on the back of the glass substrate. That is, after the display panel completes all the manufacturing processes, a highly reflective metal film (such as aluminum or silver) or a dielectric reflective film is deposited on the back (outer surface) of the glass substrate.

[0066] Process flow:

[0067] Complete all front-end processes, including thin-film transistor arrays, organic light-emitting diode devices, and packaging.

[0068] Clean the back of the panel;

[0069] A reflective film is deposited on the back side of a glass substrate by physical vapor deposition or chemical vapor deposition.

[0070] Optionally, a protective layer (such as silicon dioxide) can be applied to the reflective film.

[0071] This method achieves a reflective layer with a reflectivity exceeding 95%, maximizing light energy utilization. Although it adds a back-side deposition process, the thickness can be controlled to the sub-micron level. It fully utilizes the inherent structure of the organic light-emitting diode panel to achieve optical path folding, and the increased physical thickness is negligible compared to the original panel. More importantly, due to the doubled optical path length, the effective diffraction distance is doubled, and the spectral resolution is approximately doubled. Furthermore, the reflective layer can also be implemented using an existing cathode layer.

[0072] The back-reflective optical path solution is particularly suitable for organic light-emitting diode display products that pursue ultra-thinness and high integration, such as foldable screen phones, smartwatches, and augmented reality / virtual reality headsets, giving them the dual functions of "display + spectral sensing".

[0073] 2.2 Photosensitive unit integration scheme;

[0074] This invention proposes two photosensitive unit integration schemes, corresponding to different process routes and performance requirements.

[0075] Option A: In-situ integrated photodiode;

[0076] This method involves fabricating photodiodes on the same or adjacent layer as the pixel driving circuit on the thin-film transistor backplane, forming an in-situ photosensitive array. The process flow is described below:

[0077] A buffer layer is formed on the glass substrate;

[0078] Fabrication of thin-film transistor devices (including gate, source / drain, and active layer);

[0079] Key steps: On the same substrate, a PIN junction photodiode is formed through additional semiconductor deposition and doping steps. The N-region, I-region, and P-region of the photodiode can be achieved through ion implantation or epitaxial growth.

[0080] An interlayer dielectric layer is fabricated, and a hole is made to connect the photodiode electrode and the readout circuit.

[0081] Fabrication of organic light-emitting diode (OLED) device layers such as planarization layer, anode, light-emitting layer, and cathode;

[0082] Above the corresponding photodiode location, a diffraction microstructure is formed on a metal layer (such as a cathode layer or a light-shielding layer) through mask design.

[0083] The photodiode can be located on the same plane as the thin-film transistor (TFT), or it can be vertically stacked above or below the TFT. The preferred configuration is vertical stacking, with the photodiode positioned above the TFT (below the planarization layer) and connected to the TFT readout circuitry via vias. The diffraction microstructure is located on a metal layer directly above the photodiode. Its advantages include complete integration within the panel with no additional thickness; vertical alignment of the photosensitive unit and the diffraction microstructure, resulting in the shortest optical path and strongest signal; and full utilization of the TFT array's readout circuitry for high frame rate acquisition.

[0084] It should be noted that for back-reflective optical paths, the photosensitive unit is integrated on the front of the panel.

[0085] Option B: Under-display integrated type;

[0086] This solution employs a standard complementary metal-oxide-semiconductor (CMOS) image sensor chip, which is mounted on the back of the glass substrate of the display panel. The diffraction microstructure is located in a metal layer inside the panel; diffracted light passes through the glass substrate and is received by the CMOS chip. The specific implementation is as follows:

[0087] The display panel is manufactured normally, and diffraction microstructures are formed on the metal layer of the thin-film transistor backplane.

[0088] The display panel is bonded to the complementary metal-oxide-semiconductor image sensor chip using optical adhesive.

[0089] Refractive index matching is performed at the bonding interface to reduce Fresnel reflection loss;

[0090] Synchronization between display driving and complementary metal-oxide-semiconductor (CMOS) acquisition is achieved through timing control.

[0091] The advantages of this approach are that it can fully utilize existing mature complementary metal-oxide-semiconductor (CMOS) technology, offering excellent photosensitivity and reducing process complexity, thus facilitating high-pixel, high-dynamic-range spectral acquisition. The challenges are: high-precision bonding and alignment are required to ensure the alignment of the microstructure with the CMOS pixels; simultaneously, the adhesive thickness introduced during bonding between the glass substrate and the CMOS may cause diffraction pattern broadening, requiring algorithmic compensation.

[0092] In practice, the above solutions or their complementary solutions can be selected according to design requirements.

[0093] III. Spectral Reconstruction Algorithm;

[0094] The diffraction microstructure used in this invention features "arbitrary shape and no need for periodic arrays," which places two core requirements on the spectral reconstruction algorithm: first, high tolerance for geometric deviations of the microstructure; and second, no need to pre-store complex calibration data across the entire wavelength range. Only a single quasi-monochromatic illumination is required to obtain a reference diffraction pattern, and a point spread function across the entire wavelength range can be generated through numerical mapping. Furthermore, it is insensitive to the specific shape of the microstructure, which highly matches the high process tolerance hardware design concept of this invention. The basic description process is as follows:

[0095] 3.1 Point spread function generation;

[0096] Through a single quasi-monochromatic light irradiation (wavelength) Collect reference diffraction patterns. Based on the wavelength scaling property of Fraunhofer diffraction, the point spread function corresponding to any wavelength can be generated through affine transformation:

[0097] ;

[0098] in, This involves scaling and interpolating the reference diffraction pattern to match the pixel size of the photosensitive unit. This step can be completed in real time within the computational reconstruction module, without requiring any pre-stored full-band calibration data, which is fundamentally different from existing technologies that require complex pre-calibration.

[0099] 3.2 Mathematical model for spectral reconstruction;

[0100] In spectral acquisition mode, the broadband diffraction intensity recorded by the photosensitive unit array This can be viewed as a weighted sum of the spread functions at different wavelengths:

[0101] ;

[0102] in, Let be the spectral distribution function of the incident light to be determined. Discretizing the above integral equation, we obtain a system of linear equations in matrix form: ;in, A vector composed of diffraction intensity data collected by the photosensitive unit; This is a measurement matrix, where each column corresponds to a discrete wavelength. The point spread function vector; Let be the spectral vector to be determined, with dimension n being the number of discrete wavelengths.

[0103] Due to measurement noise and point spread function mapping errors, the above equations are usually ill-conditioned and cannot be solved directly.

[0104] In some embodiments, a reference diffraction pattern is acquired through a single quasi-monochromatic illumination. Based on the wavelength scaling characteristics of Fraunhofer diffraction, a point spread function matrix for the entire wavelength range is generated through numerical affine transformation. The broadband diffraction intensity collected by the photosensitive unit array is vectorized into a measurement vector, and a system of linear equations is established between the measurement vector and the spectral vector to be determined. The measurement matrix of the linear equation system is composed of the point spread function matrix. The system of linear equations is solved using the Tikhonov regularization method. By introducing a regularization term to suppress noise amplification, the spectral vector is obtained. The spectral vector is then linearly interpolated and spectrally calibrated to obtain the spectral distribution function of the incident light.

[0105] This invention employs the Tikhonov regularization method for solving the problem. By introducing a regularization term, noise amplification is suppressed. The formula is as follows: The solution obtained After linear interpolation and spectral calibration, the spectral distribution function of the incident light is obtained. Simultaneously, this invention employs a collaborative design of the algorithm and display driving timing to solve the display light interference problem, achieving a high degree of integration between the algorithm and hardware. While maintaining reconstruction accuracy, it effectively suppresses display light interference, ensuring the reliability of detection during display operation.

[0106] IV. Display and Perception Collaborative Driving Method;

[0107] To address the interference of display pixel emission on spectral acquisition, this invention proposes a spatiotemporally decoupled drive control scheme.

[0108] 4.1 Space reuse;

[0109] The spatial reuse in this invention refers to the spatial isolation and coordinated layout of spectral sensing and image display functions within the physical space of the display panel. Its core idea is to construct a low-noise spectral acquisition channel by utilizing the inherent non-display area and adjustable light-emitting units of the display panel without sacrificing the display aperture ratio.

[0110] In some embodiments, the diffraction microstructure is disposed in the non-opening region between pixel units or in the light-shielding metal layer region above the thin-film transistor device, and the feature size of the diffraction microstructure is smaller than the pixel pitch. In the spectral acquisition mode, the pixel units within a preset range around the diffraction microstructure are turned off by the display driving circuit to form a local dark field environment. A dedicated spectral acquisition time period is divided within the display frame period, during which pixel emission is paused or the pixels are placed in a reset state, while the photosensitive unit array is controlled to sample, thus isolating display light interference in time.

[0111] Specifically, the diffraction microstructure is placed in the non-aperture area between pixel units (corresponding to the area below the black matrix) or the microstructure is arranged in the light-shielding metal layer area inherent above the thin film transistor device. This area is opaque and its reuse does not affect the display effect, ensuring that the display aperture ratio is not affected. In the spectral acquisition mode, the drive control module turns off the pixels adjacent to the diffraction microstructure area to further reduce local stray light.

[0112] The spatial layout can adopt a single-point layout, that is, selecting a number of pixel gaps in a specific area of ​​the panel (such as the center or corners), and arranging an independent diffraction microstructure in each gap. This method is suitable for local spectral detection needs. Alternatively, an array layout can be adopted, in which multiple diffraction microstructures are evenly or randomly distributed throughout the entire display area to form a coding array, thereby improving the signal-to-noise ratio and accuracy of spectral reconstruction.

[0113] It should be noted that since the pixel pitch of modern high-resolution display panels is typically between 20μm and 100μm, while the feature size (aperture or slit width) of diffractive microstructures ranges from 0.1μm to 10μm, it can be fully accommodated within the pixel gap without affecting the display aperture ratio. For self-emissive display technologies such as organic light-emitting diodes (OLEDs) or Mini-LEDs, there are also non-emissive areas within the pixels (such as driving circuit wiring areas and thin-film transistor device areas). These areas can also be used to arrange microstructures, especially suitable for spectral imaging applications requiring higher spatial resolution.

[0114] 4.2 Dynamic Spatial Cooperative Driving Method (Temporal Control);

[0115] While static spatial isolation can prevent microstructures from directly blocking the light-emitting area, it cannot completely eliminate stray light diffraction and scattering generated by the emission of surrounding pixels. Therefore, this invention further proposes a dynamic spatial cooperative driving scheme. A dedicated spectral acquisition time period is defined within a display frame cycle. During the acquisition time period, the display driving circuit pauses pixel emission or resets it, while simultaneously controlling the photosensitive unit to perform sampling, thus isolating display light interference in time.

[0116] Specifically, in spectral acquisition mode, the display driving circuit shuts down pixel units within a certain range around the diffraction microstructure, creating a localized dark field environment. The smallest range is a ring of pixels directly adjacent to the microstructure (approximately a 3×3 pixel area), eliminating over 90% of near-field stray light. Furthermore, the stray light propagation range can be calculated based on the panel thickness and diffraction distance, dynamically adjusting the size of the shut-off area. Using thin-film transistor driving circuits, it is entirely possible to shut down only specific rows and columns of pixels without altering the display content of other areas.

[0117] For display panels using zone-driven technology (such as Mini-LED), the backlight zone containing the microstructure can be directly turned off. For display panels where individual pixels cannot be turned off (such as some LCD panels), pixel reset or black-state insertion methods can be used: pixel reset involves setting the driving voltage of the target pixel to a reset level during the acquisition period, making it appear black. Alternatively, black frame insertion involves inserting a completely black frame into the display frame sequence, and completing the spectral acquisition within that frame.

[0118] Furthermore, in some embodiments, within the first display frame, the pixel unit is controlled to display normally and acquire the combined signal of ambient light and display light; within the second display frame, the pixel units surrounding the diffraction microstructure are controlled to turn off or insert black frames, acquiring only the ambient light signal; the combined signal acquired in the first display frame and the ambient light signal acquired in the second display frame are differentially processed to obtain a pure incident light signal component to be detected.

[0119] Through the above-described collaborative driving method, the present invention can achieve high signal-to-noise ratio spectral signal acquisition without affecting the normal display effect.

[0120] In some alternative embodiments: a double-layer diffraction structure is employed, in which diffraction microstructures are fabricated separately on different metal layers (such as the gate layer and source / drain layers) to form a vertically stacked double-layer diffraction element. The two microstructures can be designed to be coaxially aligned, i.e., the two microstructures are vertically aligned to form a composite diffraction aperture, enhancing the optical field manipulation capability. Alternatively, they can be staggered: the two microstructures are arranged in a staggered manner to produce richer diffraction patterns and increase the amount of spectral encoded information.

[0121] The spatial reuse scheme proposed in this invention is compatible with mainstream thin-film transistor processes, including but not limited to mask design, photolithography, black matrix, planarization, etc.

[0122] Through spatial reuse design, the following technical effects were achieved: zero aperture ratio loss, low stray light interference, high spatial freedom, and no additional process steps.

[0123] This embodiment takes a computational spectrometer system integrated in an organic light-emitting diode display panel as an example to describe in detail its physical structure, driving timing and reconstruction algorithm.

[0124] refer to Figure 2 and Figure 3 This invention also provides a computational spectral detection method integrated into a display driver backplane, implemented based on any of the above-described systems, the method comprising the following steps:

[0125] S100, determine the incident light signal to be detected and the display panel. The display panel includes, in sequence along the optical path, a display function layer, a diffraction modulation layer, a photosensitive unit array, and a calculation and reconstruction module. The display function layer contains multiple pixel units. The diffraction modulation layer reuses at least one opaque metal layer in the thin film transistor array. At least one diffraction microstructure is formed on the opaque metal layer by photolithography. The photosensitive unit array is integrated on the thin film transistor substrate and located below the diffraction modulation layer. The calculation and reconstruction module is electrically connected to the photosensitive unit array.

[0126] S200, the incident light signal to be detected is input to the display panel, the incident light signal to be detected passes through the display functional layer in sequence to the diffraction modulation layer, the diffraction microstructure performs Fraunhofer diffraction modulation on the incident light signal to be detected to generate a diffraction light signal, the diffraction light signal continues to propagate downward and reaches the photosensitive unit array.

[0127] S300 receives the diffracted light signal through the photosensitive unit array, converts the diffracted light signal into an electrical signal, and transmits the electrical signal to the calculation and reconstruction module;

[0128] S400, the electrical signal is processed by the calculation and reconstruction module, and the spectral distribution function of the incident light is reconstructed by a regularization transformation algorithm to obtain the spectral detection result.

[0129] 1. Physical structure integration scheme;

[0130] like Figure 1 and Figure 4 As shown, this spectrometer system deeply reuses the existing layered structure of the display panel, eliminating the need to increase the panel thickness. In-situ reuse of the diffraction modulation layer: The system directly utilizes the light-shielding metal layer in the thin-film transistor drive circuit layer. During the photolithography stage of panel manufacturing, a diffraction micro-aperture is formed by etching on this opaque metal film, below the black matrix avoiding the pixel openings. The micro-aperture has an arbitrary irregular polygonal shape and is distributed in the corners of the panel (e.g., ...). Figure 5 As shown in the diagram, this design reduces the precision requirements of the manufacturing process, allowing for processing deviations in the microstructure. Folded optical path design: External incident light passes through micropores and then enters the transparent glass substrate for diffraction propagation. Photosensitive unit configuration: The diffracted light spot distribution is acquired by a complementary metal-oxide-semiconductor image sensor attached to the back of the panel, which is electrically connected to the back-end computing module.

[0131] 2. Spatiotemporally decoupled collaborative driving timing;

[0132] To eliminate interference from the display panel's own light emission on ambient light detection, this system employs a spatiotemporally decoupled driving scheme (such as...). Figure 6(As shown): Black frame insertion: After the normal image display frame, the display driving circuit inserts a specific detection frame. Within this detection frame, the pixels are in a black state (do not emit light). Precise sampling window: The system controller only issues a sensor enable signal within the detection frame, opening a microsecond-level sampling window. At this time, there is no self-illumination interference in the display area, and the photosensitive unit only collects the ambient light signal entering through the diffraction micro-aperture, achieving complete decoupling of the signal on the time axis.

[0133] 3. Spectral reconstruction process based on numerical calculation;

[0134] This system employs a closed-loop algorithm that combines offline preprocessing with online reconstruction (such as...). Figure 7 (As shown) Detection Completion: Single Reference Wavelength Calibration: During system initialization, only one monochromatic reference light source (such as a 532 nm laser) is needed to illuminate the micropore and record the diffraction pattern. The calculation module uses numerical affine transformation to derive the full-band point spread function from this single reference image based on the ratio between wavelength and diffraction pattern size, constructing a complete response matrix A. This step avoids the cumbersome full-band calibration process of traditional spectrometers. Regularized Reconstruction Calculation: During online sensing, the broadband diffraction distribution acquired by the photosensitive unit is vectorized into measured values. The calculation module calls the Tikhonov regularization algorithm to solve the linear equation system, ultimately calculating the one-dimensional environmental spectral distribution curve.

[0135] 4. Summary of technical effects;

[0136] Through the above embodiments, the present invention achieves a high degree of integration between spectral detection function and display panel. Its core advantage lies in:

[0137] Zero aperture ratio loss: The sensor and microstructure are hidden in the non-light-emitting area, which does not affect the screen brightness.

[0138] High process tolerance: Allows for irregular diffraction aperture shapes, reducing the difficulty of mass production.

[0139] Strong anti-interference capability: The problem of light interference in the display is solved by timing control, ensuring the accuracy of spectral detection.

[0140] Although the description of this disclosure has been quite detailed and particularly focused on several of the described embodiments, it is not intended to limit itself to any of these details or embodiments or any particular embodiment, but should be considered as effectively covering the intended scope of this disclosure by referring to the appended claims and taking into account the broad possible interpretations of these claims provided by the prior art. Furthermore, the foregoing description of this disclosure with respect to embodiments foreseeable by the inventors is intended to provide a useful description, and non-substantial modifications to this disclosure that have not yet been foreseen may still represent equivalent modifications.

Claims

1. A computational spectral detection system integrated into a display driver backplane, characterized in that, include: The display panel, along the optical path direction, includes a display function layer, a diffraction modulation layer, a photosensitive unit array, and a computational reconstruction module in sequence. The display function layer contains multiple pixel units for image display; The diffraction modulation layer reuses at least one opaque metal layer in the thin-film transistor array, and at least one diffraction microstructure is formed on the opaque metal layer by photolithography. The diffraction microstructure is used to perform Fraunhofer diffraction modulation on the incident light signal. The photosensitive unit array is integrated on the thin-film transistor substrate and located below the diffraction modulation layer, and is used to receive diffracted light signals and convert them into electrical signals. The computational reconstruction module is electrically connected to the photosensitive unit array and is used to process the electrical signal and reconstruct the spectral distribution function of the incident light.

2. The system according to claim 1, characterized in that, The diffraction modulation layer multiplexes at least one opaque metal layer in the thin-film transistor array, comprising: The diffraction modulation layer reuses the source and drain metal layers, and while forming the source and drain electrodes, it retains the micro-hole pattern in the non-display area corresponding to the pixel gap. Alternatively, the diffraction modulation layer may reuse the gate metal layer to form a diffraction slit in a specific region while forming the gate scan line; The diffraction microstructure is a through-hole or slit penetrating the opaque metal layer. The geometric dimensions of the diffraction microstructure are on the same order of magnitude as the wavelength of the spectrum to be detected. The diffraction microstructure has an arbitrary shape and can be single or sparsely distributed.

3. The system according to claim 1, characterized in that, The system also includes: A reflective layer is disposed on the back side of the glass substrate to reflect the downward-propagating diffracted light signal back upward. The photosensitive unit array includes photosensitive units integrated on the front of the display panel for receiving diffracted light signals reflected by the reflective layer.

4. The system according to claim 1, characterized in that, The diffraction microstructure is disposed in the non-opening region between pixel units or in the light-shielding metal layer region above the thin-film transistor device. The feature size of the diffraction microstructure is smaller than the pixel pitch. The diffraction microstructure is a through hole or gap penetrating the opaque metal layer. The diffraction microstructure has an arbitrary shape and can be single or sparsely distributed.

5. A computational spectral detection method integrated into a display driver backplane, implemented based on the system described in any one of claims 1 to 4, characterized in that, The method includes the following steps: The system determines the incident light signal to be detected and the display panel. The display panel includes, in sequence along the optical path, a display function layer, a diffraction modulation layer, a photosensitive unit array, and a computational reconstruction module. The display function layer contains multiple pixel units. The diffraction modulation layer reuses at least one opaque metal layer from the thin-film transistor array. At least one diffraction microstructure is formed on the opaque metal layer by photolithography. The photosensitive unit array is integrated on the thin-film transistor substrate and located below the diffraction modulation layer. The computational reconstruction module is electrically connected to the photosensitive unit array. The incident light signal to be detected is input to the display panel. The incident light signal to be detected passes through the display functional layer in sequence and reaches the diffraction modulation layer. The diffraction microstructure performs Fraunhofer diffraction modulation on the incident light signal to be detected to generate a diffraction light signal. The diffraction light signal continues to propagate downward and reaches the photosensitive unit array. The diffracted light signal is received by the photosensitive unit array, converted into an electrical signal, and transmitted to the calculation and reconstruction module. The electrical signal is processed by the computational reconstruction module, and the spectral distribution function of the incident light is reconstructed using a regularization transformation algorithm to obtain the spectral detection result.

6. The method according to claim 5, characterized in that, The incident light signal to be detected passes sequentially through the display functional layer to reach the diffraction modulation layer, including: The incident light signal to be detected enters from above the display panel, passes sequentially through the encapsulation layer, cathode layer, organic light-emitting layer, anode layer, planarization layer, and interlayer dielectric layer, and reaches the diffraction microstructure located on the metal layer of the thin film transistor backplane; The diffraction microstructure performs Fraunhofer diffraction on the incident light signal to be detected. The diffracted light signal continues to propagate downward through the gate insulating layer, the buffer layer, and the glass substrate, and is finally received by the photosensitive unit array located below the glass substrate.

7. The method according to claim 5, characterized in that, The method further includes: A reflective film is deposited on the back of the glass substrate to form a reflective layer. The diffracted light signal is reflected upwards by the reflective layer after propagating to the lower surface of the glass substrate. By receiving the reflected diffracted light signal through a photosensitive unit integrated on the front of the display panel, the effective diffraction distance is doubled.

8. The method according to claim 5, characterized in that, The step of processing the electrical signal through the calculation and reconstruction module and reconstructing the spectral distribution function of the incident light using a regularization transformation algorithm includes: A reference diffraction pattern is obtained by a single quasi-monochromatic illumination. Based on the wavelength scaling characteristics of Fraunhofer diffraction, a point spread function matrix for the entire wavelength band is generated by numerical affine transformation. The broadband diffraction intensity acquired by the photosensitive unit array is vectorized into a measurement vector, and a system of linear equations is established between the measurement vector and the spectral vector to be determined. The measurement matrix of the system of linear equations is composed of the point spread function matrix. The linear equations are solved using the Tikhonov regularization method. By introducing a regularization term to suppress noise amplification, the spectral vector is obtained. The spectral vector is then linearly interpolated and spectrally calibrated to obtain the spectral distribution function of the incident light.

9. The method according to claim 5, characterized in that, The method further includes: The diffraction microstructure is disposed in the non-opening region between pixel units or in the light-shielding metal layer region above the thin-film transistor device, wherein the feature size of the diffraction microstructure is smaller than the pixel pitch. In spectral acquisition mode, the pixel units within a preset range around the diffraction microstructure are turned off by the display driving circuit to form a local dark field environment; A dedicated spectral acquisition time period is defined within the display frame cycle. During the spectral acquisition time period, pixel emission is paused or the pixel is reset. At the same time, the photosensitive unit array is controlled to perform sampling, thus isolating display light interference in time.

10. The method according to claim 9, characterized in that, The method further includes: Within the first display frame, the pixel unit is controlled to display normally and to acquire a combined signal of ambient light and display light; Within the second display frame, the pixel units surrounding the diffraction microstructure are controlled to be turned off or inserted into black frames, so that only ambient light signals are collected; The composite signal acquired from the first display frame and the ambient light signal acquired from the second display frame are differentially processed to obtain a pure incident light signal component to be detected.

Images