High-sensitivity hyperspectral microscopic calculation imaging system and device

Abstract

The invention provides a high-sensitivity hyperspectral microscopic calculation imaging system, and the system comprises a microscope objective module which is used for carrying out the imaging amplification of a sample, and outputting first incident light; the adjustable wide-spectrum transmission mask module is used for carrying out wavelength selective truncation and high-transmittance transmission on the first incident light to obtain second incident light; the spectrum modulation mask module is used for carrying out structured wavelength coding and compression acquisition on the second incident light to obtain a modulated coded light signal; the image sensor module is used for receiving the coded light signal and converting the coded light signal into two-dimensional compressed image data; and the spectrum decoupling reconstruction algorithm module is used for performing decoupling reconstruction processing on the two-dimensional compressed image data and outputting a three-dimensional hyperspectral data cube containing a spatial dimension and a spectrum dimension. The invention provides a hyperspectral microscopic calculation imaging scheme with high sensitivity, high resolution, high-speed imaging and quantitative analysis capabilities, the system complexity and cost are reduced, and the practical application value is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computational microscopy imaging technology, and in particular to a high-sensitivity hyperspectral computational microscopy imaging system and device. Background Technology

[0002] Hyperspectral microscopy is a core technology in the interdisciplinary field that integrates spectral analysis, microscopic observation, and imaging techniques. It transcends the limitations of traditional microscopy, which only provides morphological information, enabling a unified "map-spectrum" analysis of the chemical composition, material distribution, and interactions within micro-regions of samples. This has irreplaceable application value in fields such as biomedicine (e.g., label-free identification of metabolic components in living cells, and precise multi-target diagnosis and grading of pathological sections), materials science (e.g., micro-region component analysis and defect detection), and environmental monitoring (e.g., rapid classification and counting of microplastics, and spatial distribution monitoring of microbial communities in soil / water).

[0003] Among related technologies, the technical solutions for achieving hyperspectral microscopy imaging include grating-based hyperspectral microscopy systems, tunable filter-based hyperspectral microscopy systems, and computational imaging fusion hyperspectral microscopy systems. In grating-based hyperspectral microscopy systems, the spectral dispersion process leads to significant dispersion of single-wavelength light energy, causing signal intensity attenuation and low sensitivity for low signal-to-noise ratio samples such as those with weak fluorescence or weak reflection. In tunable filter-based hyperspectral microscopy systems, the low transmittance of the LCTF / AOTF results in a significant drop in signal-to-noise ratio (SNR) for weak signal samples due to the combined effect of low transmittance and successive signal attenuation. Furthermore, the inherent design contradiction between the spectral bandwidth and peak transmittance of the filters makes it difficult to maintain high light flux while ensuring high spectral resolution. In computational imaging fusion hyperspectral microscopy systems, the algorithm model and optical hardware design are independent, lacking deep adaptation, and reconstruction is prone to spectral drift and spatial artifacts, resulting in insufficient accuracy in weak signal scenarios. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the first objective of this disclosure is to propose a high-sensitivity hyperspectral microscopy computational imaging system. Through a microscope objective module, an adjustable broadband transmission mask module, a spectral modulation mask module, an image sensor module, and a spectral decoupling and reconstruction algorithm module, it provides a hyperspectral microscopy computational imaging scheme that combines high sensitivity, high resolution, high speed imaging, and quantitative analysis capabilities, thereby reducing system complexity and cost and enhancing practical application value.

[0006] To achieve the above objectives, a first aspect of this disclosure proposes a high-sensitivity hyperspectral microscopic computational imaging system. The system includes a microscope objective module, an tunable broadband transmission mask module, a spectral modulation mask module, an image sensor module, and a spectral decoupling and reconstruction algorithm module. The microscope objective module is used to image and magnify the sample and output the first incident light; The adjustable broadband transmission mask module is connected to the microscope objective module and is used to selectively cut off the wavelength of the first incident light and transmit it with high transmission to obtain the second incident light. The spectral modulation mask module is connected to the tunable broadband transmission mask module and is used to perform structured wavelength encoding and compression acquisition on the second incident light to obtain the modulated encoded optical signal. The image sensor module is connected to the spectral modulation mask module and is used to receive the encoded optical signal and convert the encoded optical signal into two-dimensional compressed image data. The spectral decoupling reconstruction algorithm module is connected to the image sensor module and is used to perform decoupling reconstruction processing on the two-dimensional compressed image data, outputting a three-dimensional hyperspectral data cube containing spatial and spectral dimensions.

[0007] The high-sensitivity hyperspectral microscopic computational imaging system of this invention may also have the following additional technical features: Optionally, the tunable broadband transmission mask module includes a substrate, a liquid crystal tunable layer, and an antireflection film stacked sequentially. The substrate is made of quartz material with a preset light transmittance range; The liquid crystal adjustable layer has an array-type pixel structure, with the size of a single pixel on the order of micrometers. The antireflection film covers the surface of the liquid crystal tunable layer and is used to achieve an antireflection rate greater than a preset value within a preset wavelength range.

[0008] Optionally, the step of performing wavelength selective cutoff and high-transmission of the first incident light to obtain the second incident light includes: The first incident light passes through the substrate to obtain the third incident light corresponding to the preset light transmission range; The target voltage applied to the liquid crystal tunable layer is obtained, and the deflection angle of the liquid crystal molecules in the liquid crystal tunable layer is controlled based on the target voltage to optically cut off the wavelength of the third incident light, and the second incident light is obtained through high-transmission transmission through the antireflection film.

[0009] Optionally, the adjustable broadband transmission mask module includes at least one set of preset fixed cutoff wavelength modes, including visible light band modes or near-infrared band modes.

[0010] Optionally, the spectral modulation mask module includes a two-dimensional coding array composed of multiple layers of heterogeneous materials, wherein each coding unit matches the pixel size of the image sensor module. The multilayer heterogeneous material stack comprises at least two dielectric material layers or metal-dielectric composite layers with different dielectric constants. The optical response of the two-dimensional coded array exhibits Fano-type transmittance spectral characteristics with asymmetric resonance peaks.

[0011] Optionally, when the multilayer heteromaterial stack includes at least two dielectric material layers with different dielectric constants, the multilayer heteromaterial stack includes a SiO2 layer, a TiO2 layer, and a Si3N4 layer stacked sequentially.

[0012] Optionally, the encoding method of the two-dimensional encoding array is random mask encoding or photolithographic custom mask encoding; The random mask encoding is naturally formed by the thickness fluctuations or interface roughness differences of the heterogeneous material stacks during the processing. The photolithographic custom mask coding uses nanofabrication technology to control the interlayer structure of individual coding units and preset their Fano resonance peak position and transmittance curve.

[0013] Optionally, the decoupling and reconstruction processing of the two-dimensional compressed image data to output a three-dimensional hyperspectral data cube containing spatial and spectral dimensions includes: The two-dimensional compressed image data is subjected to noise reduction and dark current correction to obtain the first compressed image data; Based on the preset encoding matrix of the spectral modulation mask module, the mixed wavelength signal in the first compressed image data is separated by an iterative optimization algorithm to obtain the second compressed image data; Based on the Transformer neural network model, the second compressed image data is mapped and reconstructed to obtain the reconstructed spectral data; A standard spectral library is introduced to perform intensity calibration on the reconstructed spectral data, and a three-dimensional hyperspectral data cube containing spatial and spectral dimensions is output.

[0014] Optionally, the hyperspectral microscopy computational imaging system can be applied to multi-label fluorescence quantitative analysis of biological samples or quantitative detection of micro-region components of materials.

[0015] To achieve the above objectives, a first aspect of this disclosure provides a high-sensitivity hyperspectral microscopy computational imaging device, the device comprising the high-sensitivity hyperspectral microscopy computational imaging system.

[0016] In summary, the high-sensitivity hyperspectral microscopic computational imaging system and apparatus provided in this disclosure include a microscope objective module, an tunable broadband transmission mask module, a spectral modulation mask module, an image sensor module, and a spectral decoupling reconstruction algorithm module. The microscope objective module is used to image and magnify the sample and output a first incident light. The tunable broadband transmission mask module, connected to the microscope objective module, is used to selectively cut off the wavelength of the first incident light and achieve high-transmission to obtain a second incident light. The spectral modulation mask module, connected to the tunable broadband transmission mask module, is used to perform structured wavelength encoding and compression acquisition of the second incident light to obtain a modulated coded light signal. The image sensor module, connected to the spectral modulation mask module, is used to receive the coded light signal and convert it into two-dimensional compressed image data. The spectral decoupling reconstruction algorithm module, connected to the image sensor module, is used to perform decoupling reconstruction processing on the two-dimensional compressed image data and output a three-dimensional hyperspectral data cube containing spatial and spectral dimensions. This disclosure provides a hyperspectral microscopy computational imaging scheme that combines high sensitivity, high resolution, high speed imaging, and quantitative analysis capabilities through a microscope objective module, an adjustable broadband transmission mask module, a spectral modulation mask module, an image sensor module, and a spectral decoupling reconstruction algorithm module. This reduces system complexity and cost, and enhances practical application value.

[0017] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which: Figure 1 This is a schematic diagram of the structure of a high-sensitivity hyperspectral microscopic computational imaging system provided in an embodiment of the present disclosure; Figure 2 This is a schematic diagram of the structure of a high-sensitivity hyperspectral microscopic computational imaging device provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram showing the results of an example of fluorescent microsphere imaging provided in an embodiment of this disclosure. Detailed Implementation

[0019] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0020] In related technologies, grating-based hyperspectral microscopy systems integrate dispersive elements such as gratings or prisms into the microscope's optical path to spatially disperse incident light from the sample according to wavelength. A planar array detector synchronously records the spectral information corresponding to a specific spatial line, acquiring a complete three-dimensional data cube through mechanical scanning or detector movement. The advantages of this approach are high spectral resolution (typically reaching the nanometer scale) and a wide spectral coverage. However, its inherent drawbacks are significant: the dispersion process leads to substantial dispersion of single-wavelength light energy, resulting in a system light flux utilization rate generally below 10%, causing severe signal intensity attenuation and extremely low sensitivity for low signal-to-noise ratio samples such as those with weak fluorescence or weak reflectance. Furthermore, the necessary mechanical scanning or complex encoding process makes acquiring a single hyperspectral image take several seconds or even longer, resulting in slow imaging speed, susceptibility to vibration artifacts, and information coupling distortion between the spectral and spatial dimensions. This makes it unsuitable for dynamic biological process observation or high-speed online detection.

[0021] Furthermore, tunable filter-based hyperspectral microscopy systems employ liquid crystal tunable filters (LCTFs) or acousto-optic tunable filters (AOTFs) as core spectroscopic elements. Their transmission center wavelength is controlled by electrical signals, sequentially acquiring two-dimensional spatial images at different wavelengths in a "band-by-band scanning" manner. These images are then stacked to create a hyperspectral data cube. This approach is relatively compact, with no macroscopically moving mechanical parts. However, its drawbacks are equally apparent: LCTFs typically have a light transmittance of only 30%-50%, while AOTFs suffer from limited spectral range (usually covering only the visible light or part of the near-infrared band) and insufficient out-of-band suppression. Their "band-by-band imaging" mode is essentially a serial acquisition process, inherently limiting efficiency; imaging speed is constrained by the filter switching rate and single-band integration time. For weak signal samples, the combination of low transmittance and successive signal attenuation further deteriorates the signal-to-noise ratio (SNR) of the final image. More importantly, there is an inherent design contradiction between the spectral bandwidth (which determines the spectral resolution) and the peak transmittance of the filter, making it difficult to maintain high luminous flux while ensuring high spectral resolution.

[0022] Furthermore, computational imaging fusion hyperspectral microscopy systems aim to simplify the front-end optical beam-splitting structure (e.g., using random masks or simple dispersive elements), acquire coded and modulated compressed two-dimensional images at the detector end, and then reconstruct a complete hyperspectral data cube from the undersampled data using complex mathematical models and reconstruction algorithms (e.g., deep learning). However, in the above schemes, the algorithm model and optical hardware design are often independent of each other, lacking deep adaptation, which makes the reconstruction process prone to introducing spectral distortion (e.g., wavelength drift) or spatial artifacts, and the reconstruction accuracy heavily depends on prior assumptions. In weak signal scenarios, since the signal-to-noise ratio of the compressed observations is already very low, the uncertainty and error of the algorithm reconstruction are drastically amplified. In addition, to ensure the feasibility and speed of reconstruction, it is often necessary to sacrifice the spectral or spatial resolution of the final output, and the reconstruction capability for weak signals is limited, failing to fundamentally solve the problem of insufficient sensitivity caused by low light energy utilization.

[0023] In summary, existing hyperspectral microscopy systems generally suffer from a technical contradiction where sensitivity, spectral resolution, and imaging speed are mutually exclusive: pursuing high spectral resolution can lead to light energy dispersion or reduced transmittance, resulting in a significant decrease in sensitivity and making them unsuitable for detecting weak signal samples; increasing sensitivity requires increasing light flux, often at the cost of sacrificing spectral resolution or narrowing the spectral range; even when combined with computational imaging technology, existing solutions have not yet overcome the dual bottlenecks of "optical hardware limitations + algorithm reconstruction accuracy," making it difficult to simultaneously meet the practical application requirements of high-sensitivity capture, high-spectral resolution, and efficient imaging.

[0024] Furthermore, the existing systems' optical and computational modules are independent, lacking an integrated design, resulting in high optical path loss and high data transmission latency, further limiting the overall system performance. For scenarios such as live-cell observation with low fluorescence quantum yields in biomedicine and micro-area weak reflectance component analysis in materials science, existing technologies still cannot provide effective solutions, necessitating a novel hyperspectral microscopy imaging system that can synergistically optimize optical hardware and computational algorithms.

[0025] The high-sensitivity hyperspectral microscopic computational imaging system of this disclosure will be described in detail below with reference to specific embodiments.

[0026] Figure 1 This disclosure provides a high-sensitivity hyperspectral microscopic computational imaging system. For example... Figure 1 As shown, this high-sensitivity hyperspectral microscopic computational imaging system may include a microscope objective module 101, an adjustable broadband transmission mask module 102, a spectral modulation mask module 103, an image sensor module 104, and a spectral decoupling and reconstruction algorithm module 105, wherein... The microscope objective module 101 is used to image and magnify the sample and output the first incident light; The adjustable broadband transmission mask module 102 is connected to the microscope objective module 101 and is used to selectively cut off the wavelength of the first incident light and transmit it with high transmission to obtain the second incident light. The spectral modulation mask module 103 is connected to the tunable broadband transmission mask module 102 and is used to perform structured wavelength encoding and compression acquisition of the second incident light to obtain the modulated encoded optical signal. The image sensor module 104 is connected to the spectral modulation mask module 103 and is used to receive coded light signals and convert the coded light signals into two-dimensional compressed image data. The spectral decoupling reconstruction algorithm module 105 is connected to the image sensor module 104 and is used to perform decoupling reconstruction processing on two-dimensional compressed image data, outputting a three-dimensional hyperspectral data cube containing spatial and spectral dimensions.

[0027] In one embodiment of this disclosure, the aforementioned tunable broadband transmission mask module may include a substrate, a liquid crystal tunable layer, and an antireflection film stacked sequentially. The substrate is made of quartz material with a preset light transmittance range, which can be 300-2500 nm, and a refractive index uniformity ≤1×10⁻⁶. -6 The liquid crystal tunable layer has an array-type pixel structure with a single pixel size on the order of micrometers. The pixel size corresponding to the pixel structure is 5×5μm. An anti-reflection film is applied to the surface of the liquid crystal tunable layer to achieve an anti-reflection rate greater than a preset value within a preset wavelength range. The preset wavelength range can be 400-1700 nm, and the preset value can be 99%.

[0028] In one embodiment of this disclosure, a method for selectively cutting off and transmitting high-transmittance light to obtain a second incident light based on the above-mentioned substrate, liquid crystal tunable layer and antireflection film may include: the first incident light passes through the substrate to obtain a third incident light corresponding to a preset light transmission range; a target voltage applied to the liquid crystal tunable layer is obtained, and the deflection angle of the liquid crystal molecules in the liquid crystal tunable layer is controlled based on the target voltage to optically cut off the wavelength of the third incident light, and the second incident light is obtained through high-transmittance transmission via the antireflection film.

[0029] In one embodiment of this disclosure, the deflection angle of liquid crystal molecules can be controlled by applying a gradient voltage (0-5 V) to the liquid crystal tunable layer, thereby changing the optical cutoff wavelength of the mask and achieving continuous tunability within a preset wavelength range (adjustment step size ≤ 5 nm).

[0030] In one embodiment of this disclosure, for specific wavelength band requirements, the aforementioned tunable broadband transmission mask module may further include at least one preset set of fixed cutoff wavelength modes, including visible light band modes (such as visible light band 400-760nm) or near-infrared band modes (near-infrared band 760-1700nm). The switching response time is ≤10 ms.

[0031] In one embodiment of this disclosure, the aforementioned adjustable broadband transmission mask module can replace the traditional narrowband filter structure. It achieves selective cutoff of emitted light by precisely adjusting the cutoff wavelength, while ensuring high transmittance (≥85%) over a wide spectral range (e.g., 400-1700nm), reducing weak signal loss. It has no mechanical movement, a compact structure, and is suitable for the miniaturization requirements of microscopic optical paths. The adjustable design can flexibly match the spectral characteristics of different samples (e.g., emission peaks of fluorescent dyes, absorption bands of material characteristics), improving the system's sensitivity to capturing low-fluorescence and weak-reflection samples, and adapting to multi-band imaging scenarios.

[0032] In one embodiment of this disclosure, the spectral modulation mask module may include a two-dimensional coding array composed of multiple heterogeneous material stacks, wherein a single coding unit matches the pixel size of the image sensor module, wherein the multiple heterogeneous material stacks include at least two dielectric material layers or metal-dielectric composite layers with different dielectric constants; the optical response of the two-dimensional coding array exhibits Fano-type transmittance spectral characteristics with asymmetric resonance peaks.

[0033] In one embodiment of this disclosure, when the multilayer heteromaterial stack includes at least two dielectric material layers with different dielectric constants, the multilayer heteromaterial stack includes a SiO2 layer, a TiO2 layer and a Si3N4 layer stacked sequentially.

[0034] In one embodiment of this disclosure, coding units (e.g., 3.75μm×3.75μm) with dimensions matching the pixels of an image sensor can be formed by precisely controlling the thickness of each layer (10-200 nm) and the interface roughness, with an array size of 2048×2048 pixels.

[0035] In one embodiment of this disclosure, the optical response of the multilayer heterogeneous material stack exhibits Fano-type transmittance spectral characteristics: due to the optical interference and resonant coupling effects between different material layers, the transmittance of the same coding unit for different wavelengths of light exhibits an asymmetric resonance peak (peak width ≤ 50 pm), and the curve of transmittance changing with wavelength has significant uniqueness, which can replace the traditional binary switch response.

[0036] In one embodiment of this disclosure, the above-mentioned spectral modulation mask module receives the second incident light from the adjustable broadband transmission mask module, and compresses and collects the second incident light through structured coding. In a single exposure, the integration of spatial information and multi-wavelength spectral signals is completed simultaneously, breaking the efficiency limitation of band-by-band acquisition, and achieving ultra-high spectral resolution at the picometer level (≤100 pm), taking into account both imaging speed and resolution performance.

[0037] In one embodiment of this disclosure, the Fano resonance (peak width ≤ 50 pm) of the above-mentioned heterogeneous material stacks achieves a spectral resolution of ≤ 100 pm in a micrometer-level field of view, which is 1-2 orders of magnitude higher than the traditional technology. This allows for precise analysis of fine micro-region features such as spectral differences of intracellular organelles and fluctuations in material grain boundary composition.

[0038] In one embodiment of this disclosure, the encoding method of the above-mentioned two-dimensional encoding array is random mask encoding or photolithographic custom mask encoding. Random mask encoding is naturally formed by the thickness fluctuation or interface roughness difference of the heterogeneous material stack during the processing. Photolithographic custom mask encoding controls the interlayer structure of a single encoding unit through nanofabrication technology and presets its Fano resonance peak position and transmittance curve.

[0039] In one embodiment of this disclosure, the aforementioned random mask can utilize the thickness fluctuations or interface roughness differences naturally generated during the processing of heterogeneous material stacking (such as magnetron sputtering or atomic layer deposition) to enable each coding unit to form a unique Fano transmittance spectral response, achieving randomized wavelength coding without additional design; the aforementioned photolithography-customized mask can precisely control the interlayer structure of a single heterogeneous stacked unit (such as local etching to form nanopores or steps) through electron beam lithography or nanoimprint technology, preset its Fano resonance peak position and transmittance curve, and form a deterministic coding array.

[0040] In one embodiment of this disclosure, the above encoding methods all utilize the strong correlation between wavelength and transmittance of heterogeneous stacks to enable the same spatial pixel position to simultaneously carry characteristic transmission signals of multiple wavelengths (different wavelengths correspond to different transmittances), thereby achieving parallel acquisition of multi-wavelength information in a single exposure and overcoming the efficiency limitations of band-by-band scanning.

[0041] In one embodiment of this disclosure, the method by which the above-described spectral decoupling reconstruction algorithm module performs decoupling reconstruction processing on two-dimensional compressed image data and outputs a three-dimensional hyperspectral data cube containing spatial and spectral dimensions may include the following steps: Step 1: Denoise reduction and dark current correction are performed on the two-dimensional compressed image data to obtain the first compressed image data; Step 2: Based on the preset encoding matrix of the spectral modulation mask module, the mixed wavelength signal in the first compressed image data is separated by an iterative optimization algorithm to obtain the second compressed image data; Step 3: Based on the Transformer neural network model, the second compressed image data is mapped and reconstructed to obtain the reconstructed spectral data; Step 4: Introduce a standard spectral library to perform intensity calibration on the reconstructed spectral data, and output a three-dimensional hyperspectral data cube containing spatial and spectral dimensions.

[0042] In one embodiment of this disclosure, a wavelet threshold denoising algorithm is used to denoise and correct dark current in two-dimensional compressed image data to obtain first compressed image data, thereby improving signal purity.

[0043] In one embodiment of this disclosure, based on the preset encoding matrix of the spectral modulation mask module, a second compressed image data is obtained by separating the mixed wavelength signals in the first compressed image data through an iterative optimization algorithm, thereby eliminating crosstalk between different bands. In one embodiment of this disclosure, the iterative optimization algorithm can be the alternating direction multiplier method.

[0044] In one embodiment of this disclosure, the Transformer neural network model can be a trained SPECAT.

[0045] In one embodiment of this disclosure, the aforementioned standard spectral library may be the NIST standard material spectrum.

[0046] In one embodiment of this disclosure, the above-mentioned spectral decoupling and reconstruction algorithm module accurately reconstructs picometer-resolution hyperspectral information from the compressed data output by the spectral modulation mask, effectively de-aliasing the signal, eliminating artifact interference, and simultaneously achieving quantitative calibration of the spectral signal. This upgrades the qualitative observation of traditional hyperspectral imaging to quantitative analysis, meeting the precise detection needs of fields such as biomedicine and materials science.

[0047] In one embodiment of this disclosure, the aforementioned tunable broadband transmission mask module and spectral modulation mask module are integrated and packaged into a planar spectral imaging chip, which is configured to be replaceably integrated into the imaging optical path of a conventional biological microscope to replace its original filtering or beam splitting components.

[0048] In one embodiment of this disclosure, the above-mentioned spectral imaging chip adopts a planar structure and can be directly integrated into the optical path of existing bright-field / fluorescence microscopes (such as Olympus and Nikon models) without modifying the main structure. It lowers the application threshold through low-cost mass production (magnetron sputtering + photolithography process), which is different from the specialized and high-cost design of existing high-end systems.

[0049] In one embodiment of this disclosure, the hyperspectral microscopy computational imaging system described above can be applied to multi-label fluorescence quantitative analysis of biological samples or quantitative detection of micro-region components of materials through a spectral imaging chip.

[0050] The high-sensitivity hyperspectral microscopic computational imaging system disclosed herein includes a microscope objective module, an tunable broadband transmission mask module, a spectral modulation mask module, an image sensor module, and a spectral decoupling reconstruction algorithm module. The microscope objective module is used to image and magnify the sample and output a first incident light. The tunable broadband transmission mask module, connected to the microscope objective module, is used to selectively cut off the wavelength of the first incident light and achieve high transmission to obtain a second incident light. The spectral modulation mask module, connected to the tunable broadband transmission mask module, is used to perform structured wavelength encoding and compression acquisition of the second incident light to obtain a modulated coded light signal. The image sensor module, connected to the spectral modulation mask module, is used to receive the coded light signal and convert it into two-dimensional compressed image data. The spectral decoupling reconstruction algorithm module, connected to the image sensor module, is used to perform decoupling reconstruction processing on the two-dimensional compressed image data and output a three-dimensional hyperspectral data cube containing spatial and spectral dimensions. This disclosure provides a hyperspectral microscopy computational imaging scheme that combines high sensitivity, high resolution, high speed imaging, and quantitative analysis capabilities through a microscope objective module, an adjustable broadband transmission mask module, a spectral modulation mask module, an image sensor module, and a spectral decoupling reconstruction algorithm module. This reduces system complexity and cost, and enhances practical application value.

[0051] Figure 2 A high-sensitivity hyperspectral microscopic computational imaging device provided in the embodiments of this disclosure, such as Figure 2 As shown, Using the Olympus IX83 inverted fluorescence microscope as the basic platform, the aforementioned high-sensitivity hyperspectral microscopy computational imaging system was integrated to verify high-sensitivity hyperspectral imaging. Specifically, the traditional emission filter in the original fluorescence excitation block of the microscope was removed, and the spectral imaging chip, consisting of an tunable broadband transmission mask module and a spectral modulation mask module, was integrated into the microscope's imaging focal plane. The chip's detection surface was coaxial with the original filter plane, and the high-sensitivity hyperspectral imaging experiment was completed in conjunction with the spectral decoupling and reconstruction algorithm module. The tunable broadband transmission mask module's liquid crystal tunable layer consists of a 5×5μm array of pixels, with a double-layer antireflection coating (≥99%) covering the 400-1700 nm wavelength band. In the experiment, the cutoff wavelength was adjusted to 450-750 nm (fluorescence band) by applying a 0-5 V gradient voltage, with an adjustment step size of 3 nm and a switching response time of 8 ms.

[0052] Furthermore, the aforementioned spectral modulation mask module is constructed based on a 2048×2048 pixel two-dimensional encoding array using a stack of SiO2 / TiO2 / Si3N4 heteromaterials. The thicknesses of each layer are SiO2 (50 nm), TiO2 (30 nm), and Si3N4 (80 nm), respectively. A random mask is formed through magnetron sputtering, and each unit exhibits a Fano-type transmittance response (asymmetric resonance peak width ≤ 45 pm) due to interlayer interference, enabling synchronous differential encoding of multi-wavelength signals in the 450-750 nm band.

[0053] Furthermore, the aforementioned tunable broadband transmission mask module and spectral modulation mask module are integrated with a 16-bit deep SCMOS image sensor (quantum efficiency ≥92%) to constitute the spectral imaging chip of this invention, replacing traditional filtering and beam splitting components (such as...). Figure 2 (As shown).

[0054] And, the analytical results of quantitative analysis of mixed fluorescent microspheres using the aforementioned high-sensitivity hyperspectral microscopic computational imaging device are as follows: Figure 3 As shown, for an agar sample of FITC, Cy3, and Cy5 fluorescent microspheres (5 μm in diameter) mixed in a 1:1:1 volume ratio: conventional microscopy only revealed a blurry, mixed fluorescent patch, such as... Figure 3 As shown in Figure A, a single detection can only qualitatively distinguish one type of fluorescent sphere and cannot differentiate component boundaries; for example... Figure 3 As shown in Figure B, this disclosure can directly distinguish three different types of fluorescent microspheres through a single detection. Based on the mask response model and calibration with NIST standards, the quantitatively calculated volume percentages are 33.9%, 32.7%, and 33.4% (actual percentage 33.3% ± 0.5%). Meanwhile, as... Figure 3 Figure C shows the distribution of material components in the sample obtained by picometer spectroscopy in this disclosure; as shown... Figure 3 Figure D shows the quantitative distribution of the content of each group of fluorescent microspheres and agar obtained in this disclosure, and the spatial distribution of each component is quantitatively deconstructed. Based on this, the integration feasibility of this disclosure is verified, achieving a breakthrough from "single-band qualitative analysis" to "full-spectrum quantitative analysis" on a conventional microscope, adapting to the scenario of fine analysis of biological multi-labeled samples.

[0055] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0056] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0057] This disclosure is intended to provide implementation schemes for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.

[0058] The acquisition, transmission, storage, use, and processing of data in this disclosed technical solution all comply with the relevant provisions of national laws and regulations.

[0059] It should be noted that in the embodiments disclosed herein, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary and are intended only to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used such solutions.

[0060] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0061] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0062] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0063] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0064] It should be understood that various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0065] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0066] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0067] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A high-sensitivity hyperspectral microscopic computational imaging system, characterized in that, The system includes a microscope objective module, an tunable broadband transmission mask module, a spectral modulation mask module, an image sensor module, and a spectral decoupling and reconstruction algorithm module, wherein... The microscope objective module is used to image and magnify the sample and output the first incident light; The adjustable broadband transmission mask module is connected to the microscope objective module and is used to selectively cut off the wavelength of the first incident light and transmit it with high transmission to obtain the second incident light. The spectral modulation mask module is connected to the tunable broadband transmission mask module and is used to perform structured wavelength encoding and compression acquisition on the second incident light to obtain the modulated encoded optical signal. The image sensor module is connected to the spectral modulation mask module and is used to receive the encoded optical signal and convert the encoded optical signal into two-dimensional compressed image data. The spectral decoupling reconstruction algorithm module is connected to the image sensor module and is used to perform decoupling reconstruction processing on the two-dimensional compressed image data, outputting a three-dimensional hyperspectral data cube containing spatial and spectral dimensions.

2. The system according to claim 1, characterized in that, The adjustable broadband transmission mask module includes a substrate, an adjustable liquid crystal layer, and an antireflection film stacked sequentially. The substrate is made of quartz material with a preset light transmittance range; The liquid crystal adjustable layer has an array-type pixel structure, with the size of a single pixel on the order of micrometers. The antireflection film covers the surface of the liquid crystal tunable layer and is used to achieve an antireflection rate greater than a preset value within a preset wavelength range.

3. The system according to claim 2, characterized in that, The process of selectively cutting off the wavelength of the first incident light and achieving high-transmission transmission to obtain the second incident light includes: The first incident light passes through the substrate to obtain the third incident light corresponding to the preset light transmission range; The target voltage applied to the liquid crystal tunable layer is obtained, and the deflection angle of the liquid crystal molecules in the liquid crystal tunable layer is controlled based on the target voltage to optically cut off the wavelength of the third incident light, and the second incident light is obtained through high-transmission transmission through the antireflection film.

4. The system according to claim 2, characterized in that, The adjustable broadband transmission mask module includes at least one set of preset fixed cutoff wavelength modes, including visible light band modes or near-infrared band modes.

5. The system according to claim 1, characterized in that, The spectral modulation mask module includes a two-dimensional coding array composed of multiple layers of heterogeneous materials, wherein each coding unit matches the pixel size of the image sensor module. The multilayer heterogeneous material stack comprises at least two dielectric material layers or metal-dielectric composite layers with different dielectric constants. The optical response of the two-dimensional coded array exhibits Fano-type transmittance spectral characteristics with asymmetric resonance peaks.

6. The system according to claim 5, characterized in that, When the multilayer heteromaterial stack includes at least two dielectric material layers with different dielectric constants, the multilayer heteromaterial stack includes a SiO2 layer, a TiO2 layer, and a Si3N4 layer stacked sequentially.

7. The system according to claim 5, characterized in that, The encoding method of the two-dimensional encoding array is random mask encoding or photolithographic custom mask encoding; The random mask encoding is naturally formed by the thickness fluctuations or interface roughness differences of the heterogeneous material stacks during the processing. The photolithographic custom mask coding uses nanofabrication technology to control the interlayer structure of individual coding units and preset their Fano resonance peak position and transmittance curve.

8. The system according to claim 1, characterized in that, The decoupling and reconstruction processing of the two-dimensional compressed image data outputs a three-dimensional hyperspectral data cube containing spatial and spectral dimensions, including: The two-dimensional compressed image data is subjected to noise reduction and dark current correction to obtain the first compressed image data; Based on the preset encoding matrix of the spectral modulation mask module, the mixed wavelength signal in the first compressed image data is separated by an iterative optimization algorithm to obtain the second compressed image data; Based on the Transformer neural network model, the second compressed image data is mapped and reconstructed to obtain the reconstructed spectral data; A standard spectral library is introduced to perform intensity calibration on the reconstructed spectral data, and a three-dimensional hyperspectral data cube containing spatial and spectral dimensions is output.

9. A hyperspectral microscopic computational imaging system as described in any one of claims 1-8, applied to multi-label fluorescence quantitative analysis of biological samples or quantitative detection of micro-region components of materials.

10. A highly sensitive hyperspectral microscopic computational imaging device, characterized in that, The device includes the high-sensitivity hyperspectral microscopic computational imaging system.

Images