Optical imaging system and imaging method for an unsectioned tissue sample

Abstract

The application relates to the technical field of optical imaging and nondestructive testing, and discloses an optical imaging system and imaging method for an unsliced tissue sample, which comprises a dynamic self-adaptive illumination module and an intelligent sensing system comprising a multi-dimensional superstructure surface, a neuromorphic sensor and an on-chip edge computing unit. Mixed exit light is captured asynchronously after spatial demultiplexing of multi-dimensional physical properties is realized by the superstructure surface. The on-chip computing unit performs bottom-layer time-of-flight gating, extracts effective ballistic photons and blocks multiple scattering backgrounds, simultaneously reconstructs wavefront distortion to generate two control signals. The first signal drives the superstructure surface to perform forward in-situ phase correction, and the second signal is fed back to the illumination module in reverse to avoid high scattering paths. Based on the asynchronous hardware closed-loop mechanism of the integrated sensing and calculation, the application eliminates data bus delay and realizes nondestructive and high-resolution three-dimensional optical imaging of thick tissue samples.

Description

Technical Field

[0001] This invention relates to the field of optical imaging and non-destructive testing technology, specifically to an optical imaging system and imaging method for unsectioned tissue samples. Background Technology

[0002] In fields such as biomedical diagnostics, non-destructive testing of advanced food quality, and life science research, high-resolution three-dimensional internal structural imaging of tissue samples has extremely high application value. Traditional microscopic imaging methods typically rely on cumbersome fixation, embedding, and physical slicing of tissue samples. This is not only time-consuming and labor-intensive, but also severely damages the original state of the sample and makes it difficult to reflect the true physiological morphology of the tissue in three-dimensional space. To achieve non-destructive evaluation, direct optical imaging of unsliced ​​thick tissue samples has gradually become an important development direction in this field. However, when a light beam penetrates a thick unsliced ​​tissue sample, due to the extremely complex internal composition and strong spatial inhomogeneity of the refractive index distribution, photons undergo severe multiple scattering during transmission, leading to wavefront degradation. This high-intensity multiple scattering causes the ballistic photons that truly carry the effective spatial structural information of the sample to be completely overwhelmed by a massive number of disordered background scattered photons, resulting in a sharp decrease in the imaging contrast and spatial resolution of deep tissues, making it difficult to obtain clear three-dimensional slice images.

[0003] Existing technologies typically employ traditional adaptive optics wavefront correction or computational imaging methods. However, in practical engineering applications, traditional image sensors generally rely on frame accumulation mechanisms based on globally synchronized exposure, making it difficult for the system to accurately identify and eliminate scattered photons with minute nanosecond delays at the physical hardware level. Furthermore, existing wavefront correction systems generally adopt a serial architecture with separation of sensing, transmission, computation, and control. The large amount of raw wavefront data captured by the sensor must be transmitted across devices to an external computer via a bandwidth-limited data bus. Subsequently, the host computer software performs long-cycle aberration calculations, and finally, compensation commands are sent to the wavefront modulation device. This lengthy software-level communication and calculation delay makes the system's feedback speed lag far behind the speed of photon transmission and environmental disturbances, making it difficult to achieve in-situ real-time closed-loop correction. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an optical imaging system and method for unsliced ​​tissue samples. It solves the problems that existing optical imaging of unsliced ​​thick tissues is easily affected by strong scattering background and wavefront distortion interference, and that the traditional sensor-control separation architecture has extremely high data bus latency, making it difficult to achieve in-situ real-time hardware-level closed-loop correction and parallel demultiplexing of multi-dimensional physical properties.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The first aspect of this invention provides an optical imaging method for unsectioned tissue samples, comprising the following steps: A pulsed laser beam is projected onto the unsliced ​​tissue sample at the initial excitation moment using a dynamic adaptive illumination module; The intelligent sensing system receives a mixed light beam emitted from the unsliced ​​tissue sample, which includes a dynamically tunable multidimensional metasurface, a neuromorphic image sensor with a pixel array, and an on-chip edge computing unit; the mixed light beam is spatially demultiplexed by the physical properties of the dynamically tunable multidimensional metasurface and then incident on the pixel array. The on-chip edge computing unit performs time-of-flight gating on the captured photon events, extracts valid ballistic photon events, and blocks the output of electrical signals triggered by multiple scattered photons. Based on the spatial distribution information of the effective ballistic photon events, the wavefront distortion phase is reconstructed and the compensation phase is calculated. Two control signals are generated synchronously to implement asynchronous hardware closed-loop feedback. The first control signal drives the dynamically tunable multidimensional metasurface to superimpose the compensation phase, and the second control signal is fed back to the dynamic adaptive illumination module to change the spatial distribution of the excitation beam, thereby avoiding high scattering paths. Based on the multi-channel data stream continuously output during the closed-loop feedback process, which is aggregated from the effective ballistic photon events, image reconstruction is performed to obtain a three-dimensional optical slice image.

[0006] Furthermore, the spatial demultiplexing of the hybrid beam is specifically achieved by the dynamically tunable multidimensional metasurface generating a selective phase response to the wavelength and polarization state of the incident photons. For the metasurface unit located in two-dimensional spatial coordinates, the spatial phase gradient imparted by the metasurface to the incident light field deflects and focuses incident photons with different wavelengths and polarization states into different sub-pixel clusters pre-divided on the pixel array, thereby completing the parallel demultiplexing of multidimensional physical properties.

[0007] Furthermore, in the process of extracting valid ballistic photon events, the time-of-flight gating determination is achieved through the following conditional formula: ; in: This represents the absolute value of the deviation between the actual flight time and the theoretical minimum flight time. The photon detection time recorded for each pixel in the pixel array; This is the initial excitation time of the pulsed laser beam; A global time gating threshold parameter is used, which is calibrated based on historical experience data and system electronic jitter characteristics, and is used to define the tolerance window for ballistic photons.

[0008] Theoretically, this represents the shortest flight time for ballistic photons to traverse the unsliced ​​tissue sample. It is obtained by the following integral formula: ; in: The tissue depth through which photons traverse; It is the speed of light in a vacuum. For unsliced ​​tissue samples along the transport path, at a distance The preset equivalent refractive index distribution value is obtained by fitting historical prior data of biological slices of different biological tissues.

[0009] When the above conditions are met, the corresponding photon is determined to be a valid ballistic photon carrying effective spatial structure information. Its spatial coordinates and spatial intensity information are extracted as spatial distribution information and recorded as a valid ballistic photon event. If the conditions are not met, it is determined to be a multiple scattering photon and its electrical signal output is blocked in real time at the pixel hardware layer.

[0010] Furthermore, when generating the compensated phase, a preset phase retrieval algorithm is first used to reconstruct the wavefront distortion phase caused by the non-uniform refractive index within the sample in real time. Subsequently, based on the principle of optical phase conjugation, the compensation required to achieve aberration-free imaging was calculated. The calculation formula is: ; in: The preset target wavefront phase is obtained by calibrating the system in a sample-free state. This parameter is measured and obtained during the system initialization and empty background calibration.

[0011] Furthermore, regarding the implementation of asynchronous hardware closed-loop feedback, the generation of the first control signal includes: converting the compensation phase into a driving voltage matrix and applying it to the dynamically tunable multidimensional metasurface according to a preset electro-optic mapping function, so as to change the refractive index or resonant state of the corresponding unit in real time to achieve in-situ phase correction; the generation of the second control signal includes: generating a dynamic structured light mask function based on a spatial region where the effective ballistic photon event intensity is lower than a set ballistic photon intensity threshold, instructing the spatial light modulation array in the illumination module to block the excitation light projection of the high scattering region in the next pulse cycle. The on-chip edge computing unit is embedded in-situ within the hardware layer of the neuromorphic image sensor to achieve the on-chip asynchronous execution of the above-mentioned operations and feedback.

[0012] A second aspect of the present invention provides an optical imaging system for unsectioned tissue samples, the system comprising: The dynamic adaptive illumination module is used to project a pulsed laser beam onto the unsliced ​​tissue sample at the initial excitation moment, and to change the spatial distribution of the excitation beam according to the feedback second control signal to avoid high scattering paths. A smart sensing system for receiving a mixed light beam emitted from the unsliced ​​tissue sample, comprising: A dynamically tunable multidimensional metasurface is disposed on the outer layer of the intelligent sensing system. It is used to spatially demultiplex the physical properties of the mixed beam and to achieve in-situ phase correction by controlling the superposition of compensated phase according to the feedback first control signal. A neuromorphic image sensor, disposed below the dynamically tunable multidimensional metasurface, includes a pixel array for receiving demultiplexed photons and capturing photon events using an asynchronous event triggering mechanism; An on-chip edge computing unit, integrated with the neuromorphic image sensor, is used to perform time-of-flight gating on captured photon events to extract valid ballistic photon events, and to reconstruct the wavefront distortion phase and calculate the compensation phase based on the spatial distribution information of the valid ballistic photon events. It synchronously generates a first control signal and a second control signal to implement asynchronous hardware closed-loop feedback. The first control signal is used to drive the dynamically tunable multidimensional metasurface to superimpose the compensation phase, and the second control signal is used to feed back to the dynamic adaptive illumination module to change the spatial distribution of the excitation beam to avoid high scattering paths.

[0013] This invention provides an optical imaging system and method for unsectioned tissue samples. It offers the following advantages: 1. In this invention, the system connects the image sensor and the on-chip edge computing unit through three-dimensional physical bonding. This structure eliminates the time-consuming transmission of wavefront data across the device bus. It utilizes the photon event triggering mechanism to calculate and generate control signals at the chip level, establishing a hardware-level closed-loop mechanism and reducing feedback response time.

[0014] 2. In this invention, the on-chip computing unit uses the photon detection time and excitation time to obtain the flight time, and performs judgment in combination with the threshold parameter. This mechanism extracts ballistic photon events at the pixel bottom layer and blocks the output of scattered photon electrical signals that exceed the threshold, thus removing the background scattering interference caused by the tissue sample.

[0015] 3. In this invention, the computing unit reconstructs the wavefront distortion phase based on the extracted ballistic photon distribution information and generates a first control signal. This signal acts on the multidimensional metasurface, superimposing the compensation phase onto the incident light field and performing forward in-situ phase correction.

[0016] 4. In this invention, the computing unit compares the ballistic photon intensity with the threshold to generate a structured light mask function, which is then fed back to the spatial light modulation array of the illumination module as a second control signal. This array blocks the projection of excitation light in a specific area according to the mask state, and guides the light beam to penetrate the tissue sample along a low scattering path. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall architecture of the optical imaging system of the present invention; Figure 2 This is a schematic flowchart of the optical imaging method of the present invention; Figure 3 This is a schematic diagram of the three-dimensional physical stacking structure and the principle of multi-dimensional attribute demultiplexing of the present invention; Figure 4 This is a functional logic block diagram of the asynchronous hardware closed-loop feedback control mechanism of the present invention. Detailed Implementation

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see the appendix Figure 1 To be continued Figure 4 This invention provides an optical imaging system and method for unsliced ​​tissue samples, which enables high-resolution three-dimensional optical imaging of unsliced ​​tissue samples to be tested.

[0020] The optical imaging system for the unsliced ​​tissue sample includes a dynamic adaptive illumination module and an intelligent sensing system.

[0021] The dynamic adaptive illumination module internally includes a pulsed laser source and a spatial light modulation array. The pulsed laser source emits short-pulse laser light. To meet the requirements of extremely low latency hardware closed-loop control, the spatial light modulation array employs a high-speed digital micromirror device based on a microelectromechanical system (MEMS) or a high-speed ferroelectric liquid crystal spatial light modulator to support microsecond-level ultra-high frame rate modulation. The spatial light modulation array receives the short-pulse laser light and performs spatial energy distribution shaping. The shaped pulsed laser beam is then incident on the unsliced ​​tissue sample.

[0022] The dynamic adaptive illumination module receives a second control signal from the intelligent sensing system via a high-speed inter-chip differential signal bus (such as an LVDS interface). The spatial light modulation array responds in microseconds according to the second control signal, changing the spatial distribution of the excitation beam to avoid high scattering paths inside the unsliced ​​tissue sample.

[0023] The intelligent sensing system receives a mixed light beam emitted from an unsliced ​​tissue sample. The intelligent sensing system adopts a three-dimensional physical stacked structure design, and from top to bottom, it includes a dynamically tunable multi-dimensional metasurface, a neuromorphic image sensor, and an on-chip edge computing unit.

[0024] A dynamically tunable multidimensional metasurface is placed on the outermost layer of the intelligent sensing system. The mixed beam is first incident on the dynamically tunable multidimensional metasurface, which performs spatial demultiplexing of the physical properties of the mixed beam. The dynamically tunable multidimensional metasurface also receives a first control signal from the underlying hardware. The dynamically tunable multidimensional metasurface performs in-situ phase correction by superimposing and compensating the phase according to the first control signal.

[0025] The neuromorphic image sensor is located beneath a dynamically tunable multidimensional metasurface. Inside the neuromorphic image sensor is a pixel array that receives photons after spatial demultiplexing. The neuromorphic image sensor operates according to an asynchronous event triggering mechanism. Based on an address event representation protocol, when an independent pixel in the pixel array detects the arrival of a photon and triggers avalanche breakdown, it skips the traditional frame accumulation and global readout process. Instead, the logic gates within the pixel instantly generate a pulse signal containing its physical coordinate address, independently and asynchronously recording the photon detection information.

[0026] The on-chip edge computing unit is integrated with the neuromorphic image sensor. In a three-dimensional physical stack architecture, the neuromorphic image sensor is located on the top photosensitive substrate, and the on-chip edge computing unit is located on the bottom logic substrate. The two substrates are interconnected at the pixel level through through-silicon vias (TSVs) and copper-copper hybrid bonding processes to achieve pixel-level face-to-face high-density vertical electrical interconnection. Based on this physical connection structure, the on-chip edge computing unit is embedded in situ within the hardware layer inside the neuromorphic image sensor, and receives photon events captured by the pixel array in real time through the aforementioned vertical interconnect channels in a bus-delay-free manner.

[0027] The on-chip edge computing unit performs low-level hardware-level calculations on the captured photon events. The on-chip edge computing unit first performs time-of-flight gating to extract valid ballistic photon events.

[0028] In the system's hardware-level timing synchronization and event triggering mechanism, photon detection satisfies the following basic timing relationship formula: ; in: The moment when the photon event is triggered is recorded in real time by the pixel-level high-frequency hardware clock inside the neuromorphic image sensor when the photon is captured. The initial excitation moment is obtained by the pulse drive circuit of the dynamic adaptive lighting module, which is synchronously output to the intelligent sensing system. This is the actual physical flight time of the photon in the unsliced ​​tissue sample, which is used for subsequent gating decisions.

[0029] After extracting valid ballistic photon events, the on-chip edge computing unit obtains the spatial distribution information of these events. Based on this spatial distribution information, the on-chip edge computing unit reconstructs the wavefront distortion phase and calculates the corresponding compensation phase.

[0030] The on-chip edge computing unit synchronously generates a first control signal and a second control signal. The first control signal drives the dynamic tunable multidimensional metasurface to superimpose and compensate the phase, while the second control signal is fed back to the dynamic adaptive illumination module to change the spatial distribution of the excitation beam. The two signals constitute the asynchronous hardware closed-loop feedback of the system.

[0031] The optical imaging method for the unsliced ​​tissue sample may include the following steps: S1: A pulsed laser beam is projected onto the unsliced ​​tissue sample at the initial excitation moment via a dynamic adaptive illumination module.

[0032] S2: The intelligent sensing system receives a mixed beam emitted from an unsliced ​​tissue sample, which includes a dynamically tunable multidimensional metasurface, a neuromorphic image sensor with a pixel array, and an on-chip edge computing unit; the mixed beam is spatially demultiplexed by the dynamically tunable multidimensional metasurface to obtain the physical properties, and then incident on the pixel array.

[0033] S3: The on-chip edge computing unit performs time-of-flight gating on the captured photon events, extracts valid ballistic photon events, and blocks the output of electrical signals triggered by multiple scattered photons.

[0034] S4: Based on the spatial distribution information of effective ballistic photon events, reconstruct the wavefront distortion phase and calculate the compensation phase.

[0035] S5: Synchronously generate two control signals to implement asynchronous hardware closed-loop feedback. The first control signal drives the dynamic tunable multidimensional metasurface to superimpose and compensate the phase. The second control signal is fed back to the dynamic adaptive illumination module to change the spatial distribution of the excitation beam, thereby avoiding high scattering paths.

[0036] S6: Based on the multi-channel data stream continuously output during the closed-loop feedback process, which is aggregated from effective ballistic photon events, image reconstruction is performed to obtain a three-dimensional optical slice image. In this embodiment, the mixed beam contains photons emitted from the unsliced ​​tissue sample, which carry multiple wavelength components and multiple polarization state components. The intelligent sensing system processes the mixed beam to perform spatial demultiplexing of multidimensional physical properties.

[0037] In step S2, the hybrid beam passes through the dynamically tunable multidimensional metasurface to spatially demultiplex its physical properties.

[0038] Dynamically tunable multidimensional metasurfaces consist of metasurface units arranged in a two-dimensional array. These metasurfaces exhibit selective phase responses to the wavelength and polarization state of incident photons.

[0039] The aforementioned selective phase response is achieved by applying total modulation phase to the incident photon through the metasurface unit. (Located in two-dimensional spatial coordinates) Dynamically tunable multidimensional metasurface units at specific wavelengths and specific polarization state The total modulation phase generated by the incident photon satisfies the following formula: ; in: The total modulation phase applied to the metasurface unit is obtained by numerically solving the metasurface unit model using full-wave electromagnetic simulation software. In this embodiment, the metasurface unit adopts an anisotropic cuboid nanopillar structure composed of a high refractive index dielectric (such as amorphous silicon or titanium dioxide). The geometric phase is obtained by setting the in-plane rotation angle of the major axis of the cuboid nanopillar structure relative to the base coordinate system; To transmit the phase, this parameter is obtained by setting the geometric dimensions of the cuboid nanopillar structure, such as its length, width, and height.

[0040] The dynamically tunable multidimensional metasurface unit uses the above-mentioned total modulation phase to impart a spatial phase gradient to the incident light field. This spatial phase gradient causes incident photons of different wavelengths and polarization states to produce optical path deflection at different angles.

[0041] To achieve spatial separation, the pixel array inside the neuromorphic image sensor is spatially divided into multiple different sub-pixel clusters. In a specific embodiment, adjacent 2×2 or 4×4 physical pixels can be defined as a macro-pixel unit. Within this macro-pixel unit, sub-pixels with fixed coordinates are reserved to receive combinations of different wavelengths (such as 400nm, 500nm, 600nm, etc.) and different polarization states (such as 0 degrees, 45 degrees, 90 degrees, 135 degrees linear polarization). Different wavelength and polarization state combinations form a one-to-one mapping relationship with different sub-pixel clusters to achieve multi-channel synchronous demultiplexing at the physical level.

[0042] Dynamically tunable multidimensional metasurface units impart a spatial phase gradient to the incident light field that satisfies a spatial focusing objective function. This spatial focusing objective function guides the directional propagation of deflected photons. The spatial focusing objective function is constructed using the following formula: ; in: To target wavelength and target polarization state The target spatial phase gradient required for the incident photon is calculated using the above formula; To design the focal length, this parameter is set based on the interlayer vertical physical distance between the dynamically tunable multidimensional metasurface and the neuromorphic image sensor; For pre-divided target wavelengths for receiving and target polarization state The corresponding physical coordinates of the center of the sub-pixel cluster are obtained from the layout mask design file of the pixel array.

[0043] For a dynamically tunable multidimensional metasurface unit located in two-dimensional spatial coordinates, the spatial phase gradient imparted to the incident light field will deflect incident photons with different wavelengths and polarization states.

[0044] The incident photons, after being deflected, are separated in space. The separated incident photons are then focused into different sub-pixel clusters pre-divided on the pixel array according to their respective wavelengths and polarization states. The intelligent sensing system uses this physical deflection and directional focusing process to complete the parallel demultiplexing of multidimensional physical properties.

[0045] In this embodiment, the on-chip edge computing unit performs time-of-flight gating on the photon events captured by the pixel array, extracts valid ballistic photon events, and simultaneously blocks the output of electrical signals triggered by multiple scattered photons.

[0046] The neuromorphic image sensor uses an asynchronous event triggering mechanism to capture demultiplexed photons. The pixel array inside the neuromorphic image sensor generates an electrical pulse when the photon arrives. The on-chip edge computing unit records the initial excitation time of the pulsed laser beam and the photon detection time corresponding to the electrical pulse generated by each pixel in the pixel array.

[0047] The on-chip edge computing unit calculates the actual flight time of the photon, which is the time difference between the photon detection time and the initial excitation time.

[0048] The on-chip edge computing unit obtains the theoretically shortest flight time for ballistic photons to traverse an unsliced ​​tissue sample. This shortest flight time is calculated using the following integral formula: ; in: This represents the theoretically shortest flight time for a ballistic photon to traverse an unsliced ​​tissue sample. The depth of the unsliced ​​tissue sample along the transmission path is obtained based on the physical coordinate system of the imaging platform's axial scanning displacement stage. The speed of light in vacuum is constant. According to Fermat's principle and the propagation characteristics of light in inhomogeneous media, the shortest geometric path for a ballistic photon is the straight line between two points penetrating the sample. Any deflection and multiple scattering of the photon within the unsliced ​​tissue sample will increase its actual propagation path length; therefore, the actual flight time of the photon (…) Physically, it must be strictly greater than or equal to the shortest flight time. The introduction of absolute value calculation into the time-of-flight gating formula aims to be compatible with and tolerate the unavoidable synchronization sampling jitter between the underlying photodetector and the hardware clock, which occurs on the order of picoseconds.

[0049] These are the integration variable parameters along the transmission path direction; For unsliced ​​tissue samples along the transport path, at a distance The preset equivalent refractive index distribution value is obtained by fitting historical experience data from optical section mapping of similar biological tissue samples.

[0050] The on-chip edge computing unit acquires a preset global time gating threshold parameter, which is used to define the ballistic photon tolerance window. The global time gating threshold parameter is set based on the system's electronic clock jitter deviation data and empirical data on the mean scattering free path of unsliced ​​tissue samples. In this embodiment, the specific setting range of the global time gating threshold parameter is 10 picoseconds to 50 picoseconds to ensure that multiple scattered photons with nanosecond-level delays after multiple scattering can be effectively filtered out, thereby extracting pure ballistic photons from the underlying hardware level.

[0051] The on-chip edge computing unit determines whether the absolute value of the difference between the actual flight time and the shortest flight time is less than or equal to the global time gating threshold parameter. The flight time gating condition satisfies the following determination formula: ; in: This is the moment of photon detection; This is the initial excitation time; The shortest flight time is calculated based on the aforementioned integral formula; The preset global time gating threshold parameter.

[0052] If the determination conditions of the above formula are met, the on-chip edge computing unit determines that the corresponding photon is a valid ballistic photon carrying valid spatial structure information. The on-chip edge computing unit extracts the spatial coordinates and spatial intensity information of the valid ballistic photon as spatial distribution information. The on-chip edge computing unit records the spatial distribution information as a valid ballistic photon event and writes the event data into the underlying hardware cache.

[0053] If the determination conditions of the above formula are not met, the on-chip edge computing unit determines that the corresponding photon is a multiple scattered photon, and the on-chip edge computing unit immediately blocks the output of the electrical signal corresponding to the multiple scattered photon at the pixel hardware layer.

[0054] In this embodiment, the on-chip edge computing unit performs the wavefront distortion phase reconstruction step and the compensation phase calculation step. The on-chip edge computing unit reads the valid ballistic photon events extracted in the aforementioned steps from the underlying hardware cache. The valid ballistic photon events include spatial coordinate information and corresponding spatial intensity information.

[0055] There are differences in refractive index within the unsliced ​​tissue sample. When photons pass through the unsliced ​​tissue sample, wavefront phase degradation occurs. The on-chip edge computing unit reconstructs the wavefront distortion phase based on the spatial intensity information contained in the effective ballistic photon event. The on-chip edge computing unit calculates the wavefront distortion phase using a preset phase recovery algorithm.

[0056] The reconstruction process of wavefront distortion phase satisfies the following algorithm mapping formula: ; in: The spatial coordinate parameters of the photon event are extracted based on the physical location addressing matrix of the pixel array. coordinates The reconstructed wavefront distortion phase; For effective ballistic photon events in coordinates The spatial intensity information is obtained based on the electrical signal pulse count or charge accumulation quantization value output by the pixel array. The preset phase retrieval algorithm operator is obtained based on the transmission intensity equation solver pre-programmed into the firmware layer of the on-chip edge computing unit. Specifically, the on-chip edge computing unit calculates the spatial first and second finite difference derivatives of the intensity between adjacent pixels through a hardware accelerator, and introduces preset von Neumann boundary conditions to directly perform the discretization solution of partial differential equations and the physical process inversion at the hardware level.

[0057] The on-chip edge computing unit acquires the preset target wavefront phase in the system's sample-free state. During the pre-scanning stage, the system removes unsliced ​​tissue samples, emits a calibration beam, and records the wavefront data received by the sensor. The system writes this wavefront data into a non-volatile memory as the aforementioned preset target wavefront phase.

[0058] The on-chip edge computing unit calculates the compensation phase based on the optical phase conjugation principle. The on-chip edge computing unit subtracts the wavefront distortion phase from the preset target wavefront phase to obtain the inverse compensation amount that the system needs to apply.

[0059] The calculation of the compensation phase satisfies the following mathematical algebraic formula: ; in: coordinates The compensation phase at that point is used to subsequently drive the outer metasurface; coordinates The preset target wavefront phase is obtained by measuring the aforementioned system initialization empty background calibration procedure. The wavefront distortion phase is obtained by reconstructing based on the aforementioned mapping formula.

[0060] The on-chip edge computing unit completes the calculation of the compensation phase at all spatial coordinate points, and the on-chip edge computing unit latches the generated compensation phase matrix into the output register.

[0061] In this embodiment, the on-chip edge computing unit synchronously generates a first control signal and a second control signal based on the calculated compensation phase and the spatial distribution information of effective ballistic photon events. The on-chip edge computing unit uses the two control signals to implement asynchronous hardware closed-loop feedback.

[0062] The on-chip edge computing unit obtains a preset electro-optic mapping function. The on-chip edge computing unit uses this electro-optic mapping function to convert the compensation phase into a driving voltage matrix. The conversion of this driving voltage matrix satisfies the following mapping relationship formula: ; in: The calculated application to the coordinates The driving voltage value of the dynamically tunable multidimensional metasurface unit; The compensation phase calculated in the preceding steps; The pre-defined electro-optic mapping function operator is obtained through system-level experimental calibration by applying a stepped voltage to the physical material of a dynamically tunable multidimensional metasurface and measuring the rate of change of phase response.

[0063] The on-chip edge computing unit outputs a driving voltage matrix as the first control signal, which is then applied to the dynamically tunable multidimensional metasurface on the outer layer of the intelligent sensing system. Specifically, the dynamically tunable multidimensional metasurface is constructed using an electro-optic tunable material, comprising an anisotropic nematic liquid crystal layer or a lithium niobate thin film layer embedded between two layers of indium tin oxide (ITO) transparent conductive electrodes. Each metasurface unit forms an independently addressed capacitor structure with the ITO electrodes via an underlying active matrix substrate. Responding to the driving voltage matrix applied between the electrodes, the dynamically tunable multidimensional metasurface unit induces molecular orientation deflection or the Pockels effect through an electrostatic field, altering the refractive index or resonant state of its structural material. The dynamically tunable multidimensional metasurface physically superimposes the corresponding compensation phase onto the incident light field, thereby achieving in-situ phase correction.

[0064] The on-chip edge computing unit extracts photon intensity information of effective ballistic photon events in each spatial coordinate region. The on-chip edge computing unit obtains the set ballistic photon intensity threshold, compares the photon intensity of each spatial region with the ballistic photon intensity threshold, and determines the spatial region where the intensity of effective ballistic photon events is lower than the threshold. This low-intensity spatial region corresponds to the high scattering path inside the unsliced ​​tissue sample.

[0065] The on-chip edge computing unit generates a dynamic structured light mask function based on the above comparison results. The generation logic of this structured light mask function satisfies the following condition judgment formula: ; in: For dynamic structured light mask functions in coordinates The state value at that location; For effective ballistic photon events in coordinates Spatial intensity information at the location; The set ballistic photon intensity threshold satisfies the following mathematical relationship: ,in, The cumulative value of the average dark current background noise of the intelligent sensing system is measured when the excitation beam is blocked; The average multiple scattering background noise intensity of the current unsliced ​​tissue sample is obtained through multiple random sampling calibrations. This is the signal-to-noise ratio tolerance coefficient, ranging from 2.0 to 3.5. This ensures the accurate setting of the Gaussian confidence interval for the dark current background noise level of the intelligent sensing system and the sample background signal-to-noise ratio measurement data.

[0066] The on-chip edge computing unit generates a second control signal containing the aforementioned dynamic structured light mask function, and feeds the second control signal back to the dynamic adaptive illumination module. The spatial light modulation array inside the dynamic adaptive illumination module receives the second control signal.

[0067] Based on the second control signal, the spatial light modulation array blocks the projection of excitation light into the spatial region where the mask state value is zero in the next pulse cycle. The spatial light modulation array changes the spatial distribution of the excitation beam, guiding the incident energy of the excitation beam to penetrate through the low scattering path where the mask state value is one.

[0068] The on-chip edge computing unit is embedded in the hardware layer inside the neuromorphic image sensor. It directly schedules the above-mentioned judgment and signal generation operations at the bottom layer of the image sensor. The on-chip edge computing unit synchronously outputs the first control signal for forward wavefront correction and the second control signal for reverse illumination avoidance.

[0069] In this embodiment, during the execution of the above-mentioned asynchronous hardware closed-loop feedback, the system continuously collects multi-channel data streams. The multi-channel data streams originate from the pixel array inside the neuromorphic image sensor. The pixel array contains different sub-pixel clusters that are pre-divided. Each sub-pixel cluster continuously outputs event data streams. The data streams output by different sub-pixel clusters correspond to different wavelength parameters and polarization state parameters.

[0070] The on-chip edge computing unit aggregates effective ballistic photon events, generates the aforementioned multi-channel data stream, and transmits the multi-channel data stream to the system's image processing unit. The image processing unit receives the multi-channel data stream and performs spatial coordinate alignment operations on the multi-channel data stream.

[0071] The aforementioned dynamically tunable multidimensional metasurface spatially deflects photons of different wavelengths and polarization states during the spatial demultiplexing stage. This results in physical positional offsets in the multi-channel data streams across the pixel array. The image processing unit performs inverse coordinate compensation based on preset offset matrix parameters. The spatial coordinate alignment process for the multi-channel data satisfies the following coordinate mapping formula: ; in: The values ​​of the aligned 3D voxels; This represents the cumulative original intensity value output for the corresponding sub-pixel cluster. and For the corresponding target wavelength and target polarization state The physical space offset is obtained by extracting the design parameters of the spatial focusing objective function of the aforementioned metasurface. The depth coordinates of the unsliced ​​tissue sample are obtained based on the displacement stage tomography calibration parameters during the system scanning process.

[0072] The image processing unit completes the spatial coordinate alignment of all channel data, performs multi-channel data fusion rendering, and reconstructs the three-dimensional structural feature data inside the sample using the aligned three-dimensional spatial voxel intensity values.

[0073] The image processing unit generates a three-dimensional optical slice image, which presents the spatial morphology inside the unsliced ​​tissue sample. The three-dimensional optical slice image contains optical response characteristic data of the sample under multiple wavelengths and multiple polarization states. The system outputs the three-dimensional optical slice image to a display terminal or storage device.

[0074] This invention utilizes an on-chip edge computing unit embedded in the hardware layer to schedule photon capture events. The system performs time-of-flight gating and wavefront distortion phase reconstruction operations at the sensing layer. The system blocks the electrical signal output of multiple scattered photons, eliminating the need for bus transmission of invalid background data. The on-chip edge computing unit synchronously outputs a first control signal and a second control signal. The first control signal drives a dynamically tunable multidimensional metasurface to perform forward physical phase correction, while the second control signal controls a dynamic adaptive illumination module to perform inverse spatial excitation region avoidance. This integrated sensing and computing mechanism enables optical imaging of unsliced ​​tissue samples, acquiring a three-dimensional optical slice image containing multidimensional physical property information of the unsliced ​​tissue sample.

Claims

1. An optical imaging method for unsectioned tissue samples, characterized in that, include: S1: A pulsed laser beam is projected onto the unsliced ​​tissue sample at the initial excitation moment via a dynamic adaptive illumination module; S2: The intelligent sensing system receives a mixed light beam emitted from the unsliced ​​tissue sample, which includes a dynamically tunable multidimensional metasurface, a neuromorphic image sensor with a pixel array, and an on-chip edge computing unit; the mixed light beam is incident on the pixel array after the spatial demultiplexing of the physical properties of the dynamically tunable multidimensional metasurface. S3: The on-chip edge computing unit performs time-of-flight gating on the captured photon events, extracts valid ballistic photon events, and blocks the output of electrical signals triggered by multiple scattered photons; S4: Based on the spatial distribution information of the effective ballistic photon events, reconstruct the wavefront distortion phase and calculate the compensation phase; S5: Synchronously generate two control signals to implement asynchronous hardware closed-loop feedback. The first control signal drives the dynamically tunable multidimensional metasurface to superimpose the compensation phase, and the second control signal feeds back to the dynamic adaptive illumination module to change the spatial distribution of the excitation beam, thereby avoiding high scattering paths. S6: Based on the multi-channel data stream continuously output during the closed-loop feedback process, which is aggregated from the effective ballistic photon events, image reconstruction is performed to obtain a three-dimensional optical slice image.

2. The method according to claim 1, characterized in that, In step S2, the hybrid beam undergoes spatial demultiplexing of its physical properties through the dynamically tunable multidimensional metasurface, specifically including: The dynamically tunable multidimensional metasurface exhibits a selective phase response to the wavelength and polarization state of the incident photon; For the dynamically tunable multidimensional metasurface unit located in two-dimensional spatial coordinates, the spatial phase gradient imparted to the incident light field will deflect and focus incident photons with different wavelengths and polarization states into different sub-pixel clusters pre-divided on the pixel array, thus completing the parallel demultiplexing of multidimensional physical properties.

3. The method according to claim 1, characterized in that, In step S3, the time-of-flight gating determination specifically includes: Record the initial excitation time of the pulsed laser beam and the photon detection time recorded by each pixel in the pixel array, and calculate the actual flight time of the photon; Obtain the preset global time gating threshold parameter used to define the ballistic photon tolerance window; Determine whether the absolute value of the difference between the actual flight time and the theoretically shortest flight time for ballistic photons to traverse the unsliced ​​tissue sample is less than or equal to the global time gating threshold parameter; If the conditions are met, the corresponding photon is determined to be a valid ballistic photon carrying effective spatial structure information. Its spatial coordinates and spatial intensity information are extracted as the spatial distribution information and recorded as the valid ballistic photon event. If the condition is not met, it is determined to be a multiple scattered photon, and its electrical signal output is blocked immediately at the pixel hardware layer.

4. The method according to claim 3, characterized in that, Theoretically, the shortest flight time for a ballistic photon to traverse the unsliced ​​tissue sample is calculated by integrating the preset equivalent refractive index distribution value of the unsliced ​​tissue sample along the transmission path direction by the speed of light in vacuum.

5. The method according to claim 1, characterized in that, In step S4, the reconstruction of the wavefront distortion phase and the calculation of the compensation phase specifically include: Based on the spatial intensity information contained in the spatial distribution information of the effective ballistic photon events, the wavefront distortion phase caused by the non-uniform refractive index inside the unsliced ​​tissue sample is reconstructed in real time using a preset phase recovery algorithm. Based on the principle of optical phase conjugation, the compensation phase required to achieve aberration-free imaging is calculated by subtracting the wavefront distortion phase from the preset target wavefront phase obtained by the system calibration in the sample-free state.

6. The method according to claim 5, characterized in that, In step S5, the first control signal drives the dynamically tunable multidimensional metasurface to superimpose the compensation phase, specifically including: According to the preset electro-optic mapping function, the compensation phase is converted into a driving voltage matrix; The driving voltage matrix is ​​applied to the dynamically tunable multidimensional metasurface as the first control signal to change the refractive index or resonance state of the corresponding dynamically tunable multidimensional metasurface unit in real time, thereby physically superimposing the compensation phase onto the subsequent incident light field to achieve in-situ phase correction.

7. The method according to claim 1, characterized in that, In step S5, the second control signal is fed back to the dynamic adaptive illumination module to change the spatial distribution of the excitation beam, thereby avoiding high scattering paths, specifically including: Based on the spatial distribution information of the effective ballistic photon events, a spatial region where the intensity of the effective ballistic photon events is lower than a set ballistic photon intensity threshold is determined. Based on the ballistic photon intensity threshold, a dynamic structured light mask function is generated, and a second control signal is generated to instruct the spatial light modulation array in the dynamic adaptive illumination module to block the excitation light projection in the spatial region in the next pulse cycle, guiding the incident energy of the excitation beam to penetrate through the low scattering path.

8. The method according to claim 2, characterized in that, In step S6, the image reconstruction based on the multi-channel data stream continuously output during the closed-loop feedback process and aggregated from the effective ballistic photon events specifically includes: During the execution of the asynchronous hardware closed-loop feedback, the multi-channel data streams from the different sub-pixel clusters are collected, wherein the data streams output by the different sub-pixel clusters correspond to different wavelengths and polarization states respectively; The system receives the multi-channel data stream and performs spatial coordinate alignment and fusion rendering of the multi-channel data to generate a three-dimensional optical slice image that reflects the three-dimensional spatial morphology of the unsliced ​​tissue sample.

9. The method according to claim 1, characterized in that, The on-chip edge computing unit is embedded in the hardware layer inside the neuromorphic image sensor to realize the on-chip asynchronous execution of the time-of-flight gating determination, the wavefront distortion phase reconstruction, and the generation of the two control signals.

10. An optical imaging system for an unsliced ​​tissue sample, used to implement the optical imaging method as described in any one of claims 1 to 9, characterized in that, The system includes: The dynamic adaptive illumination module is used to project a pulsed laser beam onto the unsliced ​​tissue sample at the initial excitation moment, and to change the spatial distribution of the excitation beam according to the feedback second control signal to avoid high scattering paths. A smart sensing system for receiving a mixed light beam emitted from the unsliced ​​tissue sample, comprising: A dynamically tunable multidimensional metasurface is disposed on the outer layer of the intelligent sensing system. It is used to spatially demultiplex the physical properties of the mixed beam and to achieve in-situ phase correction by controlling the superposition of compensated phase according to the feedback first control signal. A neuromorphic image sensor, disposed below the dynamically tunable multidimensional metasurface, includes a pixel array for receiving demultiplexed photons and capturing photon events using an asynchronous event triggering mechanism; An on-chip edge computing unit, integrated with the neuromorphic image sensor, is used to perform time-of-flight gating on captured photon events to extract valid ballistic photon events, and to reconstruct the wavefront distortion phase and calculate the compensation phase based on the spatial distribution information of the valid ballistic photon events. It synchronously generates a first control signal and a second control signal to implement asynchronous hardware closed-loop feedback. The first control signal is used to drive the dynamically tunable multidimensional metasurface to superimpose the compensation phase, and the second control signal is used to feed back to the dynamic adaptive illumination module to change the spatial distribution of the excitation beam to avoid high scattering paths.

Images