Super-resolution fluorescence lifetime microscopy imaging method, apparatus, device, and program product

Abstract

The application is suitable for the field of fluorescence lifetime imaging technology, and provides an ultrahigh-resolution fluorescence lifetime microscopic imaging method, device, equipment and program product. The method comprises the following steps: obtaining N compressed sampling images through a digital micromirror device and a stripe camera, obtaining N reference images through a reference camera, performing reconstruction processing on each compressed sampling image to obtain a fluorescence decay image sequence, and performing statistical processing and fitting processing on the intensity decay distribution of the fluorescence molecules in the fluorescence decay image sequence to obtain the fluorescence lifetime of each fluorescence molecule in the compressed sampling image; performing peak detection processing on each reference image to obtain the center positions corresponding to each fluorescence molecule in the reference image; and generating an ultrahigh-resolution fluorescence lifetime image according to the fluorescence lifetime and the center position of each fluorescence molecule. The scheme can realize ultrahigh-resolution fluorescence lifetime microscopic imaging and improve the performance of fluorescence lifetime microscopic imaging.

Description

Technical Field

[0001] This application belongs to the field of fluorescence lifetime imaging technology, and in particular relates to a super-resolution fluorescence lifetime microscopy imaging method, device, equipment and program product. Background Technology

[0002] Fluorescence lifetime microscopy, a key tool in biomedical research, can reveal dynamic information about intracellular microenvironment characteristics such as pH, viscosity, and molecular interactions by measuring the time required for fluorescent molecule intensity to decay to 1 / e of its initial value. This technique is insensitive to fluorophore concentration and excitation intensity, providing a unique perspective for understanding the molecular dynamics of biological systems.

[0003] However, current mainstream technologies face significant bottlenecks. While traditional fluorescence lifetime microscopy based on time-correlated single-photon counting is widely used, its data acquisition process requires repeated excitation and sampling, resulting in excessively long imaging times and low photon acquisition efficiency. More critically, when the fluorescent sample is under non-single-photon excitation conditions, photon accumulation effects severely distort the fluorescence decay curve, causing the measured lifetime value to deviate from the true physical properties and leading to misjudgments of the molecular environment.

[0004] On the other hand, streak cameras, with their picosecond-level time resolution, offer a new approach to fluorescence lifetime measurement. A single exposure can completely capture the fluorescence decay process, fundamentally avoiding interference from photon accumulation effects and significantly improving data acquisition efficiency. However, existing streak camera systems are limited by their one-dimensional spatial resolution. Although they have achieved two-dimensional wide-field fluorescence lifetime imaging through compressed ultrafast imaging technology, they still lack effective integration with super-resolution imaging principles. While single-molecule localization super-resolution technology can break through the optical diffraction limit and improve spatial resolution to the nanoscale, its combination with fluorescence lifetime information still relies on the inefficient time-correlated single photon counting (TCSPC) architecture.

[0005] Currently, no complete solution exists that can simultaneously integrate the advantages of wide-field imaging with streak cameras, the efficient data acquisition characteristics of compressed sensing theory, and the single-molecule sparse excitation localization principle. This has led to a long-standing dual dilemma in the field of super-resolution fluorescence lifetime microscopy: insufficient spatial resolution and inaccurate lifetime measurements. This technological gap makes it difficult to resolve the dynamic fluorescence lifetime of subcellular fine structures, severely restricting the accurate observation of molecular interactions in living cells and ultimately reducing the performance of fluorescence lifetime microscopy. Summary of the Invention

[0006] In view of this, embodiments of this application provide a super-resolution fluorescence lifetime microscopy imaging method to solve the technical problem of low performance of existing fluorescence lifetime microscopy imaging techniques.

[0007] In a first aspect, embodiments of this application provide a super-resolution fluorescence lifetime microscopy imaging method, applied to a fluorescence lifetime microscopy imaging system. The fluorescence lifetime microscopy imaging system includes a fluorescent sample, a pulsed laser, a digital micromirror device, a streak camera, and a reference camera. The pulsed laser is used to emit laser light onto the fluorescent sample. In each time period, some fluorescent molecules in the fluorescent sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different in each time period. The digital micromirror device and the streak camera are used together to sequentially perform encoding processing, photoelectric conversion processing, deflection processing, and time integration processing based on the fluorescence emitted by the fluorescent sample in each time period to obtain N compressed sampling images. The reference camera is used to acquire N reference images based on the fluorescence emitted by the fluorescent sample in each time period, where N is greater than 1. The method includes: For each compressed sampled image, based on the reference image corresponding to the compressed sampled image, the compressed sampled image is reconstructed to obtain a sequence of several frames of fluorescence decay images corresponding to the compressed sampled image. The intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence is statistically processed and fitted to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampled image. For each of the reference images, peak detection processing is performed on the reference image to obtain the center position of each fluorescent molecule in the reference image. A super-resolution fluorescence lifetime image corresponding to the fluorescent sample is generated based on the fluorescence lifetime of each fluorescent molecule in each compressed sample image and the center position of each fluorescent molecule in each reference image; the super-resolution fluorescence lifetime image includes the fluorescence lifetime corresponding to each fluorescent molecule in the fluorescent sample.

[0008] Optionally, the digital micromirror device is specifically used for: Based on the fluorescence emitted by the fluorescent sample in each time period, a corresponding original image sequence for each time period is generated, and each original image sequence is encoded using an encoding matrix to obtain N encoded image sequences. The stripe camera includes a photocathode, a scanning electric field, a fluorescent screen, and a photosensitive chip. Specifically, the stripe camera is used for: With the slit in the stripe camera fully open, the N encoded image sequences in the form of light signals are acquired; The N encoded image sequences in optical signal form are converted into the N encoded image sequences in electrical signal form using a photocathode. The scanning electric field is used to deflect the N encoded image sequences in each electrical signal form, and the N deflected image sequences in electrical signal form are obtained through the fluorescent screen. The N compressed sampled images are obtained by time-integrating the N deflected image sequences using the photosensitive chip.

[0009] Optionally, the step of reconstructing the compressed sampled image based on the reference image corresponding to the compressed sampled image to obtain a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image includes: Determine a sensing matrix to describe the encoding effect of the digital micromirror device, the deflection effect of the scanning electric field, and the time integration effect of the photosensitive chip; Determine a first relational expression to describe the relationship between the sensing matrix, the compressed sampled image, and the original image sequence corresponding to the compressed sampled image; Based on the prior knowledge of the original image sequence obtained in advance, the first relation is transformed into a second relation; the second relation is used to describe the relationship between the local optimal solution of the fluorescence attenuation image sequence and the fidelity term, the first prior term, and the regularization coefficient. The fidelity term is determined based on the compressed sampled image, the sensing matrix, and the original image sequence. The first prior term is determined based on the prior knowledge. The regularization coefficient is used to adjust the weight between the fidelity term and the first prior term. Based on the reference image corresponding to the compressed sampled image, the local optimal solution of the second relation is solved, and the local optimal solution is converted into a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image.

[0010] Optionally, the step of solving the local optimal solution of the second relation based on the reference image corresponding to the compressed sampled image, and converting the local optimal solution into a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image, includes: Obtain auxiliary variables for denoising the compressed sampled image; Based on the auxiliary variables, the second relation is transformed into a third relation; the second relation is the relation corresponding to the unconstrained optimization problem, and the third relation is the relation corresponding to the constrained optimization problem. Based on the third relation, a first sub-optimization relation, a second sub-optimization relation, and a third sub-optimization relation are determined; the first sub-optimization relation is used to make the fluorescence attenuation image sequence as close as possible to the compressed sampled image, the second sub-optimization relation is used to perform image noise reduction processing, and the third sub-optimization relation is used to correct the error corresponding to the first sub-optimization relation and the error corresponding to the second sub-optimization relation. Based on the reference image corresponding to the compressed sampled image, the first sub-optimization relation, the second sub-optimization relation, and the third sub-optimization relation are iteratively operated on until the preset convergence condition is reached. Based on the local optimal solution of the second sub-optimization relation obtained at the end, a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image is determined.

[0011] Optionally, the second sub-optimization relation can be operated on in the following manner: The second sub-optimization relation is transformed into a fourth sub-optimization relation; the second sub-optimization relation includes a second prior term, which is determined based on the prior knowledge of the auxiliary variable; the fourth sub-optimization relation does not include any prior term. The fourth sub-optimization relation is calculated, and the preliminary calculation result is corrected by using the reference image corresponding to the compressed sampled image during the calculation process.

[0012] Optionally, the step of performing peak detection processing on the reference image to obtain the center position corresponding to each fluorescent molecule in the reference image includes: Multiple sub-reference images are obtained from the reference image by using a sliding window image segmentation method; For each sub-reference image, if the brightness of the center pixel of the sub-reference image is higher than the brightness of all the pixels surrounding the target in the sub-reference image, then the center pixel of the sub-reference image is determined as the center position of a fluorescent molecule in the reference image; the pixels surrounding the target are the pixels among the non-center pixels of the sub-reference image that have not been determined as the center position of the fluorescent molecule.

[0013] Optionally, generating a super-resolution fluorescence lifetime image corresponding to the fluorescent sample based on the fluorescence lifetime of each fluorescent molecule in each compressed sampled image and the center position of each fluorescent molecule in each reference image includes: For each fluorescent molecule, the fluorescent molecule is labeled at its center position according to its fluorescence lifetime. After labeling all fluorescent molecules, all fluorescent molecules are synthesized into the same image to obtain the super-resolution fluorescence lifetime image.

[0014] Secondly, embodiments of this application provide a super-resolution fluorescence lifetime microscopy imaging device, applied to a fluorescence lifetime microscopy imaging system. The fluorescence lifetime microscopy imaging system includes a fluorescent sample, a pulsed laser, a digital micromirror device, a streak camera, and a reference camera. The pulsed laser is used to emit laser light onto the fluorescent sample. In each time period, some fluorescent molecules in the fluorescent sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different in each time period. The digital micromirror device and the streak camera are used together to sequentially perform encoding processing, photoelectric conversion processing, deflection processing, and time integration processing based on the fluorescence emitted by the fluorescent sample in each time period to obtain N compressed sampling images. The reference camera is used to acquire N reference images based on the fluorescence emitted by the fluorescent sample in each time period, where N is greater than 1. The device includes: The compressed sampling image processing unit is used to perform reconstruction processing on each compressed sampling image according to the reference image corresponding to the compressed sampling image to obtain a sequence of several frames of fluorescence decay images corresponding to the compressed sampling image, and to perform statistical processing and fitting processing on the intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampling image. The reference image processing unit is used to perform peak detection processing on each of the reference images to obtain the center position of each fluorescent molecule in the reference image. A super-resolution fluorescence lifetime image generation unit is used to generate a super-resolution fluorescence lifetime image corresponding to the fluorescent sample based on the fluorescence lifetime of each fluorescent molecule in each compressed sample image and the center position of each fluorescent molecule in each reference image; the super-resolution fluorescence lifetime image includes the fluorescence lifetime corresponding to each fluorescent molecule in the fluorescent sample.

[0015] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the super-resolution fluorescence lifetime microscopy imaging method as described in any of the first aspects above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the super-resolution fluorescence lifetime microscopy method as described in any of the first aspects above.

[0017] Fifthly, embodiments of this application provide a computer program product that, when run on a display device, causes the display device to perform each step of the super-resolution fluorescence lifetime microscopy imaging method as described in any of the first aspects above.

[0018] The beneficial effects of the super-resolution fluorescence lifetime microscopy imaging method provided in this application are as follows: The super-resolution fluorescence lifetime microscopy imaging method provided in this application is applied to a fluorescence lifetime microscopy imaging system, which includes a fluorescent sample, a pulsed laser, a digital micromirror device, a streak camera, and a reference camera. The pulsed laser is used to emit laser light onto the fluorescent sample. In each time period, some fluorescent molecules in the fluorescent sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different in each time period. The digital micromirror device and the streak camera are used together to perform encoding processing, photoelectric conversion processing, deflection processing, and time integration processing sequentially based on the fluorescence emitted by the fluorescent sample in each time period to obtain N compressed sampling images. The reference camera is used to acquire N reference images based on the fluorescence emitted by the fluorescent sample in each time period, where N is greater than 1. The method includes: For each compressed sampled image, the compressed sampled image is reconstructed based on the reference image corresponding to it to obtain a sequence of several frames of fluorescence decay images corresponding to the compressed sampled image. The intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence is statistically processed and fitted to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampled image. For each reference image, peak detection processing is performed on the reference image to obtain the center position of each fluorescent molecule in the reference image. Based on the fluorescence lifetime of each fluorescent molecule in each compressed sample image and the center position of each fluorescent molecule in each reference image, a super-resolution fluorescence lifetime image corresponding to the fluorescent sample is generated; the super-resolution fluorescence lifetime image includes the fluorescence lifetime of each fluorescent molecule in the fluorescent sample.

[0019] This method enables super-resolution fluorescence lifetime microscopy, improving the performance of fluorescence lifetime microscopy. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1A schematic diagram of a fluorescence lifetime microscopy imaging system provided in an embodiment of this application; Figure 2 A flowchart illustrating the implementation of a super-resolution fluorescence lifetime microscopy method provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a super-resolution fluorescence lifetime microscopy imaging device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] It should be noted that the terminology used in the embodiments of this application is only for explaining specific embodiments of this application and is not intended to limit this application. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, "at least one" or "one or more" means one, two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0023] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0024] In traditional fluorescence lifetime microscopy, time-correlated single-photon counting methods require repeated excitation and sampling, resulting in insufficient data acquisition efficiency. Furthermore, under high photon flux conditions, they are susceptible to photon stacking effects, which compress the initial portion of the fluorescence decay curve, causing the fitted fluorescence lifetime value to deviate from the true value. The photon stacking effect directly leads to inaccurate fluorescence lifetime measurements, failing to accurately reflect the dynamic changes in the microenvironment of fluorescent molecules. Moreover, while existing streak camera-based fluorescence lifetime microscopy can capture the complete fluorescence decay process in a single exposure, its spatial resolution limits its ability to resolve fine structures at the subcellular scale, such as distinguishing boundary features between adjacent organelles.

[0025] The above problems result in low performance of fluorescence lifetime microscopy, leading to systematic biases in the acquisition of fluorescence lifetime information. This causes the assessment of intracellular microenvironment parameters such as pH, viscosity, and intermolecular interactions to fail, thereby affecting the understanding of the molecular mechanisms of biological processes and hindering the progress of biomedical research based on fluorescence lifetime.

[0026] To address this issue, this application proposes a super-resolution fluorescence lifetime microscopy imaging method based on compressed ultrafast imaging, applied to a fluorescence lifetime microscopy imaging system. This system includes a fluorescent sample, a pulsed laser, a digital micromirror device (DMM), a streak camera, and a reference camera. The pulsed laser emits light onto the fluorescent sample. At each time interval, some fluorescent molecules in the sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different at each time interval. The DMM and the streak camera work together to perform encoding, photoelectric conversion, deflection, and time integration processing on the fluorescence emitted by the sample at each time interval, obtaining N compressed sampling images. The reference camera acquires N reference images based on the fluorescence emitted by the sample at each time interval, where N is greater than 1. In practical applications, a typical value for N can be 10000.

[0027] In one possible implementation, the principle that "at each time interval, a portion of the fluorescent molecules in the sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different at each time interval" can be achieved by placing the fluorescent sample in an imaging buffer containing thiol. In this environment, at any given moment, only a very small number of fluorophores in the sample are in an "on" state, responding to the excitation of the pulsed laser and emitting fluorescence. Under this "sparse excitation" state, adjacent fluorophores are imaged under different exposures, thereby achieving spatial separation of different fluorophores in the sample. The fluorophores in the buffer switch their emission "light switches" on a millisecond timescale, with different fluorophores transitioning to the "on" state at different times.

[0028] In one possible implementation, the digital micromirror device is specifically used to: generate original image sequences corresponding to each time period based on the fluorescence emitted by the fluorescent sample in each time period, and encode each original image sequence through an encoding matrix to obtain N encoded image sequences; A stripe camera consists of a photocathode, a scanning electric field, a fluorescent screen, and a photosensitive chip. Specifically, a stripe camera is used for: With the slit in the streak camera fully open, N encoded image sequences in the form of optical signals are acquired; the N encoded image sequences in the form of optical signals are converted into N encoded image sequences in the form of electrical signals through a photocathode; each of the N encoded image sequences in the form of electrical signals is deflected by a scanning electric field, and the N deflected image sequences in the form of electrical signals are acquired through a fluorescent screen; the N deflected image sequences are time-integrated by a photosensitive chip to obtain N compressed sampled images.

[0029] Using the above methods, digital micromirror devices and streak cameras can acquire N compressed sampled images.

[0030] As an example, a schematic diagram of a fluorescence lifetime microscopy imaging system is provided below. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of a fluorescence lifetime microscopy imaging system provided in an embodiment of this application. Figure 1 As shown, the fluorescence lifetime microscopy imaging system mainly consists of an optical system composed of a picosecond pulsed laser, a DMD (digital micromirror device), a high repetition rate fringe camera, a reference camera, and various optical components.

[0031] Among them, picosecond pulsed lasers are A picosecond laser (typically 1 MHz) is emitted and illuminates a fluorescent sample through a lens and objective, exciting the fluorescent molecules in the sample to emit fluorescence. The light from the fluorescent sample, after passing through the objective, is filtered by a dichroic mirror to remove reflected light from the picosecond laser, retaining only the fluorescence excited by the sample. The filtered fluorescence is split into two paths by a beam splitter. One path goes directly to a reference camera and is acquired at a low speed using time integration. The other path's fluorescence is focused onto a DMD by a lens and spatially encoded, then passes through the camera lens into a high-repetition-rate streak camera with the slit fully open. The streak camera is also synchronously triggered at a frequency of 1 MHz, and the scanning circuit operates. The fluorescence image encoded by the DMD reaches the photocathode of the streak camera, is converted into a photoelectron image, and is accelerated by the accelerating electric field of the grid before entering the scanning electric field. The photoelectrons are subjected to a vertically inclined ramp-pulse scanning electric field, causing the photoelectron images from different times to be deflected to varying degrees. After MCP multiplication, they bombard different positions in the vertical direction of the fluorescent screen. A deflected, compressed sampled image is formed on the fluorescent screen and is ultimately acquired by the photosensitive chip. The field of view of the reference camera is identical to that of the image sensor in the streak camera. Unlike the image sensor in the streak camera, the reference camera performs direct time-integration imaging of fluorescence images from multiple excitation cycles. The resulting reference images will be used for subsequent CUP reconstruction calculations and single-molecule localization super-resolution calculations.

[0032] Furthermore, during each laser pulse cycle, when the fluorescent sample in the buffer environment is irradiated with a picosecond laser, only a very small number of fluorophores are excited in the field of view, forming diffuse spots that decay exponentially over time. The photosensitive chips of both the reference camera and the streak camera operate at low repetition frequencies. The camera operates at a frequency of 20 Hz to acquire images. This means that for each acquired image, the camera's integration period lasts... One laser pulse cycle (typically 50,000 cycles). Therefore, the resulting image is... The accumulation of signals generated by the secondary excitation event ensures that enough optical signals are collected.

[0033] The above provides an example of how to acquire N compressed sampled images and N reference images using a fluorescence lifetime microscopy system. The following provides how to generate a super-resolution fluorescence lifetime image corresponding to a fluorescent sample based on the acquired N compressed sampled images and N reference images.

[0034] Please see Figure 2 , Figure 2 The flowchart illustrates the implementation of a super-resolution fluorescence lifetime microscopy method provided in this application embodiment. This super-resolution fluorescence lifetime microscopy method may include steps S101-S103, detailed below: In S101, for each compressed sampled image, the compressed sampled image is reconstructed based on the reference image corresponding to the compressed sampled image to obtain a sequence of several frames of fluorescence decay images corresponding to the compressed sampled image. The intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence is statistically processed and fitted to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampled image.

[0035] In this embodiment, it is first necessary to reconstruct each compressed sampled image based on the reference image corresponding to that compressed sampled image to obtain a sequence of several frames of fluorescence attenuation images corresponding to that compressed sampled image.

[0036] In one possible implementation, steps a to d can be used to "reconstruct the compressed sampled image based on the reference image corresponding to the compressed sampled image to obtain a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image", as detailed below: In step a, a sensing matrix is ​​determined to describe the encoding effect of the digital micromirror device, the deflection effect of the scanning electric field, and the time integration effect of the photosensitive chip.

[0037] The generation process of the compressed sampled image is as follows: First, the two-dimensional image signal is encoded using a DMD, which is equivalent to multiplying the image signal by a pseudo-randomly distributed 0-1 encoding matrix. The slit of the streak camera is fully opened, allowing the encoded two-dimensional image signal to enter the streak camera completely and pass sequentially through the photocathode, scanning electric field, and fluorescent screen before finally reaching the photosensitive chip. Due to the deflection from the vertical scanning electric field, the encoded images at different times produce different magnitudes of vertical displacement on the fluorescent screen, which are then superimposed, forming a sampled signal that is acquired by the photosensitive chip.

[0038] Therefore, the compressed sampled image is successively affected by the encoding effect of the DMD, the shifting effect of the scanning electric field, and the time integration effect of the photosensitive chip. These effects can be described as follows: , , Let the sensing matrix used to describe the encoding effect of the digital micromirror device, the deflection effect of the scanning electric field, and the time integration effect of the photosensitive chip be represented by H, i.e. Thus, the sensing matrix can be obtained.

[0039] In step b, a first relational expression is determined to describe the relationship between the sensing matrix, the compressed sampled image, and the original image sequence corresponding to the compressed sampled image.

[0040] As can be seen from the above steps, the formula for obtaining a compressed sampled image is:

[0041] in, Represented as a compressed sampled image, Let n represent the original image sequence, and n represent the noise.

[0042] because and will and Simplified as and Then, the first relational expression describing the relationship between the sensing matrix, the compressed sampled image, and the original image sequence corresponding to the compressed sampled image can be obtained as follows:

[0043] In step c, the first relation is transformed into the second relation based on the prior knowledge of the pre-acquired original image sequence.

[0044] By reversing the first relation, it is possible to "reconstruct the compressed sampled image to obtain a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image".

[0045] However, solving the first relation in reverse is an underdetermined problem, and an exact solution cannot be obtained by directly solving the system of equations. Therefore, compressed sensing theory needs to be introduced to transform the problem into a constrained optimization problem using prior knowledge of the original image signal. That is, the first relation needs to be transformed into the second relation based on the prior knowledge of the pre-acquired original image sequence.

[0046] In one possible implementation, the second relation describes the relationship between the local optimum of the fluorescence attenuation image sequence and the fidelity term, the first prior term, and the regularization coefficient. The fidelity term is determined based on the compressed sampled image, the sensor matrix, and the original image sequence. The first prior term is determined based on prior knowledge. The regularization coefficient is used to adjust the weights between the fidelity term and the first prior term. For example, the second relation could be:

[0047] In the formula, For authenticity purposes; The first prior term contains prior knowledge of the original image; The regularization coefficient is . This is a locally optimal solution.

[0048] In step d, based on the reference image corresponding to the compressed sampled image, the local optimal solution of the second relation is solved, and the local optimal solution is converted into a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image.

[0049] After obtaining the second relation, the local optimal solution of the second relation can be obtained based on the reference image corresponding to the compressed sampled image, and then a series of fluorescence attenuation images corresponding to the compressed sampled image can be obtained.

[0050] In one possible implementation, auxiliary variables can be obtained for denoising the compressed sampled image; for example, the auxiliary variables can be derived from... express.

[0051] Then, based on the auxiliary variables, the second relation can be transformed into the third relation; the second relation is the relation corresponding to the unconstrained optimization problem, and the third relation is the relation corresponding to the constrained optimization problem.

[0052] For example, the third relation can be:

[0053] in, The constraints, that is, the constraints of the third relation, are: .

[0054] Subsequently, in order to solve the third relation, we can first determine the first sub-optimization relation, the second sub-optimization relation, and the third sub-optimization relation based on the third relation. The first sub-optimization relation is used to make the fluorescence attenuation image sequence as close as possible to the compressed sampled image, the second sub-optimization relation is used for image denoising, and the third sub-optimization relation is used to correct the errors corresponding to the first and second sub-optimization relations.

[0055] For example, the first sub-optimization relation can be:

[0056] For example, the second sub-optimization relation can be:

[0057] For example, the third sub-optimization relation can be:

[0058] The third relation can be solved by iteratively solving the first, second, and third sub-optimization relations.

[0059] Specifically, based on the reference image corresponding to the compressed sampled image, the first sub-optimization relation, the second sub-optimization relation, and the third sub-optimization relation can be iteratively calculated until the preset convergence condition is reached. Based on the local optimal solution of the second sub-optimization relation obtained at the end, a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image can be determined.

[0060] In one possible implementation, the computation of the second sub-optimization relation can be based on a certain image prior. Image denoising operations. Based on this understanding, the calculation of the second sub-optimization relation can be directly replaced by other advanced denoising algorithms. Specifically, the second sub-optimization relation can be calculated in the following way: The second sub-optimization relation is transformed into the fourth sub-optimization relation. The second sub-optimization relation includes the second prior term, which is determined based on the prior knowledge of the auxiliary variable. The fourth sub-optimization relation does not include any prior term.

[0061] For example, the fourth sub-optimization relation can be:

[0062] In the formula For noise reduction operators, representing a certain noise reduction device's... The noise reduction process.

[0063] Next, the fourth sub-optimization relation is calculated, and the preliminary calculation results are corrected by using the reference image corresponding to the compressed sampled image during the calculation process.

[0064] During the calculation of the fourth sub-optimization relation, the results may exhibit "illusion" phenomena: excessive smoothing during image denoising and the generation of artifacts. To eliminate these effects, the initial calculation results can be corrected by using a reference image corresponding to the compressed sampled image during the calculation process, thereby correcting the denoised image.

[0065] Based on the reference image corresponding to the compressed sampled image, the first sub-optimization relation, the second sub-optimization relation, and the third sub-optimization relation are iteratively calculated until the preset convergence condition is met. Then, the local optimal solution of the second sub-optimization relation is used to determine a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image.

[0066] After obtaining a sequence of several frames of fluorescence decay images corresponding to the compressed sampled image, statistical processing and fitting processing can be performed on the intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampled image.

[0067] In S102, for each reference image, peak detection processing is performed on the reference image to obtain the center position of each fluorescent molecule in the reference image.

[0068] In the embodiments of this application, "performing peak detection processing on the reference image to obtain the center position corresponding to each fluorescent molecule in the reference image" can be achieved in the following manner, detailed below: Multiple sub-reference images are obtained by image segmentation using a sliding window. For each sub-reference image, if the brightness of the center pixel of the sub-reference image is higher than the brightness of all pixels surrounding the target in the sub-reference image, then the center pixel of the sub-reference image is determined as the center position of a fluorescent molecule in the reference image. The pixels surrounding the target are the pixels among the non-center pixels of the sub-reference image that have not been determined as the center position of the fluorescent molecule.

[0069] Sliding window image segmentation is an image processing technique that divides an original image into multiple local regions, or sub-reference images, by moving a fixed-size window across the image with a preset step size. This technique improves processing efficiency and accuracy by breaking down complex global image analysis tasks into independent analyses of local regions. For example, rectangular or circular windows can be used for sliding, and the window size can be flexibly adjusted according to the expected size and density of fluorescent molecules. Another implementation is that the sliding window can adaptively adjust its size and shape based on the characteristics of the image content to better accommodate fluorescent molecule signals of different sizes or shapes.

[0070] A sub-reference image is a local image region extracted from the original reference image through sliding window image segmentation. Each sub-reference image contains pixel information within that region, and its function is to provide an independent analysis unit for subsequent local peak detection.

[0071] The core criterion for determining whether a pixel is the center of a fluorescent molecule is that the brightness of the center pixel is higher than that of all pixels surrounding the target. This criterion is based on the characteristic that fluorescent molecule signals usually exhibit local brightness peaks.

[0072] When the brightness of the center pixel of a sub-reference image is higher than that of all other pixels in a specific region around it, the center pixel is considered to be the point of maximum brightness in that local region, and is thus initially determined to be the center location of the fluorescent molecule.

[0073] Pixels surrounding the target refer to pixels in the sub-reference image that, excluding the center pixel, have not yet been identified as the center of other fluorescent molecules. Their purpose is to provide a clear reference range for brightness comparison of the center pixel, ensuring that pixels already identified as centers of other fluorescent molecules are not incorrectly included in the comparison range during local peak detection, thus avoiding duplicate detection or misjudgment. For example, pixels already marked as centers of fluorescent molecules can be excluded during brightness comparison. Another implementation is to mark the pixels at each fluorescent molecule center location and within a certain radius as processed after determination; subsequent peak detection will then disregard these processed pixels.

[0074] This application employs a sliding window image segmentation technique to decompose the original reference image into multiple sub-reference images, thereby transforming the complex task of global peak detection into the independent analysis of multiple local regions. This decomposition strategy effectively reduces processing complexity and improves detection efficiency. For each sub-reference image, by comparing the brightness of its central pixel with the brightness of all pixels surrounding the target, local brightness peaks can be accurately identified. The filtering of pixels surrounding the target—that is, excluding pixels already identified as the center location of other fluorescent molecules—ensures that each fluorescent molecule is detected only once, effectively avoiding misjudgments caused by overlapping fluorescent molecule signals. This local peak detection strategy can effectively identify the precise center location of each fluorescent molecule in the reference image, providing relatively accurate localization results even when fluorescent molecules are densely distributed or have uneven brightness. Combining this method with super-resolution fluorescence lifetime microscopy provides reliable spatial localization information for subsequent fluorescence lifetime calculation and super-resolution image generation, thereby improving the accuracy and resolution of the final super-resolution fluorescence lifetime image.

[0075] In S103, a super-resolution fluorescence lifetime image corresponding to the fluorescent sample is generated based on the fluorescence lifetime of each fluorescent molecule in each compressed sampling image and the center position of each fluorescent molecule in each reference image; the super-resolution fluorescence lifetime image includes the fluorescence lifetime of each fluorescent molecule in the fluorescent sample.

[0076] In the embodiments of this application, the "generating a super-resolution fluorescence lifetime image corresponding to the fluorescent sample based on the fluorescence lifetime of each fluorescent molecule in each compressed sampled image and the center position of each fluorescent molecule in each reference image" can be achieved in the following manner, detailed below: For each fluorescent molecule, the fluorescent molecule is labeled at its center position according to its fluorescence lifetime. After labeling all fluorescent molecules, all fluorescent molecules are synthesized into the same image to obtain a super-resolution fluorescence lifetime image.

[0077] The "labeling process" aims to correlate the calculated fluorescence lifetime value with the precise spatial location of the fluorescent molecule. Specifically, this can involve directly storing the fluorescence lifetime value of the fluorescent molecule at the corresponding pixel in the image.

[0078] "After completing the labeling process of all fluorescent molecules, all fluorescent molecules are synthesized into the same image to obtain a super-resolution fluorescence lifetime image." The aim is to integrate all individually labeled fluorescent molecules into a complete image, thereby forming a super-resolution image that can intuitively display the spatial distribution of fluorescence lifetime.

[0079] The above methods achieve the following technical effects: 1. Breaking the diffraction limit in spatial resolution: This invention, combined with the single-molecule localization principle, achieves a nearly tenfold improvement in spatial resolution (approximately 20 nm), enabling clear resolution of subcellular structures and breaking through the resolution limitation of approximately 200 nanometers in traditional fluorescence microscopy. 2. High-precision fluorescence lifetime measurement: Compared to time-correlated single-photon counting technology, this method utilizes the ability of a streak camera to acquire fluorescence lifetime in a single image. By directly measuring the fluorescence decay process, fluorescence lifetime is obtained, avoiding photon accumulation effects, significantly improving the accuracy of fluorescence lifetime measurement, and effectively distinguishing fluorophores with different lifetime values. 3. Significantly improved imaging efficiency: Utilizing the wide-field imaging and single-shot lifetime measurement capabilities of compressed ultrafast imaging technology, spatiotemporal resolution information can be obtained with only a small amount of data acquisition, greatly reducing the data acquisition burden and overcoming the limitations of time-correlated single-photon counting technology, which suffers from long data acquisition time and high storage pressure.

[0080] As can be seen from the above, this method can achieve super-resolution fluorescence lifetime microscopy imaging, thus improving the performance of fluorescence lifetime microscopy imaging.

[0081] Based on the super-resolution fluorescence lifetime microscopy imaging method provided in the above embodiments, this application further provides a super-resolution fluorescence lifetime microscopy imaging apparatus for implementing the above method embodiments. Please refer to... Figure 3 , Figure 3 This is a schematic diagram of a super-resolution fluorescence lifetime microscopy imaging device provided in an embodiment of this application. This device is applied to a fluorescence lifetime microscopy imaging system, which includes a fluorescent sample, a pulsed laser, a digital micromirror device (DMD), a streak camera, and a reference camera. The pulsed laser emits laser light onto the fluorescent sample. In each time period, some fluorescent molecules in the sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different in each time period. The DMD and the streak camera are used together to sequentially encode, photoelectrically convert, deflect, and integrate the fluorescence emitted by the sample in each time period to obtain N compressed sampling images. The reference camera is used to acquire N reference images based on the fluorescence emitted by the sample in each time period, where N is greater than 1.

[0082] like Figure 3 As shown, the super-resolution fluorescence lifetime microscopy imaging device 30 may include: a compressed sampling image processing unit 31, a reference image processing unit 32, and a super-resolution fluorescence lifetime image generation unit 33. Wherein: The compressed sampling image processing unit 31 is used to reconstruct each compressed sampling image based on the reference image corresponding to the compressed sampling image, to obtain a sequence of several frames of fluorescence decay images corresponding to the compressed sampling image, and to perform statistical processing and fitting processing on the intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampling image.

[0083] The reference image processing unit 32 is used to perform peak detection processing on each reference image to obtain the center position of each fluorescent molecule in the reference image.

[0084] The super-resolution fluorescence lifetime image generation unit 33 is used to generate a super-resolution fluorescence lifetime image corresponding to the fluorescent sample based on the fluorescence lifetime of each fluorescent molecule in each compressed sample image and the center position of each fluorescent molecule in each reference image; the super-resolution fluorescence lifetime image includes the fluorescence lifetime corresponding to each fluorescent molecule in the fluorescent sample.

[0085] Optionally, digital micromirror devices are specifically used for: Based on the fluorescence emitted by the fluorescent sample in each time period, the original image sequence corresponding to each time period is generated, and each original image sequence is encoded by an encoding matrix to obtain N encoded image sequences. A stripe camera consists of a photocathode, a scanning electric field, a fluorescent screen, and a photosensitive chip. Specifically, a stripe camera is used for: With the slit in the streak camera fully open, acquire N encoded image sequences in the form of light signals; The N encoded image sequences in optical signal form are converted into N encoded image sequences in electrical signal form using a photocathode. The N encoded image sequences in the form of each electrical signal are deflected by scanning an electric field, and the N deflected image sequences in the form of electrical signals are acquired through a fluorescent screen. By performing time integration on N deflection-processed image sequences using a photosensitive chip, N compressed sampled images are obtained.

[0086] Optionally, the compressed sampling image processing unit 31 is specifically used for: Determine the sensing matrix used to describe the encoding effect of the digital micromirror device, the deflection effect of the scanning electric field, and the time integration effect of the photosensitive chip; Determine a first relational expression to describe the relationship between the sensing matrix, the compressed sampled image, and the original image sequence corresponding to the compressed sampled image; Based on the prior knowledge of the original image sequence obtained in advance, the first relation is transformed into the second relation. The second relation is used to describe the relationship between the local optimal solution of the fluorescence attenuation image sequence and the fidelity term, the first prior term, and the regularization coefficient. The fidelity term is determined based on the compressed sampled image, the sensing matrix, and the original image sequence. The first prior term is determined based on prior knowledge. The regularization coefficient is used to adjust the weight between the fidelity term and the first prior term. Based on the reference image corresponding to the compressed sampled image, the local optimal solution of the second relation is obtained, and the local optimal solution is converted into a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image.

[0087] Optionally, the compressed sampling image processing unit 31 is specifically used for: Obtain auxiliary variables for denoising the compressed sampled image; Based on the auxiliary variables, the second relation is transformed into the third relation; the second relation is the relation corresponding to the unconstrained optimization problem, and the third relation is the relation corresponding to the constrained optimization problem. Based on the third relation, the first sub-optimization relation, the second sub-optimization relation, and the third sub-optimization relation are determined. The first sub-optimization relation is used to make the fluorescence attenuation image sequence as close as possible to the compressed sampled image. The second sub-optimization relation is used to perform image noise reduction processing. The third sub-optimization relation is used to correct the error corresponding to the first sub-optimization relation and the error corresponding to the second sub-optimization relation. Based on the reference image corresponding to the compressed sampled image, the first sub-optimization relation, the second sub-optimization relation, and the third sub-optimization relation are iteratively calculated until the preset convergence condition is reached. Based on the local optimal solution of the second sub-optimization relation, a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image is determined.

[0088] Optionally, the compressed sampling image processing unit 31 is specifically used for: The second sub-optimization relation is transformed into the fourth sub-optimization relation; the second sub-optimization relation includes the second prior term, which is determined based on the prior knowledge of the auxiliary variable; the fourth sub-optimization relation does not include any prior term. The fourth sub-optimization relation is calculated, and the preliminary calculation results are corrected by using the reference image corresponding to the compressed sampled image during the calculation process.

[0089] Optionally, the reference image processing unit 32 is specifically used for: Multiple sub-reference images are obtained from the reference image by using a sliding window image segmentation method; For each sub-reference image, if the brightness of the center pixel of the sub-reference image is higher than the brightness of all the pixels surrounding the target in the sub-reference image, then the center pixel of the sub-reference image is determined as the center position of a fluorescent molecule in the reference image; the pixels surrounding the target are the pixels among the non-center pixels of the sub-reference image that have not been determined as the center position of a fluorescent molecule.

[0090] Optionally, the super-resolution fluorescence lifetime image generation unit 33 is specifically used for: For each fluorescent molecule, the fluorescent molecule is labeled at its center position according to its fluorescence lifetime. After labeling all fluorescent molecules, all fluorescent molecules are synthesized into a single image to obtain a super-resolution fluorescence lifetime image.

[0091] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of this application. Their specific functions and technical effects can be referred to the method embodiments section, and will not be repeated here.

[0092] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 4 provided in this embodiment may include: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40, such as a program corresponding to the super-resolution fluorescence lifetime microscopy method. When the processor 40 executes the computer program 42, it implements the steps described above applied in the embodiment of the super-resolution fluorescence lifetime microscopy method, for example... Figure 2 S101~S103 are shown. Alternatively, when processor 40 executes computer program 42, it implements the functions of each module / unit in the above-described embodiment of the super-resolution fluorescence lifetime microscopy device, for example... Figure 3 The functions of units 31-33 shown.

[0093] For example, computer program 42 can be divided into one or more modules / units, one or more of which are stored in memory 41 and executed by processor 40 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 42 in electronic device 4. For example, computer program 42 can be divided into a compressed sampling image processing unit 31, a reference image processing unit 32, and a super-resolution fluorescence lifetime image generation unit 33. For the specific functions of each unit, please refer to... Figure 3 The relevant descriptions in the corresponding embodiments are not repeated here.

[0094] Those skilled in the art will understand that Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or combine certain components, or use different components.

[0095] The processor 40 can be a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0096] The memory 41 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM. The memory 41 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card. Furthermore, the memory 41 can include both internal and external storage units of the electronic device 4. The memory 41 is used to store computer programs and other programs and data required by the electronic device. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units is merely an example. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the super-resolution fluorescence lifetime microscopy imaging device can be divided into different functional units to complete all or part of the functions described above. The functional units in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0099] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0100] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

[0101] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0102] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A super-resolution fluorescence lifetime microscopy imaging method, characterized in that, An application is made in a fluorescence lifetime microscopy imaging system, which includes a fluorescent sample, a pulsed laser, a digital micromirror device, a streak camera, and a reference camera. The pulsed laser emits laser light onto the fluorescent sample. At each time interval, some fluorescent molecules in the sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different at each time interval. The digital micromirror device and the streak camera are used together to sequentially perform encoding processing, photoelectric conversion processing, deflection processing, and time integration processing based on the fluorescence emitted by the sample at each time interval to obtain N compressed sampling images. The reference camera is used to acquire N reference images based on the fluorescence emitted by the sample at each time interval, where N is greater than 1. The method includes: For each compressed sampled image, based on the reference image corresponding to the compressed sampled image, the compressed sampled image is reconstructed to obtain a sequence of several frames of fluorescence decay images corresponding to the compressed sampled image. The intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence is statistically processed and fitted to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampled image. For each of the reference images, peak detection processing is performed on the reference image to obtain the center position of each fluorescent molecule in the reference image. A super-resolution fluorescence lifetime image corresponding to the fluorescent sample is generated based on the fluorescence lifetime of each fluorescent molecule in each compressed sample image and the center position of each fluorescent molecule in each reference image; the super-resolution fluorescence lifetime image includes the fluorescence lifetime corresponding to each fluorescent molecule in the fluorescent sample. The digital micromirror device is specifically used for: Based on the fluorescence emitted by the fluorescent sample in each time period, a corresponding original image sequence for each time period is generated, and each original image sequence is encoded using an encoding matrix to obtain N encoded image sequences. The stripe camera includes a photocathode, a scanning electric field, a fluorescent screen, and a photosensitive chip. Specifically, the stripe camera is used for: With the slit in the stripe camera fully open, the N encoded image sequences in the form of light signals are acquired; The N encoded image sequences in optical signal form are converted into the N encoded image sequences in electrical signal form using a photocathode. The scanning electric field is used to deflect the N encoded image sequences in each electrical signal form, and the N deflected image sequences in electrical signal form are obtained through the fluorescent screen. The N compressed sampled images are obtained by time-integrating the N deflected image sequences using the photosensitive chip. The step of reconstructing the compressed sampled image based on the reference image corresponding to the compressed sampled image to obtain a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image includes: Determine a sensing matrix to describe the encoding effect of the digital micromirror device, the deflection effect of the scanning electric field, and the time integration effect of the photosensitive chip; Determine a first relational expression to describe the relationship between the sensing matrix, the compressed sampled image, and the original image sequence corresponding to the compressed sampled image; Based on the prior knowledge of the original image sequence obtained in advance, the first relation is transformed into a second relation; the second relation is used to describe the relationship between the local optimal solution of the fluorescence attenuation image sequence and the fidelity term, the first prior term, and the regularization coefficient. The fidelity term is determined based on the compressed sampled image, the sensing matrix, and the original image sequence. The first prior term is determined based on the prior knowledge. The regularization coefficient is used to adjust the weight between the fidelity term and the first prior term. Based on the reference image corresponding to the compressed sampled image, the local optimal solution of the second relation is solved, and the local optimal solution is converted into a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image. The step of performing peak detection processing on the reference image to obtain the center position of each fluorescent molecule in the reference image includes: Multiple sub-reference images are obtained from the reference image by using a sliding window image segmentation method; For each sub-reference image, if the brightness of the center pixel of the sub-reference image is higher than the brightness of all the pixels surrounding the target in the sub-reference image, then the center pixel of the sub-reference image is determined as the center position of a fluorescent molecule in the reference image; the pixels surrounding the target are the pixels among the non-center pixels of the sub-reference image that have not been determined as the center position of the fluorescent molecule.

2. The method according to claim 1, characterized in that, The step of solving the local optimal solution of the second relation based on the reference image corresponding to the compressed sampled image, and converting the local optimal solution into a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image, includes: Obtain auxiliary variables for denoising the compressed sampled image; Based on the auxiliary variables, the second relation is transformed into a third relation; the second relation is the relation corresponding to the unconstrained optimization problem, and the third relation is the relation corresponding to the constrained optimization problem. Based on the third relation, a first sub-optimization relation, a second sub-optimization relation, and a third sub-optimization relation are determined; the first sub-optimization relation is used to make the fluorescence attenuation image sequence as close as possible to the compressed sampled image, the second sub-optimization relation is used to perform image noise reduction processing, and the third sub-optimization relation is used to correct the error corresponding to the first sub-optimization relation and the error corresponding to the second sub-optimization relation. Based on the reference image corresponding to the compressed sampled image, the first sub-optimization relation, the second sub-optimization relation, and the third sub-optimization relation are iteratively operated on until the preset convergence condition is reached. Based on the local optimal solution of the second sub-optimization relation obtained at the end, a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image is determined.

3. The method according to claim 2, characterized in that, The second sub-optimization relation is calculated in the following manner: The second sub-optimization relation is transformed into a fourth sub-optimization relation; the second sub-optimization relation includes a second prior term, which is determined based on the prior knowledge of the auxiliary variable; the fourth sub-optimization relation does not include any prior term. The fourth sub-optimization relation is calculated, and the preliminary calculation result is corrected by using the reference image corresponding to the compressed sampled image during the calculation process.

4. The method according to any one of claims 1 to 3, characterized in that, The step of generating a super-resolution fluorescence lifetime image corresponding to the fluorescent sample based on the fluorescence lifetime of each fluorescent molecule in each compressed sample image and the center position of each fluorescent molecule in each reference image includes: For each fluorescent molecule, the fluorescent molecule is labeled at its center position according to its fluorescence lifetime. After labeling all fluorescent molecules, all fluorescent molecules are synthesized into the same image to obtain the super-resolution fluorescence lifetime image.

5. A super-resolution fluorescence lifetime microscopy imaging device, characterized in that, An application is made in a fluorescence lifetime microscopy imaging system, which includes a fluorescent sample, a pulsed laser, a digital micromirror device, a streak camera, and a reference camera. The pulsed laser is used to emit laser light onto the fluorescent sample. In each time period, some fluorescent molecules in the fluorescent sample respond to the laser and emit fluorescence, and the fluorescent molecules that respond to the laser and emit fluorescence are different in each time period. The digital micromirror device and the streak camera are used together to perform encoding processing, photoelectric conversion processing, deflection processing, and time integration processing sequentially based on the fluorescence emitted by the fluorescent sample in each time period to obtain N compressed sampling images. The reference camera is used to acquire N reference images based on the fluorescence emitted by the fluorescent sample in each time period, where N is greater than 1. The device includes: The compressed sampling image processing unit is used to perform reconstruction processing on each compressed sampling image according to the reference image corresponding to the compressed sampling image to obtain a sequence of several frames of fluorescence decay images corresponding to the compressed sampling image, and to perform statistical processing and fitting processing on the intensity decay distribution of fluorescent molecules in the fluorescence decay image sequence to obtain the fluorescence lifetime of each fluorescent molecule in the compressed sampling image. The reference image processing unit is used to perform peak detection processing on each of the reference images to obtain the center position of each fluorescent molecule in the reference image. A super-resolution fluorescence lifetime image generation unit is used to generate a super-resolution fluorescence lifetime image corresponding to the fluorescent sample based on the fluorescence lifetime of each fluorescent molecule in each compressed sample image and the center position of each fluorescent molecule in each reference image; the super-resolution fluorescence lifetime image includes the fluorescence lifetime corresponding to each fluorescent molecule in the fluorescent sample. The digital micromirror device is specifically used for: Based on the fluorescence emitted by the fluorescent sample in each time period, a corresponding original image sequence for each time period is generated, and each original image sequence is encoded using an encoding matrix to obtain N encoded image sequences. The stripe camera includes a photocathode, a scanning electric field, a fluorescent screen, and a photosensitive chip. Specifically, the stripe camera is used for: With the slit in the stripe camera fully open, the N encoded image sequences in the form of light signals are acquired; The N encoded image sequences in optical signal form are converted into the N encoded image sequences in electrical signal form using a photocathode. The scanning electric field is used to deflect the N encoded image sequences in each electrical signal form, and the N deflected image sequences in electrical signal form are obtained through the fluorescent screen. The N deflected image sequences are integrated over time by the photosensitive chip to obtain the N compressed sampled images; The compressed sampling image processing unit is specifically used for: Determine a sensing matrix to describe the encoding effect of the digital micromirror device, the deflection effect of the scanning electric field, and the time integration effect of the photosensitive chip; Determine a first relational expression to describe the relationship between the sensing matrix, the compressed sampled image, and the original image sequence corresponding to the compressed sampled image; Based on the prior knowledge of the original image sequence obtained in advance, the first relation is transformed into a second relation; the second relation is used to describe the relationship between the local optimal solution of the fluorescence attenuation image sequence and the fidelity term, the first prior term, and the regularization coefficient. The fidelity term is determined based on the compressed sampled image, the sensing matrix, and the original image sequence. The first prior term is determined based on the prior knowledge. The regularization coefficient is used to adjust the weight between the fidelity term and the first prior term. Based on the reference image corresponding to the compressed sampled image, the local optimal solution of the second relation is solved, and the local optimal solution is converted into a sequence of several frames of fluorescence attenuation images corresponding to the compressed sampled image. The reference image processing unit is specifically used for: Multiple sub-reference images are obtained from the reference image by using a sliding window image segmentation method; For each sub-reference image, if the brightness of the center pixel of the sub-reference image is higher than the brightness of all the pixels surrounding the target in the sub-reference image, then the center pixel of the sub-reference image is determined as the center position of a fluorescent molecule in the reference image; the pixels surrounding the target are the pixels among the non-center pixels of the sub-reference image that have not been determined as the center position of the fluorescent molecule.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the super-resolution fluorescence lifetime microscopy method as described in any one of claims 1 to 4.

7. A computer program product, characterized in that, When the computer program product is executed by a processor, it implements the steps of the super-resolution fluorescence lifetime microscopy method as described in any one of claims 1 to 4.

Images