Super-resolution imaging method, device and equipment

Abstract

The invention discloses a super-resolution imaging method and device, a medium and equipment. The method comprises the following steps: based on a point scanning imaging system, at least adopting two point spread functions to carry out fluorescence image acquisition on the same region of interest, and obtaining fluorescence images corresponding to the point spread functions; carrying out denoising processing and deconvolution processing on the basis of each fluorescence image to obtain a super-resolution image; and determining whether the super-resolution image satisfies a predetermined imaging condition, and when determining that the predetermined imaging condition is not satisfied, performing denoising processing and deconvolution processing again based on each fluorescence image to obtain a current super-resolution image until the current super-resolution image satisfies the predetermined imaging condition, and obtaining a target super-resolution image. According to the invention, super-resolution image imaging can be rapidly carried out.

Description

Technical Field

[0001] This invention relates to the field of microscopic imaging technology, and in particular to a super-resolution imaging method, apparatus, and device. Background Technology

[0002] In recent years, various super-resolution imaging techniques have emerged in fields such as biology and medicine. Super-resolution imaging has broken the optical diffraction limit, enabling researchers to study the fine features of subcellular structures at the nanoscale resolution, thus gaining a deeper understanding of the details of life processes. However, existing point spread function-modified techniques, such as stimulated emission depletion microscopy (STED), require precise alignment of two beams, demanding sophisticated equipment, using high-energy light, and exhibiting significant phototoxicity to samples. Furthermore, in practical applications, they often fail to reach their theoretical resolution limits. Laser confocal microscopy offers advantages such as large imaging scale, high resolution, high imaging speed, and high robustness, but for many important biological processes at the hundred-nanometer resolution scale, further resolution improvements are still needed.

[0003] Therefore, there is an urgent need for a point-scan super-resolution imaging method to solve the problem of super-resolution imaging under low excitation power in existing technologies. Summary of the Invention

[0004] This invention provides a super-resolution imaging method, apparatus, and device, with the main objective of solving the current problem of difficulty in achieving point scanning super-resolution imaging at low excitation power.

[0005] To address the above problems, this application provides a super-resolution imaging method, comprising:

[0006] Based on the point scanning imaging system, at least two point spread functions are used to acquire fluorescence images of the same sample region to obtain fluorescence images corresponding to each point spread function.

[0007] Denoising and deconvolution processing are performed on the fluorescence images to obtain super-resolution images;

[0008] Determine whether the super-resolution image meets the predetermined imaging conditions. If the predetermined imaging conditions are not met, perform denoising and deconvolution processing on each fluorescence image again to obtain the current super-resolution image. The process continues until the current super-resolution image meets the predetermined imaging conditions to obtain the target super-resolution image.

[0009] Optionally, the step of performing denoising and deconvolution processing on each of the fluorescence images to obtain a super-resolution image specifically includes:

[0010] Determine the weights of the point spread functions corresponding to the current deconvolution process;

[0011] The denoised image is determined based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function.

[0012] The denoised image is deconvolved to obtain the super-resolution image.

[0013] Optionally, determining the denoised image based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function specifically includes:

[0014] Based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function, the sub-denoising spectrum corresponding to each fluorescence image is determined.

[0015] The total denoised spectrum is obtained by merging the denoised spectra of each sub-denoised spectrum.

[0016] The denoised image is determined based on the total denoised spectrum.

[0017] Optionally, the method further includes: determining the denoising operator corresponding to the point spread function, specifically including:

[0018] Determine the second spectrum of each point spread function;

[0019] The denoising operator for each point spread function is determined based at least on the second spectrum of each point spread function.

[0020] Optionally, the method further includes: determining a suppression parameter corresponding to the current deconvolution process;

[0021] The step of determining the denoising operator for each point spread function based at least on the second spectrum of each point spread function specifically includes:

[0022] Based on the second spectrum of each point spread function and the suppression parameter, determine the denoising operator of the point spread function corresponding to the current deconvolution process.

[0023] Optionally, the predetermined imaging conditions include a predetermined signal-to-noise ratio threshold or a predetermined resolution threshold.

[0024] Optionally, the scanning imaging system includes, but is not limited to, any one of the following: Gaussian beam imaging system, Bessel beam imaging system, and vortex beam imaging system.

[0025] Optionally, the scanning imaging system includes: a scanning imaging system based on point scanning imaging technology, which includes, but is not limited to, any one of the following: a scanning imaging system based on confocal fluorescence microscopy and a scanning imaging system based on laser-depleted imaging technology;

[0026] The scanning imaging system of the confocal fluorescence microscopy technology includes any one of the following: a single-point scanning confocal imaging system, a rotary confocal imaging system, and a line scanning confocal imaging system.

[0027] To address the above problems, this application provides a super-resolution imaging device, comprising:

[0028] The acquisition module is used to acquire fluorescence images of the same region of interest using at least two point spread functions based on a point scanning imaging system, and to obtain fluorescence images corresponding to each point spread function.

[0029] The processing module is used to perform denoising and deconvolution processing on each of the fluorescence images to obtain a super-resolution image;

[0030] The acquisition module is used to determine whether the super-resolution image meets the predetermined imaging conditions. If it is determined that the predetermined imaging conditions are not met, the module re-performs denoising and deconvolution processing based on each fluorescence image to obtain the current super-resolution image. The process continues until the current super-resolution image meets the predetermined imaging conditions, at which point the target super-resolution image is obtained.

[0031] To address the aforementioned problems, this application provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the super-resolution imaging methods described above.

[0032] To address the aforementioned problems, this application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of any of the super-resolution imaging methods described above.

[0033] The super-resolution imaging method, apparatus, medium, and device described in this application acquire fluorescence images with different point spread functions using a point scanning imaging system. This eliminates the need for precise alignment of the two light beams along their optical axes; instead, it simplifies the process by repeatedly denoising and deconvolving each fluorescence image. This allows for the acquisition of super-resolution structural information with a resolution superior to that of confocal microscopy, significantly improving imaging resolution and speed. Furthermore, the method presented in this application exhibits strong robustness and demonstrates good reconstruction performance even for fluorescence images with low signal-to-noise ratios.

[0034] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0035] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0036] Figure 1 This is a flowchart illustrating a super-resolution imaging method according to an embodiment of this application;

[0037] Figure 2 (a) is the original image generated by the simulation;

[0038] Figure 2 (b) For the purpose of using the method of this application to Figure 2 (a) Full width at half maximum (FWHM) analysis of the marked portion after reconstruction;

[0039] Figure 3 (a) is another original image generated by simulation;

[0040] Figure 3 (b) for using the method in this application to Figure 3 (a) Gray value analysis of the marked part after reconstruction;

[0041] Figure 4 (a) is another original image generated by the simulation;

[0042] Figure 4 (b) for using the method in this application to Figure 4 (a) Gray value analysis of the diameter of the marked part after reconstruction;

[0043] Figure 5 (a) is the original image that was actually taken;

[0044] Figure 5 (b) For the purpose of using the method of this application to Figure 5 (a) Grayscale analysis of the marked portion after reconstruction;

[0045] Figure 6 This is another embodiment of the present application, a structural block diagram of a super-resolution imaging device. Detailed Implementation

[0046] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0047] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0048] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0049] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0050] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0051] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0052] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0053] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0054] This application provides a super-resolution imaging method, the flowchart of which is shown below. Figure 1 As shown, it includes the following steps:

[0055] Step S101: Based on the point scanning imaging system, at least two point spread functions are used to acquire fluorescence images of the same sample area to obtain fluorescence images corresponding to each point spread function;

[0056] In this step, the sample area can specifically be the cell structure to be observed, including but not limited to: endoplasmic reticulum, mitochondria, chloroplasts, centrosomes, Golgi apparatus, morphology of nucleolus, ribosomes, lysosomes, etc.

[0057] In practice, fluorescent molecules can be used to pre-label the structure / region of interest to be imaged. Then, a point-scan imaging system with multiple point spread functions is used to acquire fluorescence images of the same labeled region, obtaining fluorescence images corresponding to each point spread function. For example, fluorescence image g1 can be acquired using a Gaussian beam scanning confocal imaging system, and fluorescence image g2 can be acquired using a vortex beam scanning confocal imaging system. During fluorescence image acquisition, the Gaussian beam and vortex beam scan independently without needing to be aligned; the intensity ratio of the two beams is not required during acquisition.

[0058] In addition, in the specific implementation of this step, the same beam scanning imaging system can be used to acquire multiple fluorescence images by using multiple point spread functions generated by different optical devices, or two or more fluorescence images generated by an algorithm based on a single fluorescence image can also be regarded as fluorescence images based on imaging systems with different point spread functions.

[0059] Step S102: Denoising and deconvolution processing are performed on each of the fluorescence images to obtain a super-resolution image;

[0060] In this step, after obtaining at least two fluorescence images, such as fluorescence image g1 and fluorescence image g2, the corresponding fluorescence images can be denoised in the frequency domain using the point spread function weights corresponding to the point spread function, obtaining the sub-denoised spectrum for each fluorescence image. Subsequently, the denoised image f' can be obtained by processing the sub-denoised spectrum of each fluorescence image. After obtaining the denoised image f', deconvolution processing can be performed on the denoised image f' to obtain the super-resolution image. During the denoising process, any of the following denoising methods can be used: filtering-based methods (e.g., mean filtering, Wiener filtering, etc.), model-based denoising methods (i.e., image prior modeling methods including sparse models, gradient models, etc.), and deep learning-based methods (e.g., denoising based on convolutional neural networks (CNN), denoising based on generative adversarial networks, etc.).

[0061] Step S103: Determine whether the super-resolution image meets the predetermined imaging conditions. If the predetermined imaging conditions are not met, perform denoising and deconvolution processing on each fluorescence image again to obtain the current super-resolution image. Obtain the target super-resolution image when the current super-resolution image meets the predetermined imaging conditions.

[0062] In this step, a signal-to-noise ratio (SNR) threshold or a resolution threshold can be preset as predetermined imaging conditions. By setting these conditions, after denoising and deconvolution to obtain a super-resolution image, it can be determined whether the super-resolution image meets the resolution or SNR threshold. For example, if the super-resolution image's resolution is greater than the resolution threshold, or its SNR is greater than the SNR threshold, then the predetermined imaging conditions are met, and it can be used as the target super-resolution image. Conversely, if the predetermined imaging conditions are not met, denoising and deconvolution processing can be repeated on each fluorescence image. This process continues until the current super-resolution image meets the predetermined imaging conditions, or the number of denoising and deconvolution operations reaches a predetermined threshold, at which point the target super-resolution image is obtained.

[0063] The super-resolution imaging method in this application acquires fluorescence images under different point spread functions using a point scanning imaging system. It eliminates the need for two beams to be aligned; simply by repeatedly denoising and deconvolving each fluorescence image, it obtains super-resolution structural information superior to that obtained by confocal microscopy, significantly improving imaging resolution. Furthermore, the method in this application exhibits strong robustness and good reconstruction results even for fluorescence images with low signal-to-noise ratios. Based on the characteristic that fluorescence images captured by different point spread function systems have the same signal but different noise, the method employs repeated denoising and deconvolution processing to achieve multi-kernel deconvolution based on multiple fluorescence images, minimizing the sum of noise obtained from each single-kernel denoising. After obtaining the denoised image, further deconvolution is performed to obtain the target super-resolution image.

[0064] Based on the above embodiments, another embodiment of this application provides an imaging effect of a super-resolution imaging method, which uses simulated generated images for super-resolution imaging, and the imaging effect is as follows. Figure 2-4 As shown. This embodiment specifically includes the following steps:

[0065] Step S201: Based on the point scanning imaging system, at least two point spread functions are used to acquire fluorescence images of the same sample region of interest, and fluorescence images corresponding to each point spread function are obtained.

[0066] In this step, the fluorescence image is acquired using point-scan-based imaging techniques, including confocal fluorescence microscopy and laser-damped imaging. Confocal fluorescence microscopy includes single-point scanning confocal imaging systems, rotating disk confocal imaging systems, and line-scan confocal imaging systems. Imaging systems used to generate different point spread functions include Gaussian beam imaging systems, Bessel beam imaging systems, and vortex beam imaging systems. In other words, a single-point scanning confocal imaging system can be used to acquire a fluorescence image by generating a Gaussian beam, or a rotating disk confocal imaging system can be used to acquire a fluorescence image by generating a Bessel beam.

[0067] When acquiring fluorescence images corresponding to different point spread functions, two or more point spread functions can be obtained based on the same type of imaging system using different parameters, thereby acquiring fluorescence images corresponding to each point spread function; two or more point spread functions can be obtained by combining the above-mentioned different imaging systems, thereby acquiring fluorescence images corresponding to each point spread function; or a single original image can be used to generate two or more different fluorescence images through an algorithm.

[0068] Step S202: Determine the weights of the point spread functions corresponding to the current deconvolution process;

[0069] Step S203: Determine the denoised image based on the weight of each point spread function, the denoising operator L of the point spread function, and the first spectrum G of the fluorescence image corresponding to the point spread function.

[0070] In this step, when determining the denoised image, the sub-denoised spectrum wLG corresponding to each fluorescence image can be determined based on the weight ω of each point spread function, the denoising operator L of the point spread function, and the first spectrum G of the fluorescence image corresponding to the point spread function; then, the sub-denoised spectra wLG are merged to obtain the total denoised spectrum F; and the denoised image f' is determined based on the total denoised spectrum.

[0071] In this embodiment, the total denoised spectrum F∧ can be calculated using the following formula:

[0072]

[0073] Step S204: Perform deconvolution processing on the denoised image to obtain the super-resolution image;

[0074] In this step, after obtaining the denoised image f', RL deconvolution can be performed on the denoised image to obtain the super-resolution image. In this embodiment, RL deconvolution is an existing method, officially called Richardson-Lucy deconvolution. This method is an iterative algorithm based on maximum likelihood estimation. It treats image pixel values ​​as probabilities, and uses Bayes' theorem to calculate the conditional probability from the real image pixels to the ideal image pixels. During the process, the ideal image estimate calculated in n-1 iterations is used to approximate the conditional probability of the nth iteration, further obtaining the ideal image estimate of the nth iteration, thus obtaining the super-resolution image.

[0075] Step S205: Determine whether the super-resolution image meets the predetermined imaging conditions. If it is determined that the predetermined imaging conditions are not met, repeat step S203. If it is determined that the predetermined imaging conditions are met, obtain the target super-resolution image.

[0076] In this embodiment, before determining the denoised image f', that is, before executing step S203, the denoising operator corresponding to the point spread function can be determined first. Specifically, the second spectrum H of each point spread function can be determined; at least based on the second spectrum of each point spread function, the denoising operator of each point spread function can be determined. That is, the second spectrum H of each point spread function can be determined first, and then based on the second spectrum of each point spread function and the suppression parameter K, the denoising operator of the point spread function corresponding to the current deconvolution process can be determined. In this embodiment, by determining the denoising operator L of the point spread function corresponding to the current deconvolution process, it is convenient to perform subsequent denoising processing based on the denoising operator L of each point spread function, thereby obtaining the denoised image f', laying the foundation for obtaining a super-resolution image based on the subsequent processing of the denoised image f'. Furthermore, the denoising operator L corresponding to each point spread function can be calculated using the following denoising operator calculation formula based on the second spectrum H of each point spread function:

[0077]

[0078] Where i represents the i-th point spread function; M represents the number of point spread functions; K represents the suppression parameter; ω i H represents the weight of the i-th point spread function under the current denoising process; H represents the spectrum of the point spread function; L represents the denoising operator.

[0079] In this embodiment, Multi-PSF Wiener (multiple point spread function Wiener filtering) can be used to denoise and deconvolve each fluorescence image. In this embodiment, the intermediate result, i.e., the denoised image f′, is obtained by applying the Multi-PSF Wiener method to each fluorescence image. This denoising process involves three parameters: First, the limitations of the Wiener deconvolution (Wiener filtering) method itself require adding a parameter K when solving the denominator to avoid division by zero. Furthermore, the actual denoising effect is achieved by adjusting the K parameter to suppress high-frequency information, thereby improving the denoising effect and suppressing deconvolution. Second, the Multi-PSF Wiener method involves multiple point spread functions (PSFs), so the final weighted stacking involves the weights ω of multiple PSFs. In practical applications, adjusting these weights indirectly affects the deconvolution effect in the second step. Therefore, in this embodiment, each time a denoising process is performed, it is necessary to first determine the weight and suppression parameter K of each point spread function corresponding to that denoising process, and then combine the weight and suppression parameter K of the point spread function to determine the denoising operator L of the point spread function. Then, based on the denoising operator and weight of the point spread function, the denoising process can be performed reasonably and accurately, making the denoising result more accurate and reliable.

[0080] In this embodiment, the original image generated by simulation is used. Figure 2 (a) Figure 3 (a) and Figure 4 (a)) is used to evaluate the reconstruction effect of this method. Figure 2 Showing simulated data images ( Figure 2 (a) The full width at half maximum (FWHM) of the reconstructed marked portion is 145 nm. Figure 2 (b)). Figure 3 Show the marked portions of the original image at different distances generated by the simulation ( Figure 3 (a) The reconstructed image and grayscale analysis of the marked part ( Figure 3 (b)). Figure 4 The images reconstructed by the method of this disclosure using coils at different distances are shown. Figure 4 (a) and perform grayscale analysis on the diameter of the marked portion. Figure 4 (b)).

[0081] Based on the above embodiments, another embodiment of this application provides a reconstruction effect of a super-resolution imaging method, using experimentally captured 100nm fluorescent beads as the original image, and the reconstruction effect is as follows. Figure 5 As shown. This embodiment specifically includes the following steps:

[0082] Step 1: Use fluorescent molecules to fluorescently label the region of interest in the sample;

[0083] Step 2: Based on the confocal scanning system, two point spread functions are used to acquire fluorescence images of the fluorescently labeled region of interest, obtaining fluorescence images g1 and g2 corresponding to each point spread function;

[0084] In this step, a point scanning system can be used to acquire a fluorescence image of the region of interest using the point spread function PSF1 to obtain a fluorescence image g1; then, the point spread function PSF2 can be used to acquire a fluorescence image of the region of interest to obtain a fluorescence image g2.

[0085] In this step, the fluorescence image is acquired using point-scan-based imaging techniques, including confocal fluorescence microscopy and laser-damped imaging. Confocal fluorescence microscopy includes single-point scanning confocal imaging systems, rotating disk confocal imaging systems, and line-scan confocal imaging systems. Imaging systems used to generate different point spread functions include Gaussian beam imaging systems, Bessel beam imaging systems, and vortex beam imaging systems.

[0086] In this embodiment, when acquiring fluorescence images corresponding to different point spread functions, two or more point spread functions can be obtained based on the same type of imaging system using different parameters, thereby acquiring fluorescence images corresponding to each point spread function; two or more point spread functions can be generated by combining the above-mentioned different imaging systems, thereby acquiring fluorescence images corresponding to each point spread function; or a single original image can be used to generate two or more different fluorescence images through an algorithm.

[0087] Step 3: Determine the weights of each point spread function corresponding to the current denoising process;

[0088] In this step, we first determine the weights ω1 and ω2 corresponding to the point spread function PSF1 and PSF2, respectively, for the denoising process. The sum of ω1 and ω2 is 1. The energy proportion of the PSFs is used as the initial values ​​for the two PSF weights. Subsequently, the final processing effect is judged based on indicators such as the signal-to-noise ratio and resolution of the reconstructed image, and the best processed image and corresponding parameters are selected accordingly.

[0089] Step 4: Determine the suppression parameter K corresponding to the current denoising process;

[0090] In this step, an initial value is set in advance based on experience, and then the parameter K is continuously changed. The effect of denoising is judged based on the signal-to-noise ratio, resolution and other indicators of the denoised image, and the best denoised image and corresponding parameters are selected accordingly.

[0091] Step 5: Determine the second spectrum H of each point spread function; based on the second spectrum of each point spread function and the suppression parameter K, determine the denoising parameters of the point spread function corresponding to the current deconvolution process.

[0092] In this step, the second spectrum H1 corresponding to the point spread function PSF1 can be determined, and the second spectrum H2 corresponding to the point spread function PSF2 can be determined. Then, the denoising operator L1 corresponding to the point spread function PSF1 and the denoising operator L2 corresponding to the point spread function PSF2 can be calculated using the above denoising operator calculation formula (i.e., formula (2)). That is, Where ω1 represents the weight corresponding to the point spread function PSF1; ω2 represents the weight corresponding to the point spread function PSF2; H1 represents the second spectrum corresponding to the point spread function PSF1; H2 represents the second spectrum corresponding to the point spread function PSF2; and K represents the suppression parameter.

[0093] Step 6: Based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function, determine the sub-denoised spectrum of each fluorescence image;

[0094] In this step, for fluorescence images g1 and g2, we first determine the first spectrum G1 corresponding to fluorescence image g1, and then determine the first spectrum G2 corresponding to fluorescence image g2. Then, using the first spectrum G1 corresponding to fluorescence image g1, the second spectrum H1 corresponding to the point spread function PSF1, and the weight ω1 corresponding to the point spread function PSF1, we calculate the sub-denoised spectrum corresponding to fluorescence image g1. Similarly, the sub-denoised spectrum corresponding to the fluorescence image g2 can be calculated. Right now,

[0095] Step 7: Merge the sub-denoised spectra to obtain the total denoised spectrum;

[0096] In this step, the individual denoised spectra can be summed to obtain the total denoised spectrum. Right now

[0097] Step 8: Determine the denoised image based on the total denoised spectrum;

[0098] In this step, after obtaining the total denoised spectrum, it can be transformed to convert it from the frequency domain to the spatial domain, thus obtaining the denoised image f′.

[0099] Step 9: Perform deconvolution processing on the denoised image to obtain a super-resolution image;

[0100] In this step, the deconvolution process includes: Richardson-Lucy algorithm, Landweber deconvolution, Wiener deconvolution, and Richardson-Lucy deconvolution (i.e., RL deconvolution). In specific implementation, after obtaining the denoised image f′, RL deconvolution can be used to deconvolve the denoised image f′ to obtain the super-resolution image f.

[0101] Step 10: Determine whether the super-resolution image meets the predetermined imaging conditions; if it is determined that the predetermined imaging conditions are not met, repeat steps 3 to 9; otherwise, if it is determined that the predetermined imaging conditions are met, the target super-resolution image can be obtained.

[0102] In this step, if the super-resolution image resolution does not reach the predetermined resolution threshold, or the signal-to-noise ratio does not reach the predetermined signal-to-noise ratio threshold, then the super-resolution image does not meet the predetermined imaging conditions. At this point, it is necessary to return to step three, that is, to redetermine the weights of the spread function at each point used for denoising. Simultaneously, in the subsequent step four, it is also necessary to redetermine the suppression parameter K used for denoising.

[0103] For original images / fluorescence images of the same structure captured on two or more point spread function (PSF) systems that have identical signals but different noise levels, this application employs a method of mutually constrained multi-kernel deconvolution to minimize the sum of noise from each individual kernel's denoising. This ensures minimal loss of high-frequency information. By performing frequency domain transformation on multiple PSF images, multiplying them with corresponding operators, summing the results, and then transforming them back to the spatial domain, a denoised image with a high signal-to-noise ratio is obtained. This method effectively reduces background noise and suppresses noise-induced artifacts during convolution.

[0104] After obtaining the denoised image, it can be further deconvolved to obtain the super-resolution image. This means applying the denoised image to the subsequent optimized deconvolution algorithm, which can effectively suppress initial noise during deconvolution and thus improve the quality of deconvolution. Simultaneously, artifacts that may be introduced in practice due to edge effects, inaccurate PSF calibration, and noise are tested one by one to improve the robustness of the method.

[0105] The method in this application, based on the characteristics of general-purpose equipment in the biological field, can easily improve existing equipment compared to existing technologies, greatly reducing development and usage costs. The two scanning beams do not need to be aligned, the equipment is easy to implement, and there is no limitation on the size of the imaging area, which can meet the needs of large field-of-view live sample imaging in application scenarios.

[0106] The super-resolution imaging method in this application is actually a mutually constrained denoising and deconvolution algorithm. Its basic principle is to use multiple frames of images (fluorescence images g1, g2, g3, g4, g5, g6) captured by multiple PSFs (point spread functions).i... Mutual constraints. In this application, the derivation process of the total denoised spectrum calculation formula (1) and the denoising operator calculation formula (2) is as follows:

[0107] For Wiener deconvolution in the case of multiple kernels (i.e., multiple PSFs), it is called Multi-PSF Wiener deconvolution. The formula for the single-kernel (single PSF) case needs slight modification. Similarly, assuming f is the object distribution, g... i For the distribution of the i-th blurred image (the i-th fluorescence image), h i Let i be the i-th PSF distribution, then the objective function of the Multi-PSF wiener is:

[0108]

[0109] In the formula ω i The weights of the spread function at the i-th point are generally determined by h. i The energy percentage is used to determine this, and by differentiating the above equation and performing a Fourier transform, we obtain:

[0110]

[0111] In the formula, F and H i G i f and h respectively i g i The spectrum For the Fourier transform operator, setting the above formula to 0, we can obtain:

[0112]

[0113] Similar to the single-core case, to avoid division by zero in the spectrum, a parameter K is introduced, resulting in:

[0114]

[0115] The parameter K in the formula is an important parameter related to the signal-to-noise ratio (SNR) in the multi-core case, also known as the signal-to-noise ratio. The above formula can be rewritten as:

[0116]

[0117] In the formula, L i This represents the deconvolution operator.

[0118] Therefore, it is clear that the spectrum of the deconvolutioned image can be considered as a weighted sum of images convolved with different PSFs. Applying this method to mutual constraint denoising, i.e., through the above derivation, formula (5) can be used as the formula for calculating the total denoised spectrum, and simultaneously obtaining the denoising operator calculation formula in this application. This lays the foundation for subsequent reasonable and accurate denoising and deconvolution processing.

[0119] Figure 5 This demonstrates an evaluation of the reconstruction performance of actual imaging using 100nm fluorescent beads. Figure 5 (a) The original image captured by the Gaussian PSF imaging system. Figure 5 (b) shows the grayscale analysis of the reconstructed marked portion.

[0120] Based on the above embodiments, another embodiment of this application provides a super-resolution imaging device, such as... Figure 6 As shown, it includes:

[0121] Acquisition module 11 is used to acquire fluorescence images of the same region of interest using at least two point spread functions based on a point scanning imaging system, and to obtain fluorescence images corresponding to each point spread function;

[0122] Processing module 12 is used to perform denoising and deconvolution processing on each of the fluorescence images to obtain a super-resolution image;

[0123] The module 13 is used to determine whether the super-resolution image meets the predetermined imaging conditions. When it is determined that the predetermined imaging conditions are not met, the module re-performs denoising and deconvolution processing based on each fluorescence image to obtain the current super-resolution image. The process continues until the current super-resolution image meets the predetermined imaging conditions, at which point the target super-resolution image is obtained.

[0124] In this embodiment, the processing module includes:

[0125] The weight determination unit is used to determine the weights of each point spread function corresponding to the current deconvolution process;

[0126] The denoising unit is used to determine the denoised image based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function;

[0127] The deconvolution unit is used to perform deconvolution processing on the denoised image to obtain the super-resolution image.

[0128] In this embodiment, the denoising unit is specifically used to: determine the sub-denoised spectrum corresponding to each fluorescence image based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function; merge the sub-denoised spectra to obtain the total denoised spectrum; and determine the denoised image based on the total denoised spectrum.

[0129] In this embodiment, the super-resolution imaging device further includes a denoising operator determination module. This module is used to determine the denoising operator corresponding to the point spread function, specifically for:

[0130] Determine the second spectrum of each point spread function;

[0131] The denoising operator for each point spread function is determined based at least on the second spectrum of each point spread function.

[0132] In this embodiment, the method further includes a super-resolution imaging device that also includes a suppression parameter determination module, which is used to: determine the suppression parameter corresponding to the current deconvolution process;

[0133] The denoising operator determination module is specifically used to: determine the denoising operator of the point spread function corresponding to the current deconvolution process based on the second spectrum of each point spread function and the suppression parameter.

[0134] In this embodiment, the predetermined imaging conditions include a predetermined signal-to-noise ratio threshold or a predetermined resolution threshold.

[0135] In this embodiment, the scanning imaging system includes any one of the following: a Gaussian beam imaging system, a Bessel beam imaging system, and a vortex beam imaging system.

[0136] In the specific implementation of this embodiment, the scanning imaging system includes: a scanning imaging system based on point scanning imaging technology, which includes any one of the following: a scanning imaging system based on confocal fluorescence microscopy imaging technology and a scanning imaging system based on laser loss imaging technology;

[0137] The scanning imaging system of the confocal fluorescence microscopy technology includes any one of the following: a single-point scanning confocal imaging system, a rotary confocal imaging system, and a line scanning confocal imaging system.

[0138] The super-resolution imaging device in this application acquires fluorescence images with different point spread functions using a point scanning imaging system. It eliminates the need for two beams to be aligned; simply by repeatedly denoising and deconvolving each fluorescence image, it obtains super-resolution structural information superior to that obtained by confocal microscopy, significantly improving imaging resolution and speed. Furthermore, the method in this application exhibits strong robustness and good reconstruction results even for fluorescence images with low signal-to-noise ratios.

[0139] Another embodiment of this application provides a storage medium storing a computer program, which, when executed by a processor, implements the following method steps:

[0140] Step 1: Based on the point scanning imaging system, at least two point spread functions are used to acquire fluorescence images of the same region of interest, and fluorescence images corresponding to each point spread function are obtained;

[0141] Step 2: Perform denoising and deconvolution processing on each of the fluorescence images to obtain a super-resolution image;

[0142] Step 3: Determine whether the super-resolution image meets the predetermined imaging conditions. If the predetermined imaging conditions are not met, perform denoising and deconvolution processing on each fluorescence image again to obtain the current super-resolution image. Continue until the current super-resolution image meets the predetermined imaging conditions to obtain the target super-resolution image.

[0143] The specific implementation process of the above method steps can be found in the embodiments of the above-mentioned super-resolution imaging methods, and will not be repeated here.

[0144] The storage medium in this application acquires fluorescence images with different point spread functions using a point scanning imaging system. Without aligning two beams, it only requires repeated denoising and deconvolution processing of each fluorescence image to obtain super-resolution structural information superior to that obtained by confocal microscopy, significantly improving imaging resolution and speed. Furthermore, the method in this application exhibits strong robustness and good reconstruction results even for fluorescence images with low signal-to-noise ratios.

[0145] Another embodiment of this application provides an electronic device, including at least a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program in the memory, performs the following method steps:

[0146] Step 1: Based on the point scanning imaging system, at least two point spread functions are used to acquire fluorescence images of the same region of interest, and fluorescence images corresponding to each point spread function are obtained;

[0147] Step 2: Perform denoising and deconvolution processing on each of the fluorescence images to obtain a super-resolution image;

[0148] Step 3: Determine whether the super-resolution image meets the predetermined imaging conditions. If the predetermined imaging conditions are not met, perform denoising and deconvolution processing on each fluorescence image again to obtain the current super-resolution image. Continue until the current super-resolution image meets the predetermined imaging conditions to obtain the target super-resolution image.

[0149] The specific implementation process of the above method steps can be found in the embodiments of the above-mentioned super-resolution imaging methods, and will not be repeated here.

[0150] The electronic device described in this application acquires fluorescence images with different point spread functions using a point scanning imaging system. Without requiring the alignment of two light beams, it only needs to repeatedly denoise and deconvolve each fluorescence image to obtain super-resolution structural information superior to that obtained by confocal microscopy, significantly improving imaging resolution and speed. Furthermore, the method described in this application exhibits strong robustness and good reconstruction results even for fluorescence images with low signal-to-noise ratios.

[0151] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A super-resolution imaging method, characterized in that, include: Based on the point scanning imaging system, at least two systems with different point spread functions are used to acquire fluorescence images of the same sample to obtain fluorescence images corresponding to each point spread function; Denoising and deconvolution processing are performed on the fluorescence images to obtain super-resolution images; Determine whether the super-resolution image meets the predetermined imaging conditions. If the predetermined imaging conditions are not met, perform denoising and deconvolution processing on each fluorescence image again to obtain the current super-resolution image. The process continues until the current super-resolution image meets the predetermined imaging conditions to obtain the target super-resolution image.

2. The method as described in claim 1, characterized in that, The step of performing denoising and deconvolution processing on each of the fluorescence images to obtain a super-resolution image specifically includes: Determine the weights of the point spread functions corresponding to the current deconvolution process; The denoised image is determined based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function. The denoised image is deconvolved to obtain the super-resolution image.

3. The method as described in claim 2, characterized in that, The process of determining the denoised image based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function specifically includes: Based on the weight of each point spread function, the denoising operator of the point spread function, and the first spectrum of the fluorescence image corresponding to the point spread function, the sub-denoising spectrum corresponding to each fluorescence image is determined. The total denoised spectrum is obtained by merging the denoised spectra of each sub-denoised spectrum. The denoised image is determined based on the total denoised spectrum.

4. The method according to any one of claims 2-3, characterized in that, The method further includes: determining the denoising operator corresponding to the point spread function, specifically including: Determine the second spectrum of each point spread function; The denoising operator for each point spread function is determined based at least on the second spectrum of each point spread function.

5. The method as described in claim 4, characterized in that, The method further includes: determining the suppression parameter corresponding to the current deconvolution process; The step of determining the denoising operator for each point spread function based at least on the second spectrum of each point spread function specifically includes: Based on the second spectrum of each point spread function and the suppression parameter, determine the denoising operator of the point spread function corresponding to the current deconvolution process.

6. The method as described in claim 1, characterized in that, The predetermined imaging conditions include a predetermined signal-to-noise ratio threshold or a predetermined resolution threshold.

7. The method as described in claim 1, characterized in that, The point scanning imaging system includes, but is not limited to, any one of the following: Gaussian beam imaging system, Bessel beam imaging system, and vortex beam imaging system.

8. The method as described in claim 1, characterized in that, The point scanning imaging system includes: a scanning imaging system based on point scanning imaging technology, which includes, but is not limited to, any one of the following: a scanning imaging system based on confocal fluorescence microscopy and a scanning imaging system based on laser loss imaging technology; The scanning imaging system of the confocal fluorescence microscopy technology includes any one of the following: a single-point scanning confocal imaging system, a rotary confocal imaging system, and a line scanning confocal imaging system.

9. A super-resolution imaging device, characterized in that, include: The acquisition module is used to acquire fluorescence images of the same sample based on a point scanning imaging system, using at least two imaging systems with different point spread functions, to obtain fluorescence images corresponding to each point spread function. The processing module is used to perform denoising and deconvolution processing on each of the fluorescence images to obtain a super-resolution image; The acquisition module is used to determine whether the super-resolution image meets the predetermined imaging conditions. If it is determined that the predetermined imaging conditions are not met, the module re-performs denoising and deconvolution processing based on each fluorescence image to obtain the current super-resolution image. The process continues until the current super-resolution image meets the predetermined imaging conditions, at which point the target super-resolution image is obtained.

10. An electronic device, characterized in that, It includes at least a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the super-resolution imaging method according to any one of claims 1-8 when executing the computer program in the memory.

Images