A method and device for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images

Abstract

The present invention discloses a method and device for enhancing the quality of super-resolution reconstruction of fluorescence microscopic images, which belongs to the field of underlying visual technology in biomedical image processing, including: acquiring fluorescence microscopic images under weak conditions; performing denoising and enhancement operations on the fluorescence microscopic images to obtain denoised and enhanced result images; estimating a flat-field image of the denoised and enhanced result images according to a two-dimensional polynomial orthogonal basis function set, and inversely deriving an image after light field correction; performing super-resolution reconstruction processing on the image after light field correction, and outputting a high-quality, high-spatial-resolution image. The method and device significantly improve the quality of images reconstructed by super-resolution algorithms, reduce reconstruction artifacts, improve the continuity of foreground reconstruction, reduce the imaging conditions of fluorescence microscopic images, and thereby reduce the influence of phototoxicity and reduce the fluorescence bleaching of samples, which is beneficial to long-term super-resolution reconstruction of living cells.

Description

Technical Field

[0001] The present invention relates to the field of underlying visual technology in biomedical image processing, and in particular to a method and device for enhancing the quality of super-resolution reconstruction of fluorescence microscopic images. Background Art

[0002] Fluorescence microscopy is an imaging technique widely used in biomedical research. It uses organic and inorganic fluorophores as molecular dyes to label specific cellular components or to attach engineered fluorescent proteins to molecules of interest to obtain microscopic images with high spatial resolution. However, due to the optical diffraction limit, the image resolution obtained by traditional fluorescence imaging technology has an upper limit, which limits the observation and understanding of subcellular structures by biomedical practitioners.

[0003] The emergence of super-resolution fluorescence microscopy provides sufficient details for the visualization of many subcellular structures, breaking this limitation and revolutionizing the field of biological imaging. Currently, super-resolution fluorescence microscopy techniques can be roughly divided into three categories: based on nonlinear fluorescence response, based on structured light illumination, and based on the optical switching or light blinking properties of fluorescent molecules.

[0004] The first type of technology uses nonlinear fluorescence response to improve resolution, such as stimulated emission depletion microscopy (STED). The principle can be summarized as first shining a beam of light on the biological sample to generate fluorescence, and then shining a second beam of light on the sample to annihilate the peripheral fluorescence. At this time, only a small area inside still has fluorescence, so the imaging resolution is improved. However, STED has relatively high requirements for equipment, requires strong excitation light, has low energy utilization, and is prone to photobleaching.

[0005] The second type of technology is based on structured light illumination, such as structured light illumination microscopy (SIM), which uses moiré fringes to move high-frequency information that originally exceeds the cutoff frequency of the imaging system to the low-frequency part so that it can be detected by the system, breaking through the diffraction limit to achieve improved resolution. Although SIM has a fast imaging speed and is not easy to damage cell samples, its resolution improvement is lower than that of technologies such as STED and is prone to artifacts.

[0006] The third category relies on the optical switching or light blinking properties of fluorescent molecules, exchanging temporal resolution for spatial resolution to improve the localization accuracy of single molecules. This type of method is called single-molecule localization microscopy (SMLMs), including photoactivated localization imaging microscopy (PALM) and stochastic optical reconstruction microscopy (STORM). In addition, the image post-processing algorithm super-resolution optical fluctuation imaging (SOFI) relies on the physically distinguishable state cycles of fluorescent molecules and also belongs to this category. Obviously, the characteristic of this type of technology that trades temporal resolution for spatial resolution is very unfriendly to living cell imaging.

[0007] In recent years, some super-resolution computational imaging post-processing algorithms based on new principles and priors have received widespread attention. These algorithms have made a trade-off between the improvement of temporal resolution and spatial resolution and have good biocompatibility. For example, the sparse deconvolution algorithm combined with SIM uses gradient information to complete spatial analysis and then combines the super-resolution radial fluctuation algorithm SRRF and eSRRF based on the blinking characteristics of fluorescent molecules. These algorithms can achieve good super-resolution effects under relatively acceptable temporal resolution. However, the common problem of these methods is that they have poor effects on low-quality fluorescence data and may produce obvious artifacts after reconstruction.

[0008] Although the sf-MSSR algorithm overcomes the problem of poor temporal resolution of traditional algorithms and has a certain effect on low signal-to-noise ratio images, most of the super-resolution methods mentioned above, such as SRRF, still have limitations such as being unsuitable for low signal-to-noise ratio images and being prone to artifacts. Even for the sf-MSSR algorithm, if you want to produce good super-resolution results, it is necessary to require a strong signal for the original image as a whole, otherwise the result is usually too discrete, and even truncation and disappearance of foreground structures may occur. Stronger fluorescence signals usually mean stronger laser power and longer exposure time, which will reduce temporal resolution and increase problems such as sample fluorescence bleaching. Existing microscopic image analysis methods deal with the low quality of fluorescence images by adjusting image contrast or Gaussian blurring, etc., but the effect is limited and may even change the original structural information of the image, resulting in reconstruction errors of the super-resolution algorithm. Summary of the invention

[0009] In view of the above, the purpose of the present invention is to provide a method and device for enhancing the super-resolution reconstruction quality of fluorescence microscopy images, which combines morphological filtering and orthogonal polynomials to enhance low-quality fluorescence microscopy images, aiming to improve the super-resolution reconstruction quality of images.

[0010] To achieve the above-mentioned object of the invention, the embodiment further provides a method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images, comprising the following steps:

[0011] Acquire fluorescence microscopy images under weak conditions;

[0012] Denoising and enhancing the fluorescence microscopy image to obtain a denoised and enhanced result image;

[0013] The flat-field image is estimated from the denoised and enhanced result image according to the two-dimensional polynomial orthogonal basis function set, and the image after light field correction is obtained by reverse deduction.

[0014] The light field corrected image is subjected to super-resolution reconstruction to output an image with high quality and high spatial resolution.

[0015] Preferably, a morphological filtering algorithm is used to perform denoising and enhancement operations on the fluorescence microscopy image to obtain a denoising and enhanced result image T w (f), the specific process is:

[0016]

[0017] Among them, T w (f) represents the result image after denoising and enhancement, f represents the fluorescence microscopy image, b represents the morphological filter kernel generated by the structural element, represents the morphological opening operation, represents the corrosion and expansion of the image f by the structural element b, ⊙ represents the morphological corrosion, Indicates morphological expansion.

[0018] Preferably, the process of estimating the flat-field image of the denoising and enhancement result image according to the two-dimensional polynomial orthogonal basis function set, and inversely deriving the image after light field correction is as follows:

[0019] Construct a two-dimensional polynomial orthogonal basis function set. From the separability of orthogonal polynomials, it can be seen that the two-dimensional polynomial kernel can be obtained from the one-dimensional polynomial kernel, that is:

[0020] P mn (x, y) = P m (x)P n (y), 0≤m≤M, 0≤n≤N

[0021] Among them, P m (x), P n (y) are one-dimensional M+1 and N+1 order orthogonal polynomials, respectively, and their forms are recursively derived from the following formula:

[0022] P0(x)=1

[0023] P1(x)=x

[0024]

[0025] Here, let M = N, then the formula P mn (x, y) = P m (x)P n (y) is a two-dimensional orthogonal polynomial of order N+1, which is used to estimate the flat-field image of the denoised and enhanced result image

[0026]

[0027] x i ,y jRepresents the coordinate position in space, used to traverse the pixels in the image, i = 1, 2, .., l, j = 1, 2, .., s, l and s represent the height and width of the image in pixels, respectively, a m(N+1)+n represents the coefficient, which is estimated by the least squares method;

[0028] By solving the coefficient a m(N+1)+n Get the estimated flat field image The image I obtained by inverting the microscopic imaging process after light field correction is:

[0029]

[0030] Wherein, ε is an adjustment parameter, and its value is 1%-10% of the maximum value of the current image pixel.

[0031] Preferably, the super-resolution reconstruction of the image after light field correction includes:

[0032] It is proposed to use the spatial analysis algorithm of super-resolution radial fluctuation (SRRF) to perform single-frame super-resolution reconstruction on the image after light field correction to obtain high-quality and high-resolution single-frame reconstruction results. The specific process is as follows:

[0033] First, the gradient G in the x and y directions is calculated based on the light field corrected image I(x,y) x and G y for:

[0034]

[0035] Then, the sub-pixels are divided in the pixel by Fourier interpolation algorithm to define a series of spatial position points (x c ,y c ) is used to calculate the radial degree and define the annular coordinate system (x i ′ ,y i ′ ), and through G x and G y Get the sub-pixel gradient G xi and G yi , the gradient line equation is obtained as:

[0036] (xx i ′ )G yi -(yy i ′ )G xi =0

[0037] Then we get (x c ,y c ) and the vertical distance from the gradient line is:

[0038]

[0039] The gradient line is defined at the position point (x c ,y c )'s convergence c i for:

[0040]

[0041] Finally, the quality-enhanced super-resolution image I is obtained SRRF (x,y) is:

[0042]

[0043] Where N represents the number of circular coordinate points, r xi ,r yi Represents the circular coordinate point (x i ′ ,y i ′ ) to (x c ,y c ) distance.

[0044] Preferably, the super-resolution reconstruction of the image after light field correction includes:

[0045] The single-frame mean shift super-resolution reconstruction (sf-MSSR) algorithm was performed on the light field corrected images using the MSSR reconstruction plug-in on the microscopic image processing software ImageJ to obtain high-quality and high-resolution single-frame reconstruction results.

[0046] To achieve the above-mentioned purpose of the invention, the embodiment further provides a device for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images, comprising a data acquisition unit, a denoising enhancement unit, a correction unit, and a super-resolution reconstruction unit.

[0047] The data acquisition unit is used to acquire fluorescence microscopic images under weak conditions;

[0048] The denoising and enhancing unit is used to perform denoising and enhancing operations on the fluorescence microscopy image to obtain a denoising and enhanced result image;

[0049] The correction unit is used to estimate the flat field image of the denoising and enhancement result image according to the two-dimensional polynomial orthogonal basis function set, and inversely deduce to obtain the image after light field correction;

[0050] The super-resolution reconstruction unit is used to perform super-resolution reconstruction processing on the image after light field correction, and output an image with high quality and high spatial resolution.

[0051] In order to achieve the above-mentioned purpose of the invention, an embodiment of the present invention further provides a computing device, characterized in that it includes a memory and a processor, the memory is used to store a computer executable program for executing enhanced super-resolution fluorescence microscopy image reconstruction quality, and when the processor executes the computer executable program, it implements the above-mentioned method for enhancing the super-resolution reconstruction quality of fluorescence microscopy images.

[0052] To achieve the above-mentioned purpose of the invention, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is processed and executed, the above-mentioned method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images is implemented.

[0053] Compared with the prior art, the technical effects of the present invention include at least:

[0054] Aiming at the problem that low signal-to-noise ratio fluorescence microscopy images under weak conditions are prone to artifacts after reconstruction by super-resolution algorithms such as SRRF, a method based on morphological filtering is proposed to achieve denoising and enhancement, which improves the quality of the original fluorescence image, greatly reduces the artifacts after super-resolution algorithm reconstruction, and improves the continuity of foreground reconstruction.

[0055] To address the problem that the foreground fluorescence signal of the original fluorescence microscopy image under weak conditions is unevenly collected, resulting in the truncation and invisibility of some structures after reconstruction by super-resolution algorithms such as sf-MSSR, it is proposed to estimate the flat field through orthogonal polynomials and then perform light field correction, so that the overall foreground structure of the original fluorescence image is highlighted, thereby improving the reconstruction effect of the super-resolution algorithm in dark areas.

[0056] By combining morphological filtering-based denoising enhancement and orthogonal polynomial-based light field correction, the quality of original fluorescence microscopy images under weak conditions is improved, the quality of the resulting images reconstructed by the super-resolution algorithm is significantly enhanced, and the imaging conditions of fluorescence microscopy images are reduced, thereby weakening the impact of phototoxicity and reducing fluorescence bleaching of samples, which is beneficial to the long-term super-resolution reconstruction of living cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 is a flow chart of a method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images provided by an embodiment of the present invention;

[0059] Figure 2This is an example diagram of the gradual enhancement effect in the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images provided by an embodiment of the present invention;

[0060] Figure 3 This is an example diagram of denoising enhancement and light field correction results in the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images provided in an embodiment of the present invention;

[0061] Figure 4 This is an example diagram of SRRF spatial analysis results in the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images provided in an embodiment of the present invention;

[0062] Figure 5 This is an example diagram of sf-MSSR results in the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopic images provided in an embodiment of the present invention;

[0063] Figure 6 is an enhancement curve diagram of the super-resolution reconstruction quality of a fluorescence microscopy image provided by an embodiment of the present invention;

[0064] Figure 7 It is a structural diagram of a device for enhancing the quality of super-resolution reconstruction of fluorescence microscopic images provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0066] In order to solve the problem that low signal-to-noise ratio fluorescence microscopy images under weak conditions are prone to artifacts after reconstruction using super-resolution algorithms such as SRRF, and that the foreground fluorescence signal of the original fluorescence microscopy images under weak conditions is unevenly collected, resulting in truncation and invisibility of some structures after reconstruction using super-resolution algorithms such as sf-MSSR, an embodiment of the present invention provides a method and device for enhancing the super-resolution reconstruction quality of fluorescence microscopy images. The enhancement method and device can significantly improve the reconstruction quality of the super-resolution result image by filtering and light field correcting the low-quality fluorescence microscopy image and then performing super-resolution algorithm reconstruction.

[0067] Figure 1 It is a flow chart of a method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images provided by an embodiment of the present invention. Figure 2 1 is an example diagram of the gradual enhancement effect in the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images provided by an embodiment of the present invention. Figure 1 and Figure 2 As shown, the enhancement method provided in the embodiment includes the following steps:

[0068] S110, acquiring a fluorescence microscopic image under weak conditions.

[0069] In the embodiment, the fluorescence microscopy image obtained under weak conditions may have obvious noise in the background and invisible structures at the edge of the image, such as Figure 3 Neutron graph (a) row (1), Figure 2 The original image under the weak condition shown in Figure 3 Enlarged view of the white boxed portion in row (1) of the neutron image (a).

[0070] S120, performing denoising and enhancement operations on the fluorescence microscopy image to obtain a denoising and enhanced result image.

[0071] In the embodiment, a morphological filtering algorithm is used to perform denoising and enhancement operations on the fluorescence microscopy image under weak conditions to obtain a denoising and enhanced result image T w (f), the specific process is:

[0072]

[0073] Among them, T w (f) represents the result image after denoising and enhancement, f represents the fluorescence microscopy image, b represents the morphological filter kernel generated by the structural element, represents the morphological opening operation, represents the corrosion and expansion of the image f by the structural element b, ⊙ represents the morphological corrosion, Represents morphological dilation. The morphological opening operation can separate the bright pixels in the image, enhance the local low-brightness area, and smooth the boundaries of larger objects without significantly changing their area. Therefore, when the original image is subtracted from the result, the denoising and enhancement effect of retaining the foreground as much as possible can be achieved.

[0074] For example, the structure after denoising and enhancement by morphological filtering is as follows: Figure 3 As shown in row (1) of sub-image (b), the enlarged screenshot of the corresponding white frame is Figure 3 In the neutron image (b) row (2), we can see that the denoising enhancement effect is obvious.

[0075] S130, estimating a flat field image for the denoising and enhancement result image according to a two-dimensional polynomial orthogonal basis function set, and inversely deriving an image after light field correction.

[0076] In the embodiment, a further light field correction process is performed on the denoising and enhancement result image based on a two-dimensional polynomial orthogonal basis function set. It can be seen from the physical process of microscopic imaging that the captured image is mainly affected by the flat field compared to the real image. Therefore, if the imaging flat field can be accurately estimated, the real microscopic image can be inferred, thereby improving the image quality.

[0077] In an embodiment, estimating a flat field image from a denoising and enhancement result image according to a two-dimensional polynomial orthogonal basis function set includes:

[0078] Construct a two-dimensional polynomial orthogonal basis function set. From the separability of orthogonal polynomials, it can be seen that the two-dimensional polynomial kernel can be obtained from the one-dimensional polynomial kernel, that is:

[0079] P mn (x, y) = P m (x)P n (y), 0≤m≤M, 0≤n≤N

[0080] Among them, P m (x), P n (y) are one-dimensional M+1 and N+1 order orthogonal polynomials, respectively, and their forms are recursively derived from the following formula:

[0081] P0(x)=1

[0082] P1(x)=x

[0083]

[0084] Here, let M = N, then the formula P mn (x, y) = P m (x)P n (y) is a two-dimensional orthogonal polynomial of order N+1, which is used to estimate the flat-field image of the denoised and enhanced result image

[0085]

[0086] x i ,y j Represents the coordinate position in space, used to traverse the pixels in the image, i = 1, 2, .., l, j = 1, 2, .., s, l and s represent the height and width of the image in pixels respectively, a m(N+1)+n represents the coefficient, which is estimated by the least squares method.

[0087] Specifically, the least squares method estimates a m(N+1)+n The process is:

[0088] First, Written in matrix form:

[0089]

[0090] at this time Represented as a ls×1-order constant column vector, A is (N+1) 2 ×1-order coefficient column vector, composed of a m(N+1)+n Composition, P is ls×(N+1) calculated from the pixel position2 The column vector E of the fitting error at each pixel position is written as:

[0091]

[0092] The sum of squares of the error terms is:

[0093]

[0094] In order to minimize the MSE, we take partial derivatives of the coefficients in A and set them to 0 to obtain:

[0095]

[0096] Then the least squares estimate of coefficient A is

[0097] In solving for the coefficient a m(N+1)+n On the basis of Estimated flat field image The image I obtained by inverting the microscopic imaging process after light field correction is:

[0098]

[0099] Wherein, ε is an adjustment parameter, and its value is 1%-10% of the maximum value of the current image pixel.

[0100] For example, the result after light field correction is as follows: Figure 3 As shown in row (1) of sub-image (c), the enlarged screenshot of the corresponding white frame is Figure 3 In sub-image (c) row (2), it can be seen that the structure of the dark area at the edge of the fluorescence image is further highlighted, and the image quality is further improved.

[0101] S140, performing super-resolution reconstruction processing on the image after light field correction, and outputting an image with high quality and high spatial resolution.

[0102] Based on the image obtained after light field correction, a super-resolution reconstruction process is performed on the image after light field correction to output an image with high quality and high spatial resolution.

[0103] In the embodiment, it is proposed to use a spatial analysis algorithm of super-resolution radial fluctuation (SRRF) to perform single-frame super-resolution reconstruction processing on the image after light field correction to obtain a high-quality, high-resolution single-frame reconstruction result to demonstrate the quality enhancement effect of the present invention on the super-resolution image. The specific process is as follows:

[0104] First, the gradient G in the x and y directions is calculated based on the light field corrected image I(x,y) x and G y for:

[0105]

[0106] Then, the sub-pixels are divided in the pixel by Fourier interpolation algorithm to define a series of spatial position points (x c ,y c ) is used to calculate the radial degree and define the annular coordinate system (x i ′ ,y i ′ ), and through G x and G y Get the sub-pixel gradient G xi and G yi , the gradient line equation is obtained as:

[0107] (xx i ′ )G yi -(yy i ′ )G xi =0

[0108] Then we get (x c ,y c ) and the vertical distance from the gradient line is:

[0109]

[0110] The gradient line is defined at the position point (x c ,y c )'s convergence c i for:

[0111]

[0112] Finally, the quality-enhanced super-resolution image I is obtained SRRF (x,y) is:

[0113]

[0114] Where N represents the number of circular coordinate points, r xi ,r yi Represents the circular coordinate point (x i ′ ,y i ′ ) to (x c ,y c ) distance.

[0115] For example, further implementation results of the SRRF spatial analysis algorithm are shown in the following example: Figure 4 . Figure 4Neutron image (a) is the fluorescence microscopy image obtained under weak conditions and the enlarged image of the corresponding white frame; Figure 4 Neutron image (b) is the result of directly performing SRRF spatial analysis on the fluorescence microscopy image under weak conditions. Figure 4 It can be seen from row (2) of subimage (b) that the reconstruction result has obvious artifacts, and the structure on the right side of the image is almost invisible; Figure 4 The neutron image (c) is the result of performing SRRF spatial analysis on the image after S120 and S130. Figure 4 It can be seen from row (2) of subimage (c) that the artifacts of the reconstruction result are eliminated, and the structure on the right side of the image is revealed, and the quality of the super-resolution reconstruction result is significantly enhanced.

[0116] In the embodiment, it is also proposed to perform super-resolution reconstruction processing on the image after light field correction, including:

[0117] The single-frame mean shift super-resolution reconstruction (sf-MSSR) algorithm was performed on the light field corrected images using the MSSR reconstruction plug-in on the microscopic image processing software ImageJ to obtain high-quality and high-resolution single-frame reconstruction results.

[0118] For example, further implementation results of the sf-MSSR algorithm are shown in the following example: Figure 5 . Figure 5 Neutron image (a) is the fluorescence microscopy image obtained under weak conditions and the enlarged image of the corresponding white frame; Figure 5 Figure (b) shows the result of directly executing the sf-MSSR algorithm on the fluorescence microscopy image under weak conditions. Figure 5 It can be seen from row (2) of sub-image (b) that there are obvious discontinuities in the reconstruction results, and a large number of foreground structures on the right side of the image are almost invisible; Figure 5 Figure (c) shows the result of executing the sf-MSSR algorithm on the image after S120 and S130. Figure 5 From the neutron image (c) row (2), it can be seen that the discontinuity of the reconstruction results is significantly improved, the overall foreground signal is highlighted, and the quality of the super-resolution reconstruction results is significantly enhanced.

[0119] For example, Figure 6 Raw representation in neutron image (a) Figure 4 The grayscale normalized intensity curve distribution under the white oblique line indicated by the arrow in row (2) of the neutron image (a) shows that the SRRF spatial analysis and SRRF spatial analysis (enhanced) represent Figure 4The grayscale normalized intensity curve distributions of the corresponding positions in sub-image (b) row (2) and sub-image (c) row (2) are shown; it can be seen that SRRF spatial analysis and SRRF spatial analysis (enhancement) achieve almost equal resolution improvements, and the SRRF spatial analysis (enhancement) in the present invention suppresses noise artifacts.

[0120] Figure 6 Raw representation in neutron image (b) Figure 5 The grayscale normalized intensity curve distribution under the white oblique line indicated by the arrow in row (2) of the neutron image (a), sf-MSSR and sf-MSSR (enhanced) represent Figure 5 The grayscale normalized intensity curve distributions of the corresponding positions in sub-image (b) row (2) and sub-image (c) row (2) show that sf-MSSR and sf-MSSR (enhanced) achieve substantially equal resolution improvements, and sf-MSSR (enhanced) in the present invention increases the grayscale peak value of the dark area structure, allowing the dark area structure to be visualized.

[0121] The method for enhancing the quality of super-resolution reconstruction of fluorescence microscopic images provided in the above embodiment is to perform denoising and enhancement based on morphological filtering, perform light field correction based on orthogonal polynomials, perform single-frame super-resolution reconstruction verification through SRRF spatial analysis and sf-MSSR for the obtained fluorescence microscopic images under weak conditions, and obtain the quality enhancement result of the super-resolution image. It should be noted that the order of denoising and enhancement based on morphological filtering and light field correction based on orthogonal polynomials can be adjusted, that is, first perform light field correction based on orthogonal polynomials in S130, and then perform morphological filtering in S120.

[0122] The method for enhancing the super-resolution reconstruction quality of fluorescence microscopy images provided in the above embodiments can retain and enhance the foreground structure information of the original fluorescence image for fluorescence microscopy images under weak imaging conditions, significantly improve the quality of images reconstructed by the super-resolution algorithm, reduce reconstruction artifacts, and improve the continuity of foreground reconstruction; at the same time, it can also reduce the imaging conditions of the fluorescence microscopy image, thereby weakening the influence of phototoxicity and reducing fluorescence bleaching of the sample, which is beneficial to the long-term super-resolution reconstruction of living cells.

[0123] Based on the same inventive concept, the embodiment further provides a device 700 for enhancing the reconstruction quality of a super-resolution fluorescence microscopic image, comprising a data acquisition unit 710, a denoising enhancement unit 720, a correction unit 730, and a super-resolution reconstruction unit 740.

[0124] Among them, the data acquisition unit 710 is used to acquire the fluorescence microscopy image under weak conditions; the denoising and enhancement unit 720 is used to perform denoising and enhancement operations on the fluorescence microscopy image to obtain a denoised and enhanced result image; the correction unit 730 is used to estimate the flat field image of the denoised and enhanced result image according to a two-dimensional polynomial orthogonal basis function set, and inversely deduce to obtain the image after light field correction; the super-resolution reconstruction unit 740 is used to perform super-resolution reconstruction processing on the image after light field correction, and output a high-quality, high-spatial resolution image.

[0125] It should be noted that the device for enhancing the reconstruction quality of super-resolution fluorescence microscopic images provided in the above embodiments should be illustrated by the division of the above functional units when performing fluorescence microscopic image reconstruction quality. The above functions can be assigned to different functional units as needed, that is, the internal structure of the terminal or server is divided into different functional units to complete all or part of the functions described above. In addition, the device for enhancing the reconstruction quality of super-resolution fluorescence microscopic images provided in the above embodiments and the embodiment of the method for enhancing the super-resolution reconstruction quality of fluorescence microscopic images belong to the same concept. The specific implementation process is detailed in the embodiment of the method for enhancing the super-resolution reconstruction quality of fluorescence microscopic images, which will not be repeated here.

[0126] Based on the same inventive concept, an embodiment further provides a computing device, including a memory and a processor, wherein the memory is used to store a computer program, and when the processor executes the computer program, a method for enhancing the quality of super-resolution reconstruction of a fluorescence microscopic image provided in the above embodiment is implemented, including the following steps:

[0127] S110, acquiring fluorescence microscopy images under weak conditions;

[0128] S120, performing denoising and enhancement operations on the fluorescence microscopy image to obtain a denoising and enhanced result image;

[0129] S130, estimating a flat field image for the denoising and enhancement result image according to a two-dimensional polynomial orthogonal basis function set, and inversely deriving an image after light field correction;

[0130] S140, performing super-resolution reconstruction processing on the image after light field correction, and outputting an image with high quality and high spatial resolution.

[0131] In an embodiment, the memory may be a proximal volatile memory such as RAM, or a non-volatile memory such as ROM, FLASH, a floppy disk, a mechanical hard disk, etc., or a remote storage cloud. The processor may be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), that is, the above-mentioned method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images may be implemented by these processors.

[0132] Based on the same inventive concept, an embodiment further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is processed and executed, the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images described above is implemented, comprising the following steps:

[0133] S110, acquiring fluorescence microscopy images under weak conditions;

[0134] S120, performing denoising and enhancement operations on the fluorescence microscopy image to obtain a denoising and enhanced result image;

[0135] S130, estimating a flat field image for the denoising and enhancement result image according to a two-dimensional polynomial orthogonal basis function set, and inversely deriving an image after light field correction;

[0136] S140, performing super-resolution reconstruction processing on the image after light field correction, and outputting an image with high quality and high spatial resolution.

[0137] Among them, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0138] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images, comprising the following steps: Acquire fluorescence microscopy images under weak conditions; Denoising and enhancing the fluorescence microscopy image to obtain a denoised and enhanced result image; According to the two-dimensional polynomial orthogonal basis function set, the flat field image is estimated for the denoising and enhancement result image, and the image after light field correction is obtained by inverse deduction. The specific process is: construct a two-dimensional polynomial orthogonal basis function set. From the separability of the orthogonal polynomial, it can be seen that the two-dimensional polynomial kernel can be obtained from the one-dimensional polynomial kernel, that is: P mn (x,y)=P m (x)P n (y),0≤m≤M,0≤n≤N Among them, P m (x), P n (y) are one-dimensional M+1 and N+1 order orthogonal polynomials, respectively, and their forms are recursively derived from the following formula: P0(x)=1 P1(x)=x Here, let M = N, then the formula P mn (x, y) = P m (x)P n (y) is a two-dimensional orthogonal polynomial of order N+1, which is used to estimate the flat-field image of the denoised and enhanced result image x i ,y j Represents the coordinate position in space, used to traverse the pixels in the image, i = 1, 2, .., l, j = 1, 2, .., s, l and s represent the height and width of the image in pixels, respectively, a m(N+1)+n represents the coefficient, which is estimated by the least squares method; By solving the coefficient a m(N+1)+n Get the estimated flat field image The image I obtained by inverting the microscopic imaging process after light field correction is: Among them, ε is the adjustment parameter, which is 1%-10% of the maximum value of the current image pixel. w (f) represents the denoising and enhancement result image; The image after light field correction is subjected to super-resolution reconstruction to output a high-quality, high-spatial-resolution image, including: proposing a spatial analysis algorithm of super-resolution radial fluctuations to perform single-frame super-resolution reconstruction on the image after light field correction to obtain a high-quality, high-resolution single-frame reconstruction result. The specific process is as follows: First, the gradient G in the x and y directions is calculated based on the light field corrected image I(x,y) x and G y for: Then, the sub-pixels are divided in the pixel by Fourier interpolation algorithm to define a series of spatial position points (x c ,y c ) is used to calculate the radial degree and define the annular coordinate system (x i ′ ,y i ′ ), and through G x and G y Get the sub-pixel gradient G xi and G yi , the gradient line equation is obtained as: (x-x i ′ )G yi -(y-y i ′ )G xi =0 Then we get (x c ,y c ) and the vertical distance from the gradient line is: The gradient line is defined at the position point (x c ,y c )'s convergence c i for: Finally, the quality-enhanced super-resolution image I is obtained SRRF (x,y) is: Where N represents the number of circular coordinate points, r xi ,r yi Represents the circular coordinate point (x i ′ ,y i ′ ) to (x c ,y c ) distance.

2. The method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images according to claim 1, characterized in that: The morphological filtering algorithm is used to denoise and enhance the fluorescence microscopy image to obtain the denoised and enhanced result image T w (f), the specific process is: Among them, T w (f) represents the result image after denoising and enhancement, f represents the fluorescence microscopy image, b represents the morphological filter kernel generated by the structural element, represents the morphological opening operation, represents the corrosion and expansion of the image f by the structural element b, ⊙ represents the morphological corrosion, Indicates morphological expansion.

3. The method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images according to claim 1, characterized in that: The super-resolution reconstruction of the image after light field correction also includes: The MSSR reconstruction plug-in on the microscopic image processing software ImageJ was used to perform a single-frame mean shift super-resolution reconstruction algorithm on the image after light field correction to obtain high-quality, high-resolution single-frame reconstruction results.

4. A device for enhancing the quality of super-resolution reconstruction of fluorescence microscopic images, characterized in that: It includes data acquisition unit, denoising and enhancement unit, correction unit and super-resolution reconstruction unit. The data acquisition unit is used to acquire fluorescence microscopic images under weak conditions; The denoising and enhancing unit is used to perform denoising and enhancing operations on the fluorescence microscopy image to obtain a denoising and enhanced result image; The correction unit is used to estimate the flat field image of the denoising and enhancement result image according to the two-dimensional polynomial orthogonal basis function set, and inversely obtain the image after light field correction. The specific process is: construct a two-dimensional polynomial orthogonal basis function set. According to the separability of the orthogonal polynomial, it can be known that the two-dimensional polynomial kernel can be obtained from the one-dimensional polynomial kernel, that is: P mn (x,y)=P m (x)P n (y),0≤m≤M,0≤n≤N Among them, P m (x), P n (y) are one-dimensional M+1 and N+1 order orthogonal polynomials, respectively, and their forms are recursively derived from the following formula: P0(x)=1 P1(x)=x Here, let M = N, then the formula P mn (x, y) = P m (x)P n (y) is a two-dimensional orthogonal polynomial of order N+1, which is used to estimate the flat-field image of the denoised and enhanced result image x i ,y j Represents the coordinate position in space, used to traverse the pixels in the image, i = 1, 2, .., l, j = 1, 2, .., s, l and s represent the height and width of the image in pixels, respectively, a m(N+1)+n represents the coefficient, which is estimated by the least squares method; By solving the coefficient a m(N+1)+n Get the estimated flat field image The image I obtained by inverting the microscopic imaging process after light field correction is: Among them, ε is the adjustment parameter, which is 1%-10% of the maximum value of the current image pixel. w (f) represents the denoising and enhancement result image; The super-resolution reconstruction unit is used to perform super-resolution reconstruction processing on the image after light field correction, and output a high-quality, high-spatial-resolution image, including: proposing to use a spatial analysis algorithm of super-resolution radial fluctuations to perform single-frame super-resolution reconstruction processing on the image after light field correction, so as to obtain a high-quality, high-resolution single-frame reconstruction result, the specific process is: First, the gradient G in the x and y directions is calculated based on the light field corrected image I(x,y) x and G y for: Then, the sub-pixels are divided in the pixel by Fourier interpolation algorithm to define a series of spatial position points (x c ,y c ) is used to calculate the radial degree and define the annular coordinate system (x i ′ ,y i ′ ), and through G x and G y Get the sub-pixel gradient G xi and G yi , the gradient line equation is obtained as: (x-x i ′ )G yi -(y-y i ′ )G xi =0 Then we get (x c ,y c ) and the vertical distance from the gradient line is: The gradient line is defined at the position point (x c ,y c )'s convergence c i for: Finally, the quality-enhanced super-resolution image I is obtained SRRF (x,y) is: Where N represents the number of circular coordinate points, r xi ,r yi Represents the circular coordinate point (x i ′ ,y i ′ ) to (x c ,y c ) distance.

5. A computing device comprising a memory and a processor, wherein the memory is used to store a computer executable program for performing a process for enhancing the quality of super-resolution fluorescence microscopy image reconstruction, characterized in that: When the processor executes the computer executable program, the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is processed and executed, the method for enhancing the quality of super-resolution reconstruction of fluorescence microscopy images according to any one of claims 1 to 3 is implemented.

Images