A method for localizing fluorescent molecules

Abstract

The application discloses a fluorescent molecule positioning method, comprising the following steps: acquiring a single-frame original image of a living cell to be observed; denoising the single-frame original image; performing interpolation processing on the denoised single-frame original image according to a preset interpolation number; replacing the interpolated single-frame original image to obtain a new image; and reserving the largest local extreme point in the new image, which is the position of the fluorescent molecule. The method can realize real-time visual non-destructive super-resolution microscopic imaging of living cells and intracellular structures, and accurately determine the position of the fluorescent molecule.

Description

Technical Field

[0001] This invention relates to the field of live biological sample observation technology, and in particular to a method for localizing fluorescent molecules. Background Technology

[0002] Far-field optical microscopy in the visible light band offers advantages such as being non-contact, non-destructive, and capable of probing the interior of samples, making it an important tool for studying organs, tissues, and cells. Due to diffraction, a point source light source passing through a microscopy imaging system cannot form a point image on the focal plane; instead, it forms a diffuse spot (i.e., an Airy disc). Due to the diffraction limit, the lateral and longitudinal resolutions of point source microscopy are only 200 nm and 500 nm, respectively.

[0003] While electron and X-ray microscopy can achieve nanometer or even higher resolutions, they are difficult to use for observing live biological samples. Near-field imaging, based on evanescent field detection, can reveal local changes at distances much smaller than the wavelength, achieving nanometer-level resolution. Atomic force microscopy and scanning tunneling microscopy also scan along the surface of an object and reconstruct the image. Although these techniques avoid the Abbe resolution limit caused by far-field diffraction, they are limited to imaging cell membrane surfaces and cannot be used for imaging living cells and intracellular structures.

[0004] For densely overlapping fluorescent molecules, sparse excitation and time-division imaging are employed. Thousands of raw images of discretely randomly distributed fluorescent molecules are acquired. In a single raw image, only a small number of sparsely distributed fluorescent molecules emit photons. Within the diffraction-limited range, only a single fluorescent molecule is excited.

[0005] Therefore, based on existing observations of live biological samples, how to obtain super-resolution microscopic images of living cells and their interiors that break through the diffraction limit, and accurately determine the location of fluorescent molecules within living cells, has become a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of the above problems, the present invention proposes a fluorescent molecule localization method that at least solves some of the above technical problems. This method can generate super-resolution microscopic images of living cells that break through the diffraction limit and accurately determine the position of fluorescent molecules in living cells.

[0007] This invention provides a method for localizing fluorescent molecules, comprising the following steps:

[0008] S1. Acquire a single-frame raw image of the living cells to be observed;

[0009] S2. Denoise the single-frame original image;

[0010] S3. Based on the preset number of interpolation steps, perform interpolation processing on the denoised single-frame original image, and then replace the interpolated single-frame original image to obtain a new image.

[0011] S4. Based on the new image, retain the largest local extremum point, which is the location of the fluorescent molecule.

[0012] Furthermore, in step S2, three-dimensional block matching filtering or broad-spectrum denoising is used to denoise the single-frame original image.

[0013] Further, in step S3, the interpolated single-frame original image is permuted to obtain a new image, including:

[0014] S31. Obtain the maximum and minimum grayscale values ​​of each pixel in the interpolated single-frame original image, and calculate the threshold.

[0015] S32. Subtract the threshold from the pixels of the interpolated single-frame original image, and replace the pixel values ​​less than 0 with 0 to obtain a new image.

[0016] Furthermore, the cri is calculated using the following formula:

[0017] cri=((max-min)*c+min)

[0018] In the above formula, cri represents the threshold; max represents the maximum gray value of each pixel in the interpolated original single-frame image; min represents the minimum gray value of each pixel in the interpolated original single-frame image; c represents a preset coefficient, and the value of c ranges from [0-1].

[0019] Further, in step S4, retaining the largest local extremum point based on the new image includes:

[0020] S41. Compare each non-zero pixel in the new image with a preset pixel that is adjacent to the pixel; if all pixels are greater than the preset pixel, then retain the pixel as a local extreme point.

[0021] S42. Compare the local extreme point with other local extreme points and retain the local extreme point with the largest value; the other local extreme points are other local extreme points within the range of the full width at half maximum or the standard deviation of the point spread function centered on the local extreme point.

[0022] Furthermore, step S4 also includes:

[0023] Subtract the estimated background noise value of the new image from the largest retained local extremum point to obtain a new local extremum;

[0024] Based on the new local extrema and the maximum pixel value of the Gaussian or Bessel function corresponding to the interpolated pixel size of the new image, the number of photons emitted by the fluorescent molecule is calculated and output.

[0025] Furthermore, the formula for calculating the number of photons is:

[0026] y = maxgb / maxk*x

[0027] In the above formula, y represents the number of photons emitted by the fluorescent molecule; maxgb represents the maximum pixel value of the Gaussian or Bessel function corresponding to the interpolated pixel size of the new image; maxk represents the new local extremum; and x represents the theoretical value of the number of photons emitted by the fluorescent molecule.

[0028] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following:

[0029] This invention provides a method for locating fluorescent molecules, comprising the following steps: acquiring a single-frame original image of a living cell to be observed; denoising the single-frame original image; interpolating the denoised single-frame original image according to a preset number of interpolation iterations; replacing the interpolated single-frame original image to obtain a new image; retaining the largest local extremum point in the new image, which is the location of the fluorescent molecule. This method enables real-time, non-destructive super-resolution microscopic imaging of living cells and intracellular structures, and accurately determines the location of fluorescent molecules.

[0030] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0031] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0032] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0033] Figure 1 This is a flowchart of the fluorescent molecule localization method provided in an embodiment of the present invention;

[0034] Figure 2 This is a 7*7 original image of a single fluorescent molecule diffuse spot formed according to an embodiment of the present invention;

[0035] Figure 3(a) is a noise-free original image containing only a single fluorescent molecule provided in an embodiment of the present invention;

[0036] Figure 3(b) shows the original image after adding Poisson noise and Gaussian noise with a variance of 0.01, provided in the embodiment of the present invention.

[0037] Figure 3(c) shows the original image after WSD denoising provided in an embodiment of the present invention;

[0038] Figure 3(d) shows the original image after denoising using BM3D according to an embodiment of the present invention;

[0039] Figure 3(e) shows the noise-free original image after interpolation provided in the embodiment of the present invention;

[0040] Figure 3(f) shows the noisy original image after interpolation provided in the embodiment of the present invention;

[0041] Figure 3(g) shows the original image after WSD denoising of the interpolated noisy original image provided in an embodiment of the present invention;

[0042] Figure 3(h) shows the original image after denoising the noisy original image after interpolation based on BM3D, provided by an embodiment of the present invention. Detailed Implementation

[0043] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0044] This invention provides a method for localizing fluorescent molecules, referring to... Figure 1 As shown, it includes the following steps:

[0045] S1. Input N frames of original images of the living cells to be observed, and extract a single frame of original image;

[0046] S2. Denoise the original single-frame image;

[0047] S3. Within the range of preset interpolation times, perform interpolation processing on the denoised single-frame original image, and replace the interpolated single-frame original image to obtain a new image.

[0048] S4. Based on the new image, retain the largest local extremum point, which is the location of the fluorescent molecule.

[0049] In a noise-free environment, when the original image contains only a single fluorescent molecule, the center of the diffuse spot in the original image is the location of the fluorescent molecule. If the resolution of the original image is high enough, the pixels are small enough, and the noise is low enough, then the pixel with the highest value in the original image is the true location of the fluorescent molecule. This embodiment provides a fluorescent molecule localization method that enables real-time, non-destructive super-resolution microscopic imaging of living cells and intracellular structures, and accurately determines the location of fluorescent molecules. Through denoising and interpolation processing of the original image, the noise and pixel size of the original image are minimized, and the location of the brightest pixel is the location of the fluorescent molecule.

[0050] The fluorescent molecule localization method provided in this embodiment will be explained and verified in detail through several specific examples below:

[0051] The overall execution process is as follows:

[0052] Input N frames of original images; set the iteration number j to the range of [1, N];

[0053] In the j-th iteration, perform the following steps;

[0054] 1.1) Denoise the original image of a single frame using denoising algorithms such as BM3D or WSD;

[0055] 1.2) Perform interpolation on the denoised single-frame original image, with the number of interpolations ranging from 0 to 1 million. After interpolation, each pixel can be divided into 1 to 1 million small pixels.

[0056] 1.3) Obtain the maximum value max and minimum value min of each pixel grayscale value of the interpolated single frame image, and obtain the threshold cri = ((max-min)*c+min), where c is a preset coefficient, and the value range of c is [0-1] or cri is a constant; wherein, the threshold cri can be directly taken as a constant based on personal experience, or it can be calculated according to the above formula. Technicians can choose flexibly, and this embodiment does not limit it.

[0057] 1.4) Subtract cri from the pixels of a single frame image, and replace pixel values ​​less than 0 with 0 to obtain a new image;

[0058] 1.5) For non-zero pixels in the new image, each pixel is compared with its neighboring (m) pixels. 2 Compare -1) pixels, if it is compared with the nearest (m) pixels 2 If all -1) pixels are large, then they are retained as local extrema. The value of m ranges from 3 to 1,000,000. m is an odd number (the area of ​​the square centered on a certain pixel is m). 2 (There are 10 pixels, m can only be an odd number because there is only one pixel in the very center).

[0059] 1.6) Analyze local extrema one by one. Using local extrema as the center, search for other local extrema within the full width at half maximum (FWHM) of the point spread function or the standard deviation of the point spread function. Compare the local extrema with other local extrema and retain only the largest local extrema. The full width at half maximum (FWHM), also known as the half-height full width, half-peak full width, or half-height width, refers to the distance between two points whose function values ​​are equal to half the peak value within a peak of the function.

[0060] 1.7) Subtract the background noise estimate d of the new image after interpolation from all retained local extrema points to obtain the new local extrema maxk. The value range of d is [min, ((max-min)*c+min)] or d is a constant. It can be flexibly selected according to the actual situation. This embodiment does not limit it.

[0061] 1.8) Obtain the maximum pixel value maxgb of the Gaussian or Bessel function of the pixel size corresponding to the interpolation of the new image. Calculate the actual number of photons emitted by the fluorescent molecule using maxgb / maxk*the theoretical value of the number of photons emitted by the fluorescent molecule, and obtain the corresponding single-frame super-resolution image.

[0062] Super-resolution images from each frame are superimposed to generate a single super-resolution microscopic image, which is then output.

[0063] The method will be explained in detail below:

[0064] First, original images were acquired to generate noise-free original images. Experimental parameters used were from the TubulinsLong Sequence dataset on the EPFL website: microscope numerical aperture (NA) 1.3, wavelength fluorescence 690nm, and effective CCD pixel size 100nm. (Refer to...) Figure 2 The image shown is a 7x7 original image formed by a single fluorescent molecule diffuse spot, with each pixel measuring 100nm x 100nm. The original image with a pixel size of 100nm is divided into an oversampled grid with a grid size of 12.5nm and an oversampling factor of 8 (the ratio between pixel size and grid spacing). In the simulation experiment, the fluorescent molecules are located within the grid; the 16x16 grid represents the distribution of the fluorescent molecules. Fluorescent molecules at different grid positions can generate 256 frames of noise-free original images.

[0065] Next, background, Poisson noise, and Gaussian noise with a variance of 0.01 are added to each generated noiseless original image. After 100 iterations, each noiseless original image generates 100 noisy original images. A total of 25,600 noisy original images are generated. Then, BM3D and WSD are used to denoise the noisy original images. Finally, interpolation processing is performed on each noiseless original image, the noisy original image, and the denoised original image.

[0066] Referring to Figures 3(a), 3(b), 3(c), 3(d), 3(e), 3(f), 3(g), and 3(h), an image is formed by interpolation and denoising of a single frame of the original image. Scale bar: 1 μm. Figures 3(a), 3(b), 3(c), and 3(d) are the images before interpolation, and Figures 3(e), 3(f), 3(g), and 3(h) are the images after interpolation. WSD and BM3D are two different denoising methods. Block-matching and 3D filtering (BM3D) and widespectrum denoising (WSD) are two algorithms that can effectively remove noise from the original image. Interpolation of the original image can effectively reduce the pixel size.

[0067] Specifically, referring to Figure 3(a), a noise-free original image containing only a single fluorescent molecule, with a size of 20*20, is shown in a frame based on parametric simulation. In this simulation, each fluorescent molecule emits 3,000 photons, and each pixel emits 64 photons of the background.

[0068] Referring to Figure 3(b), this is the original image after adding Poisson noise and Gaussian noise with a variance of 0.01. The original image containing a single fluorescent molecule has a maximum pixel value of approximately 364 (i.e., 364 fluorescent molecules). Due to the small number of photons, the noise is significant. The diffuse spot and background are locally blurred, and the SNR (signal-to-noise ratio) is only 3.986 dB.

[0069] Referring to Figures 3(c) and 3(d), the original images were denoised using WSD (broadband denoising) and BM3D (3D block-matched filtering denoising), respectively, resulting in an SNR improvement of 6.341 dB and 20.223 dB. The denoising effect is significant. In Figures 3(c) and 3(d), due to the large pixel size (up to 100 nm), the pixels in the diffuse spots exhibit a clear discrete block structure. The background pixels in Figure 3(b) also show a clear discrete block structure. Although the background in Figure 3(c) is relatively smooth, the discrete block structure of the pixels is still very obvious.

[0070] Figures 3(e), 3(f), 3(g), and 3(h) show the original images after cubic interpolation. That is, one pixel in the original image is bicubic interpolated into 8*8 pixels. Because the interpolated pixels are very small, the speckle and background pixels are no longer discrete, blocky structures, but rather nearly smooth, continuous structures. As shown in Figure 3(e), the SNR of the noise-free original image after interpolation is 18.769 dB. As shown in Figure 3(f), the SNR of the noisy original image is 5.812 dB, an improvement of 1.826 dB. As shown in Figure 3(g), the SNR of the noisy original image after WSD denoising is almost unchanged from that after WSD denoising (Figure 3(c)), only increasing from 10.327 dB to 10.474 dB. As shown in Figure 3(h), the SNR of the noisy original image after interpolation and denoising using BM3D is reduced by about 5dB compared to the SNR of the noisy original image after denoising using BM3D (Figure 3(d)), but its SNR is still as high as 19.183dB. The background of Figures 3(f) and 3(g) shows a smooth, mottled structure.

[0071] Furthermore, after denoising and interpolation, if the noise and pixel count are sufficiently small, the maximum pixel position of the original image containing only a single fluorescent molecule is the true position location of the fluorescent molecule. The number of photons in the fluorescent molecule can then be calculated using the following formula:

[0072]

[0073] In the above formula, num is the calculated number of photons in the fluorescent molecule; Pix max It represents the maximum value of the image pixels after interpolation; num ref This is the actual number of photons emitted by each fluorescent molecule; Pix ref It is the maximum value of the actual image pixels.

[0074] Photon count accuracy is calculated using the following formula:

[0075]

[0076] In the above formula, PNP represents the photon count precision; num represents the calculated number of photons in the fluorescent molecule; num ref It represents the actual number of photons emitted by each fluorescent molecule.

[0077] Photon count accuracy is used to assess the error in photon count calculation. The mean and standard deviation of the interpolated positioning accuracy and photon count accuracy are shown in Table 1. Here, Raw refers to the original image with noise after interpolation; Noiseless refers to the original image without noise after interpolation; WSD and BM3D are two different denoising algorithms for the original image with noise after interpolation. The mean positioning accuracy before denoising is 29.742 nm, and the standard deviation is 17.813 nm. The mean positioning accuracy after denoising with BM3D and WSD are 18.469 nm and 17.3778 nm, respectively, with standard deviations of 10.053 nm and 8.9877 nm, respectively. The positioning accuracy is improved by approximately 10 nm, and the standard deviation is improved by approximately 8 nm. BM3D and WSD have similar capabilities in improving positioning accuracy. The mean positioning accuracy of the noiseless original image is 8.8388 nm. The positioning accuracy of the original image after denoising is nearly twice that of the ideal value (noiseless original image).

[0078] Table 1. Positioning accuracy and photon count accuracy of a single molecule

[0079]

[0080] The mean photon count accuracy after denoising using BM3D and WSD were 17.36 and 16.8, respectively, with standard deviations of 5.77 and 7.52. The mean photon count accuracy of the noise-free original image was 16.1, with a standard deviation of 11.6. The photon count accuracy of the denoised original image was close to the ideal value (noiseless original image), with a smaller standard deviation. BM3D and WSD showed similar improvements in positioning accuracy. The mean photon count accuracy of the noisy original image (Raw) was only 6.15, exhibiting the worst positioning accuracy; therefore, the noisy original image had the worst positioning ability. After denoising and interpolation, BM3D and WSD effectively improved the positioning accuracy of single molecules while maintaining a basically unchanged photon count accuracy.

[0081] Finally, the fluorescent molecule localization method provided in this embodiment was validated using Gaussian fitting. Single-molecule localization, i.e., fluorescent molecule localization, was performed on 2500 frames of original images from the Tubulins Long Sequence dataset using Gaussian fitting, resulting in super-resolution images and magnified local images. The super-resolution images were obtained based on the original images before interpolation, the super-resolution images obtained after denoising the original images using BM3D and WSD respectively, the super-resolution images obtained after interpolation of the original images, and the super-resolution images obtained after denoising the original images using BM3D and WSD respectively and then interpolating. Comparative analysis of the super-resolution images and magnified local images showed that the localization capabilities were very similar, with no significant differences.

[0082] Before interpolation, the localization speed after denoising is significantly improved. Based on BM3D and WSD, the localization time is improved from 324.134s to 289.83s and 306.447s, respectively. After interpolation, due to the increase in data volume, the computation time is greatly extended, reaching 1325.86s, 1509.188s, and 860.745s, respectively.

[0083] Single-molecule localization methods based on Gaussian fitting inherently possess strong noise resistance. Therefore, denoising has little impact on their localization ability. Interpolation not only fails to significantly improve localization ability but also greatly extends the processing time.

[0084] Furthermore, based on the super-resolution image obtained from the original image before interpolation, and the super-resolution images obtained after applying BM3D and WSD denoising to the original image respectively, it is concluded that the microtubes in the super-resolution image are very thick due to the large pixel size of the original image.

[0085] The super-resolution image is obtained by interpolating the original image, and the super-resolution image is obtained by denoising the original image using BM3D and WSD respectively, followed by interpolation. Compared with the original image, the microtubules in the super-resolution image are very fine, and the localization effect is comparable to that of the Gaussian fitting method.

[0086] Comparative analysis of magnified local images of the super-resolution images reveals similar localization effects from both denoising methods, with intracellular microtubule structures appearing clearer than in the original images before denoising. This demonstrates that both denoising and interpolation contribute to improving single-molecule (fluorescent molecule) localization. After interpolation, the localization times based on BM3D and WSD are 135.332 s and 187.918 s, respectively, significantly shorter than the 324.134 s and 289.83 s based on Gaussian fitting. Therefore, the fluorescence molecule localization method (DIL) based on denoising, interpolation, and local maxima can be considered an effective new method for fluorescence molecule localization. Simulation results show that the single-molecule localization accuracy provided in this embodiment can reach approximately 18 nm. Real-world experimental results demonstrate that the fluorescence molecule localization method provided in this embodiment has comparable localization capabilities to the Gaussian fitting algorithm, but with significantly reduced time consumption.

[0087] The simulation results provided in this embodiment show that the positions of fluorescent molecules can be easily determined after denoising and interpolation, achieving good positioning accuracy and photon count accuracy. Although fluorescent molecules are sparsely distributed in the actual original image, the number of fluorescent molecules is usually multiple, or even more. The method provided in this embodiment can easily extract the local maxima from the denoised and interpolated original image. The location of the local maxima is the position of the fluorescent molecule, and the number of photons emitted by the fluorescent molecule can be calculated from it.

[0088] The fluorescent molecule localization method provided in this embodiment is based on the process of observing the dynamic microscopic life of living cells in the visible light band, enabling real-time, non-destructive super-resolution microscopic imaging. The position of fluorescent molecules in the original image can be precisely determined through the diffuse spot method. Superimposing the localization points of all fluorescent molecules yields a super-resolution microscopic image that breaks the diffraction limit. By acquiring thousands of frames of original images showing the discrete and random distribution of fluorescent molecules, real-time, non-destructive super-resolution microscopic imaging can be achieved to accurately determine the position of fluorescent molecules within living cells. Superimposing the localization points of all fluorescent molecules yields a super-resolution microscopic image that breaks the diffraction limit, thereby enabling super-resolution microscopic imaging of intracellular structures such as microtubules, microfilaments, and mitochondria.

[0089] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for localizing fluorescent molecules, characterized in that, Includes the following steps: S1. Acquire a single-frame raw image of the living cells to be observed; S2. Denoise the single-frame original image; S3. Based on the preset number of interpolation steps, perform interpolation processing on the denoised single-frame original image, and then replace the interpolated single-frame original image to obtain a new image. S4. Based on the new image, retain the largest local extremum point, which is the location of the fluorescent molecule; In step S2, three-dimensional block matching filtering or broad-spectrum denoising is used to denoise the single-frame original image, respectively. The interpolated single-frame original image is permuted to obtain a new image, including: S31. Obtain the maximum and minimum grayscale values ​​of each pixel in the interpolated single-frame original image, and calculate the threshold. S32. Subtract the threshold from the pixels of the interpolated single-frame original image, and replace pixel values ​​less than 0 with 0 to obtain a new image; In step S4, retaining the largest local extremum point based on the new image includes: S41. For each non-zero pixel in the new image, compare it pixel by pixel with the m neighboring pixels. 2 -1 pixels are compared; if all of the pixels are greater than the nearest m 2 If -1 pixel is found, then the pixel is retained as a local extremum point. The value of m ranges from 3 to 1,000,000, and m is an odd number. S42. Analyze each of the local extreme points. Using the local extreme point as the center, search for other local extreme points within the range of the full width at half maximum (FWHM) or the standard deviation of the point spread function. Compare the local extreme point with other local extreme points and retain the local extreme point with the largest value, which is the position of the fluorescent molecule. Step S4 further includes: The new local extremum is obtained by subtracting the estimated background noise value of the interpolated new image from the largest retained local extremum point. Based on the new local extrema and the maximum pixel value of the Gaussian or Bessel function of the pixel size after interpolation of the new image, the actual number of photons emitted by the fluorescent molecule is calculated and output to obtain the corresponding single-frame super-resolution image; the super-resolution images of each frame are superimposed to generate a super-resolution microscopic image and output. The formula for calculating the number of photons is: y = maxgb / maxk*x In the above formula, y represents the number of photons emitted by the fluorescent molecule; maxgb represents the maximum pixel value of the Gaussian or Bessel function corresponding to the interpolated pixel size of the new image; maxk represents the new local extremum; and x represents the theoretical value of the number of photons emitted by the fluorescent molecule.

2. The fluorescent molecule localization method as described in claim 1, characterized in that, The threshold is calculated using the following formula: cri=((max-min)*c+min) In the above formula, cri represents the threshold; max represents the maximum gray value of each pixel in the interpolated original single-frame image; min represents the minimum gray value of each pixel in the interpolated original single-frame image; c represents a preset coefficient, and the value of c ranges from [0-1].

Images