Two-dimensional fluorescent microscopic image self-supervision denoising method based on image representation

Abstract

The invention discloses a two-dimensional fluorescent microscopic image self-supervision denoising method based on image representation, which combines INR and a self-supervision neural network structure, and aims to solve the problem of serious image noise caused by low-dose irradiation and other factors in fluorescent microscopic imaging. The method comprises the following steps: firstly, carrying out preliminary modeling and denoising on a two-dimensional fluorescent microscopic image by utilizing an INR model, and effectively recovering an image structure and continuity; and then contrast enhancement is carried out on an INR result, and the INR result is guided as a pseudo reference image to be combined with a network model based on U-Net to carry out self-supervised denoising training, so that the image definition and the detail reduction degree are further improved. The method does not depend on paired clear images, high cost of traditional supervised learning is avoided, and the imaging requirements under the conditions of high time resolution and low irradiation dose are kept at the same time. Experiments prove that the method achieves an excellent image quality improvement effect on a plurality of real fluorescence image data sets, and has good practicability and popularization value.

Description

Technical Field

[0001] This invention belongs to the field of fluorescence microscopy imaging technology, specifically relating to a self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation. Background Technology

[0002] Fluorescence microscopy has been widely applied in various biomedical research fields, including in vivo imaging and cell structure observation. In two-dimensional fluorescence microscopy, high-intensity background noise is often generated due to factors such as the weak fluorescence signal of the sample itself, limited excitation light exposure time, and light attenuation caused by imaging depth. Gaussian noise is the most typical type of noise, significantly affecting the signal-to-noise ratio, detailed structural reconstruction, and the effectiveness of subsequent image processing tasks.

[0003] Traditional image denoising methods, such as Gaussian filtering, median filtering, and nonlocal means algorithms, can reduce high-frequency noise in images under certain conditions, but they often sacrifice image details and edge information, resulting in image blurring. In recent years, with the development of deep learning, neural network-based image denoising methods have become a research hotspot. For example, the literature [K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, "Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising," in IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, July 2017] proposed DnCNN (Denoising Convolutional Neural Network), which utilizes residual learning and batch normalization mechanisms to effectively improve the modeling ability of different types of noise, outperforming traditional models. This method models noise by training a deep convolutional network and has been successfully applied to the removal of various noise types, such as Gaussian and Poisson noise. The literature [Ronneberger, O., Fischer, P., Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds) Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015] proposed U-Net (U-shaped convolutional neural network), whose encoder-decoder structure has been widely used in low-level visual tasks such as image denoising, and is particularly suitable for small sample scenarios. The skip connection design preserves the spatial structure of the image and effectively prevents the loss of feature information. Its derived network structures have all shown good denoising performance in the fields of medical and biological imaging.

[0004] However, the aforementioned methods generally rely on supervised learning frameworks, requiring training with pairs of "clean-noisy" images, which is often difficult to obtain in real fluorescence microscopy imaging. On the one hand, due to the non-reproducibility and phototoxicity of biological samples, it is impossible to perform repeated imaging to obtain "clear images"; on the other hand, it is also difficult to construct sufficiently representative synthetic datasets under real experimental conditions. Therefore, unsupervised or self-supervised image denoising methods have become a research focus in recent years. For example, the literature [T. Huang, S. Li, X. Jia, H. Lu and J. Liu, "Neighbor2Neighbor: A Self-Supervised Framework for Deep Image Denoising," in IEEE Transactions on Image Processing, vol. 31, pp. 4023-4038, 2022] proposes a self-supervised training strategy based on the redundancy information within the image. By randomly sampling neighboring pixels, label-free denoising is achieved. This method can train a competitive neural network denoising model without the need for clean images, and is particularly suitable for real biological imaging data.

[0005] Nevertheless, these methods are often limited by the insufficient ability of network structures to model image distributions, and their recovery capabilities remain limited for fluorescent images with complex structures and blurred edges. The emerging INR (Implicit Neural Representation) technique in recent years has provided a new approach to image denoising. INR encodes images as spatial functions (such as mappings from coordinates to pixel values) without requiring explicit pixel grid input, thus enabling the reconstruction of image structures with higher spatial continuity. It is particularly adept at handling structural recovery tasks under the influence of Gaussian noise. Summary of the Invention

[0006] In view of the above, the present invention provides a self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation. First, the INR model is used to perform preliminary modeling and restoration of high-noise images, which significantly improves the overall structural clarity. Then, a model based on the U-Net structure is introduced to further optimize the INR results without relying on clean images, thereby obtaining clearer and more detailed images.

[0007] A self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation includes the following steps:

[0008] (1) Fluorescently label biological samples and use a fluorescence microscope to acquire images of biological samples under low light dose imaging conditions to obtain a series of two-dimensional fluorescence microscopic images with low signal-to-noise ratio.

[0009] (2) The pre-trained INR model is used to perform image representation on the two-dimensional fluorescence microscopy image with low signal-to-noise ratio, generate a pre-denoised image, and then perform exponential image contrast enhancement processing on the pre-denoised image to obtain the enhanced pre-denoised image.

[0010] (3) The two-dimensional fluorescence microscopy image is downsampled twice using a neighborhood pixel extraction strategy to obtain a set of downsampled image pairs. The same strategy is used to downsample the corresponding enhanced pre-denoised image of the two-dimensional fluorescence microscopy image twice to obtain a set of enhanced pre-denoised downsampled image pairs.

[0011] (4) According to step (3), perform the same operation on all two-dimensional fluorescence microscopy images to obtain multiple sets of downsampled image pairs and their corresponding enhanced pre-denoised downsampled image pairs. Each downsampled image pair and its corresponding enhanced pre-denoised downsampled image pair is used as a set of samples, and then all samples are divided into training set and validation set.

[0012] (5) Construct a deep learning network model, train the model using the training set, and test the model performance using the validation set;

[0013] (6) Input the newly acquired low signal-to-noise ratio two-dimensional fluorescence microscopy image into the trained model, and a high-quality reconstructed image can be output.

[0014] Furthermore, the low-dose imaging condition in step (1) refers to low excitation light intensity or short exposure time, and the acquired two-dimensional fluorescence micrograph contains a large amount of noise. The imaging model expression is as follows:

[0015] i = P(o) + N

[0016] Where: o represents two-dimensional noise-free image data, N represents Gaussian noise, P() represents Poisson noise, and i represents a two-dimensional fluorescence microscopy image with low signal-to-noise ratio.

[0017] Furthermore, the INR model in step (2) adopts a multilayer perceptron structure. By learning the image coordinate encoding, it utilizes the continuity of local structures within the image to achieve pre-denoising and generate a pre-denoised image. The specific expression is as follows:

[0018] I(x,y)=MLP(x,y)→i(x,y)

[0019] Where: (x,y) represents the spatial coordinates in the image, i(x,y) represents the pixel intensity at spatial coordinates (x,y) in the two-dimensional fluorescence microscopy image, MLP represents the multilayer perceptron used in the INR model, → indicates approaching, and I(x,y) represents the pixel intensity at spatial coordinates (x,y) in the pre-denoised image generated by the INR model.

[0020] Furthermore, the specific expression for the exponential image contrast enhancement processing in step (2) is as follows:

[0021] I′(x,y)=α·α (β·I(x,y)) -γ

[0022] Where: I(x,y) represents the pixel intensity at spatial coordinate (x,y) in the pre-denoised image generated by the INR model, I′(x,y) is the pixel intensity at spatial coordinate (x,y) in the enhanced pre-denoised image, α, β, and γ are hyperparameters for adjusting image brightness and contrast, and e is a natural constant.

[0023] Furthermore, the neighborhood pixel extraction strategy in step (3) involves dividing the two-dimensional fluorescence microscopy image into multiple 2×2 pixel blocks, randomly selecting one pixel from each pixel block, and stitching the extracted pixels together to obtain a downsampled image with a length and width that are half of the original image. Repeating this downsampling operation twice yields a set of downsampled image pairs.

[0024] Furthermore, the deep learning network model in step (5) adopts the U-Net model structure, using any one of the downsampled images in the downsampled image pair as input and the other downsampled image as the label for self-supervised training.

[0025] Furthermore, in step (5), the following loss function is used to train the model:

[0026]

[0027] in: The total loss function of the model, For self-supervised loss function, As the auxiliary loss function, λ1 and λ2 are the set weight coefficients, g1 and g2 are downsampled image pairs in the training set samples, and g1′ and g′2 are enhanced pre-denoised downsampled image pairs in the training set samples, corresponding to g1 and g2 respectively, f θ (g1) represents the reconstructed image obtained by inputting g1 into the model and outputting it.

[0028] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the above-described self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation.

[0029] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation.

[0030] This invention innovatively combines implicit neural representation with a self-supervised denoising framework to construct a two-stage processing mechanism suitable for high-noise fluorescence microscopy images. First, it uses INR to perform preliminary modeling and restoration of image structures, effectively improving the basic image clarity and detail continuity. Then, through self-supervised optimization of the enhanced image, it further improves image quality, significantly outperforming traditional U-Net-based denoising methods. Without relying on clean images, this invention effectively improves the spatial resolution and biological structure recognizability of images while maintaining temporal resolution and irradiation dose. It provides high-quality data support for the post-processing and intelligent analysis of fluorescence microscopy imaging data, demonstrating broad application prospects and practical value. Attached Figure Description

[0031] Figure 1 This is a schematic flowchart of the self-supervised denoising method for two-dimensional fluorescence microscopy images of the present invention.

[0032] Figure 2 This is a schematic diagram of the network architecture of the self-supervised denoising method for two-dimensional fluorescence microscopy images of the present invention.

[0033] Figure 3 This diagram illustrates the comparison between the reconstruction results of traditional methods and the present invention under high-noise simulation images. The top left image is the original simulation image, the top right image is the noisy image, the bottom left image is the reconstructed image using the self-supervised denoising method based on U-Net, and the bottom right image is the reconstructed image using the method of the present invention.

[0034] Figure 4 The diagram shows a comparison of experimental results between traditional methods and the present invention on noisy medical images. The top image is the noisy image, the bottom left image is the image reconstructed by the self-supervised denoising method based on U-Net, and the bottom right image is the image reconstructed by the method of the present invention. Detailed Implementation

[0035] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] In fluorescence microscopy, the excitation light path of a fluorescence microscope emits a laser beam that illuminates a biological sample that has been fluorescently labeled or exhibits autofluorescence, causing it to fluoresce and be collected by a detector to obtain a two-dimensional image. Biological samples are typically sensitive to light intensity; excessive light can cause photobleaching and photodamage to cells. Therefore, in order to extend the imaging time as much as possible and image live cells for extended periods, it is necessary to reduce the excitation light intensity or exposure time for each acquisition, resulting in a significant amount of noise in the acquired images.

[0037] To reconstruct such low-quality images, this embodiment provides a self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation. The entire process is as follows: Figure 1As shown:

[0038] Step S1: Data Acquisition. The data can be real or simulated data. For real data, a sequence of original two-dimensional images is acquired using a fluorescence microscope under low irradiation dose conditions to obtain a high-noise image dataset. For simulated data, image data is generated using MATLAB. Specifically, a certain number of points, lines, spheres (hollow / solid), etc., are randomly generated in three-dimensional space and then sliced ​​in two dimensions to obtain simulated images that can accurately represent the edges and centers of the mitochondrial images used in the final method. Gaussian and Poisson noise are added to simulate a low signal-to-noise ratio image environment. The resulting data is denoted as i.

[0039] Low-dose imaging conditions involve low excitation light intensity (<1mW) or short exposure time (<100ms), resulting in two-dimensional images containing significant noise. The imaging model expression is as follows:

[0040] i = P(o) + N

[0041] Where: o represents two-dimensional noise-free data expressed in spatial coordinates (x,y), N is Gaussian noise, P is Poisson noise, and i represents a two-dimensional image with low signal-to-noise ratio (PSNR < 15).

[0042] Step S2: Data preprocessing. Normalize the input data to facilitate subsequent representation and training.

[0043] Step S3: INR Data Representation and Augmentation. The original high-noise image is modeled and reconstructed using an implicit neural representation model, forming a reconstructed dataset. This reconstructed dataset can then be directly used for training in subsequent iterations. For example... Figure 2 As shown, the INR network numbers each pixel i(x,y) in the image and uses its coordinates as input. Employing a multilayer perceptron structure, it learns the corresponding grayscale value I(x,y) as output. The input position is then Fourier-encoded to regress the image's grayscale value. Training is performed using mean squared error loss to construct a mapping from coordinates to grayscale values. The image restored by INR is structurally smoother and has enhanced edge continuity, which is helpful for guiding subsequent models.

[0044] INR data representation involves learning image coordinate encoding and utilizing the continuity of local structures within the image to achieve pre-denoising, resulting in the pre-denoising result I(x,y). The expression for the representation process is as follows:

[0045] I(x,y)=MLP(x,y)→i(x,y)

[0046] Where: (x,y) represents the spatial coordinate information in image i, i(x,y) represents the pixel intensity information corresponding to the (x,y) coordinate in image i, MLP represents the multilayer perceptron model used, and I(x,y) represents the image representation pre-denoising result corresponding to the (x,y) coordinate.

[0047] The location encoding method of INR in this embodiment is as follows:

[0048] μ(x i ,y i )=[sin(2 0 πx),cos(2 0 πx),…,sin(2 L-1 πx),cos(2 L-1 πx)],[sin(2 0 πy),cos(2 0 πy),…,sin(2 L-1 πy),cos(2 L-1 πy)]

[0049] Where: μ(x) i ,y i ) is a tensor formed by encoding pixels in the original data.

[0050] Network output result I(x) i ,y i ) is represented as:

[0051] I(x i ,y i )=MLP(μ(x i ,y i )|φ)

[0052] Wherein: MLP is a multilayer perceptron, consisting of 6 fully connected layers, with ReLU as the nonlinear activation function, and φ is the network learning parameter.

[0053] The mathematical form of the nonlinear activation function ReLU is:

[0054] f(x) = max(0,x)

[0055] Loss function design:

[0056]

[0057] Where: N is the total number of pixels in the image.

[0058] The INR output image is then exponentially enhanced to improve contrast, enabling it to function as a pseudo-reference image. This enhancement is expressed as an exponential function adjustment of the output intensity, used to highlight the target region and facilitate guidance during subsequent training.

[0059] I′(x,y)=α·e (β·I(x,y)) -γ

[0060] Where: I(x,y) is the intensity value of pixel (x,y) in the denoised image, I′(x,y) is the enhanced pixel intensity, and α, β, γ are hyperparameters for adjusting image brightness and contrast.

[0061] Step S4: Dataset Construction. For example... Figure 2 As shown, the enhanced image and the original image are input together into a U-Net-based self-supervised denoising network. This network first divides the low signal-to-noise ratio data i into 2×2 pixel adjacent regions and randomly selects one pixel from them to obtain downsampled results g1 and g2, which are half the size of the original image. The constructed image pair is {g1, g2}. The result obtained from INR representation enhancement is also downsampled twice in the same way, and the constructed image pair is {g′1, g′2}.

[0062] Step S5: Network Construction. Reconstruct one of the results using U-Net to obtain f. θ (g1), together with another g2, calculates the basic neighborhood prediction loss:

[0063]

[0064] Use f θ (g1), g2, g′1, and g′2 are used together to calculate the self-supervised guided loss:

[0065]

[0066] Total loss function:

[0067]

[0068] Where λ1 and λ2 are adjustable hyperparameters.

[0069] Step S6: Network Training. After obtaining the total loss function in step S5, a closed loop can be formed to train and improve the network.

[0070] Step S7: Data Reconstruction. In the last iteration of each training loop, the trained network is applied to the newly acquired low signal-to-noise ratio data to obtain the reconstruction results.

[0071] The effectiveness of this invention is verified through simulation and experiments. In this verification example, we performed denoising processing on both the simulated image and the real mitochondrial image. Figure 3The paper presents a comparison between the self-supervised denoising method based on U-Net and the method of this invention under high-noise simulation images. It can be seen that the image clarity and artifact reduction after restoration by the method of this invention are significantly improved compared to the self-supervised denoising method based on U-Net alone. Figure 4 The results of the reconstruction experiments on real images are shown, and it can be seen that the reconstructed images of the present invention are clearer and have a significant improvement in detail, which proves that the method of the present invention is effective.

[0072] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation, characterized in that, Includes the following steps: (1) Fluorescently label biological samples and use a fluorescence microscope to acquire images of biological samples under low light dose imaging conditions to obtain a series of two-dimensional fluorescence microscopic images with low signal-to-noise ratio. (2) The pre-trained INR model is used to perform image representation on the two-dimensional fluorescence microscopy image with low signal-to-noise ratio, generate a pre-denoised image, and then perform exponential image contrast enhancement processing on the pre-denoised image to obtain the enhanced pre-denoised image. (3) The two-dimensional fluorescence microscopy image is downsampled twice using a neighborhood pixel extraction strategy to obtain a set of downsampled image pairs. The same strategy is used to downsample the corresponding enhanced pre-denoised image of the two-dimensional fluorescence microscopy image twice to obtain a set of enhanced pre-denoised downsampled image pairs. (4) According to step (3), perform the same operation on all two-dimensional fluorescence microscopy images to obtain multiple sets of downsampled image pairs and their corresponding enhanced pre-denoised downsampled image pairs. Each downsampled image pair and its corresponding enhanced pre-denoised downsampled image pair is used as a set of samples, and then all samples are divided into training set and validation set. (5) Construct a deep learning network model, train the model using the training set, and test the model performance using the validation set; (6) Input the newly acquired low signal-to-noise ratio two-dimensional fluorescence microscopy image into the trained model, and a high-quality reconstructed image can be output.

2. The self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation according to claim 1, characterized in that: The low-dose imaging condition in step (1) refers to low excitation light intensity or short exposure time. The acquired two-dimensional fluorescence microscopy images contain a large amount of noise. The imaging model expression is as follows: i = P(o) + N Where: o represents two-dimensional noise-free image data, N represents Gaussian noise, P() represents Poisson noise, and i represents a two-dimensional fluorescence microscopy image with low signal-to-noise ratio.

3. The self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation according to claim 1, characterized in that: The INR model in step (2) adopts a multilayer perceptron structure. By learning the image coordinate encoding, it utilizes the continuity of local structures within the image to achieve pre-denoising and generate a pre-denoised image. The specific expression is as follows: I(x,y)=MLP(x,y)→i(x,y) Where: (x,y) represents the spatial coordinates in the image, i(x,y) represents the pixel intensity at spatial coordinates (x,y) in the two-dimensional fluorescence microscopy image, MLP represents the multilayer perceptron used in the INR model, → indicates approaching, and I(x,y) represents the pixel intensity at spatial coordinates (x,y) in the pre-denoised image generated by the INR model.

4. The self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation according to claim 1, characterized in that: The specific expression for the exponential image contrast enhancement processing in step (2) is as follows: I′(x,y)=α·e (β·I(x,y)) -c Where: I(x,y) represents the pixel intensity at spatial coordinate (x,y) in the pre-denoised image generated by the INR model, I′(x,y) is the pixel intensity at spatial coordinate (x,y) in the enhanced pre-denoised image, α, β, and γ are hyperparameters for adjusting image brightness and contrast, and e is a natural constant.

5. The self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation according to claim 1, characterized in that: The neighborhood pixel extraction strategy in step (3) involves dividing the two-dimensional fluorescence microscopy image into multiple 2×2 pixel blocks, randomly selecting one pixel from each pixel block, and stitching the extracted pixels together to obtain a downsampled image with a length and width that are half of the original image. Repeating this downsampling operation twice yields a set of downsampled image pairs.

6. The self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation according to claim 1, characterized in that: The deep learning network model in step (5) adopts the U-Net model structure, and uses one downsampled image from the downsampled image pair as input and the other downsampled image as label for self-supervised training.

7. The self-supervised denoising method for two-dimensional fluorescence microscopy images based on image representation according to claim 6, characterized in that: In step (5), the following loss function is used to train the model: in: The total loss function of the model, For self-supervised loss function, The auxiliary loss function is defined by λ1 and λ2 as set weight coefficients, g1 and g2 are downsampled image pairs in the training set, and g′1 and g′2 are enhanced pre-denoised downsampled image pairs in the training set, corresponding to g1 and g2 respectively. θ (g1) represents the reconstructed image obtained by inputting g1 into the model and outputting it.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: The processor is used to execute the computer program to implement the image representation-based self-supervised denoising method for two-dimensional fluorescence microscopy images as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the image representation-based self-supervised denoising method for two-dimensional fluorescence microscopy images as described in any one of claims 1 to 7.

Images