Self-adaptive image denoising method and system

Abstract

The invention discloses a self-adaptive image denoising method and system, belongs to the technical field of computer vision and artificial intelligence, and solves the problems of background artifacts, structural distortion, discontinuous boundaries and the like caused by improper normalization strategies and image block splicing when an existing self-supervised denoising method is used for processing an image with a large-area background or sparse signals. Comprising the steps of collecting an original noisy image, performing risk assessment on the original noisy image to obtain a high-risk block proportion, and determining a normalization strategy; normalizing the image according to the high-risk block proportion; extracting an image block from the normalized image as a training block, processing the training block to obtain a self-supervised training data pair, taking the self-supervised training data pair as an input and a label, selecting an existing denoising neural network, and training the existing denoising neural network to obtain a trained denoising model; and de-noising an image to be de-noised by using the trained de-noising model to obtain a final de-noised image. The method is suitable for high-requirement quantitative biological imaging analysis scenes.

Description

An Adaptive Image Denoising Method and System Technical Field

[0001] This invention belongs to the field of computer vision and artificial intelligence technology, and specifically relates to a digital image denoising method based on deep learning. Background Technology

[0002] Fluorescence microscopy is a core tool for visualizing cell dynamics and structures. However, there is an inherent trade-off between high-resolution imaging and low-light-damage imaging. To achieve long-term, high spatiotemporal resolution observation of live cells, researchers must reduce the light dose to minimize photobleaching and phototoxicity, but this inevitably leads to a significant decrease in the signal-to-noise ratio of the acquired images.

[0003] To address this challenge, self-supervised denoising methods, such as self-inspired noise-to-noise (SN2N), have emerged. These methods do not require paired clean / noisy image data; they can train the denoising model using only the information redundancy of a single noisy image, greatly improving photon efficiency and making it possible to obtain high-quality images under low-light conditions.

[0004] Despite this, existing self-supervised denoising methods still face a key bottleneck in practical applications: background artifacts. During the data preprocessing stage of model training, the commonly used patch-wise normalization strategy, while enhancing local contrast, can drastically amplify weak background noise when processing images containing large background areas or sparsely distributed signals. This leads the model to learn and generate unrealistic background textures or pseudo-structures. Furthermore, during the model inference stage, segmenting large images into small blocks and then simply stitching them together easily produces obvious stitching marks or intensity discontinuities at the block boundaries, severely impacting the overall image quality and the accuracy of subsequent quantitative analyses (such as cell segmentation, particle tracking, and protein expression quantification). These artifacts are deceptive and are often misjudged as real biological structures, leading to incorrect biological interpretations.

[0005] Therefore, there is an urgent need for a new denoising method that can retain the high photon efficiency advantage of self-supervised learning while fundamentally solving the problems of background artifacts and stitching artifacts, so as to generate more realistic and reliable high-quality images. Summary of the Invention

[0006] This invention provides an adaptive image denoising method and system, which aims to solve the problems of background artifacts, structural distortion and boundary discontinuity introduced by existing self-supervised denoising methods when processing images with large background areas or sparse signals (especially biological microscopic images) due to inappropriate normalization strategies and image patch stitching.

[0007] In a first aspect, the present invention aims to provide an adaptive image denoising method, comprising the following steps: S1: acquiring an original noisy image, performing a risk assessment on the original noisy image to obtain the proportion of high-risk blocks, and determining a normalization strategy; S2: normalizing the original noisy image according to the proportion of high-risk blocks; S3: extracting image blocks from the normalized image as training blocks, processing the training blocks to obtain self-supervised training data pairs; S4: using the self-supervised training data pairs as input and labels, selecting an existing denoising neural network, training it, and obtaining a trained denoising model; S5: using the trained denoising model to denoise the image to be denoised, obtaining the final denoised image.

[0008] Furthermore, a preferred embodiment is provided: the original noisy image is a high-resolution biological microscopic image obtained through microscopic imaging technology.

[0009] Furthermore, a preferred embodiment is provided: S1 includes: S11: extracting several overlapping image blocks from the density of the original noisy image; S12: calculating the global statistical index of the entire original noisy image as a benchmark for subsequent evaluation, and calculating the statistical index of each image block; S13: performing a risk assessment on each image block, classifying the image blocks into high-risk blocks and / or low-risk blocks; S14: calculating the proportion of high-risk blocks, counting the number of high-risk blocks, and calculating the proportion of all high-risk blocks in all image blocks as the high-risk block proportion.

[0010] Furthermore, a preferred solution is provided: the criteria for determining the high-risk block are as follows: ,in, Represents the local mean of an image patch. This represents the global mean of the original noisy image. This represents the relative coefficient of the block mean, with a value range of 0.80 to 1.00.

[0011] Furthermore, a preferred solution is provided: the criteria for determining the high-risk block are as follows: and and ,in, This represents the relative coefficient of the block mean, with a value ranging from 0.01 to 0.30. Represents the local standard deviation of an image patch. This represents the global standard deviation of the original noisy image. This represents the relative coefficient of the block standard deviation, with a value ranging from 0.10 to 0.60. Indicates the local skewness of an image patch. Represents the global skewness of the original noisy image. This represents the relative coefficient of block skewness, with a value range of 0.50 to 2.00.

[0012] Furthermore, a preferred solution is provided: in S3, the processing of the training block includes: diagonal resampling, Fourier interpolation, and normalization.

[0013] Furthermore, a preferred solution is provided: S5 includes: extracting several image patches from the image to be processed using a sliding window method; normalizing each extracted image patch; inputting the normalized image patch into the trained denoising model to obtain the corresponding denoised prediction block; generating several corresponding two-dimensional Gaussian weight maps with the same size as the image patches according to the size of the several image patches; performing Gaussian weighted fusion to calculate the final denoised image.

[0014] Secondly, the purpose of this invention is to propose an adaptive image denoising system. This system is based on an adaptive image denoising method as described in any one or more of the above-mentioned schemes. The system includes: a risk assessment module for acquiring the original noisy image, performing a risk assessment on the original noisy image, obtaining the proportion of high-risk blocks, and determining a normalization strategy; a normalization module for normalizing the original noisy image according to the proportion of high-risk blocks; a pre-training module for extracting image blocks from the normalized image as training blocks, processing the training blocks to obtain self-supervised training data pairs; a model training module for using the self-supervised training data pairs as input and labels, selecting an existing denoising neural network, training it, and obtaining a trained denoising model; and a denoising module for using the trained denoising model to denoise the image to be denoised, obtaining the final denoised image.

[0015] Thirdly, the present invention aims to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, and when the processor runs the computer program stored in the memory, the processor executes an adaptive image denoising method according to any one or more of the above-described schemes.

[0016] Fourthly, the present invention aims to provide a computer-readable storage medium for storing a computer program that executes an adaptive image denoising method as described in any one or more of the above-described schemes.

[0017] Compared with existing technologies, the advantages of this invention are as follows: 1. Suppressing background artifacts and improving structural realism: This invention, through a unique risk assessment adaptive normalization strategy, can intelligently determine the characteristics of image content before model training. For sparse signals or large background images prone to artifacts, it automatically adopts globally consistent image-level normalization, effectively avoiding excessive amplification of background noise, suppressing the generation of background artifacts from the source, ensuring the smoothness and cleanliness of the background after denoising, and improving the realism of the foreground structure.

[0018] 2. Eliminating stitching artifacts and ensuring image continuity: In the model inference stage, this invention adopts a Gaussian weighted fusion overlapping block prediction mechanism. By performing a smooth weighted average on the prediction results of the overlapping areas, it completely eliminates the grid-like stitching artifacts and boundary intensity jump problems common in traditional block processing methods, generating a visually seamless and spatially continuous high-quality full-size image.

[0019] Third, improved reliability of quantitative analysis: By effectively suppressing various artifacts, the images output by this invention have more accurate pixel intensity relationships and clearer structural boundaries. This greatly improves the accuracy and reproducibility of subsequent quantitative analysis tasks (such as automatic cell segmentation, molecular localization and tracking, fluorescence intensity quantification, etc.), providing a more reliable data foundation for precise biological research.

[0020] IV. Wide Adaptability and High Degree of Automation: The adaptive normalization mechanism of this invention can automatically adapt to images of different types and contents, eliminating the need for tedious manual parameter adjustments for different samples, thus improving the automation and universality of the processing workflow. This framework does not rely on a specific neural network structure and can be combined with various mainstream networks, exhibiting good scalability.

[0021] This invention is applicable to demanding quantitative bioimaging analysis scenarios. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0023] Figure 1 is a flowchart of model training in an adaptive image denoising method according to a specific embodiment of the present invention; Figure 2 is a flowchart of image denoising using the trained model in an adaptive image denoising method according to a specific embodiment of the present invention. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0025] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0027] This implementation method provides an adaptive image denoising method. First, by performing a risk assessment on the local statistical characteristics of the input image, the optimal normalization strategy (image-level or block-level) is adaptively selected for the image. Second, using a self-supervised learning approach, training data pairs are generated through diagonal resampling and Fourier interpolation, and a denoising neural network is trained using a loss function containing self-constraint terms. Finally, a Gaussian weighted fusion overlapping block prediction mechanism is used in the inference stage. As shown in Figure 1, the specific steps include: Step S1: Provide the original noisy image. Obtain at least one original noisy digital image for model training. The images are high-resolution biological microscopic images, such as images obtained by techniques such as rotating disk confocal structured light microscopy (SD-SIM), structured light microscopy (SIM), stimulated emission depletion microscopy (STED), or dilatational microscopy (ExM).

[0028] Step S1.1: Perform a risk assessment to determine the normalization strategy. To determine the effect of image... The optimal normalization strategy is used to execute an artifact risk assessment process. This process includes the following sub-steps: Step S1.1.1: Dense extraction of image patches. The image patch size (e.g., 128×128 pixels) and sliding step size (e.g., 64 pixels) are set from the original image... Dense extraction of a series of overlapping image patches .

[0029] Step S1.1.2: Calculate statistical indicators. First, calculate the statistical indicators for the entire original image. Global statistical indicators, including the global mean Global standard deviation and global skewness This serves as a benchmark for subsequent evaluation. Then, for each extracted image patch... Calculate its local statistical indicators, including local means. Local standard deviation and local skewness .

[0030] Step S1.1.3: Perform a risk assessment on each image patch. Based on a preset series of relative thresholds, assess the risk of each image patch. The blocks are classified as either high-risk or low-risk. An image block is classified as high-risk if it meets one of the following conditions: (1) Low signal and low contrast background features: The pixel intensity within the block is generally low and changes gradually. The determination criteria are: and and .in In this embodiment, the relative coefficient of the block mean is... The preferred value range is 0.90~0.98, and the preferred value is 0.96; In this embodiment, the standard deviation relative coefficient is used. The preferred value range is 0.65~0.85, with a preferred value of 0.75; In this embodiment, the relative coefficient of block skewness is... The preferred value range is 0.02~0.10, with a preferred value of 0.05. (2) Sparse strong signal characteristics: The block is mostly background, but contains a few extremely bright pixels. The judgment condition is: and and .in The relative coefficient of the block mean is preferably 0.10 to 0.30, with a preferred value of 0.10. The relative coefficient of the block standard deviation is preferably set to 0.10~0.60 in this embodiment, and more preferably to 0.30. The relative coefficient of block skewness is preferably set to 0.50~2.00 in this embodiment, with a preferred value of 1.00. Image blocks that do not meet any of the above high-risk conditions are classified as low-risk blocks.

[0031] Step S1.1.4: Calculate the proportion of high-risk blocks. Count the number of all blocks identified as high-risk. And calculate the total number of extracted image patches. The proportion of .

[0032] Step S2: Determine whether the proportion of high-risk blocks exceeds a preset threshold. The proportion of high-risk blocks calculated in step S1.1.4... With a preset ratio threshold A comparison is made. In this embodiment, the preferred range for the proportion threshold of high-risk blocks is 0.05 to 0.30. Preferably, in this embodiment... Based on experience, it is set to 0.1 (i.e., 10%).

[0033] Step S2.1: Select and perform global normalization (image-level normalization). If the judgment result is yes (i.e.) This indicates that the image content as a whole is prone to artifacts during block-level normalization. Therefore, for the entire original image... Perform uniform image-level normalization. For example, use min-max normalization to linearly map the pixel values ​​of the entire image to the interval [0, 1].

[0034] Step S2.2: Select and perform block normalization (block-level normalization). If the judgment result is negative (i.e.) If the image content shows a low risk of artifacts overall, then block-level normalization is more suitable to enhance local contrast. In this case, subsequent steps will focus on the original image. The training blocks are extracted and each training block is normalized independently.

[0035] Step S3: Generate self-supervised training data pairs. Extract image blocks for training from the image after normalization in step S21 or the original image to be normalized in step S2.2. For each training block, perform the following operations to generate training data pairs: (1) Diagonal resampling: Treat each 2×2 adjacent pixels in the training block as a unit, and generate two sub-images with the same content but different noise levels by averaging the diagonal pixels. and (2) Fourier interpolation: For two subgraphs and Perform Fourier transforms on each image, pad with zeros in the frequency domain, and then perform an inverse Fourier transform to restore the image size to the original training block size, resulting in a pair of twin images. and (3) Apply normalization: If block normalization was selected in step S2 (step S2.2), then the generated twin image pairs are normalized here. and Normalization was performed independently for each.

[0036] Step S4: Train the neural network model. Pair the twin images generated in step S3. and As input and labels, select an existing neural network (such as U-Net or its variants) and train it. The loss function used for training... Here is a composite loss function, defined as follows: ,in: Representative denoising neural network model, This represents standard loss calculations, such as L1 or L2 loss. The first two terms are standard Noise2Noise loss, which uses the prediction result of one twin image to fit another twin image. The third term is a self-constraint term, which forces the network to give consistent prediction results for two inputs with the same content but different noise levels. It is a hyperparameter that controls the weights of self-constraint terms.

[0037] Step S5: Output the trained denoising model. After the training process converges, save the weight parameters of the neural network to obtain a trained denoising model with adaptive capabilities. This is for use in subsequent image denoising applications.

[0038] Step S6: This step corresponds to Figure 2 and describes in detail the application of the model trained in S1-S5. The steps for denoising a new noisy image.

[0039] Step S6.1: Input the noisy image to be processed and the trained model. Load a large noisy image to be processed. And load the denoising model obtained through the above training method. .

[0040] Step S6.2: Apply a sliding window to extract overlapping blocks. Set a block size. (e.g., 128×128 pixels) and a step size (For example, 64 pixels, achieving a 50% overlap). A sliding window method is used to select from the image to be processed. Extract a series of overlapping image patches For image boundary areas, methods such as mirror filling can be used to ensure that all areas are completely covered.

[0041] Step S6.3: Normalize and predict each block. For each extracted image block... Perform iterative processing: (1) Apply normalization: for the image patch Normalization is performed. In this embodiment, block-level normalization is uniformly adopted to ensure the stability of the inference process. (2) Model prediction: The normalized image blocks are input into the trained denoising model. In the middle, the corresponding denoising prediction block is obtained. .

[0042] Step S6.4: Generate a Gaussian weight map. Based on the block size... A two-dimensional Gaussian weight map with the same size as the block is generated in advance. The weighted graph has the highest weight at its center and decreases smoothly towards the edges, as defined below: ,in: These are the pixel coordinates within the block. These are the center coordinates of the block, for example . It is the standard deviation of the Gaussian function, which controls the decay rate of the weights. In this implementation, its value is related to the block size. and step length Relevant, acceptable .

[0043] Step S6.5: Perform Gaussian weighted fusion. This is to combine all overlapping denoised prediction blocks. To seamlessly merge them into a single large image, the following weighted fusion steps are performed: (1) Initialize the accumulation matrix: Create two matrices with the original image to be processed. Zero matrices of the same size: an image matrix used to accumulate weighted pixel values. Another weight matrix used for accumulating weights (2) Iterative accumulation: Traverse all denoising prediction blocks For each Compare it with the Gaussian weight map generated in step S204 Perform element-wise multiplication to obtain the weighted prediction block. The weighted prediction block is then added to the image matrix. At the corresponding spatial location. Simultaneously, the weighted graph... It is also accumulated in the weight matrix. At the same position.

[0044] Step S6.6: Calculate and output the final denoised image. After traversing and accumulating all blocks, calculate the final denoised image by accumulating the image matrix. Divide element by element by the accumulated weight matrix The final denoised image is calculated. : ,in It is a very small positive number (e.g.) (), used to prevent the denominator from being zero. The final output This results in a high-quality, seamlessly stitched, complete denoised image.

[0045] The method described in this embodiment can suppress the generation of background artifacts from the source and eliminate stitching traces. While significantly improving the signal-to-noise ratio, it ensures the structural authenticity and background consistency of the image, making it particularly suitable for high-requirement quantitative biological imaging analysis.

[0046] Implementation Method Two: This implementation method provides an adaptive image denoising system based on an adaptive image denoising method as described in Implementation Method One. The system includes: a risk assessment module for acquiring the original noisy image, assessing the risk of the original noisy image, obtaining the proportion of high-risk blocks, and determining a normalization strategy; a normalization module for normalizing the original noisy image according to the proportion of high-risk blocks; a pre-training module for extracting image blocks from the normalized image as training blocks, processing the training blocks to obtain self-supervised training data pairs; a model training module for using the self-supervised training data pairs as input and labels, selecting an existing denoising neural network, training it, and obtaining a trained denoising model; and a denoising module for using the trained denoising model to denoise the image to be denoised, obtaining the final denoised image.

[0047] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. An adaptive image denoising method, characterized in that, Includes the following steps: S1: Acquire the original noisy image, conduct a risk assessment on the original noisy image, obtain the proportion of high-risk blocks, and determine the normalization strategy; S2: Normalize the original noisy image according to the proportion of high-risk blocks; S3: Extract image patches from the normalized image as training blocks, process the training blocks to obtain self-supervised training data pairs; S4: Use the self-supervised training data pairs as input and labels, select an existing denoising neural network, train it to obtain a trained denoising model; S5: Use the trained denoising model to denoise the image to be denoised to obtain the final denoised image.

2. The adaptive image denoising method according to claim 1, characterized in that, The original noisy image is a high-resolution biological microscopic image obtained through microscopic imaging technology.

3. The adaptive image denoising method according to claim 1, characterized in that, S1 includes: S11: Density extraction of several overlapping image blocks from the original noisy image; S12: Calculation of the global statistical index of the entire original noisy image as a benchmark for subsequent evaluation, and calculation of the statistical index of each image block; S13: Risk assessment of each image block, classifying the image blocks into high-risk blocks and / or low-risk blocks; S14: Calculation of the proportion of high-risk blocks, counting the number of high-risk blocks, and calculating the proportion of all high-risk blocks in all image blocks as the high-risk block proportion.

4. The adaptive image denoising method according to claim 2, characterized in that, The criteria for determining a high-risk block are as follows: ,in, Represents the local mean of an image patch. This represents the global mean of the original noisy image. This represents the relative coefficient of the block mean, with a value range of 0.80 to 1.

00.

5. The adaptive image denoising method according to claim 2, characterized in that, The criteria for determining a high-risk block are as follows: and and ,in, This represents the relative coefficient of the block mean, with a value ranging from 0.01 to 0.

30. Represents the local standard deviation of an image patch. This represents the global standard deviation of the original noisy image. This represents the relative coefficient of the block standard deviation, with a value ranging from 0.10 to 0.

60. Indicates the local skewness of an image patch. Represents the global skewness of the original noisy image. This represents the relative coefficient of block skewness, with a value range of 0.50 to 2.

00.

6. The adaptive image denoising method according to claim 1, characterized in that, In step S3, the processing of the training block includes: diagonal resampling, Fourier interpolation, and normalization.

7. The adaptive image denoising method according to claim 1, characterized in that, S5 includes: extracting several image blocks from the image to be processed using a sliding window method; normalizing each extracted image block; inputting the normalized image block into the trained denoising model to obtain the corresponding denoised prediction block; generating several two-dimensional Gaussian weight maps with the same size as the image blocks based on the size of the several image blocks; performing Gaussian weighted fusion to calculate the final denoised image.

8. An adaptive image denoising system, characterized in that, The system is implemented based on an adaptive image denoising method as described in any one of claims 1-7. The system includes: a risk assessment module for acquiring the original noisy image, assessing the risk of the original noisy image, obtaining the proportion of high-risk blocks, and determining a normalization strategy; a normalization module for normalizing the original noisy image according to the proportion of high-risk blocks; a pre-training module for extracting image blocks as training blocks from the normalized image, processing the training blocks, and obtaining self-supervised training data pairs; a model training module for using the self-supervised training data pairs as input and labels, selecting an existing denoising neural network, training it, and obtaining a trained denoising model; and a denoising module for using the trained denoising model to denoise the image to be denoised, obtaining the final denoised image.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes an adaptive image denoising method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that performs an adaptive image denoising method according to any one of claims 1-7.

Images