Self-supervised image denoising method using Gaussian process modeling

Abstract

The invention discloses a self-supervised image denoising method using Gaussian process modeling. The method comprises the following steps: performing down-sampling on a noise-containing image, and then performing random cyclic displacement to generate a plurality of noise-containing down-sampling images; modeling joint distribution among the down-sampling images by using a Gaussian process, predicting a pseudo-true value of the given down-sampling image by using other down-sampling images and corresponding de-noised images, supervising a de-noising model, and learning noise attributes in a noise image; the de-noising model uses a network architecture based on double-branch Transform, and pays attention to local and global information at the same time; a loss function is designed by minimizing negative logarithm likelihood of conditional distribution and an L2 norm between a pseudo-truth value and model prediction to optimize a training network. According to the method, the self-supervision property is utilized, and training and optimization can be carried out without a clean reference image.

Description

Technical Field

[0001] The invention relates to a self-supervised image denoising method using Gaussian process modeling, and belongs to the field of image processing. Background Art

[0002] The application of endoscopes in medicine is extremely important. It is a non-invasive or minimally invasive diagnostic and therapeutic method that can conduct in-depth inspections and operations without surgery. Compared with traditional open surgery, endoscopic surgery is performed through small incisions, which reduces surgical trauma, shortens the recovery period, and reduces the patient's pain and risk of complications. However, the quality of endoscopic images directly affects the accuracy of diagnosis and treatment. In the endoscopic operation environment, light conditions, complexity of tissue structure, and differences in equipment performance may cause image blur, low contrast, and color distortion, which affect the doctor's accurate identification of lesions. Movements such as patient breathing, heartbeat, and jitter caused by endoscopic operation may cause motion artifacts in the image, affecting image clarity. Therefore, endoscopic image denoising is the key to improving its diagnostic and therapeutic effects. Endoscopic image denoising involves many difficulties. In the denoising process, it is necessary to effectively remove noise and retain useful details in the image as much as possible, such as edge and texture information, to avoid excessive smoothing and information loss. Endoscopic images of different parts and types have different characteristics. A general denoising algorithm may not be applicable to all situations, and the algorithm needs to have good adaptability and adjustability. Although deep learning performs well in image denoising, it requires a large amount of high-quality annotated data for training. Medical image annotation is costly and highly specialized, and data scarcity is a common problem. In the process of image denoising, traditional methods require a large number of paired noisy-clean images to train the network; however, due to the complex internal structure of the camera, noise is often difficult to model accurately, so a supervised method needs to be applied. The current fully supervised method may be affected by the domain gap between synthetic noise and real-world noise in some cases, and there may be problems such as synthetic noise not fully simulating real noise and limited model generalization ability. Therefore, we consider using a self-supervised method to recover images from noisy images. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a self-supervised image denoising method using Gaussian process modeling, which can be trained and optimized without the need for clean reference images by utilizing its self-supervised nature.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A self-supervised image denoising method using Gaussian process modeling, comprising the following steps:

[0006] Step 1: Input a set of noisy endoscopic images Y, including several endoscopic images;

[0007] Step 2: Divide H / 2*W / 2 2×2 windows from the given noise image y∈Y, and obtain the noise downsampling distribution with a unit size of 2×2 by average pooling. The input noise image y is a two-dimensional array containing a large number of pixel values. The size of the output downsampled noise image y is half of the original size, achieving smooth denoising; perform random cyclic shift on the downsampled image to generate more images; where H is the image height and W is the image width;

[0008] Step 3: Use Gaussian process to develop denoising mapping function f (·), so that the image prediction value obtained by the denoising network and the original downsampled image form a joint Gaussian distribution P(f(yd)); this distribution can be expressed as Where d represents downsampling, yd is the image of the original image y after downsampling and random cyclic translation, μd is the mean vector, and Z d is a set of intermediate feature vectors obtained after the downsampled image passes through the denoising network, K(Z d ,Z d ) is the covariance matrix based on the kernel function, is a learnable parameter, I is the unit matrix;

[0009] Step 4: Use Gaussian process to learn the relationship between their corresponding denoising function mappings to predict the pseudo-true value of each downsampled image; establish a noise model and use Gaussian process to model the uncertainty of the denoising mapping function f(·); through conditional Gaussian distribution, use the information of other downsampled images and their denoising results to guide the denoising prediction of the current image and obtain the pseudo-true value of the downsampled image;

[0010] Step 5: Use a hybrid architecture consisting of a dual-branch Transformer encoder and a convolutional decoder to predict denoised images. The input image is processed by a dual-branch encoder, where the fine branch refines local features in high-resolution space and the coarse branch extracts global context; the fine branch uses three series-connected Transformer blocks and uses upsampling operations to increase the feature map resolution; the coarse branch uses Transformer blocks and downsampling modules to reduce the spatial resolution of the feature map and focus on global feature extraction. The decoder uses convolutional and upsampling layers to reconstruct the denoised image based on the features output by the encoder;

[0011] Step 6: Input the shifted downsampled image obtained in step 2 as a training sample into the network, and then perform inverse cyclic shift, adjusting it in the opposite direction and step size of the previous shift, ensuring that the predicted image is aligned with the original unshifted image to obtain the corresponding denoised downsampled image prediction value;

[0012] Step 7: The 2-norm loss L can be calculated by using the predicted value of the denoised downsampled image obtained in step 6 and the shifted downsampled image obtained in step 2 2 ;

[0013] Step 8: The Gaussian process loss L can be calculated by using the predicted value of the denoised downsampled image obtained in step 6 and the pseudo-true value of each downsampled image predicted by the Gaussian process in step 4 GP ;

[0014] Step 9: Optimize the denoising network through the total loss function. Input the noisy image into the trained denoising network to obtain the denoised image. The training consists of two losses. Calculate the total loss L total .

[0015] Furthermore, in step 1, a set of noisy images is input Where m is the input image number, y m is the mth image, and M is the total number of input noise images.

[0016] Furthermore, in step 2, the downsampled image Perform random cyclic shift to obtain a set of N downsampled cyclic shifted images: Where N is the number of downsampled images generated by cyclic shift, d represents the image after downsampling operation, i is the sequence number of the downsampled image, is the i-th image generated after downsampling and random cyclic shift of the input image y.

[0017] Furthermore, in step 3, similar attributes between the downsampled images are modeled, and the differences between the downsampled images are also considered by learning the covariance relationship between the downsampled images; for the downsampled images The corresponding intermediate vector is obtained by processing the denoising network encoder After the decoder and inverse shift operation to compensate for the information lost in the previous downsampling process, a set of denoised downsampled image prediction values ​​is finally obtained. Right now Constructed by Gaussian process and The joint Gaussian distribution between them; where d represents the downsampling operation, i represents the downsampling image sequence number, is the i-th image generated after downsampling and random cyclic translation of the input image, for The corresponding intermediate vector is obtained after being processed by the denoising network encoder. for The predicted value of the denoised downsampled image obtained by the denoising network.

[0018] Furthermore, in step 4, a Gaussian process is used to learn the relationship between their corresponding denoising mapping functions f(·), and other downsampled images are used. The denoised image is used to predict each downsampled image The pseudo-true value of ; where d represents the downsampling operation, N represents the total number of images, i and j represent the downsampling image sequence numbers, Represents the set of j-th downsampled images among the N generated downsampled images except the i-th one.

[0019] Based on the same real image x, there are two independent noise images y and s, which satisfy the conditional probability distribution: for a given y, the expectation E[y|x](y)=x and the conditional expectation value of s, E[z|x](z)=x+ε, where ε≠0 and is very small; let y=x+n 1 and s=x+ε+n 2 , where n 1 and n 2 They are respectively and The zero-mean additive noise is small and non-zero, with a mean of 0 and a variance of Gaussian noise. Define a new variable So the image s is Since ε and n 2 Independent of each other, n 1 and The joint follows a two-dimensional Gaussian distribution, and n 1 =yx, So the deduction is as follows:

[0020]

[0021] P(yx,sx)=N(0,∑ 2 )#(2)

[0022] Since x≈f(y) and x≈f(s) in the ideal denoising network, where f(·) represents the denoising mapping function, we have:

[0023] P(yf(y),sf(s))=N(0,∑ 2 )#(3)

[0024] P(f(y)|s,f(s))=N(μ y ,∑ 2 )#(4)

[0025] Among them, P(f(y)|s, f(s)) describes the conditional probability μ of the denoised result f(y) of y given s and its denoised result f(s) yTo predict the denoised image of y, the network weights are learned by minimizing the negative log-likelihood of the conditional distribution:

[0026] L GP =-logP(μ y -f(y)|s,f(s))#(5)

[0027] The Gaussian distribution of the known denoising mapping function f(·) is:

[0028]

[0029] Among them, μ d is the mean vector predicted by the Gaussian process; is the intermediate vector obtained by the denoising network encoder of the noisy image, and K(Z d , Z d ) is an RQ-based kernel matrix that defines the covariance, is a learnable parameter and I is the identity matrix.

[0030] Use another downsampled image The denoised image is used to predict each downsampled image The pseudo-true value of other downsampled images is defined similarly. and other downsampled images obtained by the encoder intermediate vector Using the joint distribution in formula (4) and (6), we get the following formula:

[0031]

[0032] in, and They are mean and covariance of the joint Gaussian distribution; This is the predicted downsampled image The pseudo-true value of .

[0033] Furthermore, in step 5, the network weights θ are trained and learned to perform image denoising; each Transformer block is equipped with a multi-head self-attention layer (MSA) and a feed-forward neural network (FFN) to calculate the self-attention features. For the input vector sequence I, the internal processing steps of the Transformer block can be concisely expressed as:

[0034] T(I)=FFN(MSA(I)+I)#(10)

[0035] The self-attention feature is passed to the FFN block, and the self-attention is calculated as follows (Q, K, V represent query, key, and value respectively, and have the same dimension, d represents the dimension):

[0036]

[0037] Among them, QK T The matching score matrix between the query and the key is obtained by multiplying the weight matrix after softmax processing with the value matrix V element by element, so that each query vector is weighted and summed with the corresponding value vector according to its matching degree with each key vector, thus obtaining the final output.

[0038] The calculation in the FFN block is as follows:

[0039] FFN(A)=MLP(GELU(DWC(MLP(A))))+A#(12)

[0040] Where A corresponds to the self-attention mechanism, DWC is a deep convolutional network, GELU is a Gaussian error linear unit, and MLP is a multi-layer perceptron.

[0041] Finally, a series of convolutional and upsampling layers in the convolutional decoder are used to output the denoised image.

[0042] Furthermore, in step 6, the shifted downsampled image obtained in step 2 is input into the network as a training sample, and then reverse cyclic shift is performed to obtain the corresponding denoised downsampled image prediction value Each downsampled image Through the encoder to obtain the intermediate vector The calculation is as follows:

[0043]

[0044] Where g represents the encoder function, θ e Represents encoder parameters.

[0045] Then the intermediate vector Forwarded to the decoder, inverse cyclic shift is performed to obtain the corresponding enhanced and denoised downsampled image The calculation is as follows:

[0046]

[0047] h represents the decoder function, θ d represents the decoder parameters, and Inv represents the inverse cyclic shift function.

[0048] Furthermore, in step 7, the denoised downsampled image prediction value (Right now ) and the shifted downsampled image obtained in step 2 The 2-norm loss L can be calculated 2 , the calculation formula is as follows:

[0049]

[0050] in, represents the denoised version of the jth downsampled image predicted by the network, is the result of the inverse cyclic shift operation on the i-th downsampled image, and N is the number of downsampled images generated by each noisy image.

[0051] Furthermore, in step 8, the denoised downsampled image prediction value and the pseudo-truth value of each downsampled image predicted by the Gaussian process (Right now ) can calculate L GP By minimizing and The generated L 2 The error is used to update the network weights, in L 2 The error introduces a gating mechanism for the predicted value and the pseudo-true value. The principle is: by inversely calculating the variance To adjust the weights, so that when training the network weights, the prediction results with higher uncertainty can be given less importance. Minimizing the variance helps the Gaussian process learn the joint distribution more accurately and thus obtain accurate pseudo-true labels. According to formula (5), the calculation formula is as follows:

[0052]

[0053] in, and are the pseudo-true values ​​and covariance matrices obtained through the Gaussian process model, Y represents the set of downsampled images, and Z represents the set of intermediate vectors through the encoder. represents the set of denoised versions of the image other than the jth downsampled image; the formula guides the network to learn an effective image denoising strategy by combining the negative log-likelihood, the weighted L2 distance of the error term, and the logarithm of the model prediction variance.

[0054] Furthermore, in step 9, the denoising network is optimized by the total loss function, and the noisy image is input into the trained denoising network to obtain a denoised image; the training consists of two losses, and the total loss is calculated:

[0055] L total =L 2 +λ GP L GP #(17)

[0056] Among them, λ GP is a predefined weight that is set to 0.03 in advance.

[0057] Beneficial effects: Considering that noise is often difficult to model accurately, the present invention proposes a new self-supervised method that can be trained and optimized without clean reference images. Using a Gaussian process, a joint Gaussian distribution is formulated between the downsampled images, and the relationship between their corresponding denoising function mappings is learned to predict the pseudo-true value of each downsampled image. This enables the network to learn the noise present in the downsampled images and achieve better denoising performance by using the joint relationship between the downsampled images with the help of the Gaussian process. The synergistic effect of the joint Gaussian distribution makes it possible to improve the accuracy and stability of the overall denoising even if the denoising result of a single image is full of uncertainty by considering the denoising information of other images related to it. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of a self-supervised image denoising method using Gaussian process modeling of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described below in conjunction with the accompanying drawings.

[0060] like Figure 1 As shown, a self-supervised image denoising method using Gaussian process modeling of the present invention comprises the following steps:

[0061] Step 1: Input a set of noisy images It includes several endoscopic images; these endoscopic images may come from multiple examination moments of the same patient or similar parts of different patients, but are all interfered by noise to varying degrees; where y is a noisy image, m is the input image number, and M is the number of noisy images.

[0062] Step 2: For each noisy image, use image downsampling techniques with regularization methods to train the network, and use additional regularizers in the loss function to consider the difference in the true value of the downsampled image. From the given noisy image y∈Y, obtain a noise downsampled distribution with a unit size of 2×2 by average pooling, and perform random cyclic shifts on the downsampled image to obtain a set of N downsampled cyclic shifted images: Where N is the number of downsampled images generated by cyclic shift, d represents the image after downsampling operation, i is the sequence number of the downsampled image, is the i-th image generated after downsampling and random cyclic shift of the input image y.

[0063] Step 3: Using Gaussian process, formulate the denoising mapping function f(·) to generate a joint Gaussian distribution between the downsampled images, describe the correlation between the downsampled images, model the similar attributes between the downsampled images, and also consider the differences between the downsampled images by learning the covariance relationship between the downsampled images. The corresponding intermediate vector is obtained by processing the denoising network encoder After the decoder and inverse shift operation to compensate for the information lost in the previous downsampling process, a set of denoised downsampled image prediction values ​​is finally obtained. Right now Constructed by Gaussian process and The joint Gaussian distribution between them; where d represents the downsampling operation, i represents the downsampling image sequence number, is the i-th image generated after downsampling and random cyclic translation of the input image, for The corresponding intermediate vector is obtained after being processed by the denoising network encoder. for The predicted value of the denoised downsampled image obtained by the denoising network.

[0064] Step 4: Use Gaussian process to learn the relationship between their corresponding denoising mapping functions f(·) and use other downsampled images The denoised image is used to predict each downsampled image The pseudo-true value of ; where d represents the downsampling operation, N represents the total number of images, i and j represent the downsampling image sequence numbers,

[0065] Represents the set of j-th downsampled images among the N generated downsampled images except the i-th one.

[0066] Based on the same real image x, there are two independent noise images y and s, which satisfy the conditional probability distribution: for a given y, the expectation E[y|x](y)=x and the conditional expectation value of s, E[z|x](z)=x+ε, where ε≠0 and is very small; let y=x+n 1 and s=x+ε+n 2 , where n 1 and n 2 They are respectively and The zero-mean additive noise is small and non-zero, with a mean of 0 and a variance of Gaussian noise. Define a new variable So the image s is Since ε and n 2 Independent of each other, n 1 and The joint follows a two-dimensional Gaussian distribution, and n 1 =y―x, So the deduction is as follows:

[0067]

[0068] P(y―x,s―x)=N(0,Σ 2 )#(2)

[0069] Since x≈f(y) and x≈f(s) in the ideal denoising network, where f(·) represents the denoising mapping function, we have:

[0070] P(y―f(y),s―f(s))=N(0,∑ 2 )#(3)

[0071] P(f(y)|s,f(s))=N(μ y ,∑ 2 )#(4)

[0072] Where P(f(y)|s,f(s)) describes the conditional probability of the denoised result f(y) of y given s and its denoised result f(s); μ y To predict the denoised image of y, the network weights are learned by minimizing the negative log-likelihood of the conditional distribution:

[0073] L GP =―log P(μ y ―f(y)│s,f(s))#(5)

[0074] The Gaussian distribution of the known denoising mapping function f(·) is:

[0075]

[0076] Among them, μ d is the mean vector predicted by the Gaussian process; is the intermediate vector obtained by the denoising network encoder of the noisy image, and K(Z d ,Z d ) is an RQ-based kernel matrix that defines the covariance, is a learnable parameter and I is the identity matrix.

[0077] Use another downsampled image The denoised image is used to predict each downsampled image The pseudo-true value of other downsampled images is defined similarly. and other downsampled images obtained by the encoder intermediate vector Using the joint distribution in formula (4) and (6), we get the following formula:

[0078]

[0079] in, and They are Mean and covariance of the joint Gaussian distribution. This is the predicted downsampled image The pseudo-true value of .

[0080] Step 5: Train and learn the network weights θ to perform image denoising. A hybrid architecture network consisting of a dual-branch Transformer encoder and a convolutional decoder is used. The encoder has a dual-branch structure, a fine branch and a coarse branch; the fine branch contains three series-connected Transformer blocks, and upsampling operations (such as bilinear interpolation) are used between each block to increase the size of the feature map, thereby performing a self-attention mechanism in the latent feature space with high spatial resolution. The coarse branch processes the input image through a series of Transformer blocks and downsampling modules, reducing the spatial resolution of the feature map to focus on global features. In particular, each Transformer block is equipped with a multi-head self-attention layer (MSA) and a feedforward neural network (FFN) to calculate self-attention features. For the input vector sequence I, the internal processing steps of the Transformer block can be concisely expressed as:

[0081] T(I)=FFN(MSA(I)+I)#(10)

[0082] The self-attention feature is passed to the FFN block, and the self-attention is calculated as follows (Q, K, V represent query, key, and value respectively, and have the same dimension, d represents the dimension):

[0083]

[0084] Among them, QK T The matching score matrix between the query and the key is obtained by multiplying the weight matrix after softmax processing with the value matrix V element by element, so that each query vector is weighted and summed with the corresponding value vector according to its matching degree with each key vector, thus obtaining the final output.

[0085] The calculation in the FFN block is as follows:

[0086] FFN(A)=MLP(GELU(DWC(MLP(A))))+A# (12)

[0087] Where A corresponds to the self-attention mechanism, DWC is a deep convolutional network, GELU is a Gaussian error linear unit, and MLP is a multi-layer perceptron.

[0088] Finally, a series of convolutional and upsampling layers in the convolutional decoder are used to output the denoised image.

[0089] Step 6: Input the shifted downsampled image obtained in step 2 into the network as a training sample, and then perform inverse cyclic shift to obtain the corresponding denoised downsampled image prediction value Each downsampled image Through the encoder to obtain the intermediate vector The calculation is as follows:

[0090]

[0091] Where g represents the encoder function, θ e Represents encoder parameters.

[0092] Then the intermediate vector Forwarded to the decoder, inverse cyclic shift is performed to obtain the corresponding enhanced and denoised downsampled image The calculation is as follows:

[0093]

[0094] h represents the decoder function, θ d represents the decoder parameters, and Inv represents the inverse cyclic shift function.

[0095] Step 7: Downsample the image by denoising and predicting the value (Right now ) and the shifted downsampled image obtained in step 2 The 2-norm loss L can be calculated 2 , the calculation formula is as follows:

[0096]

[0097] in, represents the denoised version of the jth downsampled image predicted by the network, is the result of the inverse cyclic shift operation on the i-th downsampled image, and N is the number of downsampled images generated by each noisy image.

[0098] Step 8: Downsample the image by denoising and predicting the value and the pseudo-truth value of each downsampled image predicted by the Gaussian process (Right now ) can calculate L GP By minimizing and The generated L 2 The error is used to update the network weights, in L 2The error introduces a gating mechanism for the predicted value and the pseudo-true value. The principle is: by inversely calculating the variance To adjust the weights, so that when training the network weights, the prediction results with higher uncertainty can be given less importance. Minimizing the variance helps the Gaussian process learn the joint distribution more accurately and thus obtain accurate pseudo-true labels. According to formula (5), the calculation formula is as follows:

[0099]

[0100] in, and are the pseudo-true values ​​and covariance matrices obtained through the Gaussian process model, Y represents the set of downsampled images, and Z represents the set of intermediate vectors through the encoder. represents the set of denoised versions of the image other than the jth downsampled image; the formula guides the network to learn an effective image denoising strategy by combining the negative log-likelihood, the weighted L2 distance of the error term, and the logarithm of the model prediction variance.

[0101] Step 9: Optimize the denoising network through the total loss function. Input the noisy image into the trained denoising network to obtain the denoised image. The training consists of two losses. Calculate the total loss:

[0102] L total =L 2 +λ GP L GP #(17)

[0103] Among them, λ GP is a predefined weight that is set to 0.03 in advance.

[0104] The present invention proposes a self-supervised image denoising method using Gaussian process modeling, in which a noisy image is downsampled and then random cyclic displacement is performed to generate multiple noisy downsampled images. The joint distribution between downsampled images is modeled using a Gaussian process, and the pseudo-true value of a given downsampled image is predicted by using other downsampled images and their corresponding denoised images to supervise the denoising model and learn the noise properties in the noisy image. The denoising model uses a network architecture based on a dual-branch Transformer, focusing on local and global information at the same time. The loss function is designed by minimizing the negative log-likelihood of the conditional distribution and the L2 norm between the pseudo-true value and the model prediction to optimize the training network.

[0105] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solutions formed by equivalent replacement or equivalent transformation shall fall within the protection scope required by the present invention.

Claims

1. A self-supervised image denoising method using Gaussian process modeling, characterized in that: The following steps are involved: Step 1: Input a set of noisy endoscopic images Y, including several endoscopic images; Step 2: Divide H / 2*W / 2 2×2 windows from the given noise image y∈Y, and obtain the noise downsampling distribution with a unit size of 2×2 by average pooling. The input noise image y is a two-dimensional array containing a large number of pixel values. The size of the output downsampled noise image is half of the original size, achieving smooth denoising; perform random cyclic shift on the downsampled image to generate more images; where H is the image height and W is the image width; Step 3: Use the Gaussian process to formulate a denoising mapping function f(·), so that the image prediction value obtained by the denoising network and the original downsampled image form a joint Gaussian distribution P(f(y d )); This distribution can be expressed as Among them, d represents downsampling, y d is the image after downsampling and random cyclic translation of the original image y, μ d is the mean vector, Z d is a set of intermediate feature vectors obtained after the downsampled image passes through the denoising network, K(Z d ,Z d ) is the covariance matrix based on the kernel function, is a learnable parameter, I is the unit matrix; Step 4: Use Gaussian process to learn the relationship between their corresponding denoising function mappings to predict the pseudo-true value of each downsampled image; establish a noise model and use Gaussian process to model the uncertainty of the denoising mapping function f(·); through conditional Gaussian distribution, use the information of other downsampled images and their denoising results to guide the denoising prediction of the current image and obtain the pseudo-true value of the downsampled image; Step 5: Use a hybrid architecture consisting of a dual-branch Transformer encoder and a convolutional decoder to predict denoised images; the input image is processed by a dual-branch encoder, the fine branch refines local features in high-resolution space, and the coarse branch extracts global context; the fine branch uses three series-connected Transformer blocks and uses upsampling operations to increase the resolution of feature maps; the coarse branch uses Transformer blocks and downsampling modules to reduce the spatial resolution of feature maps and focus on global feature extraction; the decoder uses convolutional and upsampling layers to reconstruct the denoised image based on the features output by the encoder; Step 6: Input the shifted downsampled image obtained in step 2 as a training sample into the network, and then perform inverse cyclic shift, adjusting it in the opposite direction and step size of the previous shift, ensuring that the predicted image is aligned with the original unshifted image to obtain the corresponding denoised downsampled image prediction value; Step 7: The 2-norm loss L2 can be calculated by using the predicted value of the denoised downsampled image obtained in step 6 and the shifted downsampled image obtained in step 2; Step 8: The Gaussian process loss L can be calculated by using the predicted value of the denoised downsampled image obtained in step 6 and the pseudo-true value of each downsampled image predicted by the Gaussian process in step 4 GP ; Step 9: Optimize the denoising network through the total loss function. Input the noisy image into the trained denoising network to obtain the denoised image. The training consists of two losses. Calculate the total loss L total .

2. The method for self-supervised image denoising using Gaussian process modeling according to claim 1, characterized in that: In step 1, a set of noisy images is input Where m is the image number, y m is the mth noise image, and M is the number of noise images.

3. The method for self-supervised image denoising using Gaussian process modeling according to claim 2, characterized in that: In step 2, the input noise image y is downsampled and randomly cyclically shifted to obtain N images Right now Among them, d represents the image after downsampling operation, i is the image number, It represents the i-th image generated after the input image is downsampled and randomly cyclically shifted, and N is the total number of generated images.

4. The method for self-supervised image denoising using Gaussian process modeling according to claim 3, characterized in that: In step 3, similar attributes between the downsampled images are modeled, and the differences between the downsampled images are also considered by learning the covariance relationship between the downsampled images; for a set of downsampled images The corresponding intermediate vector is obtained by processing the denoising network encoder After the decoder and inverse shift operation to compensate for the information lost in the previous downsampling process, a set of denoised downsampled image prediction values ​​is finally obtained. Right now Constructed by Gaussian process and The joint Gaussian distribution between them; where d represents the downsampling operation, i represents the downsampling image sequence number, represents the i-th image generated after the input image is downsampled and randomly cyclically shifted, for The corresponding intermediate vector is obtained after being processed by the denoising network encoder. for The predicted value of the denoised downsampled image obtained by the denoising network.

5. The method for self-supervised image denoising using Gaussian process modeling according to claim 4, characterized in that: In step 4, a Gaussian process is used to learn the relationship between their corresponding denoising mapping functions f(·), and other downsampled images are used The denoised image is used to predict each downsampled image The pseudo-true value of ; where d represents the downsampling operation, N represents the total number of generated images, i and j represent the downsampling image sequence numbers, Represents the set of j-th downsampled images among the N generated downsampled images except the i-th one. Based on the same real image x, there are two independent noise images y and s, which satisfy the conditional probability distribution: for a given y, the expectation E[y|x](y)=x and the conditional expectation value E[z|x](z)=x+ε, where ε≠0 and is very small; let y=x+n1 and s=x+ε+n2, where n1 and n2 are the variances of and The zero-mean additive noise is small and non-zero, with a mean of 0 and a variance of Gaussian noise; define a new variable So the image s is Since ε and n2 are independent of each other, n1 and The joint obeys a two-dimensional Gaussian distribution, and n1=y―x, So the deduction is as follows: P(y―x,s―x)=N(0,Σ 2 )#(2) Since x≈f(y) and x≈f(s) in the ideal denoising network, where f(·) represents the denoising mapping function, we have: P(y―f(y),s―f(s))=N(0,Σ 2 )#(3) P(f(y)|s,f(s))=N(μ y ,∑ 2 )#(4) Where P(f(y)|s,f(s)) describes the conditional probability of the denoised result f(y) of y given s and its denoised result f(s); μ y To predict the denoised image of y, the network weights are learned by minimizing the negative log-likelihood of the conditional distribution: L GP =―logP(μ y ―f(y)│s,f(s))#(5) The Gaussian distribution of the known denoising mapping function f(·) is: Among them, μ d is the mean vector predicted by the Gaussian process; is the intermediate vector obtained by the denoising network encoder of the noisy image, and K(Z d ,Z d ) is an RQ-based kernel matrix that defines the covariance, is a learnable parameter, I is the unit matrix; Use another downsampled image The denoised image is used to predict each downsampled image The pseudo-true value of other downsampled images is defined similarly. and other downsampled images obtained by the encoder intermediate vector Using the joint distribution in formula (4) and (6), we get the following formula: in, and They are mean and covariance of the joint Gaussian distribution; This is the predicted downsampled image The pseudo-true value of .

6. The method for self-supervised image denoising using Gaussian process modeling according to claim 5, characterized in that: In step 5, the network weights θ are trained and learned to perform image denoising; each Transformer block is equipped with MSA and FFN to calculate self-attention features; for the input vector sequence I, the internal processing steps of the Transformer block are expressed as: T(I)=FFN(MSA(I)+I)#(10) The self-attention feature is passed to the FFN block. The self-attention is calculated as follows. Q, K, V represent query, key, and value respectively, and have the same dimensions. d represents the dimension: Among them, QK T is the matching score matrix between query and key. The weight matrix after softmax processing is element-wise multiplied with the value matrix V, so that each query vector is weighted and summed with the corresponding value vector according to its matching degree with each key vector, thus obtaining the final output; The calculation in the FFN block is as follows: FFN(A)=MLP(GELU(DWC(MLP(A))))+A#(12) Where A corresponds to the self-attention mechanism, DWC is a deep convolutional network, GELU is a Gaussian error linear unit, and MLP is a multi-layer perceptron; Finally, a series of convolutional and upsampling layers in the convolutional decoder are used to output the denoised image.

7. The method for self-supervised image denoising using Gaussian process modeling according to claim 6, characterized in that: In step 6, the shifted downsampled image obtained in step 2 is input into the network as a training sample, and then reverse cyclic shift is performed to obtain the corresponding denoised downsampled image prediction value. Each downsampled image Through the encoder to obtain the intermediate vector The calculation is as follows: Where g represents the encoder function, θ e Represents encoder parameters; Then the intermediate vector Forwarded to the decoder, inverse cyclic shift is performed to obtain the corresponding enhanced and denoised downsampled image The calculation is as follows: h represents the decoder function, θ d represents the decoder parameters, and Inv represents the inverse cyclic shift function.

8. The method for self-supervised image denoising using Gaussian process modeling according to claim 7, characterized in that: In step 7, the denoising downsampled image prediction value is And the shifted downsampled image obtained in step 2 The 2-norm loss L2 can be calculated as follows: in, represents the denoised version of the jth downsampled image predicted by the network, is the result of the inverse cyclic shift operation on the i-th downsampled image, and N is the number of downsampled images generated by each noisy image.

9. The method for self-supervised image denoising using Gaussian process modeling according to claim 8, characterized in that: In step 8, the denoising downsampled image prediction value is and the pseudo-truth value of each downsampled image predicted by the Gaussian process Calculate L GP By minimizing and The generated L2 error is used to update the network weights, and a gating mechanism for the predicted value and the pseudo-true value is introduced in the L2 error. The principle is: by inversely calculating the variance To adjust the weights, so that when training the network weights, the prediction results with higher uncertainty can be given lower importance; minimizing the variance helps the Gaussian process learn the joint distribution more accurately and thus obtain accurate pseudo-true labels; according to formula (5), the calculation formula is as follows: in, and are the pseudo-true values ​​and covariance matrices obtained through the Gaussian process model, Y represents the set of downsampled images, and Z represents the set of intermediate vectors through the encoder. represents the set of denoised versions of the image other than the jth downsampled image; the formula guides the network to learn an effective image denoising strategy by combining the negative log-likelihood, the weighted L2 distance of the error term, and the logarithm of the model prediction variance.

10. The self-supervised image denoising method using Gaussian process modeling according to claim 9, characterized in that: In step 9, the denoising network is optimized by the total loss function, and the noisy image is input into the trained denoising network to obtain a denoised image; the training consists of two losses, and the total loss is calculated: L total =L2+λ GP L GP #(17) Among them, λ GP is a predefined weight that is set to 0.03 in advance.

Images