Sea surface temperature image completion method and network based on denoising diffusion generation model

Abstract

The application belongs to the technical field of image processing, and discloses a sea surface temperature image completion method and network based on a denoising diffusion generation model, which comprises a weekly mean generation module, a daily deviation generation module and a multiscale fusion module based on weekly mean and daily deviation, a denoising diffusion generation module I of the weekly mean generation module is used to generate a completed weekly mean image, a daily deviation image is obtained by using the daily deviation generation module, and a denoising diffusion generation module II of the daily deviation generation module is used to obtain a completed daily deviation image, the completed daily deviation image and the completed weekly mean image are respectively input into a multiscale deviation feature decoupling extraction module and a multiscale mean feature decoupling extraction module of the multiscale fusion module based on weekly mean and daily deviation, decoupling features of three different scales are obtained, and then the decoupling features are fused on the same scale, and finally, the fused features obtain a completed image. The method improves the details and clarity of the image, and obtains a high-quality output image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method and network for completing sea surface temperature images based on a denoising diffusion generation model. Background Technology

[0002] Traditional deep learning-based sea surface temperature (SST) completion methods learn the spatiotemporal distribution of SST by estimating it using historical ocean data. Traditional methods, such as those based on convolutional neural networks (CNNs) and encoder-decoder architectures, directly estimate and complete SST using historical ocean data. These traditional methods suffer from the following problems: unstable training and poor completion accuracy. This is because, on the one hand, traditional methods typically perform a single completion operation directly on the missing regions; on the other hand, the numerical range of SST is usually very large, and directly estimating such a large range leads to a highly unstable completion process, while also making it difficult to accurately estimate the value at each point, thus limiting the accuracy of the completion.

[0003] Currently, cutting-edge methods in deep learning-based sea surface temperature (SST) image completion employ a coarse-to-fine approach, mostly using a two-stage method. The first stage reconstructs the weekly SST using the monthly average as a coarse prediction of the missing daily SST field. The second stage learns the difference between the weekly average and the daily SST field to estimate the bias in the missing region. This bias is then superimposed with the weekly average predicted in the first stage to complete the missing region. Compared to traditional methods, this reduces the complexity of the training process. Finally, the repaired image is superimposed with the predicted short-term average image to obtain the final completed daily image. This approach ensures both a relatively stable global estimate and accurate local fine-grained estimation.

[0004] However, the above method has the following problems: First, during model training, this method uses a generative adversarial network (GAN) as its basic architecture, which not only leads to a highly unstable and difficult-to-converge training process, but also causes the loss of some key information of the image during generation, reducing the quality of the generated results. For example, a GAN consists of a generator and a discriminator. During training, the generator aims to generate "fake" images to "deceive" the discriminator, while the discriminator aims to distinguish between the "fake" images generated by the generator and real images. This training method can lead to problems such as mode collapse and mode degeneration, resulting in a very volatile training process and making it difficult for the model to converge. Second, the GAN uses an end-to-end training model to generate images by inputting random noise or conditional variables, and then only uses the discriminator to judge in order to ensure the quality of the generated images. Furthermore, the training process of the GAN is very volatile and difficult to converge, which reduces the quality of the generated results. Therefore, an advanced network architecture that can avoid these problems while still possessing generative capabilities should be selected to complete this task. Secondly, in the local modeling process, directly superimposing the estimated bias of the missing portion with the weekly average at a single scale to obtain the completion result is not only too one-sided in its perspective, but also ignores the information contained in the image at different scales, leading to poor fusion quality and reducing the quality of the final generated image. For example, sea surface temperature images contain physical information at multiple scales, and this information at different scales affects the numerical value of sea surface temperature. In a single sea surface temperature image, this information at different scales is intertwined. The fusion method at a single scale exacerbates the problem of entanglement between information at different scales, reducing the quality of the final generated image. Therefore, in the fusion process, it is necessary to consider not only the case at a single scale, but also the cases at other scales simultaneously. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a sea surface temperature image completion method and network based on a denoising diffusion generation model. The method employs a denoising diffusion generation module to progressively reduce noise in the image, starting from the initial noise level, until a high-quality output image is achieved. A multi-scale fusion module combines multi-scale and gated fusion mechanisms. It utilizes multi-scale mean features to decouple and extract information from different scales of the weekly average sea surface temperature image, and uses multi-scale deviation features to decouple and extract information from different scales of the daily deviation image. These features are then fused to fill in missing values, thereby improving image detail and clarity.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] This invention provides a sea surface temperature image completion network based on a denoising diffusion generation model, including a weekly mean generation module, a daily deviation generation module, and a multi-scale fusion module based on the weekly mean and daily deviation. The weekly mean generation module includes a denoising diffusion generation module I, which is used to process the input missing weekly mean image SST. cor The weekly mean image SST is generated using the denoising diffusion generation module I. short The daily deviation generation module includes a denoising diffusion generation module II, which is used to calculate the missing daily image SST input. day And the completed weekly mean image SST short The data deviation between them yields the daily deviation image SST. ano The completed daily deviation image SST is obtained through the denoising diffusion generation module II. ano-recon The multi-scale fusion module based on weekly mean and daily deviation includes a multi-scale deviation feature decoupling extraction module, a multi-scale mean feature decoupling extraction module, and a multi-scale feature fusion module. First, it processes the input completed daily deviation image SST... ano-recon Multi-scale deviation feature decoupling extraction is performed using a multi-scale deviation feature decoupling extraction module to obtain deviation decoupling features y1, y2, and y3 at three different scales: small-scale, medium-scale, and large-scale. These features are then applied to the input padded weekly mean image SST. short Multi-scale mean feature decoupling extraction is performed using a multi-scale mean feature decoupling module to obtain mean decoupling features z1, z2, and z3 at three different scales: small-scale, medium-scale, and large-scale. Then, the bias decoupling feature and the mean decoupling feature are fused at the same scale to obtain three fused results at small-scale, medium-scale, and large-scale. Finally The images are then fused together to obtain a complete and incomplete SST image. recon .

[0008] Furthermore, the denoising diffusion generation module I and the denoising diffusion generation module II have the same structure, including two processes: forward diffusion and backward denoising. In the forward diffusion process, for the missing input image, Gaussian noise is gradually added over T time steps, and the noise image at each step is recorded. When the image at step T becomes a pure Gaussian distribution, backward denoising is performed. The backward denoising process uses the result obtained from forward diffusion to backdifflate and denoise the pure Gaussian distribution to generate a complete image.

[0009] Furthermore, the image processing procedure of the denoising diffusion generation module I is as follows: In the forward diffusion process, the initial missing weekly mean image SST is input. cor_0 Then, Gaussian noise is gradually added over T time steps, and the noisy image is recorded at each step. The image is then SST. cor_0Convert to Gaussian white noise The formula is as follows:

[0010]

[0011] Wherein q(SST) cor_t |SST cor_t-1 ) represents the process of forward diffusion through the image SST at time t-1 at each step. cor_t-1 Adding Gaussian noise to obtain the image SST at time t cor_t The Gaussian noise is composed of the mean Sum of variances ∑ t =β t definition, Represents a Gaussian distribution, where I denotes the identity matrix; image SST cor_t By adding a variance of β at time step t t Gaussian noise, and according to the variance table, the sample SST from the previous step cor_t-1 Scaling to Received;

[0012] By using the independence of the noise added in each step of formula (1), the total noise variance is calculated as follows: Where α t =1-β t ,β s Let represent the variance of the Gaussian noise when the step size is s. Equation (1) can be rewritten as a single step.

[0013]

[0014] Wherein q(SST) cor_t |SST cor_0 This indicates that during the forward process, the missing weekly mean image SST is fed forward at time step t. cor_0 Adding Gaussian noise to obtain the image SST cor_t ;

[0015] In the backward denoising process, the input image SST at time T is... cor_T Then, denoising is performed gradually over T time steps, and the output of each step is recorded. The denoising process formula is as follows:

[0016]

[0017] Where, p θ (SST cor_t-1 |SST cor_t ) indicates that at each step of the backward pass, by removing the image SST cor_t Gaussian noise in the image SST cor_t-1During backward denoising, the mean parameter μ of the noise in the Gaussian distribution is predicted by modeling with a neural network. θ (SST cor_t ,t) and variance ∑ θ (SST cor_t ,t);

[0018] The predicted parameter mean μ θ (SST cor_t The formula for t is as follows:

[0019]

[0020] Where, ∈ θ (SST cor_t ,t) represents the step size t and the image is SST cor_t Noise at that time;

[0021] The SST of the denoised image can be obtained. cor_t-1 for,

[0022]

[0023] Furthermore, the image processing procedure of the denoising diffusion generation module II is as follows: During forward diffusion, the initial missing daily deviation image SST is input. ano_0 Then, Gaussian noise is gradually added over T time steps, and the noisy image is recorded at each step. The image is then SST. ano_0 Convert to Gaussian white noise The formula is as follows:

[0024]

[0025] Wherein q(SST) ano_t |SST ano_t-1 ) indicates that at each step of the forward pass, the image SST is passed through... ano_t-1 Adding Gaussian noise to obtain the image SST ano_t The Gaussian noise is composed of the mean Sum of variances ∑ t =β' t Definition; Image SST ano_t This is achieved by adding a variance of β' at time step t. t Gaussian noise, and according to the variance table, the previous sample SST ano_t-1 Scaling to Received;

[0026] Calculate the total noise variance Then (7) was rewritten as one step.

[0027]

[0028] Wherein q(SST) ano_t |SST ano_0 () indicates that during the forward process, the missing weekly mean image SST is passed through at time step t. ano_0 Adding Gaussian noise to obtain the image SST ano_t ;

[0029] Subsequently, the backward denoising process inputs SST. ano_T Then, denoising is performed gradually over T time steps, and the output of each step is recorded. The denoising process formula is as follows:

[0030]

[0031] Where, p θ (SST ano_t-1 |SST ano_t ) indicates that at each step of the backward pass, by removing the image SST ano_t Gaussian noise in the image SST ano_t-1 During backward denoising, the mean parameter μ of the noise in the Gaussian distribution is predicted by modeling with a neural network. θ (SST ano_t ,t) and variance ∑ θ (SST ano_t ,t);

[0032] The predicted parameter mean μ θ (SST ano_t The formula for t is as follows:

[0033]

[0034] Where, ∈ θ (SST ano_t ,t) represents the step size t and the image is SST ano_t Noise at that time;

[0035] Obtain the denoised image SST ano_t-1 for,

[0036]

[0037] Furthermore, the multi-scale fusion module based on weekly average and daily deviation inputs the completed weekly average image SST. short And the completed daily deviation image SST ano-recon Output complete daily sea surface temperature image (SST) recon First, three parallel dilated convolutional layers with different void ratios are used to analyze the predicted sea surface temperature (SST) images. shortMulti-scale mean feature decoupling extraction is performed to extract features of information at different scales. Then, the features at larger scales are subtracted from the features at smaller scales to obtain the decoupling features at larger scales. After obtaining the decoupling features at different scales of the weekly mean image and the daily deviation image, the small-scale decoupling features (y1, z1), medium-scale decoupling features (y2, z2), and large-scale decoupling features (y3, z3) corresponding to the mean and deviation are respectively input into the gated fusion module GFM for fusion.

[0038] Secondly, this invention provides a multi-scale sea surface temperature image completion method based on a denoising diffusion generation model, which is implemented using the sea surface temperature image completion network based on the denoising diffusion generation model as described above, and specifically includes the following steps:

[0039] Step S1: Input the missing weekly mean image SST cor The image is used to generate a complete weekly mean image SST using the denoising and diffusion generation module I of the weekly mean generation module. short ;

[0040] Step S2: Input the missing daily image SST day The missing daily images SSt are obtained using the daily deviation generation module. day The completed weekly mean image SST obtained in step 1 short The deviation between them yields the daily deviation image SST. ano The completed daily deviation image SST is obtained through the denoising diffusion generation module II. ano-recon ;

[0041] Step S3: The completed daily deviation image SST obtained in Step 2... ano-recon The completed weekly mean image SST obtained in step 1 short The multi-scale deviation feature decoupling extraction module and the multi-scale mean feature decoupling extraction module based on the multi-scale fusion module of weekly mean and daily deviation are respectively input. The multi-scale deviation feature decoupling extraction module extracts multi-scale deviation features to obtain deviation decoupling features y1, y2, and y3 at three different scales: small scale, medium scale, and large scale. The multi-scale mean feature decoupling extraction module extracts multi-scale mean features to obtain mean decoupling features z1, z2, and z3 at three different scales: small scale, medium scale, and large scale. Then, the deviation decoupling features and mean decoupling features are fused at the same scale to obtain three fusion results at small scale, medium scale, and large scale. Finally The images are then fused together to obtain a complete and incomplete SST image. recon .

[0042] Compared with the prior art, the advantages of this invention are:

[0043] (1) A denoising diffusion generation module is applied during the image completion process. Starting with noise, the image is progressively denoised until a high-quality output image is achieved. By adjusting the generation process through sampling from a given pixel during the backdiffusion iteration, noise in the image is effectively reduced while preserving image detail, thus improving image quality and clarity. This solves the problems of instability and difficulty in convergence during training caused by the use of generative adversarial networks as the basic architecture in the original method, as well as the problem of low-quality generated results due to the loss of some key information during the generation process, thereby improving the quality of the generated image.

[0044] (2) A multi-scale fusion module is proposed in the image completion process, which combines multi-scale and gated fusion mechanisms. Multi-scale mean features are used to decouple and extract information from different scales of the weekly average sea surface temperature image, and multi-scale deviation features are used to decouple and extract information from different scales of the daily deviation image. Features from different scales are then fused to fill in missing values, thereby improving image detail and clarity. This addresses the problem of the original method's approach of directly superimposing the completed daily deviation image and the weekly average sea surface temperature image at a single scale during local modeling, which is too one-sided and ignores information from different scales, leading to poor fusion quality. This improves the quality of the final generated image completion. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a network architecture diagram of the present invention;

[0047] Figure 2 This is a schematic diagram of the structure of the noise reduction diffusion generation module I of the present invention;

[0048] Figure 3 This is a schematic diagram of the noise reduction diffusion generation module II of the present invention;

[0049] Figure 4 This is a schematic diagram of the gating fusion mechanism of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0051] Example 1

[0052] Combination Figure 1As shown, this embodiment provides a sea surface temperature image completion network based on a denoising diffusion generation model, including a weekly mean generation module, a daily deviation generation module, and a multi-scale fusion module based on the weekly mean and daily deviation.

[0053] The following sections will introduce each module in detail:

[0054] The weekly mean generation module includes a denoising and diffusion generation module I. This module is used to generate the input image SST containing the missing weekly mean values. cor The weekly mean image SST is generated using the denoising diffusion generation module I. short .

[0055] The daily deviation generation module includes a denoising and diffusion generation module II, which is used to calculate the missing daily image SST from the input. day And the completed weekly mean image SST short The data deviation between them yields the daily deviation image SST. ano The completed daily deviation image SST is obtained through the denoising diffusion generation module II. ano-recon .

[0056] The multi-scale fusion module based on weekly mean and daily deviation includes a multi-scale deviation feature decoupling extraction module, a multi-scale mean feature decoupling extraction module, and a multi-scale feature fusion module. First, it processes the input completed daily deviation image SST. ano-recon Multi-scale deviation feature decoupling extraction is performed using a multi-scale deviation feature decoupling extraction module to obtain deviation decoupling features y1, y2, and y3 at three different scales: small-scale, medium-scale, and large-scale. These features are then applied to the input padded weekly mean image SST. short Multi-scale mean feature decoupling extraction is performed using a multi-scale mean feature decoupling module to obtain mean decoupling features z1, z2, and z3 at three different scales: small-scale, medium-scale, and large-scale. Then, the bias decoupling feature and the mean decoupling feature are fused at the same scale to obtain three fused results at small-scale, medium-scale, and large-scale. Finally The images are then fused together to obtain a complete and incomplete SST image. recon .

[0057] It should be noted that the denoising diffusion generation module I and the denoising diffusion generation module II have the same structure, including two processes: forward diffusion and backward denoising. In the forward diffusion process, for the missing input image, Gaussian noise is gradually added over T time steps, and the noise image at each step is recorded. When the image at step T becomes a pure Gaussian distribution, backward denoising is performed. The backward denoising process uses the result obtained from forward diffusion to backdifflate and denoise the pure Gaussian distribution to generate a complete image.

[0058] The following section details each module and data processing:

[0059] (1) Weekly average generation module: Input the missing weekly average sea surface temperature image SST cor_0 The denoising and diffusion generation module I outputs the predicted cloudless weekly mean sea surface temperature image (SST). short The structure of the noise reduction diffusion generation module I is as follows: Figure 2 As shown, the working principle of the inference process is based on the random noise vector SST. cor_T Sampling is performed, and noise is gradually reduced until a high-quality output image (SST) is achieved. short It includes two processes: forward diffusion and backward denoising.

[0060] The image processing procedure of the denoising diffusion generation module I is as follows: In the forward diffusion process, the initial missing weekly mean image SST is input. cor_0 Then, Gaussian noise is gradually added over T time steps, and the noisy image is recorded at each step. The image is then SST. cor_0 Convert to Gaussian white noise The formula is as follows:

[0061]

[0062] Wherein q(SST) cor_t |SST cor_t-1 ) represents the process of forward diffusion through the image SST at time t-1 at each step. cor_t-1 Adding Gaussian noise to obtain the image SST at time t cor_t The Gaussian noise is composed of the mean Sum of variances ∑ t =β t definition, Represents a Gaussian distribution, where I denotes the identity matrix; image SST cor_t By adding a variance of β at time step t t Gaussian noise, and according to the variance table, the sample SST from the previous step cor_t-1 Scaling to It's important to note that the model adds Gaussian noise with variance β to the image at each time step, and this noise gradually decreases during generation. The variance table is designed to control this gradual reduction of noise. During the training of the denoising diffusion generation model, optimization algorithms (such as stochastic gradient descent) are used to adjust the values ​​in the variance table based on the model's error or loss function. Therefore, the variance table is either learned through model training or designed based on prior knowledge and experimental experience.

[0063] By using the independence of the noise added in each step (1), the total noise variance can be calculated as follows: Where α t =1-β t ,β s This represents the variance of the Gaussian noise when the step size is s. Therefore, equation (1) can be rewritten as a one-step expression.

[0064]

[0065] Wherein q(SST) cor_t |SST cor_0 This indicates that during the forward process, the missing weekly mean image SST is fed forward at time step t. cor_0 Adding Gaussian noise to obtain the image SST cor_t ;

[0066] Subsequently, in the backward denoising process, the image SST at time T is input. cor_T Then, denoising is performed gradually over T time steps, and the output of each step is recorded. The denoising process formula is as follows:

[0067]

[0068] Where, p θ (SST cor_t-1 |SST cor_t ) indicates that at each step of the backward pass, by removing the image SST cor_t Gaussian noise in the image SST cor_t-1 During backward denoising, the mean parameter μ of the noise in the Gaussian distribution is predicted by modeling with a neural network. θ (SST cor_t ,t) and variance ∑ θ (SST cor_t ,t).

[0069] In the backward denoising process of the denoising diffusion generative model, the trainable parameters only exist in the mean of the distribution. The mean is related to the variable, but not the variance. Predictions are added to the current image SST. cor_t Accumulated noise is the best method for parameterizing the model. Therefore, this invention obtains the following predicted parameter mean μ. θ (SST cor_t ,t),

[0070]

[0071] Where, ∈ θ (SST cor_t ,t) represents the step size t and the image is SST cor_t Noise at that time;

[0072] The SST of the denoised image can be obtained. cor_t-1 for,

[0073]

[0074] (2) Daily Deviation Generation Module:

[0075] The weekly average sea surface temperature image SST was obtained. short and daily sea surface temperature images SST day Daily Deviation Image (SST) ano The formula is as follows:

[0076] SST ano =SST short -SST day (6)

[0077] The daily deviation image SST is completed by the noise reduction and diffusion generation module II. ano-recon The structure of the noise reduction diffusion generation module II is as follows: Figure 3 As shown, the working principle of the inference process is similar to that of the denoising diffusion generation module I, which processes the random noise vector SST. ano_T Sampling is performed, and noise is gradually reduced until a high-quality output image (SST) is achieved. ano-recon It also includes two processes: forward diffusion and backward denoising.

[0078] The image processing procedure of the denoising diffusion generation module II is as follows:

[0079] During forward diffusion, the initial missing daily deviation image SST is input. ano_0 Then, Gaussian noise is gradually added over T time steps, and the noisy image is recorded at each step. The image is then SST. ano_0 Convert to Gaussian white noise The formula is as follows:

[0080]

[0081] Wherein q(SST) ano_t |SST ano_t-1 ) indicates that at each step of the forward pass, the image SST is passed through... ano_t-1 Adding Gaussian noise to obtain the image SST ano_t The Gaussian noise is composed of the mean Sum of variances ∑ t =β' t Definition; Image SST ano_t This is achieved by adding a variance of β' at time step t. t Gaussian noise, and according to the variance table, the previous sample SST ano_t-1 Scaling to The result is; where α' t =1-β' t ;

[0082] Calculate the total noise variance Then (7) was rewritten as one step.

[0083]

[0084] Wherein q(SST) ano_t |SST ano_0 () indicates that during the forward process, the missing weekly mean image SST is passed through at time step t. ano_0 Adding Gaussian noise to obtain the image SST ano_t .

[0085] Subsequently, the backward denoising process inputs SST. ano_T Then, denoising is performed gradually over T time steps, and the output of each step is recorded. The denoising process formula is as follows:

[0086]

[0087] Where, p θ (SST ano_t-1 |SST ano_t ) indicates that at each step of the backward pass, by removing the image SST ano_t Gaussian noise in the image SST ano_t-1 During backward denoising, the mean parameter μ of the noise in the Gaussian distribution is predicted by a neural network model. θ (SST ano_t ,t) and variance ∑ θ (SST ano_t ,t).

[0088] Prediction added to the current image SST ano_t Accumulated noise is the best method for parameterizing the model. Therefore, this invention obtains the following predicted parameter mean μ. θ (SST ano_t ,t),

[0089]

[0090] Where, ∈ θ (SST ano_t ,t) represents the step size t and the image is SST ano_t Noise at that time.

[0091] The SST of the denoised image can be obtained. ano_t-1 for,

[0092]

[0093] (3) Multi-scale fusion module based on weekly mean and daily deviation: Input the completed weekly mean image SST short And the completed daily deviation image SST ano-recon Output complete daily sea surface temperature image (SST) recon .

[0094] This invention proposes using three parallel, dilated convolutional layers with different void ratios to predict the sea surface temperature (SST) image. short Multi-scale mean feature decoupling extraction is performed to extract features at different scales. Then, the features at larger scales are subtracted from the features at smaller scales to obtain the decoupled features at larger scales. The process is as follows:

[0095] y1=y′1 (12)

[0096] y2=y′2-y′1 (13)

[0097] y3=y′3-y′2 (14)

[0098] Where y'1, y'2, and y'3 represent the bias characteristics at small, medium, and large scales, respectively, and y1, y2, and y3 represent the bias decoupling characteristics at small, medium, and large scales, respectively.

[0099] SST for completing daily anomalous images ano-recon The process of decoupling and extracting multi-scale bias features is similar to that in formulas (12)-(14), as shown below:

[0100] z1=z′1 (15)

[0101] z2=z′2-z′1 (16)

[0102] z3=z′3-z′2 (17)

[0103] Where z'1, z'2, and z'3 represent the mean features at small, medium, and large scales, respectively, and z1, z2, and z3 represent the mean decoupling features at small, medium, and large scales, respectively.

[0104] After obtaining the decoupling features at different scales of the weekly mean image and the daily deviation image, the small-scale decoupling features (y1, z1), medium-scale decoupling features (y2, z2), and large-scale decoupling features (y3, z3) corresponding to the mean and deviation are respectively input into the gated fusion module GFM for fusion.

[0105] Among them, the gating fusion mechanism is as follows Figure 4 As shown, Figure 4 g in i and o iThis represents the intermediate variables obtained after the gate and sigmoid operations. Inputting small-scale decoupling features (y1, z1), medium-scale decoupling features (y2, z2), and large-scale decoupling features (y3, z3), the intermediate process generates (g1, o1), (g2, o2), and (g3, o3), and finally outputs three fusion results. The calculation formula is as follows:

[0106] d = concat(y) i ,z i (18)

[0107]

[0108] Where i = 1, 2, 3, concat represents the join operation, g i =σ(d), o i =σ(d), where σ represents the sigmoid operation.

[0109] Finally, the three fusion results By fusing the images together and capturing features from different regions of the image through convolution, the final completed daily sea surface temperature image (SST) is obtained. recon The fusion process is as follows:

[0110]

[0111] Example 2

[0112] This embodiment provides a multi-scale sea surface temperature image completion method based on a denoising diffusion generation model. It utilizes a sea surface temperature image completion network based on the denoising diffusion generation model. The network structure and functional principles can be found in the description in Embodiment 1, and will not be repeated here. The image completion method specifically includes the following steps:

[0113] Step S1: Input the missing weekly mean image SST cor The image is used to generate a complete weekly mean image SST using the denoising and diffusion generation module I of the weekly mean generation module. short ;

[0114] Step S2: Input the missing daily image SST day The missing daily image SST is obtained using the daily deviation generation module. dat The completed weekly mean image SST obtained in step 1 short The deviation between them yields the daily deviation image SST. ano The completed daily deviation image SST is obtained through the denoising diffusion generation module II. ano-recon Step S3: The completed daily deviation image SST obtained in step 2... ano-reconThe completed weekly mean image SST obtained in step 1 short The multi-scale deviation feature decoupling extraction module and the multi-scale mean feature decoupling extraction module based on the multi-scale fusion module of weekly mean and daily deviation are respectively input. The multi-scale deviation feature decoupling extraction module extracts multi-scale deviation features to obtain deviation decoupling features y1, y2, and y3 at three different scales: small scale, medium scale, and large scale. The multi-scale mean feature decoupling extraction module extracts multi-scale mean features to obtain mean decoupling features z1, z2, and z3 at three different scales: small scale, medium scale, and large scale. Then, the deviation decoupling features and mean decoupling features are fused at the same scale to obtain three fusion results at small scale, medium scale, and large scale. Finally The images are then fused together to obtain a complete and incomplete SST image. recon .

[0115] The loss function of this method is as follows: L 总 =L short +L ano +L MFM (twenty one)

[0116] L 总 It is divided into three parts: L short L represents the loss function of the weekly mean generation module. ano L represents the loss function of the daily deviation generation module. MFM The loss function for the multi-scale fusion module based on weekly mean and daily deviation is described in detail below:

[0117] The loss function L of the weekly mean generation module short for:

[0118]

[0119] The loss function L of the daily deviation generation module ano for:

[0120]

[0121] Where t is the time step, x0 is the initial sample, ∈ is the true noise distribution, and ∈ θ It predicts the noise distribution, x t This represents the sample obtained after a time step t.

[0122] The loss function L based on the multi-scale fusion module of weekly mean and daily deviation MFM for:

[0123]

[0124] Where, xn This is a true image of sea surface temperature. This represents the generated result, where i represents small-scale, medium-scale, and large-scale. It is x n The corresponding binary masks are the same size. N represents the number of pixels in the SST image.

[0125] X is the input to this model:

[0126] X = [SST] corruop SST day (25).

[0127] In summary, this invention uses a diffusion model as the basic architecture in the weekly mean generation module and the daily deviation generation module. This solves the pain points of the original method, which uses a generative adversarial network (GAN) as its basic architecture, resulting in an extremely unstable training process and difficulty in convergence. The training process is more intuitive and simpler, while avoiding the loss of some key image information during training, thus improving the quality of the generated results. Existing methods use GANs for complex adversarial training, which makes the entire training process very volatile and difficult to converge, prone to problems such as mode collapse and mode breakdown. At the same time, GANs use an end-to-end training mode to generate images by inputting random noise or conditional variables, and rely solely on a discriminator for discrimination, which leads to the loss of some key image information during training, reducing the quality of the generated results. In contrast, the denoising diffusion model in this method only requires a single model and does not involve adversarial training, making the training process simpler and more stable. It has a better effect on processing image noise, accurately removing noise and retaining image details and texture information by adaptively adjusting the image completion method, thus improving the progress and fidelity of the restored image.

[0128] Secondly, a multi-scale fusion module based on weekly mean and daily deviation is proposed. This module ensures the quality of image generation from multiple perspectives, addressing the problem that the original method directly superimposes the daily deviation and weekly mean at a single scale during local modeling, resulting in a one-sided perspective that ignores information at different scales in the image. This exacerbates the entanglement of information at different scales, leading to poor fusion quality and reduced quality of the final generated image. The multi-scale fusion module based on weekly mean and daily deviation in this method can fully utilize information at small, medium, and large scales, fusing information at different scales separately. This reduces the problem of information entanglement at different scales, thereby improving the quality of the final generated image and enhancing the model's generalization ability.

[0129] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A sea surface temperature image completion network based on a denoising diffusion generation model, characterized in that, The system includes a weekly mean generation module, a daily deviation generation module, and a multi-scale fusion module based on the weekly mean and daily deviation. The weekly mean generation module includes a denoising diffusion generation module I, which is used to generate the input image SST containing the missing weekly mean. cor The weekly mean image SST is generated using the denoising diffusion generation module I. short The daily deviation generation module includes a denoising diffusion generation module II, which is used to calculate the missing daily image SST input. day And the completed weekly mean image SST short The data deviation between them yields the daily deviation image SST. ano The completed daily deviation image SST is obtained through the denoising diffusion generation module II. ano-recon The denoising diffusion generation module I and denoising diffusion generation module II have the same structure, including two processes: forward diffusion and backward denoising. In the forward diffusion process, for the missing input image, Gaussian noise is gradually added over T time steps, and the noise image at each step is recorded. When the image at step T becomes a pure Gaussian distribution, backward denoising is performed. The backward denoising process uses the result obtained from forward diffusion to backdifflate and generate a complete image by denoising the pure Gaussian distribution. The multi-scale fusion module based on weekly mean and daily deviation includes a multi-scale deviation feature decoupling extraction module, a multi-scale mean feature decoupling extraction module, and a multi-scale feature fusion module. First, the input complete daily deviation image SST is processed. ano-recon Multi-scale deviation feature decoupling extraction is performed using a multi-scale deviation feature decoupling extraction module to obtain deviation decoupling features y1, y2, and y3 at three different scales: small-scale, medium-scale, and large-scale. These features are then applied to the input padded weekly mean image SST. short Multi-scale mean feature decoupling extraction is performed using a multi-scale mean feature decoupling module to obtain mean decoupling features z1, z2, and z3 at three different scales: small-scale, medium-scale, and large-scale. Then, the bias decoupling feature and the mean decoupling feature are fused at the same scale to obtain three fused results at small-scale, medium-scale, and large-scale. Finally The images are then fused together to obtain a complete and incomplete SST image. recon .

2. The sea surface temperature image completion network based on a denoising diffusion generation model according to claim 1, characterized in that, The image processing procedure of the denoising diffusion generation module I is as follows: In the forward diffusion process, the initial missing weekly mean image SST is input. cor_0 Then, Gaussian noise is gradually added over T time steps, and the noisy image is recorded at each step. The image is then SST. cor_0 Convert to Gaussian white noise The formula is as follows: Wherein q(SST) cor_t |SST cor_t-1 ) represents the process of forward diffusion through the image SST at time t-1 at each step. cor_t-1 Adding Gaussian noise to obtain the image SST at time t cor_t The Gaussian noise is composed of the mean Sum of variances ∑ t =β t definition, Represents a Gaussian distribution, where I denotes the identity matrix; image SST cor_t By adding a variance of β at time step t t Gaussian noise, and according to the variance table, the sample SST from the previous step cor_t-1 Scaling to Received; By using the independence of the noise added in each step of formula (1), the total noise variance is calculated as follows: Where α t =1-β t ,β s Let represent the variance of the Gaussian noise when the step size is s. Equation (1) can be rewritten as a single step. Wherein q(SST) cor_t |SST cor_0 This indicates that during the forward process, the missing weekly mean image SST is fed forward at time step t. cor_0 Adding Gaussian noise to obtain the image SST cor_t ; In the backward denoising process, the input image SST at time T is... cor_T Then, denoising is performed gradually over T time steps, and the output of each step is recorded. The denoising process formula is as follows: Where, p θ (SST cor_t-1 |SST cor_t ) indicates that at each step of the backward pass, by removing the image SST cor_t Gaussian noise in the image SST cor_t-1 During backward denoising, the mean parameter μ of the noise in the Gaussian distribution is predicted by modeling with a neural network. θ (SST cor_t ,t) and variance ∑ θ (SST cor_t ,t); The predicted parameter mean μ θ (SST cor_t The formula for t is as follows: Where, ∈ θ (SST cor_t ,t) represents the step size t and the image is SST cor_t Noise at that time; The SST of the denoised image can be obtained. cor_t-1 for, 3. The sea surface temperature image completion network based on a denoising diffusion generation model according to claim 1, characterized in that, The image processing procedure of the denoising diffusion generation module II is as follows: During forward diffusion, the initial missing daily deviation image SST is input. ano_0 Then, Gaussian noise is gradually added over T time steps, and the noisy image is recorded at each step. The image is then SST. ano_0 Convert to Gaussian white noise The formula is as follows: Wherein q(SST) ano_t |SST ano_t-1 ) indicates that at each step of the forward pass, the image SST is passed through... ano_t-1 Adding Gaussian noise to obtain the image SST ano_t The Gaussian noise is composed of the mean Sum of variances ∑ t =β ' t Definition; Image SST ano_t By adding a variance of β at time step t ' t Gaussian noise, and according to the variance table, the previous sample SST ano_t-1 Scaling to Received; Calculate the total noise variance Then, formula (7) is rewritten in one step. Wherein q(SST) ano_t |SST ano_0 () indicates that during the forward process, the missing weekly mean image SST is passed through at time step t. ano_0 Adding Gaussian noise to obtain the image SST ano_t ; Subsequently, the backward denoising process inputs SST. ano_T Then, denoising is performed gradually over T time steps, and the output of each step is recorded. The denoising process formula is as follows: Where, p θ (SST ano_t-1 |SST ano_t ) indicates that at each step of the backward pass, by removing the image SST ano_t Gaussian noise in the image SST ano_t-1 During backward denoising, the mean parameter μ of the noise in the Gaussian distribution is predicted by modeling with a neural network. θ (SST ano_t ,t) and variance ∑ θ (SST ano_t ,t); The predicted parameter mean μ θ (SST ano_t The formula for t is as follows: Where, ∈ θ (SST ano_t ,t) represents the step size t and the image is SST ano_t Noise at that time; Obtain the denoised image SST ano_t-1 for, 4. The sea surface temperature image completion network based on a denoising diffusion generation model according to claim 1, characterized in that, The multi-scale fusion module based on weekly mean and daily deviation inputs the completed weekly mean image SST. short And the completed daily deviation image SST ano-recon Output complete daily sea surface temperature image SST recon First, three parallel dilated convolutional layers with different void ratios are used to analyze the predicted sea surface temperature (SST) images. short Multi-scale mean feature decoupling extraction is performed to extract features of different scales. Then, the features of larger scales are subtracted from the features of smaller scales to obtain the decoupled features of larger scales. After obtaining the decoupling features at different scales of the weekly mean image and the daily deviation image, the small-scale decoupling features (y1, z1), medium-scale decoupling features (y2, z2), and large-scale decoupling features (y3, z3) corresponding to the mean and deviation are respectively input into the gated fusion module GFM for fusion.

5. A sea surface temperature image completion method based on a denoising diffusion generation model, characterized in that, The sea surface temperature image completion network based on the denoising diffusion generation model described in any one of claims 1-4 is used to achieve this, specifically including the following steps: Step S1: Input the missing weekly mean image SST cor The image is used to generate a complete weekly mean image SST using the denoising and diffusion generation module I of the weekly mean generation module. short ; Step S2: Input the missing daily image SST day The missing daily image SST is obtained using the daily deviation generation module. day The completed weekly mean image SST obtained in step 1 short The deviation between them yields the daily deviation image SST. ano The completed daily deviation image SST is obtained through the denoising diffusion generation module II. ano-recon ; Step S3: The completed daily deviation image SST obtained in Step 2... ano-recon The completed weekly mean image SST obtained in step 1 short The multi-scale deviation feature decoupling extraction module and the multi-scale mean feature decoupling extraction module based on the multi-scale fusion module of weekly mean and daily deviation are respectively input. The multi-scale deviation feature decoupling extraction module extracts multi-scale deviation features to obtain deviation decoupling features y1, y2, and y3 at three different scales: small scale, medium scale, and large scale. The multi-scale mean feature decoupling extraction module extracts multi-scale mean features to obtain mean decoupling features z1, z2, and z3 at three different scales: small scale, medium scale, and large scale. Then, the deviation decoupling features and mean decoupling features are fused at the same scale to obtain three fusion results at small scale, medium scale, and large scale. Finally The images are then fused together to obtain a complete and incomplete SST image. recon .

Images