Method and system for sea surface temperature completion based on missing correlation and physical constraint

Abstract

The application belongs to the technical field of image processing, and discloses a sea surface temperature completion method and system based on missing correlation and physical constraints. The system comprises a sea surface temperature data acquisition module, a preprocessing module, a generator, a discriminator and a loss calculation module. The generator comprises a multi-scale adaptive missing correlation module and a physical constraint space-time interaction guiding module. The multi-scale adaptive missing correlation module comprises a space-time feature extractor and a dynamic multi-level up-down sampler. The space-time feature extractor extracts space-time correlation features, and then outputs the features to the dynamic multi-level up-down sampler for image reconstruction completion. The physical constraint space-time interaction guiding module is used for correcting and embedding physical constraints on the features reconstructed and completed by the multi-scale adaptive missing correlation module through numerical mode sea surface temperature data, and outputting a complete reconstructed image. The application improves the accuracy and reliability of sea temperature image completion.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and relates to image completion methods, particularly to a sea surface temperature completion method and system based on missing correlation and physical constraints. Background Technology

[0002] Sea surface temperature (SST) is one of the most important factors in ocean hydrology, influencing the exchange of heat, momentum, and water vapor between the sea and the atmosphere. Due to cloud cover, sensor malfunctions, and other reasons, some SST remote sensing data is missing, affecting its application in marine environmental monitoring and other fields. Therefore, completing SST field data is crucial for ocean-related applications. SST completion can be considered a form of image completion, which learns the image features of SST grid points that are not obscured by clouds to generate a complete SST image that is spatiotemporally and continuously continuous with the known areas in the missing regions.

[0003] In recent years, data-driven methods based on deep learning have been widely applied in sea surface temperature (SST) completion. Compared to traditional data interpolation and assimilation methods, which are computationally complex and struggle to fully model the complex nonlinear relationships between data, these methods achieve more accurate and faster completion of missing SST values ​​by learning and mining the distribution patterns and high-dimensional evolution characteristics of large amounts of spatiotemporal SST data. Currently, most cutting-edge SST completion methods employ fixed-structure generative adversarial network (GAN) models, using multi-scale modeling or a two-stage "coarse-to-fine" image completion mechanism to improve accuracy. However, these methods have several drawbacks:

[0004] 1) The significant fluctuations in the range of missing data in sea surface temperature remote sensing data are not considered. Due to factors such as cloud cover and sensor malfunctions, sea surface temperature data varies considerably across different times and spaces. For example, in a certain sea area, the missing data rate on a given day may reach 70%, but yesterday, without cloud cover, the missing rate would be less than 5%, while the missing rate the day before yesterday reached 40%, and the distribution of missing data is completely different from that of the current day. However, existing sea surface temperature data completion methods all use the same fixed network structure to train and reconstruct data with varying degrees of missing data. When the range of missing data rate is large, the model may experience difficulty in convergence and poor robustness, inevitably leading to a decline in performance.

[0005] 2) The data completion modeling process ignores the physical constraints of the spatiotemporal evolution of sea surface temperature, resulting in a lack of physical consistency and interpretability. For example, the methods mentioned above generally use remote sensing data with random noise as input and employ a purely data-driven approach to construct deep neural networks to mine multi-scale global and local spatial features to achieve sea surface temperature completion. However, this approach relies solely on observational remote sensing data for "black box" network modeling. When the missing rate is high, the model will struggle to uncover the spatiotemporal dependencies of sea surface temperature changes and generate complete sea surface temperature data that conforms to physical mechanisms, thus limiting the reliability of sea surface temperature completion. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a sea surface temperature (SST) completion method and system based on missing correlation and physical constraints. (1) A multi-scale adaptive missing correlation deep generator network is designed. By constructing a dynamic mapping relationship between the missing input remote sensing data and the multi-level upsampler / downsampler network structure, the generator structure will adapt to the SST reconstruction task under different missing rates, thereby improving the trainability and robustness of the model. In addition, the generator fully explores the spatiotemporal evolution correlation and non-stationary characteristics of SST from both spatial and frequency domain perspectives, improving the quality and detail information of the completed image. (2) Numerical pattern data is introduced to construct a physical constraint spatiotemporal interaction guidance module, making its completion results interpretable and physically consistent. The interaction layer based on the attention mechanism can progressively embed the physical constraints and spatiotemporal correlation characteristics of numerical pattern data into the remote sensing data features, and use cross-layer residual connection to achieve full fusion of initial features and deep interaction features, thereby improving the accuracy and reliability of SST completion.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] The sea surface temperature completion method based on missing correlations and physical constraints includes the following steps:

[0009] Step 1: Acquisition of sea surface temperature data;

[0010] Step 11: Acquire raw sea surface temperature remote sensing data (SST) for historical time periods. his The current full sea surface temperature image to be supplemented x rec and its corresponding binary missing mask matrix

[0011] Step 12: Acquire numerical model sea surface temperature (SST) data from historical time periods to the current time. nmd ;

[0012] Step 2: Introduce random cloud mask noise m randApply it to the current moment's uncompleted full sea surface temperature image x rec This allows us to calculate the current-time image used as input for model training. Missing rate and its missing mask matrix classification

[0013] The model includes a generator and a discriminator. The generator is used to reconstruct and complete the missing image, and the discriminator is used to determine whether the completion result of the generator is true or false.

[0014] Step 3: Multi-scale adaptive missing association;

[0015] Extract the image at the current time. The spatiotemporal correlation features are obtained, and high-dimensional features are extracted through dynamic multi-level upsampling and downsampling to complete the initial reconstruction and completion of the image to be completed, and the reconstruction features F are output. rec ;

[0016] Step 4: Physical constraint-based spatiotemporal interaction guidance;

[0017] Sea surface temperature data from numerical model SST nmd and reconstructed features F rec Through multi-layer convolution and physical constraint interaction, the reconstructed feature F is completed. rec The results are corrected and embedded with physical constraints, ultimately outputting a complete reconstructed image.

[0018] Step 5: Reconstruct and complete the image using the generator. Original input: Sea surface temperature image to be completed x rec and binary missing mask matrix They are fed together into the discriminator to train the entire model;

[0019] Step 6: Obtain the complete sea surface temperature image x rec The final complete image y rec :

[0020]

[0021] in, Indicates using a and All-one matrices of the same size and Subtraction, ⊙ represents the Hadamard product.

[0022] Furthermore, in step 3, a spatiotemporal feature extractor is constructed to pre-extract spatiotemporal correlation features, and then its output is fed into a dynamic multi-level upsampling and downsampling generator for image reconstruction and completion, as detailed below:

[0023] Step 31: According to Determine the number of network layers s of the dynamic multi-level upsampling generator, which means that the input data needs to be downsampled s times.

[0024] Step 32: Use a spatiotemporal feature extractor to process the image at the current time. Spatiotemporal correlation features are pre-extracted to obtain spatiotemporal fusion features.

[0025] Step 33: Based on the current network layer number s, use a dynamic multi-level upsampling and downsampling device to perform spatiotemporal fusion of features. High-dimensional feature extraction is performed to complete the initial reconstruction and completion of the image to be completed, resulting in feature F. rec .

[0026] Furthermore, in step 32, specifically, the spatiotemporal feature extractor utilizes the GRU to capture temporal dependencies and obtain temporal features. Graph convolution is used to extract the spatial correlation of grid points in irregular ocean regions and obtain spatial features. Then to and By splicing and merging, we can obtain

[0027]

[0028] Here, Concat() refers to the feature concatenation function, and ReLU() is the activation function.

[0029] Furthermore, in step 33, the dynamic multi-level upsampler includes s upsamplers and s downsamplers, constructing a sampler structure that fuses spatial and frequency domains. Specifically, the spatial domain operation of the downsampler consists of convolutional layers, activation layers, and max pooling layers, while the frequency domain operation consists of Fourier transform, low-pass filter, high-pass filter, inverse Fourier transform, and max pooling layers.

[0030] The spatial domain operations of the upsampler consist of an upsampling layer, a convolutional layer, and an activation layer, while the frequency domain operations consist of an upsampling layer, a Fourier transform, a low-pass filter, a high-pass filter, and an inverse Fourier transform.

[0031] Furthermore, step 4 is detailed below:

[0032] Step 41: Compare the sea surface temperature data (SST) from the numerical model. nmd Perform random masking preprocessing to obtain

[0033] Step 42: Reconstruct the features F output in step 33 rec and The data are fed into a physically constrained interactive guided feature extractor consisting of multiple convolutional layers and interactive layers to complete the generation of reconstructed features F in step 3. rec The results are corrected and embedded with physical constraints, ultimately outputting a complete reconstructed image.

[0034] Furthermore, in step 42, F rec and The data are fed into their respective convolutional layers, and then their outputs are fed into the interaction layer; the remote sensing data feature input on the left side of each interaction layer is set to I. rec The numerical pattern data feature input on the right is I. nmd The calculation process of the interaction layer is as follows: I rec and I nmd The query matrix Q is obtained after linear transformation. rec and Q nmd Key matrix K rec and K nmd Value matrix V rec and V nmd Then the attention weight matrix M is obtained. rec and M nmd ;

[0035]

[0036] Where d represents the query matrix Q rec and Q nmd Dimensions and It is K rec and K nmd The transpose of the matrix;

[0037] Numerical model data features I nmd Attention map transfer to refine remote sensing data features rec Attention map, i.e., using M nmd To optimize and correct M rec Therefore, the completion features corrected from numerical model data are used. The calculation process is as follows:

[0038]

[0039] Where g(·) is the gate function, derived from adaptive optimization.

[0040] The final output of the current interaction layer is represented as:

[0041]

[0042] in, and Representing remote sensing data features I rec Sum of numerical pattern data features I nmd Features after attention calculation.

[0043] Furthermore, in step 1, the raw sea surface temperature remote sensing data (SST) for historical time periods... his =[SST t-3 SST t-2 SST t-1 ], which is SST his The image x representing the sea surface temperature to be completed at time t is shown. rec Image sequences of the first three historical moments (SST) t-3 SST t-2 SST t-1 Sea surface temperature data from numerical model SST nmd =[SST' t-3 SST' t-2 SST' t-1 SST' t ], SST' t SST' represents the sea surface temperature image generated by the numerical model at time t. t-3 SST' t-2 SST' t-1 These are sea surface temperature images generated by numerical models from the three historical time points preceding the current time t.

[0044] Furthermore, in step 2, the random cloud mask noise m rand The cloud mask is randomly selected from other sea surface temperature remote sensing images in the original dataset and applied to the sea surface temperature image x to be completed. rec Perform model training. The calculation is as follows:

[0045]

[0046] Where ⊙ represents the Hadamarda complex;

[0047] Missing Mask Matrix Classification It is based on Missing data and historical sea surface temperature images SST his It is determined by the average of the missing rates at multiple time points.

[0048] The present invention also provides a sea surface temperature completion system based on missing correlation and physical constraints, for implementing the sea surface temperature completion method based on missing correlation and physical constraints as described above. The system includes a sea surface temperature data acquisition module, a preprocessing module, a generator, a discriminator and a loss calculation module, wherein the generator includes a multi-scale adaptive missing correlation module and a physical constraint spatiotemporal interaction guidance module.

[0049] The sea surface temperature data acquisition module is used to acquire raw sea surface temperature remote sensing data (SST) for historical time periods. his The current full sea surface temperature image to be supplemented x rec and its corresponding binary missing mask matrix Numerical model sea surface temperature data (SST) from historical time periods to the present moment nmd ;

[0050] The preprocessing module is used to process random cloud mask noise m rand The full sea surface temperature image to be supplemented at the current moment x rec Calculate the current time-stamped image used as input for model training. Missing rate and its missing mask matrix classification

[0051] The multi-scale adaptive missing association module includes a spatiotemporal feature extractor and a dynamic multi-level upsampling and downsampling unit. The spatiotemporal feature extractor performs spatiotemporal correlation feature pre-extraction and then sends its output to the dynamic multi-level upsampling and downsampling unit for preliminary image reconstruction and completion.

[0052] The physical constraint spatiotemporal interaction guidance module includes a physical constraint interaction guidance feature extractor composed of multiple convolutional layers and interaction layers, used to extract numerical model sea surface temperature data (SST). nmd The features initially reconstructed and completed by the multi-scale adaptive missing association module are corrected and physically constrained, and finally a complete reconstructed image is output.

[0053] The loss calculation module is used to calculate reconstruction loss and generate adversarial loss.

[0054] Compared with the prior art, the advantages of the present invention are as follows:

[0055] (1) Considering that when the missing rate of sea surface temperature data varies greatly, existing network models with the same fixed structure may have difficulty converging and poor robustness, this invention designs a multi-scale adaptive missing correlation module to construct a dynamic mapping relationship between the missing status of the input remote sensing data image and the multi-level upsampling generator network structure. It can dynamically adjust the number of layers of the upsampling generator network structure according to the missing status of the input remote sensing data image to be completed, and optimize the samplers of the corresponding levels in ascending order of missing rate during training. The optimized samplers remain fixed, thereby improving the trainability and robustness of the model.

[0056] (2) In order to improve the detailed information of the completed sea surface temperature image, this invention constructs a sampler structure that integrates spatial and frequency domains, so as to fully explore the global and local spatiotemporal evolution features of the sea surface temperature image sequence from multiple perspectives. In the frequency domain channel, the sampler's feature mining capability for local non-stationary changes in sea surface temperature is enhanced by using low-pass and high-pass filters in series, thus improving the detailed information of the completed image.

[0057] (3) To address the problem that existing methods do not consider the physical mechanism constraints in the spatiotemporal evolution of sea surface temperature, this invention proposes a physical constraint spatiotemporal interaction guidance module based on attention mechanism and cross-layer residual connection method by introducing FVCOM numerical model data. This module can realize the progressive spatiotemporal correlation interaction fusion of remote sensing data features and numerical model data features, enhance the spatiotemporal correlation and physical consistency of the completed image, thereby improving the reliability of sea surface temperature completion and making its results interpretable and physically consistent. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is an architectural diagram of the method of the present invention;

[0060] Figure 2 This is a structural diagram of a single upsampler of the present invention;

[0061] Figure 3 This describes the calculation process of the interaction layer in this invention;

[0062] Figure 4 This is a system structure diagram of the present invention. Detailed Implementation

[0063] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0064] Example 1

[0065] Combination Figure 1 As shown, this embodiment provides a sea surface temperature completion method based on missing correlations and physical constraints, including the following steps:

[0066] Step 1: Acquisition of sea surface temperature data;

[0067] Step 11: Acquire raw sea surface temperature remote sensing data (SST) for historical time periods. his The current full sea surface temperature image to be supplemented x rec and its corresponding binary missing mask matrix

[0068] Step 12: Obtain sea surface temperature (SST) data from the finite-volume coastal ocean model (FVCOM) from historical time periods to the current time. nmd .

[0069] In step 1, the raw sea surface temperature remote sensing data (SST) for historical time periods... his =[SST t-3 SST t-2 SST t-1 ], which is SST his The image x representing the sea surface temperature to be completed at time t is shown. rec Image sequences of the first three historical moments (SST) t-3 SST t-2 SST t-1 Sea surface temperature data from numerical model SST nmd =[SST' t-3 SST' t-2 SST' t-1 SST' t ], SST' t SST' represents the sea surface temperature image generated by the numerical model at time t. t-3 SST' t-2 SST' t-1 These are sea surface temperature images generated by numerical models from the three historical time points preceding the current time t.

[0070] Step 2: Introduce random cloud mask noise m rand Apply it to the current moment's uncompleted full sea surface temperature image x rec This allows us to calculate the current-time image used as input for model training. Missing rate and its missing mask matrix classification

[0071] The model includes a generator and a discriminator. The generator is used to reconstruct and complete the missing image, and the discriminator is used to determine whether the completion result of the generator is true or false.

[0072] In step 2, the random cloud mask noise m rand The cloud mask is randomly selected from other sea surface temperature remote sensing images in the original dataset and applied to the sea surface temperature image x to be completed. rec Perform model training. The calculation is as follows:

[0073]

[0074] Where ⊙ represents the Hadamarda.

[0075] Missing Mask Matrix Classification It is based on Missing data and historical sea surface temperature images SST his It is determined by the average of the missing rates at multiple time points.

[0076] Step 3: Multi-scale adaptive missing association;

[0077] Extract the image at the current time. The spatiotemporal correlation features are obtained, and high-dimensional features are extracted through dynamic multi-level upsampling and downsampling to complete the initial reconstruction and completion of the image to be completed, and the reconstruction features F are output. rec .

[0078] In step 3, a spatiotemporal feature extractor is constructed to pre-extract spatiotemporal correlation features, and then its output is fed into a dynamic multi-level upsampling / downsampling generator for image reconstruction and completion. Specifically:

[0079] Step 31: According to Determine the number of network layers s in the dynamic multi-level upsampling and downsampling generator, which means that the input data needs to be downsampled s times.

[0080] In step 31, considering the large range of data missing rate changes, the structure of the upsampling and downsampling will be dynamically increased as the data missing rate increases, thereby improving the trainability and robustness of the model. If there are k categories, the range of the number of upsampling / downsampling network layers s is 1-k.

[0081] Step 32: Use a spatiotemporal feature extractor to process the image at the current time. Spatiotemporal correlation features are pre-extracted to obtain spatiotemporal fusion features.

[0082] In step 32, considering that in generative adversarial networks (GANs), when input and target data have similar spatiotemporal dependencies, the generator tends to generate data with similar distributions, a spatiotemporal feature extractor is constructed to pre-extract spatiotemporal correlation features. Its output is then fed into a dynamic multi-level upsampling / downsampling generator for image reconstruction and completion. Specifically, the spatiotemporal feature extractor utilizes a GRU to capture temporal dependencies and obtain temporal features. Graph convolution is used to extract the spatial correlation of grid points in irregular ocean regions and obtain spatial features. Then to and By splicing and merging, we can obtain

[0083]

[0084] Here, Concat() refers to the feature concatenation function, and ReLU() is the activation function.

[0085] Step 33: Based on the current network layer number s, use a dynamic multi-level upsampling and downsampling device to perform spatiotemporal fusion of features. High-dimensional feature extraction is performed to complete the initial reconstruction and completion of the image to be completed, resulting in feature F. rec .

[0086] In step 33, the dynamic multi-level upsampler includes s upsamplers and s downsamplers, such as... Figure 2 The structure of a single upsampler is shown. The spatiotemporal evolution of sea surface temperature is highly complex and non-stationary. However, existing methods only reconstruct missing data from a spatial domain perspective, which leads to performance bottlenecks and makes it difficult to fully explore its spatiotemporal evolution characteristics. Therefore, a sampler structure that integrates spatial and frequency domains is constructed to fully explore local and global spatiotemporal evolution features from multiple perspectives.

[0087] Specifically, the spatial domain operations of the downsampler consist of convolutional layers, activation layers, and max pooling layers, while the frequency domain operations consist of Fourier transform, low-pass filter, high-pass filter, inverse Fourier transform, and max pooling layers.

[0088] The spatial domain operations of the upsampler consist of an upsampling layer, a convolutional layer, and an activation layer, while the frequency domain operations consist of an upsampling layer, a Fourier transform, a low-pass filter, a high-pass filter, and an inverse Fourier transform.

[0089] A low-pass filter can filter out some noise in sea surface temperature images, and the high-pass filter that follows is responsible for extracting local details that change rapidly and non-stationarily in the features. For example... Figure 2 As shown, in the i-th sampler layer, the input of downsampler i is The output of downsampler i is Similarly, the input to upsampler i is The result of upsampler i is then combined with the input of downsampler i to obtain the output. As shown in the following equation:

[0090]

[0091] in, It is the downsampler of the i-th layer. It is the upsampler of the i-th layer, and ⊕ indicates that the corresponding elements of the feature tensor are added together.

[0092]

[0093] in, The expression represents spatial domain downsampling operation, FT(·) represents Fourier transform, LF(·) is a low-pass filter, HF(·) is a low-pass filter, and IFT(·) represents inverse Fourier transform. This indicates an upsampling operation in the spatial domain.

[0094] During training, the samplers at the corresponding levels are optimized in ascending order of missing rate, and the optimized samplers are kept fixed.

[0095] Step 4: Physical constraint-based spatiotemporal interaction guidance;

[0096] Sea surface temperature data from numerical model SST nmd and reconstructed features F rec Through multi-layer convolution and physical constraint interaction, the reconstructed feature F is completed. rec The results are corrected and embedded with physical constraints, ultimately outputting a complete reconstructed image.

[0097] As a preferred implementation, step 4 is as follows:

[0098] Step 41: Compare the sea surface temperature data (SST) from the numerical model. nmd Perform random masking preprocessing to obtain

[0099] In step 41, considering the complete numerical model data SST nmd The input completely guides the model's optimization direction and can lead to overfitting. Therefore, a random masking operation is introduced here to introduce some missing features from the numerical model data, achieving a relative balance between the features of the numerical model data and the remote sensing data, thus better enabling subsequent feature interaction learning. The SST after the random masking operation... nmd Represented as:

[0100]

[0101] Where, m' rand The cloud mask was randomly selected from other sea surface temperature remote sensing images in the original dataset.

[0102] Step 42: Reconstruct the features F output in step 33 rec and The data are fed into a physically constrained interactive guided feature extractor consisting of multiple convolutional layers and interactive layers to complete the generation of reconstructed features F in step 3. rec The results are corrected and embedded with physical constraints, ultimately outputting a complete reconstructed image.

[0103] In step 42, F rec and The data are fed into their respective convolutional layers, and then their outputs are fed into the interaction layer; the remote sensing data feature input on the left side of each interaction layer is set to I. rec The numerical pattern data feature input on the right is I. nmd The calculation process of the interaction layer is as follows Figure 3 As shown: I rec and I nmd The query matrix Q is obtained after linear transformation. rec and Q nmd Key matrix K rec and K nmd Value matrix V rec and V nmd Then the attention weight matrix M is obtained. rec and M nmd .

[0104]

[0105] Where d represents the query matrix Q rec and Q nmd Dimensions and It is K rec and K nmd The transpose of .

[0106] For remote sensing images of sea surface temperature to be completed, the loss of information in missing regions makes it difficult to fully model their spatiotemporal correlation with regions with values. However, in numerical model data, the same region will retain some values ​​in the missing region after random masking, providing physically constrained guidance for the spatiotemporal correlation between missing and valuable regions in remote sensing images. Therefore, the numerical model data feature I... nmd Attention map transfer to refine remote sensing data features rec Attention map, i.e., using M nmd To optimize and correct Mrec Therefore, the completion features corrected from numerical model data are used. The calculation process is as follows:

[0107]

[0108] Where g(·) is the gate function, derived from adaptive optimization.

[0109] The final output of the current interaction layer is represented as:

[0110]

[0111] in, and Representing remote sensing data features I rec Sum of numerical pattern data features I nmd Features after attention calculation.

[0112] Step 5: Reconstruct and complete the image using the generator. Original input: Sea surface temperature image to be completed x rec and binary missing mask matrix They are fed together into the discriminator to train the entire model;

[0113] Step 6: Obtain the complete sea surface temperature image x rec The final complete image y rec :

[0114]

[0115] in, Indicates using a and All-one matrices of the same size and Subtraction, ⊙ represents the Hadamard product.

[0116] Furthermore, it should be noted that the loss function of this invention is as follows:

[0117]

[0118] Where α is a hyperparameter, The reconstruction loss after the generator completes image reconstruction is represented, and is calculated as follows:

[0119]

[0120] in, Represents the image x to be completed rec The binary missing mask matrix, whose size is x rec same, and This represents the i-th grid point in the image. G(·) represents the current moment image of the input generator after introducing random cloud mask noise, and G(·) represents the generator network of the present invention.

[0121] Represents the adversarial loss of a generative adversarial network:

[0122]

[0123] Where D(·) represents the discriminator, and both D(·) and G(·) participate in the training.

[0124] Example 2

[0125] like Figure 4 As shown, this embodiment designs a sea surface temperature completion system based on missing correlation and physical constraints, including a sea surface temperature data acquisition module, a preprocessing module, a generator, a discriminator, and a loss calculation module. The generator includes a multi-scale adaptive missing correlation module and a physical constraint spatiotemporal interaction guidance module.

[0126] The sea surface temperature data acquisition module is used to acquire raw sea surface temperature remote sensing data (SST) for historical time periods. his The current full sea surface temperature image to be supplemented x rec and its corresponding binary missing mask matrix Numerical model sea surface temperature data (SST) from historical time periods to the present moment nmd .

[0127] The preprocessing module is used to process random cloud mask noise m rand The full sea surface temperature image to be supplemented at the current moment x rec Calculate the current time-stamped image used as input for model training. Missing rate and its missing mask matrix classification

[0128] The multi-scale adaptive missing correlation module includes a spatiotemporal feature extractor and a dynamic multi-level upsampling and downsampling unit. The spatiotemporal feature extractor pre-extracts spatiotemporal correlation features and then sends its output to the dynamic multi-level upsampling and downsampling unit for preliminary image reconstruction and completion.

[0129] The physical constraint spatiotemporal interaction guidance module includes a physical constraint interaction guidance feature extractor composed of multiple convolutional layers and interaction layers. It is used to correct and embed the features initially reconstructed and completed by the multi-scale adaptive missing correlation module using numerical model sea surface temperature data SSTnmd, and finally output a complete reconstructed image.

[0130] The loss calculation module is used to calculate reconstruction loss and generate adversarial loss.

[0131] This system is used to implement the sea surface temperature completion method as described in Example 1. The data processing procedures of each module can be found in the description of Example 1, and will not be repeated here.

[0132] In summary, most current deep learning-based methods for sea surface temperature image completion employ the same fixed network structure for training and reconstructing data images with varying degrees of missing information. However, this can lead to convergence difficulties and poor robustness when the missing information rate of sea surface temperature data varies significantly, inevitably resulting in a performance degradation. Therefore, this invention designs a multi-scale adaptive missing information association module. By constructing a dynamic mapping relationship between the missing information of the input remote sensing data image and the multi-level upsampling / downsampling generator network structure, the number of network layers in the generator dynamically adapts to the reconstruction task of the sea surface temperature image to be completed under different missing information rates. In other words, the higher the missing information rate of the image to be completed, the more network layers the generator needs, thus improving the trainability and robustness of the model.

[0133] Furthermore, this invention constructs a sampler structure that integrates spatial and frequency domains in the multi-scale adaptive missing correlation module, which fully explores local and global spatiotemporal evolution features from multiple perspectives. In the frequency domain, low-pass and high-pass filters are used to improve the sampler's ability to extract local non-stationary evolution features of sea surface temperature, thereby improving the detailed information of the completed image.

[0134] Furthermore, existing methods neglect the physical constraints of modeling the spatiotemporal evolution of sea surface temperature, resulting in a lack of interpretability and physical consistency in the completed images. While some methods treat numerical model data as ground truth and use loss functions to evaluate the physical correctness of the reconstructed sea surface temperature images, these methods do not fully exploit the physical constraints and spatiotemporal dependencies within the numerical model data. This is particularly problematic when the missing data rate is high, making it difficult to effectively improve the quality and reliability of the completed images. Therefore, this invention proposes a physically constrained spatiotemporal interaction guidance module. It designs an interaction layer based on an attention mechanism and a connection method for cross-layer residuals, progressively achieving deep spatiotemporal correlation interaction between remote sensing data features and numerical model data features. This improves the reliability of sea surface temperature completion, making the results interpretable and physically consistent.

[0135] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A sea surface temperature completion method based on missing correlations and physical constraints, characterized in that, Includes the following steps: Step 1: Acquisition of sea surface temperature data; Step 11: Acquire raw sea surface temperature remote sensing data for historical time periods The current full sea surface temperature image to be supplemented and its corresponding binary missing mask matrix ; Step 12: Obtain numerical model sea surface temperature data from historical time periods to the current moment. ; Step 2: Introduce random cloud mask noise Apply it to the current full sea surface temperature image to be supplemented. This allows us to calculate the current-time image used as input for model training. Missing rate and its missing mask matrix classification ( ); The model includes a generator and a discriminator. The generator is used to reconstruct and complete the missing image, and the discriminator is used to determine whether the completion result of the generator is true or false. Step 3: Multi-scale adaptive missing association; Extract the image at the current time. The spatiotemporal correlation features are obtained, and high-dimensional features are extracted through dynamic multi-level upsampling and downsampling to complete the initial reconstruction and completion of the image to be completed, and the reconstructed features are output. ; In step 3, a spatiotemporal feature extractor is constructed to pre-extract spatiotemporal correlation features, and then its output is fed into a dynamic multi-level upsampler for image reconstruction and completion, as follows: Step 31: According to ( Determine the number of network layers s in the dynamic multi-level upsampler, that is, the input data needs to be downsampled s times; Step 32: Use a spatiotemporal feature extractor to process the image at the current time. Spatiotemporal correlation features are pre-extracted to obtain spatiotemporal fusion features. ; Step 33: Based on the current network layer number s, use a dynamic multi-level upsampling and downsampling device to perform spatiotemporal fusion of features. High-dimensional feature extraction is performed to complete the initial reconstruction and completion of the image to be completed, and the features are obtained. In step 33, the dynamic multi-level upsampler includes s upsamplers and s downsamplers, constructing a sampler structure that fuses spatial and frequency domains. Specifically, the spatial domain operations of the downsampler consist of convolutional layers, activation layers, and max pooling layers, while the frequency domain operations consist of Fourier transform, low-pass filter, high-pass filter, inverse Fourier transform, and max pooling layers. The spatial domain operations of the upsampler consist of upsampling layers, convolutional layers, and activation layers, while the frequency domain operations consist of upsampling layers, Fourier transform, low-pass filter, high-pass filter, and inverse Fourier transform. Step 4: Physical constraint-based spatiotemporal interaction guidance; Numerical model sea surface temperature data and reconstructed features Through multi-layer convolution and physical constraint interaction, the reconstructed features are completed. The results are corrected and embedded with physical constraints, ultimately outputting a complete reconstructed image. ; Step 4 is as follows: Step 41: Compare sea surface temperature data from numerical models Perform random masking preprocessing to obtain ; Step 42: Reconstruct the features output in step 33 and The data are fed into a physically constrained interactive guided feature extractor consisting of multiple convolutional layers and interactive layers to complete the generation of reconstructed features in step 3. The results are corrected and embedded with physical constraints, ultimately outputting a complete reconstructed image. ; In step 42, and The data are fed into their respective convolutional layers, and then their outputs are fed into the interaction layer; the remote sensing data feature input on the left side of each interaction layer is set as... The numerical pattern data feature input on the right is The calculation process of the interaction layer is as follows: and The query matrix is ​​obtained after linear transformation. and Key matrix and Value matrix and Then the attention weight matrix is ​​obtained. and ; ， ； Where d represents the query matrix Dimensions and yes and The transpose of the matrix; Features of numerical model data Attention map transfer to refine remote sensing data features Attention map, that is, using To optimize and correct Therefore, the completion features corrected from numerical model data are used. The calculation process is as follows: )) in, ( ) is a gate function derived from adaptive optimization. ; The final output of the current interaction layer is represented as: ， ； in, and Representing remote sensing data features respectively Features of sum-numerical pattern data Features after attention calculation; Step 5: Reconstruct and complete the image using the generator. Original input: sea surface temperature image to be completed and binary missing mask matrix They are fed together into the discriminator to train the entire model; Step 6: Obtain the complete sea surface temperature image to be supplemented. The final complete image : ； in, Indicates using a and All-one matrices of the same size and Subtraction, It represents the Hadamardi (or Hadama) stack.

2. The sea surface temperature completion method based on missing correlation and physical constraints according to claim 1, characterized in that, In step 32, specifically, the spatiotemporal feature extractor uses the GRU to capture temporal dependencies and obtain temporal features. Graph convolution is used to extract the spatial correlation of grid points in irregular ocean areas and obtain spatial features. Then to and By splicing and merging, we can obtain : ； in, () refers to the feature concatenation function. ) is the activation function.

3. The sea surface temperature completion method based on missing correlation and physical constraints according to claim 1, characterized in that, In step 1, the raw sea surface temperature remote sensing data for historical time periods. = ， ， That is It is the current Sea surface temperature images are pending completion. Image sequences of the first three historical moments ， ， Numerical model sea surface temperature data = ， ， ， , This represents the sea surface temperature image generated by the numerical model at time t. ， ， These are the current Sea surface temperature images generated by numerical models from the three historical moments prior to this time.

4. The sea surface temperature completion method based on missing correlation and physical constraints according to claim 1, characterized in that, In step 2, random cloud mask noise The cloud mask is randomly selected from other sea surface temperature remote sensing images in the original dataset and applied to the sea surface temperature image to be completed. Perform model training. The calculation is as follows: ； in, It represents the Hadamardi (or Hadama) stack; Missing mask matrix ( ) is based on Missing data and historical sea surface temperature images It is determined by the average of the missing rates at multiple time points.

5. A sea surface temperature completion system based on missing correlations and physical constraints, characterized in that, To implement the sea surface temperature completion method based on missing correlation and physical constraints as described in any one of claims 1-4, the system includes a sea surface temperature data acquisition module, a preprocessing module, a generator, a discriminator, and a loss calculation module, wherein the generator includes a multi-scale adaptive missing correlation module and a physical constraint spatiotemporal interaction guidance module. The sea surface temperature data acquisition module is used to acquire raw sea surface temperature remote sensing data for historical time periods. Current full sea surface temperature image to be supplemented and its corresponding binary missing mask matrix Numerical model sea surface temperature data from historical periods to the present moment ; The preprocessing module is used to remove random cloud mask noise. Full sea surface temperature image to be supplemented at the current moment Calculate the current time-stamped image used as input for model training. Missing rate and its missing mask matrix classification ( ); The multi-scale adaptive missing association module includes a spatiotemporal feature extractor and a dynamic multi-level upsampling and downsampling unit. The spatiotemporal feature extractor performs spatiotemporal correlation feature pre-extraction and then sends its output to the dynamic multi-level upsampling and downsampling unit for preliminary image reconstruction and completion. The physical constraint spatiotemporal interaction guidance module includes a physical constraint interaction guidance feature extractor composed of multiple convolutional layers and interaction layers, used to extract numerical model sea surface temperature data (SST). nmd The features initially reconstructed and completed by the multi-scale adaptive missing association module are corrected and physically constrained, and finally a complete reconstructed image is output. The loss calculation module is used to calculate reconstruction loss and generate adversarial loss.

Images