Sea surface temperature completion method and system based on cross-scale spatiotemporal significant and insignificant modeling

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a sea surface temperature completion method and system based on cross-scale space-time significant and insignificant modeling, which comprises the following steps: S1, inputting a damaged SST, a weekly mean SST and a historical SST sequence, and obtaining current scale wavelet sub-band features through wavelet decomposition; S2, based on the sub-band features, extracting spatial steady-state features and abnormal features, and time steady-state features and abnormal features; S3, updating and multi-modal fusing and synergistically enhancing the four types of features of each sub-band to complete fine-scale modeling; S4, down-sampling a low-frequency sub-band to generate a coarse scale, repeatedly down-sampling and repeating steps S2-S3 on the coarse scale to complete fine-to-coarse modeling; then, layer-by-layer up-sampling high-level features and cross-scale integration are performed to complete coarse-to-fine reconstruction; and S5, inverse wavelet transformation is performed on the most fine-scale reconstructed features to obtain a complete SST image; and the present application improves the robustness and accuracy of completing sea surface temperature completion under large-area missing.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a sea surface temperature completion method and system based on cross-scale spatiotemporal saliency and non-saliency modeling. Background Technology

[0002] Using deep neural networks to complete the spatiotemporal data field of sea surface temperature is an important attempt to apply artificial intelligence to marine science in recent years. Compared with traditional data interpolation and data assimilation methods, the deep neural network-based method can learn the distribution of spatiotemporal temperature data by estimating a large amount of historical ocean data, thereby achieving the goal of more accurately completing the missing values ​​of the ocean temperature field.

[0003] Currently, cutting-edge deep learning-based methods for ocean surface temperature (SST) completion use the missing SST image and its corresponding weekly average SST image as dual inputs. They construct a dual-branch structure in the spatial domain that simultaneously extracts stability and anomalies, decoupling the modeling of key SST features. Subsequently, the spatial domain features from the two branches are mapped to the frequency domain, and adaptive frequency domain filtering and deep fusion are performed by introducing a Fourier Neural Operator (FNO). The fused frequency domain features are then returned to the spatial domain via inverse Fourier transform, and the final SST reconstruction result is generated through a feature mapping network. Its advantages lie in two aspects: firstly, it addresses the problem of traditional methods only performing representation learning of salient information, resulting in insufficient mining of non-salient (anomaly) information and limiting completion accuracy; secondly, it addresses the problem that traditional methods rely on spatial domain representation strategies from semantic and visual tasks, making it difficult to fully represent ocean data lacking clear semantic information.

[0004] However, the above methods have the following problems: First, existing methods typically model the sea surface temperature field only at a single scale in the spatial domain, making it difficult to simultaneously consider the synergistic constraints between the structure at different spatial scales and the evolutionary characteristics at different temporal scales, thus limiting the overall ability to characterize complex ocean processes. The sea surface temperature field is simultaneously affected by multi-scale dynamic processes in space; however, most existing methods only model at a single scale or single resolution in the spatial domain, ignoring the intrinsic relationship between the structure at different spatial scales and its temporal evolution. In the case of large-area data gaps, it is difficult to simultaneously consider local anomalies and global background constraints, easily leading to structural distortion or overall distribution shift. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a sea surface temperature (SST) completion method and system based on cross-scale spatiotemporal saliency and non-saliency modeling. It utilizes wavelet decomposition to consider information at different scales and extracts and fuses spatiotemporal steady-state and anomalous features in both the spatial and temporal domains, thus solving the problem of insufficient completion accuracy caused by single-scale and single-dimensional modeling in existing methods.

[0006] To achieve the above objectives, the first aspect of this invention proposes a sea surface temperature completion method based on cross-scale spatiotemporal saliency and non-saliency modeling, comprising the following steps: S1. Wavelet decomposition: Wavelet decomposition is performed on the input damaged SST image, weekly mean SST image and historical SST image sequence to obtain the wavelet sub-band features of each image at the current scale. S2. Extraction of Spatiotemporal Steady-State and Anomaly Features: For damaged SST images and weekly mean SST images: based on the wavelet subband features at the current scale, spatial steady-state features and spatial anomalous features are extracted through an attention mechanism; For historical SST image sequences: based on wavelet subband features at the current scale, time steady-state features and time anomaly features are extracted through a gating mechanism; S3. Fusion of steady-state and anomalous features: Single-mode updates are performed on spatial steady-state features, spatial anomaly features, temporal steady-state features, and temporal anomaly features. The updated features are then fused using a unified fusion operator for multi-mode fusion. The fused features are then enhanced across modes to complete fine-scale modeling and obtain the reconstructed features of each wavelet sub-band at the current scale. S4. Multi-scale feature reconstruction: Wavelet downsampling is performed on the low-frequency subbands of the wavelet subband features at the current scale of each image to generate a coarse scale. Downsampling and S2-S3 are repeated at the coarse scale until multi-scale modeling from fine to coarse is completed to obtain multi-scale wavelet subband features. The high-level scale features in the multi-scale wavelet subband features are passed to the low-level scale through layer-by-layer upsampling, and feature integration is performed between different scales to complete the multi-scale feature reconstruction from coarse to fine. S5. Inverse wavelet reconstruction: The finest scale reconstruction features are reconstructed using inverse wavelet reconstruction to obtain the final complete SST image.

[0007] Furthermore, step S1 specifically includes: Input damaged SST image Weekly mean SST image And historical SST image sequences, through two-dimensional discrete wavelet transform, wavelet decomposition is performed to obtain the wavelet sub-band features of each image at the current scale, namely the sub-band components of four different frequency bands, expressed as: ; in, Indicates a damaged SST image Weekly mean SST image Or historical SST image sequences. Represents a two-dimensional discrete wavelet transform. Indicates the low-frequency sub-band. , and These represent the high-frequency subbands in the horizontal, vertical, and diagonal directions, respectively. , , , These represent the low-frequency subband features, horizontal high-frequency subband features, vertical high-frequency subband features, and diagonal high-frequency subband features of each image at the current scale.

[0008] Furthermore, based on the wavelet subband features at the current scale, spatial steady-state features and spatial anomaly features are extracted through an attention mechanism, including: S201. Based on the wavelet subband features at the current scale, in the spatial dimension, the damaged SST image Weekly mean SST image any subband Spatial features are extracted through shared convolutional coding layers; S202. Based on spatial features, spatial steady-state features are extracted through a spatial steady-state attention mechanism, represented as follows: ; Among them, sub-band , Indicates the low-frequency sub-band. , and These represent the high-frequency sub-bands in the horizontal, vertical, and diagonal directions, respectively. This represents the normalized exponential function, used to map similarity to an attention weight distribution. Indicates a damaged SST image In sub-band Spatial characteristics, Represents the weekly mean SST image Spatial characteristics of sub-bands; This represents the feature dimension, used for scaling and normalizing the similarity results. Indicates spatial steady-state characteristics; S203. Based on spatial features, spatial anomaly features are extracted through a reverse attention mechanism, represented as follows: ; in, This refers to the reverse mapping of attention weights, i.e., the reverse attention mechanism. Indicates spatial anomaly characteristics.

[0009] Furthermore, based on the wavelet subband features at the current scale, time steady-state features and time anomaly features are extracted through a gating mechanism, including: S204. Any sub-band of historical SST image sequence based on the wavelet sub-band features at the current scale. Spatial features of historical SST image sequences are extracted through shared convolutional coding layers; S205. Based on the spatial features of historical SST image sequences, extract time-steady features with long-term consistent evolution through a time-steady-state gating mechanism. S206. Based on the spatial features of historical SST image sequences, extract temporal anomaly features through a temporal anomaly gating mechanism.

[0010] Furthermore, single-modal updates are performed on spatial steady-state features, spatial anomaly features, temporal steady-state features, and temporal anomaly features. The updated features are then fused using a unified fusion operator for multimodal fusion. Cross-modal collaborative enhancement is performed on the fused features to complete fine-scale modeling, including: S301. Based on each wavelet subband in the current scale wavelet subband features, a two-dimensional convolutional residual update unit is used for single-mode update; wherein, the two-dimensional convolutional residual update unit includes a two-dimensional convolutional layer, a normalization layer, a nonlinear activation layer, and residual connections; the single-mode update is as follows: spatial steady-state features and temporal steady-state features are input into the steady-state update unit for update, and spatial anomaly features and temporal anomaly features are input into the anomaly update unit for update; the parameters of the steady-state update unit and the anomaly update unit are shared between different wavelet subbands, and the parameters of the steady-state update unit and the anomaly update unit are independent of each other; S302. Input the updated spatial steady-state features and the updated temporal steady-state features into the steady-state fusion operator to obtain the steady-state fusion features; input the updated spatial anomaly features and the updated temporal anomaly features into the anomaly fusion operator to obtain the anomaly fusion features; wherein, the steady-state fusion operator and the anomaly fusion operator adopt the same fusion structure, both including feature concatenation, difference feature calculation, product correlation calculation, modal weight generation, and weighted fusion process; the parameters of the steady-state fusion operator and the anomaly fusion operator are shared among the wavelet subbands, and the parameters of the steady-state fusion operator and the anomaly fusion operator are independent of each other; S303. Input the steady-state fusion features and the abnormal fusion features into the cross-modal collaborative enhancement unit. Generate gated descriptive features based on the differences and correlations between the steady-state fusion features and the abnormal fusion features. The gated descriptive features are convolved to obtain gated weights. The abnormal fusion features are mapped through channels and multiplied element-wise with the gated weights and added to the steady-state fusion features to obtain collaborative fusion features. Input the collaborative fusion features into the collaborative update unit for residual optimization to complete fine-scale modeling and obtain the reconstruction features of each wavelet sub-band at the current scale.

[0011] Furthermore, the feature concatenation, differential feature calculation, product correlation calculation, modality weight generation, and weighted fusion process include: For any fusion structure The spatial input features and temporal input features are respectively and ,but: ; ; ; ; in, Indicates steady state, Indicates an anomaly. This represents the updated spatial steady-state characteristics. This represents the updated time steady-state characteristics. Indicates the updated spatial anomaly characteristics. This indicates the anomaly characteristics of the updated time. Represents the input features in the steady-state space. Steady-state time-side input features, Indicates the input features of the abnormal spatial side. Indicates input features at abnormal times; The spatial and temporal input features are concatenated, differential features are calculated, and product correlation is calculated to obtain the fused descriptive features. Based on the fusion description features, the spatial and temporal weights are generated by normalization through convolutional layers. At the same spatial location and the same channel, the spatial and temporal weights are added element by element to 1. The spatial and temporal input features are weighted and fused using the generated spatial and temporal weights, and the fused features of the corresponding path are obtained through convolution.

[0012] Furthermore, the finest-scale reconstructed features are reconstructed using inverse wavelet reconstruction to obtain the final complete SST image, including: S501. The finest scale reconstructed features are mapped back to the corresponding wavelet subband coefficient space through convolution operations to obtain the wavelet subband coefficients of any subband b after mapping. S502. The sub-band coefficients after mapping are recombined using inverse wavelet transform to recover the final complete SST image. .

[0013] To achieve the above objectives, a second aspect of the present invention proposes a sea surface temperature completion system based on cross-scale spatiotemporal saliency and non-saliency modeling, comprising: The wavelet decomposition module is used for wavelet decomposition: it performs wavelet decomposition on the input damaged SST image, weekly mean SST image and historical SST image sequence respectively, and obtains the wavelet sub-band features of each image at the current scale. The spatiotemporal steady-state and anomalous feature extraction module is used for extracting spatiotemporal steady-state and anomalous features: For damaged SST images and weekly mean SST images: Based on the wavelet subband features at the current scale, spatial steady-state features and spatial anomalous features are extracted through an attention mechanism; For historical SST image sequences: Based on the wavelet subband features at the current scale, temporal steady-state features and temporal anomalous features are extracted through a gating mechanism. The steady-state and anomalous feature fusion module is used to fuse steady-state and anomalous features: it performs single-mode updates on spatial steady-state features, spatial anomalous features, temporal steady-state features, and temporal anomalous features; it performs multi-mode fusion on the updated features using a unified fusion operator; it performs cross-mode collaborative enhancement on the fused features; it completes fine-scale modeling; and it obtains the reconstructed features of each wavelet sub-band at the current scale. The multi-scale feature reconstruction module is used for multi-scale feature reconstruction: wavelet downsampling is performed on the low-frequency sub-bands of the wavelet sub-band features at the current scale of each image to generate a coarse scale. The downsampling, spatiotemporal steady-state and anomalous feature extraction module and steady-state and anomalous feature fusion module are repeated at the coarse scale until the multi-scale modeling from fine to coarse is completed to obtain multi-scale wavelet sub-band features. The high-level scale features in the multi-scale wavelet sub-band features are passed to the low-level scale through layer-by-layer upsampling, and feature integration is performed between different scales to complete the multi-scale feature reconstruction from coarse to fine. The inverse wavelet reconstruction module is used for inverse wavelet reconstruction: the finest scale reconstruction features are reconstructed using inverse wavelet to obtain the final complete SST image.

[0014] Compared with the prior art, the advantages of this invention are: On the one hand, existing methods for sea surface temperature (SST) completion tasks typically rely on information at a single scale for modeling, often focusing on a fixed spatial resolution or local structural features. This makes it difficult to simultaneously account for small-scale anomalous disturbances and large-scale background field constraints, leading to local structural distortions or overall distribution shifts when large areas of data are missing, significantly limiting the robustness of the completion results. This invention, however, utilizes wavelet decomposition to fully consider information at different scales, improving the robustness of the model in completing SST under large-area data gaps. On the other hand, most existing methods only perform representation learning of steady-state and anomalous information in the spatial domain, neglecting the significant steady-state evolution and temporal anomalous changes that are also prevalent in the sea surface temperature field over time. This makes it difficult to achieve an effective balance between temporal continuity and abrupt changes, thus limiting the accuracy and temporal consistency of the completion results. This invention, however, fully conducts representation learning of stable and anomalous information not only spatially but also temporally, improving the accuracy of the model completion. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] In the attached diagram: Figure 1 This is a flowchart illustrating the sea surface temperature completion method based on cross-scale spatiotemporal saliency and non-saliency modeling in this specific embodiment. Figure 2 This is a schematic diagram of the spatiotemporal steady-state and anomaly feature extraction and steady-state and anomaly fusion in this specific embodiment; Figure 3 This is a schematic diagram of the sea surface temperature completion system based on cross-scale spatiotemporal saliency and non-saliency modeling in this specific embodiment; Figure 4 This is a visualization comparison of the results of the present invention and existing methods at a 68% missing rate in the sea surface temperature (SST) completion method based on cross-scale spatiotemporal saliency and non-saliency modeling in this specific embodiment; where (a) represents the damaged SST image; (b) represents the real SST image; (c) represents the AIN result; (d) represents the DINEOF result; (e) represents the Phy_INN result; (f) represents the FMambaIR result; (g) represents the DINFNN result; (h) represents the SVIFNN result; and (i) represents the result of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0018] Example 1: Please see Figure 1 and Figure 2 This invention provides a sea surface temperature completion method based on cross-scale spatiotemporal saliency and non-saliency modeling, comprising the following steps: S1. Wavelet decomposition: Wavelet decomposition is performed on the input damaged SST image, weekly mean SST image and historical SST image sequence to obtain the wavelet sub-band features of each image at the current scale. S2. Extraction of Spatiotemporal Steady-State and Anomaly Features: Damaged SST images and weekly mean SST images: Based on the wavelet subband features at the current scale, spatial steady-state features and spatial anomalous features are extracted through an attention mechanism; Historical SST image sequences: Based on wavelet subband features at the current scale, time steady-state features and time anomaly features are extracted through a gating mechanism; S3. Fusion of steady-state and anomalous features: Single-mode updates are performed on spatial steady-state features, spatial anomalous features, temporal steady-state features, and temporal anomalous features. The updated features are then fused using a unified fusion operator for multi-mode fusion. The fused features are then enhanced across modes to complete fine-scale modeling and obtain the reconstructed features of each wavelet sub-band at the current scale. S4. Multi-scale feature reconstruction: Wavelet downsampling is performed on the low-frequency sub-bands of the wavelet sub-band features at the current scale of each image to generate a coarse scale. Downsampling and S2-S3 are repeated at the coarse scale until multi-scale modeling from fine to coarse is completed to obtain multi-scale wavelet sub-band features. The high-level scale features in the multi-scale wavelet sub-band features are passed to the low-level scale through layer-by-layer upsampling, and feature integration is performed between different scales to complete multi-scale feature reconstruction from coarse to fine. S5. Inverse wavelet reconstruction: The finest scale reconstruction features are reconstructed using inverse wavelet reconstruction to obtain the final complete SST image.

[0019] Specifically, step S1 includes: Input damaged SST image Weekly mean SST image And historical SST image sequences, through two-dimensional discrete wavelet transform, wavelet decomposition is performed to obtain the wavelet sub-band features of each image at the current scale, namely the sub-band components of four different frequency bands, expressed as: ; in, Indicates a damaged SST image Weekly mean SST image Or historical SST image sequences. Represents a two-dimensional discrete wavelet transform. Indicates the low-frequency sub-band. , and These represent high-frequency subbands in the horizontal, vertical, and diagonal directions, respectively. , , , These represent the low-frequency subband features, horizontal high-frequency subband features, vertical high-frequency subband features, and diagonal high-frequency subband features of each image at the current scale, respectively. The low-frequency subband features at the current scale of each image are... Corresponding to large-scale, smoothly varying sea surface temperature background information, the horizontal high-frequency sub-band features at the current scale of each image. Vertical high-frequency subband characteristics and diagonal high-frequency sub-band characteristics These correspond to local variation information in different directions, and are used to characterize the boundary structure, detailed texture, and anomalous variation features in the sea surface temperature field.

[0020] In step S2 of this embodiment, based on the wavelet subband features at the current scale, spatial steady-state features and spatial anomaly features are extracted through an attention mechanism, including: S201. Based on the wavelet subband features at the current scale, in the spatial dimension, the damaged SST image Weekly mean SST image any subband Spatial features are extracted through shared convolutional coding layers; among which, The shared convolutional coding layer corresponds to Figure 1 Encoder 1 in the middle includes a two-dimensional convolutional layer, which is used to extract the local spatial neighborhood features of the damaged SST image and the weekly mean SST image in the current wavelet subband, and provide the basic spatial feature representation for the subsequent spatial steady-state attention mechanism and spatial anomaly attention mechanism. S202. Based on spatial features, spatial steady-state features are extracted through a spatial steady-state attention mechanism, represented as follows: ; in, This represents the normalized exponential function, used to map similarity to an attention weight distribution. Indicates a damaged SST image Any sub-band Spatial characteristics, Represents the weekly mean SST image Any sub-band Spatial characteristics; This represents the feature dimension, used for scaling and normalizing the similarity results. Indicates spatial steady-state characteristics; S203. Based on spatial features, spatial anomaly features are extracted using a reverse attention mechanism, as follows: ; in, This refers to the reverse mapping of attention weights, i.e., the reverse attention mechanism. Indicates spatial anomaly characteristics.

[0021] In some embodiments, based on wavelet subband features at the current scale, time steady-state features and time anomaly features are extracted through a gating mechanism, including: S204. Any sub-band of historical SST image sequence based on the wavelet sub-band features at the current scale. Spatial features of historical SST image sequences are extracted through shared convolutional coding layers; S205. Based on the spatial features of historical SST image sequences, extract time-steady features with long-term consistent evolution through a time-steady-state gating mechanism. S206. Based on the spatial features of historical SST image sequences, extract temporal anomaly features through a temporal anomaly gating mechanism.

[0022] Specifically, S204, historical SST image sequences based on current-scale wavelet subband features. any subband Spatial features of historical SST image sequences are extracted through shared convolutional coding layers; among which, The shared convolutional coding layer corresponds to Figure 1 The encoder 2 in the sequence includes a two-dimensional convolutional layer, which is used to extract the local spatial structure features of each time step and each wavelet sub-band in the historical SST image sequence, so as to provide the spatial feature representation of the historical sequence for the subsequent time steady state gating mechanism and time anomaly gating mechanism. S205. Based on the spatial features of historical SST image sequences, temporal steady-state features with long-term consistent evolution are extracted through a temporal steady-state gating mechanism, represented as follows: ; in, This indicates a steady-state gating mechanism. Indicates time steady-state characteristics, This represents the spatial feature representation obtained by encoding at time step t; S206. Based on the spatial features of historical SST image sequences, temporal anomaly features are extracted through a temporal anomaly gating mechanism, represented as follows: ; in, This indicates an abnormal gating mechanism. Indicates time-related abnormal characteristics. This represents the spatial feature representation obtained by encoding at time step t. This indicates the total time step.

[0023] In other embodiments, step S3 includes: S301. Based on each wavelet subband in the current scale wavelet subband features, a two-dimensional convolutional residual update unit is used for single-mode update; wherein, the two-dimensional convolutional residual update unit includes a two-dimensional convolutional layer, a normalization layer, a nonlinear activation layer, and residual connections; the single-mode update is as follows: spatial steady-state features and temporal steady-state features are input into the steady-state update unit for update, and spatial anomaly features and temporal anomaly features are input into the anomaly update unit for update; the parameters of the steady-state update unit and the anomaly update unit are shared between different wavelet subbands, and the parameters of the steady-state update unit and the anomaly update unit are independent of each other; S302. Input the updated spatial steady-state features and the updated temporal steady-state features into the steady-state fusion operator to obtain the steady-state fusion features; input the updated spatial anomaly features and the updated temporal anomaly features into the anomaly fusion operator to obtain the anomaly fusion features; wherein, the steady-state fusion operator and the anomaly fusion operator adopt the same fusion structure, both including feature concatenation, difference feature calculation, product correlation calculation, modal weight generation, and weighted fusion process; the parameters of the steady-state fusion operator and the anomaly fusion operator are shared among the wavelet subbands, and the parameters of the steady-state fusion operator and the anomaly fusion operator are independent of each other; S303. Input the steady-state fusion features and the abnormal fusion features into the cross-modal collaborative enhancement unit. Generate gated descriptive features based on the differences and correlations between the steady-state fusion features and the abnormal fusion features. Perform convolution operation on the gated descriptive features to obtain gated weights. After channel mapping, the abnormal fusion features are multiplied element-wise with the gated weights and added to the steady-state fusion features to obtain collaborative fusion features. Input the collaborative fusion features into the collaborative update unit for residual optimization to complete fine-scale modeling and obtain the reconstruction features of each wavelet sub-band at the current scale.

[0024] Specifically, S301, based on each wavelet subband in the current scale wavelet subband features, a two-dimensional convolutional residual update unit is used for single-modal update; the two-dimensional convolutional residual update unit includes a two-dimensional convolutional layer, a normalization layer, a nonlinear activation layer, and residual connections; expressed as: ; Where X represents the input features, including: spatial steady-state features, temporal steady-state features, spatial anomaly features, and temporal anomaly features. , Represents a steady-state update unit. Indicates an abnormal update unit. The parameters represent the steady-state update unit. The parameters representing the abnormal update unit, This represents a two-dimensional convolutional layer used to extract local spatial neighborhood information from the features of the current subband. Indicates the normalization layer. Represents a non-linear activation function. This indicates a channel-mapped convolutional layer, with residual connections used to preserve the original input feature information before the update and improve the stability of feature propagation.

[0025] Specifically, single-mode update involves inputting spatial steady-state features and temporal steady-state features into a steady-state update unit for updating, and inputting spatial anomaly features and temporal anomaly features into an anomaly update unit for updating; represented as: ; ; ; ; in, This represents the updated spatial steady-state characteristics. This represents the updated time steady-state characteristics. Indicates the updated spatial anomaly characteristics. Updated time anomaly characteristics; Furthermore, the parameters of the steady-state update unit and the anomalous update unit are shared among four different wavelet subbands: LL, LH, HL, and HH, respectively, while the parameters of the steady-state update unit and the anomalous update unit are independent of each other, as expressed in: ; ; in, These represent the parameters of the steady-state update unit in the LL, LH, HL, and HH subbands, respectively. These represent the parameters of the abnormal update unit in the LL, LH, HL, and HH subbands, respectively. The steady-state update unit and the abnormal update unit adopt the same two-dimensional convolutional residual structure, but their parameters are independent of each other to avoid confusion between steady-state background information and abnormal change information in the single-mode update stage.

[0026] Specifically, the steady-state update unit and the outlier update unit use the same two-dimensional convolutional residual structure, but their parameters are independent of each other, including: spatial steady-state features and temporal steady-state features share the same steady-state update unit. parameters Spatial anomaly features and temporal anomaly features share the same anomaly update unit. parameters This is to avoid the confusion between steady-state background information and anomalous change information during the single-mode update stage.

[0027] Specifically, in step S302, the updated spatial steady-state features and the updated temporal steady-state features are input into the steady-state fusion operator to obtain the steady-state fusion features; the updated spatial anomaly features and the updated temporal anomaly features are input into the anomaly fusion operator to obtain the anomaly fusion features; wherein, the steady-state fusion operator and the anomaly fusion operator adopt the same fusion structure, both including feature concatenation, difference feature calculation, product correlation calculation, modal weight generation, and weighted fusion process; the parameters of the steady-state fusion operator and the anomaly fusion operator are shared among the four different wavelet subbands LL, LH, HL, and HH, and the parameters of the steady-state fusion operator and the anomaly fusion operator are independent of each other, expressed as: ; ; ; ; in, These represent the parameters of the steady-state fusion operator in the LL, LH, HL, and HH subbands, respectively. These represent the parameters of the anomaly fusion operator in the LL, LH, HL, and HH subbands, respectively. This represents the steady-state fusion feature corresponding to the b-th wavelet sub-band. This represents the anomalous fusion feature corresponding to the b-th wavelet sub-band; Represents the steady-state fusion operator. Indicates the abnormal fusion operator; The parameters represent the steady-state fusion operator. The parameters represent the abnormal fusion operator.

[0028] In some embodiments, the feature concatenation, differential feature calculation, product correlation calculation, modality weight generation, and weighted fusion process include: For any fusion structure The spatial input features and temporal input features are respectively and ,but: ; ; ; ; in, Indicates steady state, Indicates an anomaly. This represents the updated spatial steady-state characteristics. This represents the updated time steady-state characteristics. Indicates the updated spatial anomaly characteristics. This indicates the anomaly characteristics of the updated time. Represents the input features in the steady-state space. Steady-state time-side input features, Indicates the input features of the abnormal spatial side. This represents the input features at abnormal times.

[0029] The spatial and temporal input features are concatenated, differential features are calculated, and product correlation is calculated to obtain fused descriptive features.

[0030] Based on the fusion description features, spatial and temporal weights are generated by normalization through convolutional layers. At the same spatial location and the same channel, the spatial and temporal weights are added element by element to equal 1.

[0031] The spatial and temporal input features are weighted and fused using the generated spatial and temporal weights, and the fused features of the corresponding path are obtained through convolution.

[0032] Specifically, the spatial and temporal input features are concatenated, differential features are calculated, and product correlation is calculated to obtain the fused descriptive features, represented as follows: ; in, This indicates feature concatenation along the channel dimension. This represents element-wise absolute value operation. This represents element-wise multiplication. The fusion description features are represented by: feature concatenation is used to preserve the original expression of both spatial and temporal features; differential feature calculation is used to characterize the inconsistency region between spatial and temporal features; and product correlation is used to characterize the correlation region between spatial and temporal features.

[0033] Specifically, the descriptive features are fused into a 1×1 convolutional layer to obtain spatial and temporal weight responses, and then... The function is normalized in both spatial and temporal dimensions to generate spatial and temporal weights, as follows: ; in, This represents the spatial weight of the b-th wavelet subband. This represents the time-side weight of the b-th wavelet subband. Used to normalize between spatial and temporal weights.

[0034] Specifically, to ensure that the spatial and temporal weights constitute a normalized modal contribution ratio, and to prevent instability in feature amplitudes due to unconstrained modal contribution ratios during the fusion process, this embodiment adaptively allocates the contributions of spatial and temporal features to the fusion result based on the reliability of information in the current wavelet subband, current position, and current channel. At the same spatial position and channel, the spatial and temporal weights are element-wise summed to 1, as shown below: .

[0035] Specifically, the spatial and temporal input features are weighted and fused using the generated spatial and temporal weights. The fused features for the corresponding path are then obtained through convolution, as shown below: ; Here, represents the fusion characteristics, namely steady-state fusion characteristics or abnormal fusion characteristics.

[0036] Through the above fusion method, the model can adaptively adjust the contribution ratio of spatial and temporal features according to the reliability of the current wavelet subband, thereby enhancing the consistency of large-scale background field and continuous temporal evolution in steady-state paths, and strengthening the ability to express local boundaries, abrupt regions and anomalous perturbations in anomalous paths.

[0037] Specifically, S303, let the steady-state fusion feature and the anomalous fusion feature corresponding to the b-th wavelet sub-band be respectively... and Based on steady-state fusion characteristics Abnormal fusion characteristics The differences and correlations between the two generate gated descriptive features, represented as follows: ; in, This indicates feature concatenation along the channel dimension. This represents element-wise absolute value operation. This represents element-wise multiplication. Gating describes the characteristics. It also includes steady-state fusion features, abnormal fusion features, the inconsistency region between the two, and the correlation region between the two, which are used to determine the supplementary strength of abnormal information to steady-state information in different spatial locations and different channels. Subsequently, the gating description features Input a 1×1 convolutional layer, and then... The function generates the gated weights, expressed as: ; in, This represents the gating weight corresponding to the b-th wavelet sub-band, with a value ranging from 0 to 1; when When the value is small, it indicates that the current position and current channel mainly retain steady-state fusion characteristics; when... When the value of is large, it indicates that more abnormal fusion features need to be introduced for the current position and the current channel to supplement information on local boundaries, gradient changes and abnormal perturbations.

[0038] Then, the abnormal fusion feature is channel-mapped and multiplied element-wise by the gate weights, and added to the steady-state fusion feature to obtain the collaborative fusion feature, represented as: ; in, This represents the collaborative fusion feature of the b-th wavelet subband. The above process represents using steady-state fusion features as the basic background expression and gated and filtered anomalous fusion features as supplementary information, thereby enhancing the expressive power of local boundaries, abrupt regions, and anomalous perturbations while maintaining the large-scale steady-state background and the continuous temporal evolution structure.

[0039] Finally, the collaboratively fused features are input into the collaborative update unit for residual optimization to obtain the reconstructed features of each wavelet subband, as follows: ; in, This represents the reconstructed feature corresponding to the b-th wavelet sub-band. Indicates the normalization layer. This represents a nonlinear activation function. The residual optimization process is used to learn local correction values ​​while preserving the main information of the collaboratively fused features, thereby further optimizing the collaborative representation of steady-state background information and anomalous change information.

[0040] Specifically, step S4 includes: Wavelet downsampling is performed on the low-frequency subbands of the current-scale wavelet subband features obtained by wavelet decomposition to enter the second layer to generate a coarse scale. Steps S2-S3 are then performed at the coarse scale to complete the second-layer coarse-scale modeling. Based on the second-layer coarse scale, a downsampling process is performed again to enter the third layer coarser scale. Steps S2-S3 are then performed at the coarser scale to complete the multi-scale modeling from fine to coarse, obtaining multi-scale wavelet subband features. The high-scale features in the multi-scale wavelet subband features are then passed to the low-scale features through layer-by-layer upsampling, and feature integration is performed between different scales to complete the multi-scale feature reconstruction from coarse to fine.

[0041] In some embodiments, step S5 includes: S501. The finest scale reconstructed features are mapped back to the corresponding wavelet subband coefficient space through convolution operations to obtain the wavelet subband coefficients of any subband b after mapping. S502. The sub-band coefficients after mapping are recombined using inverse wavelet transform to recover the final complete SST image. .

[0042] Specifically, the reconstructed features at the finest scale (S501) are mapped back to the corresponding wavelet subband coefficient space through convolution operations, yielding the wavelet subband coefficients of any subband b after mapping, expressed as: ; in, This represents the reconstructed features of any subband b obtained at the finest scale after multi-scale feature reconstruction. Denotes the wavelet subband coefficients of any subband b obtained by mapping. .

[0043] Specifically, in step S502, the mapped sub-band coefficients are recombined using inverse wavelet transform to recover the final, complete SST image. , is represented as: ; in, Indicates inverse wavelet transform. This represents the reconstructed low-frequency subband, corresponding to large-scale, smoothly varying background temperature information. This represents the reconstructed mid-to-high frequency sub-bands, each corresponding to local variation information in different directions, used to characterize boundary structures, detailed textures, and anomalous changes.

[0044] Loss function: Through steady-state prediction constraints, missing measurement area completion constraints, observed area consistency constraints, and steady-state-anomaly... The additivity constraint constitutes the reconstruction loss, expressed as: ; in, It is the L2 norm. This represents element-wise multiplication. For real SST images, This is a weekly mean SST image. This represents a reconstructed damaged SST image obtained by occluding a real, complete SST image with a cloud mask. and These are the predicted steady-state components and anomalous components, respectively. To reconstruct damaged SST images binary mask; This represents the steady-state prediction constraint. This indicates the constraint to complete the missing measurement area. This indicates a consistency constraint for the observed region. Represents the steady-state-anomalous additivity constraint. This represents the reconstruction loss.

[0045] binary mask Represented as: ; in, Reconstructing a damaged SST image Pixel position index in This represents the value of the binary mask at the i-th pixel position.

[0046] Performing two-dimensional discrete wavelet transforms on both the completed SST image and the original SST image, respectively, results in: ; ; in, This represents the low-frequency and high-frequency subbands obtained after the completion result undergoes a two-dimensional discrete wavelet transform. This represents the low-frequency and high-frequency subbands obtained after the two-dimensional discrete wavelet transform of a true and complete SST image.

[0047] To further constrain the consistency of the completion results in the frequency band structure, wavelet consistency loss is defined. This is used to ensure that the completion result not only closely approximates the true SST image in the spatial domain, but also maintains consistency in the low-frequency background and high-frequency details after wavelet decomposition, and is expressed as: ; in, As sub-band weights, the weights of high-frequency sub-bands are made greater than those of low-frequency sub-bands, with the specific weight relationship satisfying: This enhances the fidelity constraints on local boundaries and anomaly details. This represents a sub-band of the complete SST image that has been padded. This represents a sub-band of the real SST image. Since the low-frequency sub-band LL mainly represents large-scale background temperature information, while the high-frequency sub-bands LH, HL, and HH mainly represent local boundaries, gradient changes, and anomaly details, the weight of the high-frequency sub-band is made greater than that of the low-frequency sub-band. The specific weight relationship satisfies: ; in, Indicates the weight of low-frequency subbands. , , This indicates the weight of the high-frequency subband.

[0048] By using the above settings, the model's fidelity constraints on local boundaries and abnormal details can be enhanced, avoiding the completion result from only approximating the real image in terms of overall temperature distribution while exhibiting excessive smoothing in local structures.

[0049] Example 2: To further illustrate the specific implementation methods and technical effects of the present invention, this embodiment describes the present invention from three aspects: model input and data processing, model training process, and experimental verification.

[0050] Model introduction and data processing procedure: To verify the sea surface temperature (SST) completion effect of this invention, this embodiment selects the Level 4 SST product of the National Satellite Ocean Application Service for Region A as experimental data, with the data time range from January 2022 to April 2023. Data from January to December 2022 was used for model training, and data from January to April 2023 was used for model testing. By dividing the training and testing sets chronologically, adjacent samples can be avoided from appearing simultaneously during training and testing, thereby reducing the impact of data leakage on the experimental results.

[0051] During data preprocessing, the original SST data is first cropped to a uniform 64×64 image size, and the data range is normalized to the (-1, 1) interval. Then, a corresponding weekly average SST image is generated based on the original complete SST image, and historical SST image sequences are extracted as the time dimension input. In this embodiment, the length of the historical SST sequence is set to 6, meaning that each sample, in addition to the current damaged SST image and the weekly average SST image, also receives a sequence of SST images from 6 historical moments to learn the temporal evolution characteristics of the sea surface temperature field.

[0052] To simulate observational gaps caused by real cloud obstruction, this embodiment selects a real cloud mask from publicly available cloud data to obstruct complete SST images, constructing damaged SST images with cloud coverage rates of 8%, 25%, 46%, and 68%, respectively. To further simulate information loss during remote sensing data acquisition, noise perturbations of 0.1, 0.2, and 0.3 were added during the generation of damaged SST images, and experimental results under these noise levels were compared and presented. In other words, the experimental results shown in Tables 1 to 4 correspond to the model completion results when the noise level is 0.1.

[0053] In this embodiment, the model input includes the current damaged SST image, the corresponding weekly average SST image, and a sequence of historical SST images; the supervision label is the current real complete SST image. During model training, the damaged SST image, the weekly average SST image, and the historical SST sequence are first input into the model. After wavelet decomposition, extraction of spatiotemporal steady-state and anomalous features, fusion of steady-state and anomalous features, multi-scale feature reconstruction, and inverse wavelet reconstruction, the model outputs a completed complete SST image. Simultaneously, the model also outputs steady-state and anomalous components, used to construct steady-state prediction constraints and steady-state-anomaly additivity constraints.

[0054] Regarding model structure parameter settings, this embodiment employs Haar wavelets for two-dimensional discrete wavelet decomposition and inverse wavelet reconstruction. After wavelet decomposition, four subbands—LL, LH, HL, and HH—are formed. The LL subband is used to characterize large-scale smooth background information, while the LH, HL, and HH subbands are used to characterize local boundaries, gradient changes, and anomalous perturbation information. The number of basic feature channels in the model is set to 64, and the number of attention heads is set to 4. The spatial steady-state and anomalous feature extraction module in the model is used to extract steady-state related information and anomalous change information between the current damaged SST image and the weekly average SST image; the temporal steady-state and anomalous feature extraction module is used to extract long-term consistent evolution features and temporal anomalous change features from historical SST sequences; and the steady-state and anomalous feature fusion module is used to fuse the spatial and temporal steady-state features and anomalous features respectively, and obtain reconstructed features through cross-modal collaborative enhancement.

[0055] Model training content: Model training was performed using a deep learning server with GPU acceleration capabilities. The server operating system was Ubuntu 22.04.5 LTS, Python version 3.9.25, and the deep learning framework was PyTorch 2.5.1 + cu121. The hardware environment included an AMD EPYC 7402 processor, 52GB of RAM, and an NVIDIA GeForce RTX 4090 GPU with 24564 MiB of VRAM. The NVIDIA driver version was 550.142, the server displayed CUDA version 12.4, and the corresponding CUDA version for PyTorch was 12.1. During training, the GPU was prioritized for forward propagation, backward propagation, and parameter updates; when the GPU was unavailable, the program automatically switched to CPU execution.

[0056] Regarding training parameter settings, this embodiment uses the Adam optimizer to train the model. The initial learning rates for both the generator and discriminator are set to 0.0001, and the Adam optimizer parameters are set to... , The training rounds were set to 100, and the batch size was set to 1. Training samples were constructed under four different cloud coverage conditions, and models were trained under the corresponding missing rate conditions to verify the completion performance of the invention under different missing levels.

[0057] The training loss consists of image domain reconstruction loss, wavelet consistency loss, edge constraint loss, and discriminant loss. The reconstruction loss includes steady-state prediction constraints, missing region completion constraints, observed region consistency constraints, and steady-state-anomaly additivity constraints, used to constrain steady-state background prediction, missing region completion accuracy, effective observation region preservation ability, and the additive relationship between steady-state and anomaly components, respectively. The wavelet consistency loss constrains the consistency between the completed image and the real SST image in the wavelet frequency bands. The low-frequency subband LL mainly constrains the consistency of the large-scale background field, while the high-frequency subbands LH, HL, and HH mainly constrain the consistency of local boundaries, gradient changes, and anomaly details. In specific training, the weight of the high-frequency subband is greater than that of the low-frequency subband to enhance the model's ability to recover local details and anomaly changes. The discriminant loss improves the realism of the completed image in terms of overall spatial distribution and high-frequency details, making the model-generated completed SST image closer to the distribution of the real SST image.

[0058] In each training batch, the damaged SST image, the weekly mean SST image, and the historical SST sequence are first input into the completion model, resulting in three outputs: the predicted weekly mean image, etc. Final image completion and historical branch reconstruction images .

[0059] During training, the image domain reconstruction loss is first calculated. For the weekly mean image, the predicted weekly mean image is calculated. Image with true weekly mean SST The L1 and L2 losses are calculated between the two images; for the final completed image, the final completed image is calculated. With real complete SST image The L1 and L2 losses are used, and a binary mask is combined to constrain the observed and missing regions respectively; for the historical branch reconstruction image, the historical branch reconstruction image is calculated. Corresponding historical target image The L1 and L2 losses are applied within the effective region. The aforementioned image domain reconstruction loss ensures that the model output closely approximates the true SST data at the pixel level and serves as the basis for subsequent generation losses. Components of.

[0060] Secondly, the wavelet consistency loss is calculated. This is done for the predicted weekly mean images. Image with true weekly mean SST Final image completion With real complete SST image Historical branch reconstruction image Compared with historical target images A two-dimensional discrete wavelet transform is performed to obtain the corresponding LL, LH, HL, and HH sub-bands, and the differences between the predicted results and the actual target in each wavelet sub-band are calculated. In this embodiment, the weight of the low-frequency sub-band LL is set to 1, and the weights of the high-frequency sub-bands LH, HL, and HH are all set to 3, thereby enhancing the model's ability to recover local boundaries, gradient changes, and anomalous details.

[0061] Next, the edge constraint loss is calculated. The Sobel operator is used to extract the edge gradient features of the predicted image and the real target image, respectively, and the L1 loss and L2 loss between them are calculated. This edge constraint loss is applied to the weekly mean image, the final completed image, and the historical branch reconstructed image, respectively, to further constrain the consistency of the completed results in local boundary and spatial gradient changes, and reduce the over-smoothing phenomenon in the completed image.

[0062] Based on the above loss, the fundamental generation loss of the completion model is constructed as follows: ; in, and These represent the L1 loss and L2 loss of the weekly mean image, respectively. and These represent the L1 loss and L2 loss of the final completed image, respectively. and These represent the L1 loss and L2 loss for reconstructing the image from the historical branches, respectively. This represents wavelet consistency loss. This represents the Sobel edge constraint loss. This represents the basic generation loss of the completion model; the weights before different loss terms are used to balance the importance of steady-state background reconstruction, final completion result, historical feature reconstruction, wavelet band consistency, and edge structure preservation.

[0063] To improve the realism of the completion results, a discriminator is introduced during training. The discriminator distinguishes between the predicted weekly mean / steady-state image and the true weekly mean SST image, the final completed image and the true complete SST image, and the high-frequency subband set of the completed image and the high-frequency subband set of the true image. The discriminant loss is used to encourage the discriminator to differentiate between the real image and the model-generated image, and to conversely constrain the completion model to generate SST images that more closely approximate the true distribution. The discriminant loss is expressed as: ; ; ; ; in, , , These represent the outputs of the weekly mean image discriminator, the final completed image discriminator, and the high-frequency subband set discriminator, respectively. These represent the parameters that the three discriminators need to learn. This represents the discriminant loss corresponding to the weekly mean image. This represents the discriminative loss corresponding to the final completed image. This represents the discriminant loss corresponding to the set of high-frequency subbands. The set of high-frequency subbands is defined as follows: ; in, , , The high-frequency subbands representing the completed SST image or the real SST image are used to improve the realism of the completion results in high-frequency details (local boundaries, abnormal changes), which is consistent with the design goal of abnormal fidelity of this invention.

[0064] For the completion model, the goal is to make the generated result as close as possible to the discriminator's judgment of the real image. Therefore, the generator adversarial loss is defined as follows: ; ; ; ; Based on the basic generation loss and the generator adversarial loss, the generation loss of the completion model is expressed as: ; in, This represents the generation loss used to update and complete the model parameters. This represents the weighting coefficients of the generator against the loss.

[0065] During model training, the discriminator and the completion model are trained using an alternating optimization approach. First, the parameters of the completion model are fixed, and then the discriminator loss is adjusted accordingly. Update discriminator parameters , , Then, the discriminator parameters are fixed, and the loss is calculated accordingly. Update and complete the model parameters. This process is repeated in each training batch until the model converges.

[0066] The specific training process is as follows: First, read the damaged SST image, weekly mean SST image, historical SST sequence, and real complete SST image from the training samples; then, input the damaged SST image, weekly mean SST image, and historical SST sequence into the model to obtain the predicted weekly mean image, the completed complete SST image, the historical branch reconstruction image, the steady-state component, and the anomalous component; next, calculate the image domain reconstruction loss, wavelet consistency loss, edge constraint loss, and generator adversarial loss based on the model output and the corresponding real target to obtain the generation loss of the completion model, and simultaneously calculate the discriminator loss based on the real and generated samples; then, use an alternating optimization method, first updating the discriminator parameters based on the discriminator loss, and then updating the completion model parameters based on the generation loss. The above process is repeated 100 times on the training set. After each training round, calculate the root mean square error (RMSE), mean square error (MSE), mean absolute error (MAE), and coefficient of determination on the test set. Evaluation metrics such as Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) are used, and the optimal results for each metric are recorded. Through this training method, the model can not only reduce completion errors at the pixel numerical level, but also be constrained simultaneously in terms of wavelet band structure, local edge gradients, and overall true distribution, thereby improving the accuracy and robustness of SST completion results under large-area cloud occlusion conditions.

[0067] Through the above training method, the present invention can make full use of the effective observation information in the current damaged SST image, the steady-state background information in the weekly average SST image, and the temporal evolution information in the historical SST sequence to achieve the complete reconstruction of the complete sea surface temperature field under different cloud coverage and large-area missing conditions.

[0068] Experimental verification: Evaluation metric: Root mean square error Coefficient of determination Structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) are used as evaluation metrics to assess the reconstruction results.

[0069] Baseline Selection: To demonstrate the technical effectiveness of this invention, classic and cutting-edge methods in the field were selected for comparative experiments: AIN, DINEOF, Phy_INN, FMambaIR, DINFNN, and SVIFNN. DINEOF is a classic method in this field, using statistical interpolation to complete missing data in a spatiotemporal manner. AIN, Phy_INN, DINFNN, and SVIFNN are all cutting-edge methods based on deep neural networks in this field, while FMambaIR is a cutting-edge deep learning method for common image restoration tasks, also utilizing the Mamba architecture. Details are as follows: AIN: A GAN-based SST completion method that employs a "coarse-to-fine" strategy in the image domain. First, it predicts the weekly average using the monthly average, then predicts daily outliers based on the predicted weekly average. Finally, the reconstructed SST image is obtained by directly adding the weekly average and the daily outliers.

[0070] DINEOF: A classic missing data completion method based on empirical orthogonal functions (EOF), widely used in geophysics. It combines EOF analysis with spatial interpolation, using the identification of spatial patterns that best represent the main changes or structures in the dataset to aid in interpolation and fill in missing data.

[0071] Phy_INN: A cutting-edge SST inpainting method based on GAN. This method uses the ASPP (Spatial Pyramid Pooling with Hollows) module to learn information at different scales of weekly mean and daily anomalies in the image domain, and then performs deep fusion of feature embeddings at different scales to achieve the inpainting task of SST images.

[0072] FMambaIR: A cutting-edge method in image restoration. It combines frequency domain and Mamba within a deep learning architecture to perform denoising and restoration of a single ordinary image.

[0073] DINFNN: A cutting-edge SST completion method based on GAN. It employs a three-stream architecture, simultaneously inputting weekly averages, historical data, and current day's damaged data, and learns periodic stability information, temporal historical information, and contextual information of the current day's completed image, respectively. The three are then fused to complete the completion.

[0074] SVIFNN: A cutting-edge SST completion method based on GAN. It adopts a two-stream structure and uses a designed anomaly attention mechanism to fully preserve the anomalous patterns of negative correlation between the daily SST data and the weekly mean, while stabilizing attention focuses on the stable patterns of positive correlation between the daily SST data and the weekly mean.

[0075] Table 1 compares the RMSE index of the present invention with that of the existing method; Table 2 compares the R2 index of the present invention with that of the existing method; Table 3 compares the SSIM index of the present invention with that of the existing method; Table 4 compares the PSNR index of the present invention with that of the existing method. The results are shown in Tables 1-4 below: Table 1: Comparison of RMSE indicators between the present invention and existing methods

[0076] Table 2: Comparison of R² index between the present invention and existing methods

[0077] Table 3: Comparison of SSIM index between the present invention and existing methods

[0078] Table 4: Comparison of PSNR index between the present invention and existing methods

[0079] This invention also visualized some experimental results at a 68% missing rate, as shown below. Figure 4 As shown, (a) represents a damaged SST image; (b) represents a real SST image; (c) represents the AIN result; (d) represents the DINEOF result; (e) represents the Phy_INN result; (f) represents the FMambaIR result; (g) represents the DINFNN result; (h) represents the SVIFNN result; and (i) represents the result of this invention.

[0080] As shown in Tables 1-4, the present invention exhibits good overall reconstruction performance under different missing rates. It should be noted that the evaluation indicators have different directions of advantage or disadvantage. A smaller RMSE indicates lower error, while larger R², SSIM, and PSNR indicate better reconstruction results. Therefore, overall performance cannot be judged solely based on the magnitude of a single value. Comprehensive comparison shows that the present invention achieves the best RMSE and R² at an 8% missing rate, and the best R² and SSIM at a 46% missing rate. In scenarios with large-area missing data, especially at a 68% missing rate, the present invention achieves the highest SSIM, while RMSE, R², and PSNR are all at a high level. These results indicate that the present invention is not optimal in all individual indicators, but it exhibits good structure preservation ability and reconstruction robustness under conditions of large-area missing data, thus demonstrating superior overall performance.

[0081] also, Figure 4 An analysis combining spatial distribution consistency with the real image, preservation of warm and cold zone positions, gradient transition continuity, and artifact characteristics revealed that the AIN results exhibited significant stripe artifacts, the DINEOF results contained large areas of approximately constant values, the Phy_INN results showed blocky boundary artifacts, and the FMambaIR results exhibited strong scattered noise. In contrast, the results of this invention generated a more continuous temperature field within the missing regions with relatively fewer artifacts, indicating a certain degree of robustness.

[0082] In summary, the experimental results fully verify the technical effectiveness of the present invention in terms of reconstruction accuracy and robustness compared with existing methods, especially in scenarios with large-area data loss, where it can effectively maintain the reliability of reconstruction results.

[0083] Example 3: See Figure 3Based on the same inventive concept, this invention also proposes a sea surface temperature completion system based on cross-scale spatiotemporal saliency and non-saliency modeling, comprising: The wavelet decomposition module is used for wavelet decomposition: it performs wavelet decomposition on the input damaged SST image, weekly mean SST image and historical SST image sequence respectively, and obtains the wavelet sub-band features of each image at the current scale. The spatiotemporal steady-state and anomaly feature extraction module is used for spatiotemporal steady-state and anomaly feature extraction: damaged SST images and weekly mean SST images: based on the wavelet subband features of the current scale, spatial steady-state features and spatial anomaly features are extracted through an attention mechanism; Historical SST image sequences are used to extract time steady-state features and time anomaly features based on wavelet subband features at the current scale through a gating mechanism. The steady-state and anomalous feature fusion module is used to fuse steady-state and anomalous features: it performs single-mode updates on spatial steady-state features, spatial anomalous features, temporal steady-state features, and temporal anomalous features; it performs multi-mode fusion on the updated features using a unified fusion operator; it performs cross-mode collaborative enhancement on the fused features; it completes fine-scale modeling; and it obtains the reconstructed features of each wavelet sub-band at the current scale. The multi-scale feature reconstruction module is used for multi-scale feature reconstruction: wavelet downsampling is performed on the low-frequency sub-bands of the wavelet sub-band features at the current scale of each image to generate a coarse scale. The downsampling, spatiotemporal steady-state and anomalous feature extraction module and steady-state and anomalous feature fusion module are repeated at the coarse scale until the multi-scale modeling from fine to coarse is completed to obtain multi-scale wavelet sub-band features. The high-level scale features in the multi-scale wavelet sub-band features are passed to the low-level scale through layer-by-layer upsampling, and feature integration is performed between different scales to complete the multi-scale feature reconstruction from coarse to fine. The inverse wavelet reconstruction module is used for inverse wavelet reconstruction: the finest scale reconstruction features are reconstructed using inverse wavelet to obtain the final complete SST image.

[0084] The program product of this application for implementing the above method may employ a portable compact disk read-only memory and include program code, and may run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this application, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0085] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although this application has disclosed preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-mentioned technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application without departing from the content of the technical solution of this application shall still fall within the scope of this application.

Claims

1. A sea surface temperature completion method based on cross-scale spatiotemporal saliency and non-saliency modeling, characterized in that, Includes the following steps: S1. Wavelet decomposition: Wavelet decomposition is performed on the input damaged SST image, weekly mean SST image and historical SST image sequence to obtain the wavelet sub-band features of each image at the current scale. SST represents sea surface temperature; S2. Extraction of Spatiotemporal Steady-State and Anomaly Features: For damaged SST images and weekly mean SST images: based on the wavelet subband features at the current scale, spatial steady-state features and spatial anomalous features are extracted through an attention mechanism; For historical SST image sequences: based on wavelet subband features at the current scale, time steady-state features and time anomaly features are extracted through a gating mechanism; S3. Fusion of steady-state and anomalous features: Single-mode updates are performed on spatial steady-state features, spatial anomaly features, temporal steady-state features, and temporal anomaly features. The updated features are then fused using a unified fusion operator for multi-mode fusion. The fused features are then enhanced across modes to complete fine-scale modeling and obtain the reconstructed features of each wavelet sub-band at the current scale. S4. Multi-scale feature reconstruction: Wavelet downsampling is performed on the low-frequency subbands of the wavelet subband features at the current scale of each image to generate a coarse scale. Downsampling and S2-S3 are repeated at the coarse scale until multi-scale modeling from fine to coarse is completed to obtain multi-scale wavelet subband features. The high-level scale features in the multi-scale wavelet subband features are passed to the low-level scale through layer-by-layer upsampling, and feature integration is performed between different scales to complete the multi-scale feature reconstruction from coarse to fine. S5. Inverse wavelet reconstruction: The finest scale reconstruction features are reconstructed using inverse wavelet reconstruction to obtain the final complete SST image.

2. The method according to claim 1, characterized in that: Step S1 specifically includes: Input damaged SST image Weekly mean SST image And historical SST image sequences, through two-dimensional discrete wavelet transform, wavelet decomposition is performed to obtain the wavelet sub-band features of each image at the current scale, i.e., the sub-band components of four different frequency bands, represented as: ； in, Indicates a damaged SST image Weekly mean SST image Or historical SST image sequences. Represents a two-dimensional discrete wavelet transform. Indicates the low-frequency sub-band. , and These represent the high-frequency subbands in the horizontal, vertical, and diagonal directions, respectively. , , , These represent the low-frequency subband features, horizontal high-frequency subband features, vertical high-frequency subband features, and diagonal high-frequency subband features of each image at the current scale.

3. The method according to claim 1, characterized in that, The wavelet subband features based on the current scale extract spatial steady-state features and spatial anomaly features through an attention mechanism, including: S201. Based on the wavelet subband features at the current scale, in the spatial dimension, the damaged SST image Weekly mean SST image any subband Spatial features are extracted through shared convolutional coding layers; S202. Based on spatial features, spatial steady-state features are extracted through a spatial steady-state attention mechanism, represented as follows: ； Among them, sub-band , Indicates the low-frequency sub-band. , and These represent the high-frequency sub-bands in the horizontal, vertical, and diagonal directions, respectively. This represents the normalized exponential function used to map similarity to an attention weight distribution, representing a damaged SST image. In sub-band Spatial characteristics, Represents the weekly mean SST image In sub-band Spatial characteristics; This represents the feature dimension, used for scaling and normalizing the similarity results. Indicates spatial steady-state characteristics; S203. Based on spatial features, spatial anomaly features are extracted through a reverse attention mechanism, represented as follows: ； in, This refers to the reverse mapping of attention weights, i.e., the reverse attention mechanism. Indicates spatial anomaly characteristics.

4. The method according to claim 1, characterized in that, The wavelet subband features based on the current scale are used to extract time steady-state features and time anomaly features through a gating mechanism, including: S204. Any sub-band of historical SST image sequence based on the wavelet sub-band features at the current scale. Spatial features of historical SST image sequences are extracted through shared convolutional coding layers; S205. Based on the spatial features of historical SST image sequences, extract time-steady features with long-term consistent evolution through a time-steady-state gating mechanism. S206. Based on the spatial features of historical SST image sequences, extract temporal anomaly features through a temporal anomaly gating mechanism.

5. The method according to claim 1, characterized in that, The process involves single-modal updates of spatial steady-state features, spatial anomaly features, temporal steady-state features, and temporal anomaly features; multimodal fusion of the updated features using a unified fusion operator; and cross-modal collaborative enhancement of the fused features to complete fine-scale modeling. This includes: S301. Based on each wavelet subband in the current scale wavelet subband features, a two-dimensional convolutional residual update unit is used for single-mode update; wherein, the two-dimensional convolutional residual update unit includes a two-dimensional convolutional layer, a normalization layer, a nonlinear activation layer, and residual connections; the single-mode update is as follows: spatial steady-state features and temporal steady-state features are input into the steady-state update unit for update, and spatial anomaly features and temporal anomaly features are input into the anomaly update unit for update; the parameters of the steady-state update unit and the anomaly update unit are shared between different wavelet subbands, and the parameters of the steady-state update unit and the anomaly update unit are independent of each other; S302. Input the updated spatial steady-state features and the updated temporal steady-state features into the steady-state fusion operator to obtain the steady-state fusion features; input the updated spatial anomaly features and the updated temporal anomaly features into the anomaly fusion operator to obtain the anomaly fusion features; wherein, the steady-state fusion operator and the anomaly fusion operator adopt the same fusion structure, both including feature concatenation, difference feature calculation, product correlation calculation, modal weight generation, and weighted fusion process; the parameters of the steady-state fusion operator and the anomaly fusion operator are shared among the wavelet subbands, and the parameters of the steady-state fusion operator and the anomaly fusion operator are independent of each other; S303. Input the steady-state fusion features and the abnormal fusion features into the cross-modal collaborative enhancement unit. Generate gated descriptive features based on the differences and correlations between the steady-state fusion features and the abnormal fusion features. The gated descriptive features are convolved to obtain gated weights. The abnormal fusion features are mapped through channels and multiplied element-wise with the gated weights and added to the steady-state fusion features to obtain collaborative fusion features. Input the collaborative fusion features into the collaborative update unit for residual optimization to complete fine-scale modeling and obtain the reconstruction features of each wavelet sub-band at the current scale.

6. The method according to claim 5, characterized in that, The process of feature concatenation, differential feature calculation, product correlation calculation, modal weight generation, and weighted fusion includes: For any fusion structure The spatial input features and temporal input features are respectively and ,but: ； ； ； ； in, Indicates steady state, Indicates an anomaly. This represents the updated spatial steady-state characteristics. Represents the updated time steady-state characteristics, and represents the updated spatial anomaly characteristics. This indicates the anomaly characteristics of the updated time. Represents the input features in the steady-state space. Steady-state time-side input features, Indicates the input features of the abnormal spatial side. Indicates input features at abnormal times; The spatial and temporal input features are concatenated, differential features are calculated, and product correlation is calculated to obtain the fused descriptive features. Based on the fusion description features, the spatial and temporal weights are generated by normalization through convolutional layers. At the same spatial location and the same channel, the spatial and temporal weights are added element by element to 1. The spatial and temporal input features are weighted and fused using the generated spatial and temporal weights, and the fused features of the corresponding path are obtained through convolution.

7. The method according to claim 5, characterized in that, The final complete SST image is obtained by inverse wavelet reconstruction of the finest-scale reconstruction features, including: S501. The finest scale reconstructed features are mapped back to the corresponding wavelet subband coefficient space through convolution operations to obtain the wavelet subband coefficients of any subband b after mapping. S502. The sub-band coefficients after mapping are recombined using inverse wavelet transform to recover the final complete SST image. .

8. A sea surface temperature completion system based on cross-scale spatiotemporal saliency and non-saliency modeling, characterized in that, include: The wavelet decomposition module is used for wavelet decomposition: it performs wavelet decomposition on the input damaged SST image, weekly mean SST image and historical SST image sequence respectively, and obtains the wavelet sub-band features of each image at the current scale. The spatiotemporal steady-state and anomalous feature extraction module is used for extracting spatiotemporal steady-state and anomalous features: For damaged SST images and weekly mean SST images: Based on the wavelet subband features at the current scale, spatial steady-state features and spatial anomalous features are extracted through an attention mechanism; For historical SST image sequences, based on the wavelet subband features at the current scale, temporal steady-state features and temporal anomalous features are extracted through a gating mechanism. The steady-state and anomalous feature fusion module is used to fuse steady-state and anomalous features: it performs single-mode updates on spatial steady-state features, spatial anomalous features, temporal steady-state features, and temporal anomalous features; it performs multi-mode fusion on the updated features using a unified fusion operator; it performs cross-mode collaborative enhancement on the fused features; it completes fine-scale modeling; and it obtains the reconstructed features of each wavelet sub-band at the current scale. The multi-scale feature reconstruction module is used for multi-scale feature reconstruction: wavelet downsampling is performed on the low-frequency sub-bands of the wavelet sub-band features at the current scale of each image to generate a coarse scale. The downsampling, spatiotemporal steady-state and anomalous feature extraction module and steady-state and anomalous feature fusion module are repeated at the coarse scale until the multi-scale modeling from fine to coarse is completed to obtain multi-scale wavelet sub-band features. The high-level scale features in the multi-scale wavelet sub-band features are passed to the low-level scale through layer-by-layer upsampling, and feature integration is performed between different scales to complete the multi-scale feature reconstruction from coarse to fine. The inverse wavelet reconstruction module is used for inverse wavelet reconstruction: the finest scale reconstruction features are reconstructed using inverse wavelet to obtain the final complete SST image.

Images