Method and apparatus for long-range spatio-temporal prediction of marine elements based on error-blocking mechanism

Abstract

The application relates to a kind of ocean element long-distance space-time prediction method and equipment based on error blocking mechanism, belong to marine environmental data processing technical field, the specific steps of the method are as follows: the input containing missing area marine environmental element time series data is completed, and the completion identification mask of the area generated by the completion model in the identification data is generated;The completed data, auxiliary data and the completion identification mask are input into the prediction model;The marine environmental element prediction value of future time generated by the prediction model is output.The application improves the robustness and prediction accuracy of the model for non-perfect input data, solves the problem of insufficient receptive field of traditional convolution network in large-scale ocean dynamic process modeling, and excavates the auxiliary value of multi-source marine environmental elements for chlorophyll concentration prediction.

Description

Technical Field

[0001] This invention belongs to the field of marine environmental data processing technology, specifically relating to a method and device for long-distance spatiotemporal prediction of marine elements based on an error blocking mechanism. Background Technology

[0002] The ocean covers a large portion of the Earth's surface, and the dynamic changes in its environmental parameters (such as chlorophyll a concentration and sea surface temperature) are crucial for studying global climate change, fisheries resource management, and ecological environment monitoring. With the development of satellite remote sensing technology, humans can acquire large-scale, high spatiotemporal resolution ocean observation data. However, optical remote sensing methods are highly susceptible to the effects of clouds, aerosols, and solar flares, resulting in potentially large areas of non-random missing data. This spatiotemporal data discontinuity severely hinders the continuous monitoring and accurate prediction of marine environmental factors.

[0003] To address the aforementioned data gaps, existing technologies typically employ interpolation methods (such as DINEOF) or deep learning-based methods (such as generative adversarial networks and convolutional neural networks) to fill in the missing regions. While existing completion techniques can fill in gaps caused by cloud cover to some extent, this is merely static spatial restoration.

[0004] Therefore, when using the aforementioned ocean satellite remote sensing data for time-series forecasting, existing technologies may face the following major technical challenges: 1. The model lacks the ability to perceive differences in data quality and has an error blocking mechanism: When using the "complete first, predict later" data processing flow, existing marine element prediction models often cannot distinguish the differences between the actual observation values ​​and the algorithm-completed values ​​in the input data.

[0005] 2. Lack of feature fusion and constraint mechanisms guided by physical consistency: Existing marine element prediction models often neglect the in-depth mining and fusion of the characteristics of multi-source heterogeneous data channels at the feature extraction level. Due to the lack of targeted channel feature coordination mechanisms, the models are unable to accurately capture the nonlinear coupling relationship between different physical environmental factors and prediction targets, and may be unable to make full use of the potential physical clues provided by auxiliary variables.

[0006] 3. Limitations of Local Receptive Fields in Capturing Long-Distance Spatiotemporal Dependencies: Ocean dynamics exhibits significant nonlocal characteristics. Large-scale ocean phenomena such as mesoscale eddies and Rossby waves often involve material transport and energy transfer over a range of hundreds of kilometers, and interactions between different sea areas exhibit significant time lag effects. Traditional convolutional neural networks rely on local convolutional kernels to extract features, and are limited by the inherent size of their receptive fields, making it difficult to capture such long-distance dynamic correlations across space, resulting in a significant reduction in prediction accuracy.

[0007] In summary, how to construct a system capable of sensing data uncertainty to block error propagation and dynamically fusing multi-source auxiliary data is a key technical problem that urgently needs to be solved in the field of marine remote sensing. Summary of the Invention

[0008] To address the aforementioned issues, this invention provides a long-range spatiotemporal prediction method for marine elements based on an error blocking mechanism. This method addresses, to some extent, the problems in existing marine environmental element prediction technologies, such as the lack of confidence information in supplementary data, the absence of physical constraints, and the difficulty in capturing long-range dependencies in marine regions.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A long-range spatiotemporal prediction method for marine elements based on an error blocking mechanism, the specific steps of which are as follows: completing the input time-series data of marine environmental elements containing missing regions, and generating a completion label mask for the regions generated by the completion model in the label data; The completed data, auxiliary data, and the completion identifier mask are input together into the prediction model; The prediction model includes an improved recurrent neural network module. The improvement is that the completion identifier mask is used as a gating adjustment signal independent of the input data and historical state, and is input to the gating calculation unit of the recurrent neural network module to dynamically adjust the cell state update based on the completion identifier mask and suppress the transmission of historical errors from the completion region to future moments. Output the predicted values ​​of marine environmental elements at future times generated by the prediction model.

[0010] Furthermore, the improved recurrent neural network module is an error-blocking state-aware ConvLSTM module; the use of the completion marker mask as a gating adjustment signal specifically involves: using the completion marker mask from the previous time step... Embedded into a mask embedding vector through convolution operations. and will Learnable weight matrices are added to the linear computation of the forget gate and the input gate, respectively. The forgetting gate and input gate The calculation is expressed by the following formula: ; ; ; in, This represents the convolution operation. and is the learnable weight matrix corresponding to the mask embedding vector.

[0011] Furthermore, the specific steps for completing time-series data include: The original data and the corresponding valid data mask are input into the completion model, which includes a mask-driven spatiotemporal feature layer-by-layer encoding module. The mask-driven spatiotemporal feature layer-by-layer encoding module includes at least one partial convolutional layer, which is configured to: perform convolution calculations only on the valid regions in the input data according to the valid data mask to generate features, and synchronously update the valid data mask so that the missing regions shrink layer by layer in the feature space.

[0012] Furthermore, the completion model also includes an observation confidence adaptive time-series weighting module, which is configured to: adaptively adjust the weight of each time step feature in time-series fusion according to the effective data ratio of the input data at each time step, and assign lower weights to time steps with greater missing data.

[0013] Furthermore, the completion model also includes a multi-scale feature aggregation module, which contains multiple parallel dilated convolution branches with different dilation rates, used to aggregate spatial context information from receptive fields of different scales during the completion process.

[0014] Furthermore, the loss function for training the complete model includes a physical constraint loss term, which is constructed based on the known theoretical relationship between marine environmental elements and at least one physical variable, and is used to penalize predicted values ​​output by the model that exceed the reasonable range of the theoretical relationship.

[0015] Furthermore, the prediction model also includes a multi-source heterogeneous data channel feature fusion module, an error blocking state-aware ConvLSTM, a cross-temporal and spatiotemporal dimension global correlation dependency capture module, and a spatiotemporal decoding prediction module. The multi-source heterogeneous data channel feature fusion module receives and fuses multiple types of marine data as multi-source auxiliary data; The error blocking state perception ConvLSTM: The fused data from the multi-source heterogeneous data channel feature fusion module is input into the error blocking state perception ConvLSTM. This module dynamically adjusts the gating mechanism according to the completion mask of the previous time step to suppress the transmission of low-confidence historical memory and extract high-quality local spatiotemporal evolution features. The aforementioned global correlation dependency capture module across spatiotemporal dimensions: inputs the temporal features output by the encoder into the global correlation dependency capture module across spatiotemporal dimensions; The spatiotemporal decoding prediction is as follows: after global feature enhancement by the global correlation dependency capture module across spatiotemporal dimensions, spatiotemporal integration is performed through the second layer ConvLSTM, and finally the chlorophyll concentration prediction map for the next time step is output.

[0016] The module is configured to: jointly flatten the input features in the time and space dimensions, and calculate the correlation weight between any two points in the joint time and space dimensions through a self-attention mechanism, so as to model long-distance time and space dependencies.

[0017] Furthermore, the prediction model also includes a multi-source data channel attention fusion module, which is configured to automatically calculate and assign fusion weights to marine environmental element data features and various auxiliary data features through a channel attention mechanism, thereby achieving adaptive feature fusion.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described thereon.

[0019] The beneficial effects of this invention compared to the prior art are as follows: This invention proposes an error-blocking state-aware encoder that embeds data quality information into a recurrent neural network gating structure in the form of a mask. This structurally blocks the cumulative propagation of completion errors along the time axis, improving the model's robustness and prediction accuracy to imperfect input data. The spatiotemporal joint nonlocal module extends the attention mechanism's perception range from a two-dimensional spatial plane to a three-dimensional spatiotemporal cube, effectively capturing long-range correlations and time lag effects across time and space in the marine environment, addressing the problem of insufficient receptive field in traditional convolutional networks for modeling large-scale marine dynamic processes. The introduction of a channel attention mechanism enables the model to adaptively learn and assign different auxiliary variable weights based on the current marine environmental state, improving feature fusion efficiency, enhancing model interpretability, and exploring the auxiliary value of multi-source marine environmental factors in chlorophyll concentration prediction. Attached Figure Description

[0020] Figure 1 This is an overall flow chart of the method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the error blocking state-aware ConvLSTM structure provided in an embodiment of the present invention. Detailed Implementation

[0021] To better understand the technical solution of this method, the following will further explain the method in conjunction with the accompanying drawings and a specific embodiment for ocean element prediction. It should be noted that the described embodiments are merely some, not all, embodiments of this method. All other embodiments obtained by those skilled in the art based on the embodiments provided in this method without inventive effort should fall within the scope of protection of this method.

[0022] like Figure 1As shown, a long-range spatiotemporal prediction method for marine elements based on an error blocking mechanism is proposed. First, high-quality data completion is performed on the acquired raw satellite remote sensing data before corresponding predictions are made. The completion model includes a mask-driven spatiotemporal feature layer-by-layer encoding module, an observation confidence adaptive temporal weighting module, a multi-scale void receptive field feature aggregation module, and a loss function incorporating physical constraints. The mask-driven spatiotemporal feature layer-by-layer encoding module extracts features from the input marine environmental data and combines them with a mask that has labeled valid data. The system distinguishes between regions with real observations and missing regions, improving computational efficiency and the sensitivity of the convolutional kernel to these regions during computation. It calculates features at the center only based on valid values ​​within the receptive field, preventing missing values ​​from lowering the calculated feature values ​​and ensuring the model aggregates only truly valid observation information, avoiding the contamination caused by missing values ​​in ordinary convolution. An adaptive temporal weighting module for observation confidence assigns lower weights to individual time steps with high missing values, preventing impact on model stability. The multi-scale hollow receptive field feature aggregation module is suitable for the non-random, block-like missing values ​​of varying sizes that are common in marine satellite remote sensing data; branches with different expansion rates are beneficial for obtaining multi-scale information. A loss function incorporating physical constraints penalizes imputation values ​​exceeding the theoretical maximum value, ensuring the imputation results also have physical plausibility, as detailed below: Step 1: Acquire raw satellite remote sensing grid data, including sea surface chlorophyll a concentration (CHL) data sequences and sea surface temperature (SST) data sequences, as well as land-sea mask data. Z-score normalization was performed on the CHL data, and land and missing regions in the original data were filled with 0. Step 2: Based on the land and sea cover And for each time step, a corresponding effective data mask is created based on the data situation in the CHL sequence. The effective observation region is marked as 1, and the rest as 0. To enhance the generalization ability of the completion model, a dynamic artificial missing mask strategy is adopted during the training phase: for each batch of data, an artificial missing mask is randomly generated for each time step. Based on this mask, the random regions in the chlorophyll data of the current batch are modified to 0, and the data is updated synchronously. The artificial missing mask is used only for loss function calculation and is not used as a model input channel; it also conforms to the missing rate and missing situation in real data. Step 3: Construct a complete model and train and infer it in conjunction with physical constraints; use the preprocessed chlorophyll data, the land-sea mask, and the updated effective data mask as channel inputs, and use sea surface temperature only for loss function calculation; Step 3.1: Construct a mask-driven spatiotemporal feature layer-by-layer encoding module to prevent missing or imputed values ​​from affecting the model; for the current time t, define the input data as the sea surface chlorophyll data after the above processing. and the effective data mask for the corresponding time step ,Will and By concatenating along the channel dimension, we obtain the input tensor of the module. .

[0023] The completion model first utilizes a mask-driven spatiotemporal feature layer-by-layer encoding module to... Spatial feature extraction is performed, where Partial Conv differs from standard convolution. It only performs convolution calculations on regions marked as valid by the mask and introduces a dynamic normalization mechanism to maintain numerical scale stability. Given the convolution kernel weights... Simultaneously, the mask is updated to achieve effective data extraction and a layer-by-layer shrinkage mechanism is implemented to gradually reduce the missing area. The formula for the first layer is as follows: ; ; Among them, the numerator : indicates that input will be made With mask Element-wise multiplication (i.e., setting missing regions to 0), then multiplying with the convolution kernel. Perform a convolution operation. This step represents the Hadamard product (element-level multiplication). It extracts features using only valid observation information, avoids treating the zero values ​​of land and empty areas as real data for calculation, and prevents the model from confusing the filled zeros with the zeros in the real data after Z-score.

[0024] denominator : Calculate the total number of valid data within the coverage area of ​​the convolution kernel.

[0025] Division operation (normalization): This step is equally important, as it eliminates fluctuations in feature value magnitude caused by uneven distribution of missing values. Regardless of the number of missing values ​​within the convolutional receptive field, the output feature values ​​will remain within the same numerical range, ensuring the consistency of feature distribution.

[0026] : is a very small constant (e.g. (), used to prevent numerical overflow caused by a denominator of 0.

[0027] Mask update This layer calculates whether there is at least one valid observation within the receptive field of the convolutional kernel. If so, it outputs 1; otherwise, it outputs 0. This is used to determine which locations in the feature map are currently inferable. The overall update will result in It will also be updated and used in some subsequent convolutional layers, but the original valid data mask will still be used in subsequent modules.

[0028] In the first layer, only the edges of holes immediately adjacent to the actual observation region are updated as effective masks. In the second and subsequent layers, since the input mask already includes the effective edge regions updated by the previous layers, the receptive field of the convolutional kernel will further cover the innerer hole regions and update these inner regions as effective masks. As the number of layers increases, the hole regions continuously shrink inward from the edges in the feature space, as shown in the following formula in subsequent layers: ; ; After multiple layers of progressive contraction, the features at the current moment will be obtained. It is input into the ConvLSTM module for time-dimension state updates, denoted as... Assume the hidden state of ConvLSTM at the previous time step is Cell state is At this point, a 1×1 convolution can be used to adjust the number of channels to obtain the input features of the ConvLSTM. The structure is as follows: ; To further enhance the model's perception of land and missing regions, this study will... Compared with the original effective data mask As an additional channel, it is concatenated to the input. In this way, the model is explicitly told which regions' features at the current time are extracted from real observation data, and which are missing regions filled in by inferring information from neighbors. This makes ConvLSTM more cautious when dealing with missing regions and helps to use temporal information to help correct the corresponding features.

[0029] ConvLSTM units pass through input gates Forgotten Gate Output gate and cell state To control the flow of information, the specific calculation formula is as follows: ; ; ; ; ; ; Among them, the input gate Controls the extent to which new features extracted at the current moment (including features inferred from them) are written into memory; Forgotten Gate It controls historical information from the previous moment, which helps the model remember the long-term ocean background field during the completion task; Cell state Used to transmit long-term spatiotemporal dependency information, it is crucial for recovering large-scale missing information.

[0030] The hidden state obtained after processing by the mask-driven spatiotemporal feature layer-by-layer encoding module It contains rich spatiotemporal features. To generate a complete feature map with the same resolution as the original input, it is projected through a 1×1 convolutional layer: ; At this time, it was obtained This is the completed feature map output by the mask-driven spatiotemporal feature layer-by-layer encoding module, which will be sent to subsequent modules for further processing.

[0031] Step 3.2: Construct an adaptive time-series weighted module for observation confidence to dynamically evaluate the importance of features at different historical moments and solve the problem of data quality differences caused by uneven cloud cover distribution; After initially extracting spatiotemporal features using a mask-driven layer-by-layer encoding module, an observation confidence adaptive temporal weighting module will be introduced. This module is used to dynamically evaluate the importance of feature maps at different historical moments, especially addressing the issue that uneven cloud cover distribution in marine satellite remote sensing data can lead to significant differences in the degree of missing data. It explicitly reduces the temporal attention weight of moments with severe occlusion and large-scale missing data by using an effective data mask, thereby aggregating a more robust spatiotemporal feature vector. The specific processing flow is as follows: Assuming the output of the mask-driven spatiotemporal feature layer-by-layer encoding module contains The feature sequence at each time step is ,in This represents the feature map corresponding to the t-th time step, and the corresponding effective data mask ratio sequence is also input. ,in This represents the proportion of valid data (i.e., non-land, non-missing) at time step t. This value can be calculated from the valid data mask generated during data preprocessing.

[0032] To calculate the temporal attention score at each time step, global average pooling will be used to transform a feature map with spatial structure (H, W). Compressed into a feature vector describing the overall situation at that time step. The formula is as follows: ; The feature information of each observation scattered throughout the space is aggregated to obtain a vector that can represent the state of the ocean in the entire region at the current moment. This vector ignores specific spatial information and focuses on capturing the overall semantics at that moment.

[0033] The next step is to combine the feature vectors at all time points. spliced ​​into a sequence To capture the dependencies between time steps and compute the attention score at each time step, a one-dimensional convolution pair will be used on the sequence. The following steps will be taken to process the issue: Using a one-dimensional convolution kernel (K is the kernel size), for the sequence Perform convolution calculations and output the raw score for each time step. : ; By using one-dimensional convolution to consider the continuous changes between previous and next time steps (such as t-1, t+1), the importance of the current time step can be smoothly evaluated.

[0034] To address the issue of potentially varying degrees of data loss due to uneven cloud cover distribution in marine satellite remote sensing data, a previously obtained effective data mask ratio sequence was utilized. To match the corresponding raw scores Perform correction operations based on the missing information: ; If the missing value at time step t is very severe, then the corrected score is obtained. This will significantly reduce the reliability of the current time step, making the model more likely to perceive that the reliability of the current time step is low, thereby reducing its importance in subsequent decisions and avoiding affecting the stability of the model.

[0035] Furthermore, to make the importance more apparent, the Softmax function will be used to adjust the corrected scores. After performing a normalization operation, the final temporal attention weights are obtained. : ; Convert the corrected scores into probability distributions This more intuitively shows the importance of the features at that moment to the final completion result.

[0036] Using the calculated time attention score The original, uncompressed PConvLSTM output feature sequence Perform a weighted summation to output a feature map that incorporates temporal importance. : ; Using Time Attention Score Multiplication is performed on all channels of the feature map at time step t, and the importance of each time step for completion is dynamically evaluated based on the quality of the original data and temporal correlation.

[0037] Step 3.3: After aggregating temporal features using the observation confidence adaptive temporal weighting module, a multi-scale dilated receptive field feature aggregation module is introduced. This module constructs multi-scale receptive fields by setting dilated convolution branches with different dilation rates in parallel, thereby achieving the simultaneous extraction of local details and global background information in marine environmental data, suitable for different sizes of blocky missing data that may occur in marine remote sensing.

[0038] The feature map of the time attention output is denoted as... At this point, the feature map contains spatiotemporal information after time-series filtering, but the spatial receptive field of the feature is relatively simple, and it can only focus on information in a small surrounding area. However, in the case of missing marine remote sensing data, there may be blocky missing areas of different sizes.

[0039] The module contains multiple parallel convolutional branches. In this study, we set up a standard convolutional branch and three dilated convolutional branches with different dilation rates.

[0040] Branch 1 performs feature mapping using a standard 1×1 convolution: This branch is used to reduce the feature dimension and to capture local pixel-level correlations as the basic feature retention information of the feature map.

[0041] Branches two, three, and four are dilated convolution branches with different dilation rates. They all use 3×3 convolution kernels, but with dilation rates of r=4, 8, and 12 for dilated convolution operations. The formula for dilated convolution is: ; Where r is the dilation rate, W is the convolution kernel weight, b is the bias term, (i, j) are the coordinates of the output feature map, and the outputs of each branch are as follows: ; ; ; The dilation rate r determines the spacing between the convolution kernel elements. It can be said that r=4 means that the receptive field is expanded to 4 times its original size.

[0042] The different branches represent different scales of capture: The r=4 branch considers more medium-scale ocean textures, such as smaller vortex structures; The r=8 branch considers larger-scale physical processes, such as ocean current trends. The r=12 branch focuses more on capturing macroscopic information, such as vortex patterns.

[0043] The multi-branch design allows the model to reference both nearby information and information at different distances when completing a missing value, which to some extent makes the completion result more spatially coherent.

[0044] This module also includes a fifth branch, used for global statistics and encoding of the entire input feature map, and for processing the input features... Global average pooling is performed along the spatial dimension to compress spatial features into 1×1 feature descriptors: ; Features after pooling Use 1×1 convolutions for feature transformation, adjusting the number of channels to match the output channels of the first four convolutional branches: ; Processed features To restore the spatial dimensions to the original input, upsampling is performed using a bilinear interpolation algorithm: ; This branch assigns global information, such as the overall sea surface chlorophyll a concentration level or distribution trend, to each location in space, which helps to keep the completed area consistent with the overall macroscopic situation of the sea area during the completion task.

[0045] The processed features from the five branches are concatenated along the channel dimension, assuming that each branch outputs a channel number of... The spliced ​​feature map is : ; at this time ,and It contains local features and global context features at different scales, and contains rich spatial semantic information.

[0046] Finally, a 1×1 convolutional layer is used. Channel mixing and dimensionality reduction are performed to output the final features. : This layer is used to learn how to adaptively weight features at different scales according to the needs of different locations, and output high-quality spatiotemporal feature maps for subsequent loss calculation and prediction.

[0047] Step 3.4: The completion model extracts features that incorporate multi-scale spatial information. Next, we need to consider the model training process, specifically obtaining the actual chlorophyll concentration labels for the current training batch. and the corresponding sea surface temperature and the previously generated effective data mask and corresponding artificial missing mask .

[0048] The feature map output by the multi-scale cavity receptive field feature aggregation module is This feature map contains rich semantic information, but it has a large number of channels and its values ​​represent a high-dimensional feature space. Therefore, it should be mapped back to the dimension of the target variable in subsequent iterations. A 1×1 convolutional layer is used to compress the number of feature channels to 1, resulting in a single-channel feature map. : ; Z-score normalization was performed on the input data before model input, and the output... It is still in a standardized state. In order to obtain sea surface chlorophyll a concentration data with physical significance, an inverse normalization operation is required.

[0049] Let the mean chlorophyll concentration of the data used for model training be denoted as . The standard deviation is The chlorophyll concentration after completion for: To restore the model-predicted values ​​to the physically meaningful range of chlorophyll concentration under real-world conditions. .

[0050] Next, we will calculate the loss, starting with an artificial missing mask. Mark which regions should be included in the calculation, where 1 indicates artificially missing regions and 0 indicates other regions. The mean squared error is calculated only in artificially missing regions. ; This forces the model to learn to use the spatiotemporal information of valid data to recover values ​​for regions that were manually missing. To determine the number of artificially created defects at each corresponding time step, and to utilize an artificial defect mask. This forces the model to move beyond simply replicating valid data to reduce losses, requiring it to learn the temporal and spatial correlations and patterns of change of ocean elements, thereby truly mastering the ability to complete the entire area. This is the actual label for the corresponding observation point.

[0051] To prevent the model from generating outliers during the completion process that are numerically reasonable but do not conform to the physical laws of the corresponding region, a physical constraint based on sea surface temperature was introduced, and it was combined with... Consistent and applied only to artificially missing regions, the values ​​used in the model completion are deemed reasonable. The theoretical chlorophyll concentration for the corresponding region was calculated using a Gaussian optimal temperature response model from oceanography, determining the theoretical maximum chlorophyll concentration at each observation location at the current temperature: ; in The maximum potential chlorophyll concentration, For the optimal temperature, These are the half-width parameters for the temperature tolerance range, and they are all preset based on the statistical fitting results of real data. This refers to the actual temperature at the current observation point.

[0052] Because factors such as nutrient limitations in nature cause actual values ​​to be lower than theoretical values, in order to avoid imposing overly strict physical laws on the data, penalties are only imposed on abnormally high values ​​that exceed the theoretical values: ; Marine phytoplankton growth is temperature-limited, and therefore typically has a theoretical upper limit. This loss term penalizes high values ​​that are impossible to achieve at the current temperature, ensuring that the completed result conforms to ecological common sense.

[0053] Therefore, the final loss function is: ; in, These are the weighting coefficients for physical loss; the optimal values ​​were determined through comparative experiments. The model will be based on... The gradient is calculated, and the network parameters of the completed model are updated using the backpropagation algorithm, thereby continuously optimizing the model. Furthermore, because... Backfeeding will only be performed in areas where human error is missing, and the update direction of model parameters will focus on improving the ability to infer unknown information and the physical consistency of inference results.

[0054] Step 3.5: After completing the training of the incomplete model, use the trained model to complete the actual ocean remote sensing data, output the completed chlorophyll concentration data, and generate a mask marking the completed area. This mask will provide data confidence information for subsequent prediction models.

[0055] When using a completion model for completion, the mean obtained during the training phase should also be used. and standard deviation The original data is standardized to conform to the model's input distribution; therefore, the model output should be standardized accordingly. Perform inverse normalization and land removal data processing: ; ; Right now The final chlorophyll concentration data after model completion is used to supplement the model and can be used in subsequent predictive models or for visualization analysis experiments.

[0056] Due to the needs of subsequent forecasting work, uncertainty information should be provided to the forecasting model, and a model will also be generated that is consistent with... Binary masks with consistent spatial dimensions : ; in, =1 represents an ocean area. =0 represents a land area. =1 indicates that there is a real observation here. =0 indicates that this is an original empty area, therefore: hour This indicates that the value here is generated by the model; the rest represent land areas or true value areas. This mask acts as a bridge between the completion model and the prediction model, explicitly marking which values ​​in the entire ocean area were inferred by the completion model. Subsequent models will use this information to apply special processing to these areas, thereby reducing the impact of completion errors on predictions.

[0057] Step 4: This embodiment provides a marine chlorophyll concentration prediction model, which aims to solve the error accumulation problem in the existing technology when processing cloud occlusion data, and the problem of limited receptive field when capturing large-scale marine dynamic processes.

[0058] The prediction model described in this invention mainly consists of four parts: a multi-source heterogeneous data channel feature fusion module, an error blocking state-aware ConvLSTM module, a cross-temporal and spatiotemporal global correlation dependency capture module, and a spatiotemporal decoding prediction module. Its specific workflow is as follows: Multi-source heterogeneous data channel feature fusion module: Receives the chlorophyll concentration features (including complete data) at the current moment and fuses SST (sea surface temperature), PAR (photosynthetically active radiation), PIC (particulate inorganic carbon), POC (particulate organic carbon), and NFLH (solar-induced chlorophyll fluorescence) as multi-source auxiliary data.

[0059] Error-blocking state-aware ConvLSTM module: The fused features are input into the error-blocking state-aware ConvLSTM. This module dynamically adjusts the gating mechanism according to the completion mask of the previous time step to suppress the transmission of low-confidence historical memory and extract high-quality local spatiotemporal evolution features.

[0060] Global correlation dependency capture module across spatiotemporal dimensions: The temporal features output by the encoder are input into the global correlation dependency capture module across spatiotemporal dimensions. This module calculates global attention in a unified three-dimensional spatiotemporal domain to capture long-distance correlations (such as ocean current transport and lag effects) across time and space.

[0061] Decoding and prediction module: After global feature enhancement, spatiotemporal integration is performed through the second layer ConvLSTM, and finally the chlorophyll concentration prediction map for the next time step is output.

[0062] The inputs to the prediction model at time t include: Completed chlorophyll concentration characteristic sequence ; Multi-source auxiliary data feature sequences ,in To assist in the number of data types; land and sea mask In the diagram, 1 represents the ocean area and 0 represents the land area; Previous moment complete area identifier mask , where 1 represents the completed region with low confidence, and 0 represents the actual observed region with high confidence.

[0063] To improve the stability of model training and accelerate convergence, Z-score standardization is applied to all input marine remote sensing elements, and the generated prediction results are then de-standardized accordingly.

[0064] Step 4.1: To address the issues of missing information in cloud-covered areas and changing correlations between multi-source marine environmental factors and sea surface chlorophyll concentration in single chlorophyll data, this embodiment designs a multi-source heterogeneous data channel feature fusion module. This module aims to adaptively integrate the nonlinear coupling relationship between chlorophyll features and multi-source auxiliary data features, allowing the model to automatically learn the contribution of different marine environmental factors to chlorophyll prediction under different conditions. The specific processing flow is as follows: The input at the current time t consists of two parts: Completed CHL feature map The feature tensor obtained by convolutionally encoding the chlorophyll concentration data output from the completion model. , .

[0065] Auxiliary data feature map : A feature tensor composed of various auxiliary data (such as sea surface temperature (SST), photosynthetically active radiation (PAR), etc.) spliced ​​together and encoded. , ,in This refers to the number of auxiliary data types.

[0066] First, the target variable features and auxiliary variable features are concatenated along the channel dimension to form a fused feature map. ,in , .

[0067] To obtain global distribution information for each channel, the fused feature map is... Perform global average pooling. Compress the spatial dimension H×W to 1×1 to obtain the channel description vector. : ; in This represents the global average value of the c-th channel, reflecting the overall response intensity of that channel across the entire sea area.

[0068] To learn the nonlinear correlations between channels and generate normalized weight coefficients, the channel description vectors are input into a lightweight fully connected network. This network consists of two fully connected layers: ; in and This is the weight matrix of the fully connected layer. It is the ReLU activation function. It is the Sigmoid activation function. This is the final output channel weight vector.

[0069] The learned weight vector Multiply back to the original fused feature map On the corresponding channel, adaptive filtering of multi-source features is completed: ;in For the first The weighting coefficients of each channel, and the land and sea mask This information is fed into subsequent modules as a channel, allowing the model to gradually realize that land areas do not need to be learned.

[0070] Step 4.2: To address the issue of prediction drift caused by the accumulation of noise introduced by the supplementary data over time, such as... Figure 2 The embodiment shown here modifies the structure of the standard ConvLSTM to obtain an error blocking state-aware ConvLSTM.

[0071] An additional completion mask from the previous time step was introduced. By participating in the gating process, ConvLSTM can perceive which regions in the historical memory were generated by model completion (regions with low confidence), thus maintaining a cautious attitude towards these historical memories during state updates and preventing the accumulation of errors in the time dimension.

[0072] Let the input features at the current time be... The hidden state of the previous moment The cell state at the previous moment Introduce the completion region marker mask from the previous time step. As an additional input, a value of 1 in the mask represents the padded value at that position in the previous time step, and a value of 0 represents the true observation value. To address the mask missing problem at the beginning of the sequence, the initial mask is set to an all-zero matrix, indicating that the historical memory is assumed to be a reliable state at the beginning of the sequence.

[0073] In order to change the weight state of the previous time step Integrating into the decision-making mechanism of ConvLSTM, a 1×1 convolutional layer is first used to map the single-channel mask onto the same feature space as the hidden state, resulting in the mask embedding vector. : ; And add it to the gating operation: ; For masking attention weights, when a region was the padding value in the previous step, the corresponding... This will act as a strong signal. During training, the network will learn to use This exhibits an inhibitory effect, meaning that the model may forget unreliable historical memories derived from the completion data to prevent contaminating the current state update of that region.

[0074] ; In the current moment's completion region, to compensate for the unreliability of historical memory, the input gate may open accordingly, absorbing more auxiliary data information from the current moment's region after gating and fusion correction, thereby correcting deviations in the historical state. ; ; because Due to the inhibitory effect of the completion region, the weight of historical memory is reduced, while the weight of the current input is increased. This makes state updates rely more on currently observed environmental information, rather than on inferred values ​​from the past that contain some unavoidable errors. ; ; Unlike the standard ConvLSTM, this module introduces the aforementioned mask embedding term when calculating the input gate, forget gate, and output gate. This mechanism structurally prevents the cumulative diffusion of completion errors during long-term propagation.

[0075] Step 4.3: Construct a global dependency capture module that spans the spatiotemporal dimensions to address the problem of limited receptive field in ordinary convolution operations and the problem of traditional nonlocal blocks ignoring temporal dependencies.

[0076] Traditional nonlocal neural networks primarily model long-distance spatial dependencies within a single moment, making it difficult to characterize typical cross-temporal and cross-spatial lag-related processes in the marine environment. This invention proposes a global correlation dependency capture module that spans both temporal and spatial dimensions. By unifying the temporal and spatial dimensions into a global point set, a three-dimensional attention map is constructed, enabling the model to directly aggregate feature information from all observation locations within the ocean area at current and historical moments. This effectively captures large-scale transport processes and time-lag effects, significantly improving the modeling capability for long-term evolutionary processes.

[0077] Let the feature sequence output by the error-blocking state-aware ConvLSTM be... Where T is the time step, C is the number of channels, and H×W is the spatial dimension, the time dimension and spatial dimension are jointly flattened, and the four-dimensional tensor is reshaped into a three-dimensional tensor. Where S = T × H × W, this operation treats the entire spatiotemporal sequence as a unified set of spatiotemporal points.

[0078] To calculate global correlation, a 1×1 convolution is first applied to the input features. Three different linear transformations are performed to generate query features, key features, and value features respectively: Query characteristics: ; Key features: ; Value characteristics: ; in , and All are 1×1 convolutional kernel weights, and the transformed feature dimension is To reduce computational load, it is usually made .

[0079] Global attention is computed in the reshaped feature space, through an embedding function. and Mapping features to the embedding space: ; The obtained attention matrix Elements in the matrix It represents the correlation weight between the i-th spatiotemporal point (a certain observation position at the current time) and the j-th spatiotemporal point (a certain observation position at the historical or current time).

[0080] The calculated spatiotemporal attention map is used to perform weighted aggregation of the original features: ; Meanwhile, in order to preserve the original local features and accelerate network training, the calculated non-local features are... With the original input Perform residual linking: ; in For 1×1 convolutional layers used for feature dimension alignment, This enhanced feature, unlike traditional nonlocal blocks that only calculate spatial correlations within a single moment, can directly establish a connection between the current observation point and any historical moment and any spatial location, breaking down the boundaries between time and space. This enables the model to effectively capture time lag effects in ocean dynamic processes (such as the delayed impact of nutrient transport on chlorophyll concentration) and long-distance dependencies across regions, significantly improving its ability to model complex ocean evolution processes.

[0081] Step 4.4: Then, carry out the final feature fusion and prediction work, perform deep fusion processing on the global spatiotemporal features and temporal memory states from the feature fusion module of multi-source heterogeneous data channels, and output the final prediction results.

[0082] Let the input of this module be... This refers to the enhanced features output by the multi-source heterogeneous data channel feature fusion module. It contains enhanced global context information, and the hidden state and cell state of this ConvLSTM layer at the previous time step are denoted as... and .

[0083] In the error-blocking state-aware ConvLSTM, state-aware processing has been performed to address the uncertainty of the completed region. Furthermore, after feature enhancement by the spatiotemporal joint nonlocal module, the data confidence has been improved to a certain extent. Therefore, the second layer of ConvLSTM can adopt a standard structure, allowing it to focus on the temporal aggregation of features and the final prediction. The formula is as follows: ; ; ; ; ; ; The hidden state output by the second layer ConvLSTM The input is fed into a final convolutional layer to generate a chlorophyll concentration prediction at time t+1: ; This ConvLSTM module integrates the spatiotemporal correlation information extracted by the multi-source heterogeneous data channel feature fusion module into the time series memory. This module not only focuses on local evolution but also makes reasonable adjustments to the predicted measurements based on global physical correlations. Simultaneously, it integrates the results of all preceding modules in terms of completion correction, error blocking, and global dependency handling, ultimately outputting high-precision chlorophyll prediction results.

[0084] The loss calculation will now begin, starting with the use of a land-sea mask. Mark which regions need to be calculated, where 1 indicates the ocean region to be learned, and 0 indicates the land region. Root mean square error is calculated only in the ocean region. ; This enables the model to learn to use spatiotemporal information about the ocean region to predict values ​​at the next moment, and utilizes the land-sea shield. This enables the model to accurately focus on the ocean areas that need to be learned and predicted.

[0085] Meanwhile, to avoid outliers in the model's predictions that are numerically reasonable but do not conform to the physical laws of the corresponding region, a physical constraint based on sea surface temperature is introduced, and this constraint is consistent with... Consistent with relevant conditions, the model applies only to marine areas to ensure the reasonableness of the predicted values. When calculating the theoretical chlorophyll concentration for the corresponding region, a Gaussian optimal temperature response model from oceanography is used to calculate the theoretical maximum chlorophyll concentration at the current temperature for each observation location: ; in The maximum potential chlorophyll concentration, For the optimal temperature, These are the half-width parameters of the temperature tolerance range, and they are all preset based on the statistical fitting results of real data.

[0086] Given factors such as nutrient limitations in nature that cause actual values ​​to be lower than theoretical values, to avoid imposing overly strict physical laws on the data, only abnormally high values ​​exceeding the theoretical values ​​are processed: ; The growth of marine phytoplankton is temperature-dependent, and therefore typically has a theoretical upper limit. This loss term penalizes high values ​​that are impossible under current temperature conditions to ensure that the predictions conform to basic ecological principles.

[0087] Therefore, the final loss function is: ; in, These are the weighting coefficients for physical loss; the optimal values ​​were determined through comparative experiments. The model will be based on... The gradient is calculated, and the network parameters of the completed model are updated using the backpropagation algorithm, thereby continuously optimizing the model. Furthermore, because... Backhauling only to the ocean region will focus on improving the inference capability for the ocean region and the physical consistency of the inference results.

[0088] Step 4.5: After completing the training of the prediction model, use the trained model to perform inference and output the chlorophyll concentration data for the next time step.

[0089] When using predictive models for inference, the mean obtained from the training phase should also be used. and standard deviation The raw data is standardized to conform to the model's input distribution; therefore, the model output should also be standardized. Perform inverse normalization and land removal data processing: ; ; Right now The chlorophyll concentration data for the next time step, inferred from the prediction model, can then be used for visualization analysis experiments.

[0090] In this embodiment, to test the advantages of the completion model in this invention, traditional statistical methods, classic deep learning image inpainting methods, marine-specific methods, and spatiotemporal graph methods were selected for horizontal comparison. The completion model proposed in this invention has high accuracy in completing marine chlorophyll content, and the constraint effect of physical constraints in this invention can be seen from the physical consistency violation rate index. The comparative experimental results are shown in Table 1:

[0091] Table 1. Comparison of experimental results of the completion model. .

[0092] In this embodiment, since the input data is on a monthly scale, long-term predictions (such as predicting data six months later) are prone to error explosion. To test the advantages of the prediction model in this invention, common baselines were selected for horizontal comparison, and their evaluation indicators for predicting the 1st, 3rd, and 6th months in the future were compared. After comparison, the prediction model proposed in this invention has higher accuracy in predicting marine chlorophyll content and is less prone to prediction drift. The comparative experimental results are shown in Table 2.

[0093] Table 2. Comparison of Prediction Models: Experimental Results 。

Claims

1. A long-range spatiotemporal prediction method for ocean elements based on an error blocking mechanism, characterized in that, The specific steps of the method are as follows: the input time series data of marine environmental elements containing missing regions are completed, and a completion label mask of the region generated by the completion model in the label data is generated; the completed data, auxiliary data and the completion label mask are input into the prediction model. The prediction model includes a multi-source heterogeneous data channel feature fusion module, an error blocking state-aware ConvLSTM module, a cross-temporal and spatial dimension global correlation dependency capture module, and a spatiotemporal decoding prediction module. The multi-source heterogeneous data channel feature fusion module receives the chlorophyll concentration features at the current moment and fuses sea surface temperature, photosynthetically active radiation, particulate inorganic carbon, particulate organic carbon, and solar-induced chlorophyll fluorescence as multi-source auxiliary data. The error blocking state-aware ConvLSTM module: inputs the fused features into the error blocking state-aware ConvLSTM. This module dynamically adjusts the gating mechanism according to the completion mask of the previous time step to suppress the transmission of low-confidence historical memory and extract high-quality local spatiotemporal evolution features. The global correlation dependency capture module across spatiotemporal dimensions: inputs the temporal features output by the encoder into the global correlation dependency capture module across spatiotemporal dimensions; this module calculates global attention in a unified three-dimensional spatiotemporal domain to capture long-distance correlations across time and space; The spatiotemporal decoding and prediction module: after global feature enhancement, spatiotemporal integration is performed through the second layer of ConvLSTM, and finally outputs the chlorophyll concentration prediction map for the next moment; The differential blocking state-aware ConvLSTM module includes an improved recurrent neural network module that uses the completion identifier mask as a gating adjustment signal. Specifically, it uses the completion identifier mask from the previous time step as the gating adjustment signal. Embedded into a mask embedding vector through convolution operations. and will Learnable weight matrices are added to the linear computation of the forget gate and the input gate, respectively. The forgetting gate and input gate The calculation is expressed by the following formula. ； ； ； in, This represents the convolution operation. and This corresponds to the mask embedding vector.

2. The long-range spatiotemporal prediction method for ocean elements based on an error blocking mechanism according to claim 1, characterized in that, The specific steps for completing time-series data include: The original data and the corresponding valid data mask are input into the completion model, which includes a mask-driven spatiotemporal feature layer-by-layer encoding module; The mask-driven spatiotemporal feature layer-by-layer encoding module includes at least one partial convolutional layer, which is configured to: perform convolution calculations only on the effective regions in the input data based on the effective data mask to generate features, and synchronously update the effective data mask so that the missing regions shrink layer by layer in the feature space.

3. The long-range spatiotemporal prediction method for ocean elements based on an error blocking mechanism according to claim 2, characterized in that, The completion model also includes an observation confidence adaptive time-series weighting module. This module adaptively adjusts the weight of each time step feature in time-series fusion based on the effective data ratio of the input data at each time step, assigning lower weights to time steps with greater missing data.

4. The long-range spatiotemporal prediction method for ocean elements based on an error blocking mechanism according to claim 3, characterized in that, The completion model also includes a multi-scale feature aggregation module, which contains multiple parallel dilated convolution branches with different dilation rates, used to aggregate spatial context information from receptive fields of different scales during the completion process.

5. The long-range spatiotemporal prediction method for ocean elements based on an error blocking mechanism according to claim 4, characterized in that, The loss function used to train the complete model includes a physical constraint loss term, which is constructed based on the known theoretical relationship between marine environmental factors and at least one physical variable, and is used to penalize predicted values ​​output by the model that exceed the reasonable range of the theoretical relationship.

6. The long-range spatiotemporal prediction method for ocean elements based on an error blocking mechanism according to claim 5, characterized in that, The prediction model also includes a multi-source data channel attention fusion module. This module uses a channel attention mechanism to automatically calculate and assign fusion weights to the features of marine environmental elements and various auxiliary data features, thereby achieving adaptive feature fusion.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the long-range spatiotemporal prediction method for marine elements based on an error blocking mechanism as described in any one of claims 1-6.

Images