Polar region sea ice thickness intelligent prediction method based on multi-layer stacking space-time Transform

Abstract

The invention belongs to the technical field of prediction of the thickness of sea ice in the arctic region, and particularly relates to an intelligent prediction method for the thickness of sea ice in a polar region based on multi-layer stacking space-time Transform. The method comprises the following steps: step 1, based on polar region sea ice thickness reanalysis data information of a target sea area, extracting a feature matrix in ocean SIT reanalysis data and constructing a space-time sequence forecast supervision data set; 2, preprocessing data of a background field and a target field; 3, an HSS-Transform model is constructed, and the HSS-Transform model is constructed; 4, inputting the sea ice thickness background field space-time sequence into an HSS-Transform model to obtain a sea ice thickness prediction space-time sequence in the next 15 days; step 5, carrying out reverse normalization operation on the obtained prediction space-time sequence, and restoring the land point; according to the method, the spatial-temporal characteristics of the sea ice thickness data can be extracted and processed at the same time on multiple time scales and spatial scales, the extension period intelligent prediction of the polar region sea ice thickness spatial-temporal sequence is realized, and the prediction level of the sea ice thickness is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of sea ice thickness prediction in the Arctic region, and specifically relates to an intelligent prediction method for polar sea ice thickness based on a multi-layer stacked spatiotemporal Transformer. Background Art

[0002] Sea ice thickness (SIT) is a key variable in the polar climate system, significantly impacting global climate change, marine ecosystems, and marine resource development. Sea ice variations not only influence energy exchange, climate regulation, and material circulation in the polar regions, but also play a crucial role in regulating the global climate system. As climate change intensifies, accurately predicting changes in sea ice thickness has become a research priority in climate science, oceanography, resource development, and marine security.

[0003] Currently, sea ice thickness prediction methods rely primarily on two technical approaches: numerical prediction and statistical prediction. Currently, polar sea ice forecasts primarily rely on numerical prediction methods. These methods, based on physical equations, simulate the interactions between sea ice, the ocean, and the atmosphere to predict sea ice thickness. The prediction process demonstrates a strong dependence on initial atmospheric conditions, resulting in high computational complexity, significant resource consumption, and severely limited forecast timeliness. This makes it difficult to meet the current sea ice thickness forecasting needs of offshore platforms, underwater unmanned and manned vehicles, and ships. Statistical prediction methods, based on probability theory and mathematical statistics, are a type of prediction method. These methods do not rely on initial conditions and directly utilize statistical models to extract statistical patterns from sea ice thickness data, thereby predicting sea ice thickness. However, sea ice thickness is influenced by a variety of factors, including meteorological and ocean dynamics, and its variations exhibit strong nonlinear, highly coupled, and multidimensional characteristics. Traditional statistical methods struggle to capture complex spatiotemporal relationships, resulting in suboptimal prediction results.

[0004] In recent years, the application of artificial intelligence technology, especially deep learning methods, in the field of sea ice forecasting has made significant progress. To this end, the present invention proposes an intelligent prediction method for polar sea ice thickness based on a multi-layer stacked spatiotemporal Transformer. Combined with the deep learning model, it can automatically extract the spatiotemporal features in the data driven by massive ocean spatiotemporal series data, and can reveal the potential laws of sea ice thickness changes through data analysis, thereby improving the timeliness and accuracy of its forecast. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent prediction method for polar sea ice thickness based on a multi-layer stacked spatiotemporal Transformer. The proposed HSS-Transformer model combines technologies such as convolutional neural networks (CNN) and spatiotemporal attention mechanisms. It can simultaneously extract and process the spatiotemporal features of sea ice thickness data at multiple time scales and spatial scales, thereby realizing extended-period intelligent prediction of polar sea ice thickness spatiotemporal series, significantly improving the prediction level of sea ice thickness.

[0006] The technical solutions adopted by the present invention are as follows:

[0007] An intelligent prediction method for polar sea ice thickness based on multi-layer stacked spatiotemporal Transformers, characterized by comprising the following steps:

[0008] Step 1: Based on the polar sea ice thickness reanalysis data of the target sea area, the characteristic matrix in the ocean SIT reanalysis data is extracted and a spatiotemporal series forecast supervision dataset is constructed, which includes the spatiotemporal series of the sea ice thickness background field for model input and the spatiotemporal series of the ocean thickness target field for model weight training and updating;

[0009] Preferably, the data processing and analysis method in step 1 is as follows:

[0010] Using the SIT reanalysis data of the target sea area, the temporal and spatial variation patterns and characteristic matrix of SIT are studied. The temporal and spatial sample matrix X of the polar region SIT for each day in the past years to be analyzed is as follows:

[0011]

[0012] Where N represents the number of spatiotemporal sample matrices, the dimension of the spatiotemporal sample matrix X is N×m×n, and m and n represent the number of latitude and longitude grid points in the target sea area, respectively. represents the SIT matrix of the i-th sample;

[0013] In order to construct a spatiotemporal sequence forecast supervision dataset of marine elements, the marine spatiotemporal sample matrix is processed into a spatiotemporal sequence forecast supervision dataset of marine environmental elements. The specific processing process is as follows:

[0014] For the spatiotemporal sample matrix of ocean elements In order to ensure that the neural network contains sufficient prior information input, the background field window length of the ocean environment elements is set to T in , the target field window length is T out , then the input background field X of the marine environment elements in and output target field X out The matrix form of is as follows:

[0015]

[0016] Where B is the number of samples, B = ST in -T out +1, input background field X in The dimension is B×T in ×m×n, output target field X out The dimension is B×T out ×m×n.

[0017] Step 2: Preprocess the background and target field data, process the land points, and normalize and standardize the time series;

[0018] Preferably, in step 2, by extracting land features from the spatiotemporal matrix and marking land points in the spatiotemporal sample matrix using a numerical matrix, the extracted land matrix L is in the following form:

[0019]

[0020] in, Represents the ocean and land points in the i-th sample, and uses 0 and 1 to distinguish the land points and ocean points in the spatiotemporal sample matrix. Finally, perform matrix bitwise multiplication on the land matrix L and the spatiotemporal sample matrix X to obtain the SIT spatiotemporal matrix X after the land points are marked L :

[0021]

[0022] After the above operations, invalid land points in the SIT spatiotemporal sample matrix are assigned a value of 0, while valid points retain the SIT value. Normalization is to scale the data to a fixed range. The normalization formula is as follows:

[0023]

[0024] Among them, X L,m×n is the unnormalized SIT space-time series, X L,min is the minimum value in the SIT space-time series, X L,max is the minimum value in the SIT space-time series, X L(norm) is the normalized SIT spatiotemporal series;

[0025] The standardized formula is expressed as:

[0026]

[0027] Among them, X L,m×n is the unstandardized SIT time-space series, μ is the mean of the time-space series, σ is the standard deviation of the time-space series, XL(std) is the standardized SIT spatiotemporal series;

[0028] After the above data preprocessing operations, SIT spatiotemporal series data that can be used for neural network training and verification are finally obtained.

[0029] Step 3: Build the HSS-Transformer model, train it, and optimize the parameters to output the trained HSS-Transformer model.

[0030] The prediction unit of the HSS-Transformer used is the spatiotemporal Transformer model, and the external stacking design enables the model to learn the multi-spatial scale feature information of SIT. The spatiotemporal Transformer model is the main program for prediction, with the encoder part on the left and the decoder part on the right.

[0031] The encoder is responsible for encoding the input sequence, and the encoder layer is the component unit of the encoder. Each encoder layer extracts and transforms the input features. Specifically, the input sequence first passes through a multi-head attention sub-layer to capture the key dependencies within the input sequence. The input then passes through a normalization layer to normalize the data so that its feature values are within a reasonable range. Then there is a residual connection to add the original input to the result of the attention sub-layer to alleviate the gradient vanishing problem. Finally, a feedforward fully connected sub-layer is used to perform a nonlinear transformation on the features. The feedforward fully connected layer has a fully connected network with two linear layers. After a linear transformation, ReLU nonlinearity is performed, and then a second linear transformation is performed. After the normalization layer and residual connection, the output of the encoder is generated.

[0032] The decoder is primarily responsible for gradually generating the target sequence based on the encoder output. Similar to the encoder, the decoder layer is the basic unit of the encoder, and each decoder layer consists of three sublayers. The multi-head attention sublayer is used to capture dependencies within the target sequence, the multi-head attention sublayer focuses on the encoder output, and incorporates key source information into the generation of the target sequence. The feedforward fully connected sublayer is used for nonlinear transformation to extract higher-level feature representations.

[0033] The HSS-Transformer stacks the spatiotemporal Transformers, and performs spatiotemporal Transformer operations on the intermediate state spatiotemporal sequences after the convolution operation on the initial input SIT spatiotemporal sequence, and gradually incorporates the predicted values of each different spatial size in the process of generating the output.

[0034] Preferably, in step 3, the temporal multi-head attention mechanism is mainly used to capture the temporal correlation in the SIT spatiotemporal sequence and establish the dependency relationship between different time points; the calculation formula of the temporal attention mechanism of each attention head is as follows:

[0035]

[0036] Where Q, K and V represent query, key and value tensors respectively, d k is the dimension of the query vector Q; Q represents the input query information and is the input tensor of the attention mechanism; K is used to measure the correlation between Q and other information; V represents the actual information related to each Q; after calculating the attention of each attention head, the results of multiple heads are spliced together and a linear transformation is performed to obtain the final temporal multi-head attention output:

[0037] MultiHead(Q,K,V)=Concat(head1,...,head p )W o

[0038] Where p represents the number of attention heads, W o is a learnable weight matrix; in the prediction of SIT spatiotemporal sequences, the spatiotemporal Transformer structure uses both temporal multi-head attention and temporal multi-head self-attention; when only the dependencies within the sequence are considered, that is, when Q = K = V, the attention mechanism becomes a self-attention mechanism; the self-attention mechanism is mainly used to capture the dependencies between different positions in the sequence without involving interactions with external sequences; in addition, the model also uses the absolute position encoding in the original Transformer to indicate the position of each time point in the sequence, thereby assisting the model in understanding the relative positions of each time point in the time series.

[0039] Step 4: Input the spatiotemporal sequence of the sea ice thickness background field into the HSS-Transformer model to obtain the spatiotemporal sequence of the sea ice thickness forecast for the next 15 days;

[0040] Preferably, in step 4, the input is a SIT spatiotemporal sequence input I=R n×l×c×h×w , first reshape the input tensor to I = R (nhw)×l×c , where n represents the batch size, l represents the time step, c is the number of channels, h and w are the spatial dimensions; next, the standard Transformer is used to perform temporal attention calculation on the reshaped sequence, and the obtained spatiotemporal features will be reshaped into I'=R n×;l×c×h×w , for use.

[0041] Preferably, when processing the spatial multi-head attention mechanism, a unique code is generated for each spatial point, which can accurately represent the position of the point in the two-dimensional grid; each coordinate of the two-dimensional space (i.e., x and y coordinates) is encoded using sine and cosine functions respectively to obtain two independent encoding vectors, and these two vectors are spliced to form a unique spatial position code; let the spatiotemporal feature sequence calculated by the temporal multi-head self-attention mechanism be I'=R n×l×c×h×w , first reshape it into I'=R (nl)×(hw)×c , making it similar to a time series; then, use the same transformation matrix to perform matrix multiplication with I′ to obtain the query Q, key K and value $V$; then, add the previously generated two-dimensional spatial position code to Q and K, and embed the spatial position information into the matrix for subsequent calculation of the spatial multi-head self-attention mechanism; next, the output feature sequence after the spatial multi-head attention mechanism operation is deformed to obtain the output, which is the spatiotemporal sequence of sea ice thickness forecast for the next 15 days I”=R n×l×c×h×w .

[0042] Step 5: Denormalize the obtained predicted spatiotemporal series and restore the land points; finally, obtain the forecast results of the sea ice thickness elements in the target sea area.

[0043] Preferably, in step 5, the prediction result of the HSS-Transformer is denormalized and the land points are restored to obtain the final SIT prediction value; the denormalization and denormalization operations are opposite to the normalization and standardization operations.

[0044] The technical effects achieved by the present invention are:

[0045] This paper is based on the long-term sea ice thickness data of the target sea area, integrates multi-source ocean observation data such as air-based, shore-based, and sea-based, and uses the HSS-Transformer model to analyze and extract the spatiotemporal variation laws of sea ice thickness, thereby realizing long-term spatiotemporal joint modeling and prediction of sea ice thickness.

[0046] The present invention breaks away from the dependence of traditional numerical prediction methods on the initial atmospheric field and the limitations of forecast timeliness. It can deeply explore the spatiotemporal characteristics and nonlinear laws of sea ice thickness data, and significantly improve the extended-term forecast capability of sea ice thickness. In addition, compared with traditional statistical models, HSS-Transformer can significantly enhance the ability to extract spatiotemporal features and improve the accuracy of prediction results. Through the method of the present invention, it is possible to effectively solve the problem of spatiotemporal feature extraction of traditional numerical model methods in extended-term sea ice thickness prediction, and provide high-precision sea ice thickness prediction for offshore platforms, underwater unmanned / manned vehicles, ships and other platforms.

[0047] This paper proposes an intelligent prediction method for polar sea ice thickness based on a hierarchical stacked spatiotemporal Transformer (HSS-Transformer) model. This model primarily extracts and analyzes the spatiotemporal features of the polar sea ice thickness spatiotemporal series from the spatiotemporal Transformer model. It also leverages its internal temporal and spatial multi-head attention mechanisms to assign different influence weights to spatiotemporal features, thereby focusing on key features and achieving improved timeliness and accuracy for the SIT spatiotemporal series prediction task.

[0048] Driven by massive ocean spatiotemporal series data, the deep learning model of the present invention can automatically extract spatiotemporal features from the data, and can reveal the underlying patterns of changes in sea ice thickness through data analysis, thereby improving the timeliness and accuracy of its forecast.

[0049] The HSS-Transformer model proposed in this paper combines technologies such as convolutional neural networks (CNN) and spatiotemporal attention mechanisms. It can simultaneously extract and process the spatiotemporal features c of sea ice thickness data at multiple time scales and spatial scales, thereby realizing extended-period intelligent prediction of the spatiotemporal series of sea ice thickness in the polar region, significantly improving the prediction level of sea ice thickness. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of an intelligent prediction method for polar sea ice thickness based on multi-layer stacked spatiotemporal Transformers of the present invention;

[0051] Figure 2 This is the spatiotemporal Transformer architecture diagram in the present invention;

[0052] Figure 3 This is a diagram of the multi-layer stacked spatiotemporal Transformer architecture in the HSS-Transformer model of the present invention;

[0053] Figure 4 It is the spring forecast map in the appendix of the specific experiment of the present invention;

[0054] Figure 5 It is the autumn forecast map in the specific experimental appendix of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is described in detail below with reference to the following examples. It should be understood that the following text is only used to describe one or more specific embodiments of the present invention and does not strictly limit the scope of protection of the present invention.

[0056] like Figure 1 As shown in FIG, a method for intelligently predicting polar sea ice thickness based on a multi-layer stacked spatiotemporal Transformer includes the following steps:

[0057] Step 1: Based on the polar sea ice thickness reanalysis data of the target sea area, the characteristic matrix in the ocean SIT reanalysis data is extracted and a spatiotemporal series forecast supervision dataset is constructed, which includes the spatiotemporal series of the sea ice thickness background field for model input and the spatiotemporal series of the ocean thickness target field for model weight training and updating;

[0058] Preferably, in step 1, based on the SIT reanalysis data of the target sea area, the characteristic matrix in the ocean SIT reanalysis data is extracted and a spatiotemporal series forecast supervision dataset is constructed, which includes the spatiotemporal series of the sea ice thickness background field input to the model and the spatiotemporal series of the sea ice thickness target field used for model weight training and updating. The data processing and analysis method is as follows:

[0059] Using the SIT reanalysis data of the target sea area, the temporal and spatial variation patterns and characteristic matrix of SIT are studied. The temporal and spatial sample matrix X of the polar region SIT for each day in the past years to be analyzed is as follows:

[0060]

[0061] Where N represents the number of spatiotemporal sample matrices, the dimension of the spatiotemporal sample matrix X is N×m×n, and m and n represent the number of latitude and longitude grid points in the target sea area, respectively. Represents the SIT matrix of the i-th sample;

[0062] In order to construct a spatiotemporal sequence forecast supervision dataset of marine elements, the marine spatiotemporal sample matrix is processed into a spatiotemporal sequence forecast supervision dataset of marine environmental elements. The specific processing process is as follows:

[0063] For the spatiotemporal sample matrix of ocean elements In order to ensure that the neural network contains sufficient prior information input, the background field window length of the ocean environment elements is set to T in , the target field window length is T out , then the input background field X of the marine environment elements in and output target field X out The matrix form of is as follows:

[0064]

[0065]

[0066] Where B is the number of samples, B = ST in -T out +1, input background field X in The dimension is B×T in ×m×n, output target field X out The dimension is B×T out ×m×n.

[0067] Step 2: Preprocess the background and target field data, process the land points, and normalize and standardize the time series;

[0068] Preferably, in step 2, the background field and target field data are preprocessed, the land points are processed, and the time series is normalized and standardized. Since the SIT spatiotemporal sample matrix of the target sea area contains invalid land points, it will seriously affect the convergence of the neural network model if no processing is performed. The present invention extracts land features from the spatiotemporal matrix and marks the land points in the spatiotemporal sample matrix through a numerical matrix. The extracted land matrix L is in the following form:

[0069]

[0070] in, Represents the ocean and land points in the i-th sample, and uses 0 and 1 to distinguish the land points and ocean points in the spatiotemporal sample matrix. Finally, perform matrix bitwise multiplication on the land matrix L and the spatiotemporal sample matrix X to obtain the SIT spatiotemporal matrix X after the land points are marked L :

[0071]

[0072] After the above operations, the invalid land points in the SIT spatiotemporal sample matrix are assigned a value of 0, while the valid points retain the SIT value, thereby ensuring that the land points and ocean points can be effectively distinguished by the characteristics of the input values during the neural network training process;

[0073] In addition, to ensure that the subsequent HSS-Transformer can converge quickly and thus improve the prediction effect and stability of the model, normalization and standardization operations are also performed. Normalization is to scale the data to a fixed range, usually the interval [0, 1]. Normalization helps to eliminate the differences in the range of different eigenvalues, so that the impact of features on the model has the same scale. The normalization formula is generally expressed as follows:

[0074]

[0075] Among them, X L,m×n is the unnormalized SIT space-time series, X L,min is the minimum value in the SIT space-time series, X L,max is the minimum value in the SIT space-time series, X L(norm) is the normalized SIT spatiotemporal series;

[0076] Standardization is the process of converting data into a distribution with zero mean and unit variance. Its main goal is to change the mean of the data to 0 and the variance to 1. Standardization is usually applicable to algorithms that require data to follow a standard normal distribution, especially for models based on gradient descent such as linear regression, neural networks, and Gaussian process models such as models that assume a Gaussian distribution. The formula for standardization is:

[0077]

[0078] Among them, X L,m×n is the unstandardized SIT time-space series, μ is the mean of the time-space series, σ is the standard deviation of the time-space series, X L(std) is the standardized SIT spatiotemporal series;

[0079] After the above data preprocessing operations, SIT spatiotemporal series data that can be used for neural network training and verification are finally obtained.

[0080] Step 3: Build the HSS-Transformer model, train it, and optimize the parameters to output the trained HSS-Transformer model.

[0081] Preferably, in step 3, the prediction unit of the HSS-Transformer used is a spatiotemporal Transformer model, and the external stacking design enables the model to effectively learn the multi-spatial scale feature information of SIT. The specific structure is shown in Figure 2 and Figure 3 As shown in the figure, the spatiotemporal Transformer model is the main program used for prediction, with the encoder part on the left and the decoder part on the right.

[0082] The encoder is an important component of the spatiotemporal Transformer model, responsible for encoding the input sequence, and the encoder layer is a component unit of the encoder. Each encoder layer extracts and transforms the input features. Specifically, the input sequence first passes through a multi-head attention sub-layer to capture the key dependencies within the input sequence. The input then passes through a normalization layer to standardize the data so that its feature values are within a reasonable range. Then there is a residual connection to add the original input to the result of the attention sub-layer to alleviate the gradient disappearance problem. Finally, a feedforward fully connected sub-layer is used to perform a nonlinear transformation on the features. The feedforward fully connected layer has a fully connected network with two linear layers. It first performs a linear transformation, then a ReLU nonlinearity, and then a second linear transformation. After the normalization layer and the residual connection, the output of the encoder is generated.

[0083] The decoder is another important component of the spatiotemporal Transformer model. It is mainly responsible for gradually generating the target sequence based on the encoder output. Very similar to the encoder, the decoder layer is the basic unit of the encoder. Each decoder layer contains three sublayers. Among them, the multi-head attention sublayer is used to capture the internal dependencies of the target sequence. The multi-head attention sublayer is used to focus on the encoder output and incorporate key source information into the generation of the target sequence. The feedforward fully connected sublayer is used for nonlinear transformation to extract higher-level feature representations.

[0084] HSS-Transformer stacks spatiotemporal Transformers, and performs spatiotemporal Transformer operations on the intermediate state spatiotemporal sequences after the convolution operation on the initial input SIT spatiotemporal sequence, and gradually incorporates the prediction values of each different spatial size in the process of generating the output; the advantage of this is that the output of the HSS-Transformer model covers the prediction results of the SIT spatiotemporal sequence at each scale before, so that the output contains multi-scale feature information;

[0085] This network not only retains the dynamic gate mechanism in the LSTM network to solve the long-term sequence dependency problem, but also replaces the vector dot product operation with the matrix Hadamard product operation in the loop calculation process, so that the network can simultaneously extract the spatiotemporal features of the sequence. Its specific structure is as follows Figure 2 as well as Figure 3 shown.

[0086] The following describes the specific design of the spatiotemporal Transformer architecture. The temporal multi-head attention mechanism is used to capture temporal correlations in the SIT spatiotemporal sequence and establish dependencies between different time points. By assigning different attention weights to each time point, time points that are more important to the context receive higher weights. This temporal attention mechanism is similar to the multi-head attention mechanism in the standard Transformer. The calculation formula for the temporal attention mechanism for each attention head is as follows:

[0087]

[0088] Where Q, K and V represent query, key and value tensors respectively, d k is the dimension of the query vector Q; Q represents the input query information and is the input tensor of the attention mechanism; K is used to measure the correlation between Q and other information; V represents the actual information related to each Q; after calculating the attention of each attention head, the results of multiple heads are spliced together and a linear transformation is performed to obtain the final temporal multi-head attention output:

[0089] MultiHead(Q,K,V)=Concat(head1,...,head p )W o

[0090] Where p represents the number of attention heads, W o is a learnable weight matrix; through the multi-head attention mechanism, the model can capture the details and global dependencies in the spatiotemporal sequence from multiple different angles; in the prediction of SIT spatiotemporal sequences, the spatiotemporal Transformer structure uses both temporal multi-head attention and temporal multi-head self-attention; when only the dependencies within the sequence are considered, that is, the attention mechanism becomes a self-attention mechanism; the self-attention mechanism is mainly used to capture the dependencies between different positions in the sequence without involving interactions with external sequences; in addition, the model also uses the absolute position encoding in the original Transformer to represent the position of each time point in the sequence, thereby assisting the model in understanding the relative positions of each time point in the time series.

[0091] Step 4: Input the spatiotemporal sequence of the sea ice thickness background field into the HSS-Transformer model to obtain the spatiotemporal sequence of the sea ice thickness forecast for the next 15 days;

[0092] Preferably, in step 4, the input is the SIT space-time sequence input I = R n×l×c×h×w , first reshape the input tensor to I = R (nhw)×l×c, where n represents the batch size, l represents the time step, c is the number of channels, h and w are the spatial dimensions; next, the standard Transformer is used to perform temporal attention calculation on the reshaped sequence, and the obtained spatiotemporal features will be reshaped into I'=R n×l×c×h×w , for use.

[0093] The spatial multi-head attention mechanism in the spatiotemporal Transformer is mainly used to capture the spatial correlation in the spatiotemporal sequence of SIT and establish the interdependence between different spatial locations. By assigning different spatial attention weights to each spatial location, regions with more frequent and representative SIT changes will receive higher weights. Unlike the temporal multi-head attention mechanism, the spatial attention mechanism needs to allocate attention in two dimensions. Therefore, we will adopt the following solution to solve this problem.

[0094] When dealing with the spatial multi-head attention mechanism, the key to the problem is how to generate position codes for spatial points, rather than directly applying 1D temporal absolute position codes. Because the spatial points are two-dimensional, for example, the distance between spatial positions (0,1) and (1,0) relative to (0,0) is 1, but when using 1D position coding, the coding values of the two are the same, and the two spatial points cannot be distinguished. Therefore, we need to generate a unique code for each spatial point, which can accurately represent the position of the point in the two-dimensional grid; referring to the position coding method in the original Transformer, the present invention uses sine and cosine functions to encode each coordinate (i.e., x and y coordinates) in the two-dimensional space respectively, and obtains two independent coding vectors, and splices these two vectors to form a unique spatial position code; this method ensures that the code of each spatial point is unique and can reflect its relative position in the two-dimensional space. After solving the problem of position coding, we can further deal with the dimensionality reduction problem of the spatial multi-head attention mechanism, and solve it by borrowing the method of the temporal multi-head attention mechanism. Considering that we only focus on the spatial dependencies within the spatiotemporal sequence, the spatial attention mechanism is actually a spatial self-attention mechanism, that is, the case where Q = K = V; assuming that the spatiotemporal feature sequence calculated by the temporal multi-head self-attention mechanism is I' = R n×l×c×h×w , the present invention first reshapes it into I'=R (nl)×(hw)×c , making its form similar to a time series; then, the present invention uses the same transformation matrix to perform matrix multiplication with I′ to obtain the query Q, key K and value $V$; then, the previously generated two-dimensional spatial position code is added to Q and K, and the spatial position information is embedded in the matrix for subsequent calculation of the spatial multi-head self-attention mechanism; next, the output feature sequence after the spatial multi-head attention mechanism operation is deformed to obtain the output, which is the spatiotemporal sequence of sea ice thickness prediction for the next 15 days I”=R n×l×c×h×w; During the calculation process of the spatial multi-head attention mechanism, different heads can focus on multiple important spatial regions and comprehensively capture the relationship between different scales and spatial points, so that the model can simultaneously capture detailed information and global spatial dependencies.

[0095] Step 5: Denormalize the obtained predicted spatiotemporal series and restore the land points; finally, obtain the forecast results of the sea ice thickness elements in the target sea area.

[0096] In step 5, the prediction results of HSS-Transformer are denormalized and the land points are restored to obtain the final SIT prediction value. The denormalization and denormalization operations are opposite to the normalization and standardization operations and are not described here.

[0097] This paper is based on the long-term sea ice thickness data of the target sea area, integrates multi-source ocean observation data such as air-based, shore-based, and sea-based, and uses the HSS-Transformer model to analyze and extract the spatiotemporal variation laws of sea ice thickness, thereby realizing long-term spatiotemporal joint modeling and prediction of sea ice thickness.

[0098] The present invention breaks away from the dependence of traditional numerical prediction methods on the initial atmospheric field and the limitation of forecast timeliness, and can deeply explore the spatiotemporal characteristics and nonlinear laws of sea ice thickness data, significantly improving the extended-term forecast capability of sea ice thickness. In addition, compared with traditional statistical models, HSS-Transformer can significantly enhance the ability to extract spatiotemporal characteristics and improve the accuracy of prediction results. The method of the present invention can effectively solve the spatiotemporal feature extraction problem of traditional numerical model methods in the extended-term sea ice thickness prediction, and provide high-precision sea ice thickness prediction for offshore platforms, underwater unmanned / manned vehicles, ships and other platforms. The technical process of the present invention is as follows: Figure 1 shown.

[0099] This paper proposes an intelligent prediction method for polar sea ice thickness based on a hierarchical stacked spatiotemporal Transformer (HSS-Transformer) model. This model primarily extracts and analyzes the spatiotemporal features of the polar sea ice thickness spatiotemporal series from the spatiotemporal Transformer model. It also leverages its internal temporal and spatial multi-head attention mechanisms to assign different influence weights to spatiotemporal features, thereby focusing on key features and achieving improved timeliness and accuracy for the SIT spatiotemporal series prediction task.

[0100] Compared with the existing technology, the present invention effectively improves the extended period prediction performance of sea ice thickness. The specific experimental appendix is as follows:

[0101] To examine the prediction performance of different networks, two classic sea ice thickness prediction networks—the EOF+LSTM, ConvLSTM, and HSS-Transformer models—were compared. To explore the predictive timeliness of the encoder-decoder architecture, the present invention used three prediction methods, each with input of 3, 5, 7, and 15 days of sea ice thickness data, to generate a 15-day forecast. The results of the parameter prediction performance evaluation, calculated using the MAE, RMSE, and SACC values obtained from the best predictions from each model, are shown in Table 1.

[0102] Table 1. Results for different networks and different forecast periods

[0103]

[0104] Table 1 shows that for inputs of 3, 5, 7, and 15 days, the MAE values of the three models are all less than 0.07, the RMSE values are all less than 0.5, and the SACC values are all greater than 0.85. Therefore, all three models have predictive capabilities for inputs of 3, 5, 7, and 15 days. The SACC values of the ConvLSTM and HSS-Transformer models are both greater than 0.9, indicating that they perform significantly better than the EOF-LSTM. It can also be seen that the more days of sea ice thickness data are input, the smaller the MAE and RMSE values of the EOF-LSTM prediction method, and the larger the SACC value. This indicates that the EOF-LSTM prediction method, because it is a time-based prediction method, does not suffer from overfitting. Therefore, the greater the number of days of sea ice thickness data input, the better the prediction results. For the ConvLSTM and HSS-Transformer prediction methods, when the input number of days is 3, 5, or 7, the MAE and RMSE decrease as the number of days increases. However, when the input is 15 days, the MAE and RMSE increase. The SACC increases with the input number of days, but decreases when the input is 15 days. This indicates that the correlation between the predicted and true values decreases, indicating overfitting. Therefore, it can be concluded that the prediction model performs best when the input number is 7 days.

[0105] Next, the three prediction methods were used to make 7-day inputs to forecast the next 3, 7, 15, and 20 days. The MAE, RMSE, and SACC evaluation parameter values calculated by selecting the forecast values with better results from each model are shown in Table 2.

[0106] Table 2. Results for different networks and different forecast periods

[0107]

[0108]

[0109] Table 2 shows that the SACC value of the EOF-LSTM forecast for the next 20 days using a 7-day input is less than 0.9, while the SACC values of the ConvLSTM and HSS-Transformer forecasting methods for the next 3, 7, 15, and 20 days are all greater than 0.9. This indicates that the ConvLSTM and HSS-Transformer forecasting methods can provide short-term, medium-term, and mid-to-long-term forecasts, while the EOF-LSTM can only provide short-term and medium-term forecasts, with a significant decline in its mid-to-long-term forecasting capabilities. It can also be seen that as the number of forecast days increases, the MAE and RMSE calculation results increase, while the SACC value gradually decreases, indicating that the forecast effectiveness gradually decreases with the increase in the number of forecast days. In addition, the RMAE value of the EOF-LSTM forecast method from 7 days to 15 days differs by only 0.0469, and the SACC value differs by 0.0096, while the RMSE value from 15 days to 20 days differs by 0.1349, and the SACC value differs by 0.0267; the RMAE value of the ConvLSTM forecast method from 7 days to 15 days differs by only 0.0117, and the SACC value differs by 0.0134, while the difference from 15 days to 20 days differs by 0.0558, and the SACC value differs by 0.0246; the RMAE value of the HSS-Transformer forecast method from 7 days to 15 days differs by only 0.0317, and the SACC value differs by 0.0124, while the difference from 15 days to 20 days differs by 0.0301, and the SACC value differs by 0.0112. In summary, it can be shown that the three prediction methods have gradually worsening prediction effects on sea ice thickness forecasts for 3, 7, 15, and 20 days out of a 7-day forecast, but the root mean square error and correlation for 3, 7, and 15-day forecasts change very little. When forecasting for 20 days, the effect drops rapidly. Therefore, it can be concluded that the three prediction methods have the best effect for 15 days out of a 7-day forecast, and are also the best from a cost perspective.

[0110] In order to intuitively represent the actual performance of the prediction results, for example, 15-day prediction values and true values in spring and autumn are selected to further evaluate the network, such as Figure 4 and Figure 5 As shown in the figure, HSS-Transformer performs well in predicting sea ice extension and sea ice thickness within the ocean area. Moreover, the performance gradually decreases with the increase of forecast time, and the 15-day forecast has the best performance.

[0111] from Figure 4In the spring forecast map, it can be seen that the HSS-Transformer prediction method predicts the sea ice contour more accurately, while the EOF-LSTM prediction method can only predict the approximate contour of the sea ice and show the contour trend of the sea ice. The ConvLSTM prediction method also has a certain degree of accuracy in the sea ice contour, but its effect is not as good as the HSS-Transformer prediction method. The EOF-LSTM prediction method has a wider bandwidth at the junction of sea ice and sea water. It can be seen that there are certain errors in the prediction of details of this method. Since the accuracy of the EOF-LSTM prediction method's own network is not high, the prediction effect of the sea ice thickness in the ice-free area cannot be maintained at 0, so it is displayed as having little ice. Comparing the prediction results of the HSS-Transformer prediction method on the first day with the actual sea ice thickness value, it can be seen that the error of this prediction method is extremely small, but the later the prediction day, the lower its prediction accuracy. The ConvLSTM prediction method also depicts the contour of the junction of sea ice and sea water, but the width of the light blue in the ice-free sea area is still relatively wide. From Figure 2 The autumn forecast map shows that the EOF-LSTM, ConvLSTM, and CNN-Tranformer methods all predict sea ice thickness in the selected areas with a certain degree of accuracy. The EOF-LSTM method shows the widest light blue area in the ice-free zone, while the ConvLSTM and HSS-Transformer methods have narrower light blue areas in the ice-free zone. Overall, the three prediction methods perform significantly better on the first day than on other days. However, on the 15th day, the sea ice thickness predictions of all three methods are less accurate, especially the EOF-LSTM method, which predicts a significantly larger area than the actual sea ice thickness. The 15th-day map of the actual sea ice thickness shows that the sea ice thickness data in the area between 15°W and 18°W and 72.5°N and 75°N are all less than 0.9, and the color is relatively light. However, all three prediction methods predict a partial red area in this area, indicating that the forecast results become less accurate with later days.

[0112] The foregoing is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained herein shall, unless otherwise specified or limited, be implemented in accordance with conventional means in the art.

Claims

1. An intelligent prediction method for polar sea ice thickness based on a multi-layered spatiotemporal Transformer, characterized by: The following steps are involved: Step 1: Based on the polar sea ice thickness reanalysis data of the target sea area, the characteristic matrix in the ocean SIT reanalysis data is extracted and a spatiotemporal series forecast supervision dataset is constructed, which includes the spatiotemporal series of the sea ice thickness background field for model input and the spatiotemporal series of the ocean thickness target field for model weight training and updating; Step 2: Preprocess the background and target field data, process the land points, and normalize and standardize the time series; Step 3: Build the HSS-Transformer model, train it, and optimize the parameters to output the trained HSS-Transformer model. Step 4: Input the spatiotemporal sequence of the sea ice thickness background field into the HSS-Transformer model to obtain the spatiotemporal sequence of the sea ice thickness forecast for the next 15 days; Step 5: Denormalize the predicted spatiotemporal sequence and restore the land points; Finally, the sea ice thickness element forecast results of the target sea area are obtained.

2. The method for intelligent prediction of polar sea ice thickness based on a multi-layered spatiotemporal Transformer according to claim 1 is characterized by: The data processing and analysis method in step 1 is as follows: Using the SIT reanalysis data of the target sea area, the temporal and spatial variation patterns and characteristic matrix of SIT are studied. The temporal and spatial sample matrix X of the polar region SIT for each day in the past years to be analyzed is as follows: Where N represents the number of spatiotemporal sample matrices, the dimension of the spatiotemporal sample matrix X is N×m×n, and m and n represent the number of latitude and longitude grid points in the target sea area, respectively. represents the SIT matrix of the i-th sample; In order to construct a spatiotemporal sequence forecast supervision dataset of marine elements, the marine spatiotemporal sample matrix is processed into a spatiotemporal sequence forecast supervision dataset of marine environmental elements. The specific processing process is as follows: For the spatiotemporal sample matrix of ocean elements In order to ensure that the neural network contains sufficient prior information input, the background field window length of the ocean environment elements is set to T in , the target field window length is T out , then the input background field X of the marine environment elements in and output target field X out The matrix form of is as follows: Where B is the number of samples, B = ST in -T out +1, input background field X in The dimension is B×T in ×m×n, output target field X out The dimension is B×T out ×m×n.

3. The method for intelligent prediction of polar sea ice thickness based on multi-layered spatiotemporal Transformers according to claim 1 is characterized by: In step 2, by extracting the land features in the spatiotemporal matrix and marking the land points in the spatiotemporal sample matrix using a numerical matrix, the extracted land matrix L is in the following form: in, Represents the ocean and land points in the i-th sample, and uses 0 and 1 to distinguish the land points and ocean points in the spatiotemporal sample matrix; finally, perform matrix bitwise multiplication on the land matrix L and the spatiotemporal sample matrix X to obtain the SIT spatiotemporal matrix X after the land points are marked L : After the above operations, invalid land points in the SIT spatiotemporal sample matrix are assigned a value of 0, while valid points retain the SIT value. Normalization is to scale the data to a fixed range. The normalization formula is as follows: Among them, X L,m×n is the unnormalized SIT space-time series, X L,min is the minimum value in the SIT space-time series, X L,max is the minimum value in the SIT space-time series, X L(norm) is the normalized SIT spatiotemporal series; The standardized formula is expressed as: Among them, X L,m×n is the unstandardized SIT time-space series, μ is the mean of the time-space series, σ is the standard deviation of the time-space series, X L(std) is the standardized SIT spatiotemporal series; After the above data preprocessing operations, SIT spatiotemporal series data that can be used for neural network training and verification are finally obtained.

4. The method for intelligent prediction of polar sea ice thickness based on multi-layered spatiotemporal Transformers according to claim 1 is characterized by: In step 3, the prediction unit of the HSS-Transformer used is the spatiotemporal Transformer model, and the external stacking design enables the model to learn the multi-spatial scale feature information of SIT. The spatiotemporal Transformer model is the main program for prediction, with the encoder part on the left and the decoder part on the right; The encoder is responsible for encoding the input sequence, and the encoder layer is the component unit of the encoder. Each encoder layer extracts and transforms the input features. Specifically, the input sequence first passes through a multi-head attention sub-layer to capture the key dependencies within the input sequence. The input then passes through a normalization layer to standardize the data so that its feature values are within a reasonable range. Then, there is a residual connection that adds the original input and the result of the attention sub-layer to alleviate the gradient vanishing problem. Finally, a feed-forward fully connected sub-layer performs a nonlinear transformation on the features. The feedforward fully connected layer is a fully connected network with two linear layers. It first performs a linear transformation, then a ReLU nonlinearity, and then a second linear transformation. After the normalization layer and the residual connection, the output of the encoder is generated. The decoder is primarily responsible for gradually generating the target sequence based on the encoder output. Similar to the encoder, the decoder layer is the basic unit of the encoder, and each decoder layer consists of three sublayers. The multi-head attention sublayer is used to capture dependencies within the target sequence, the multi-head attention sublayer focuses on the encoder output, and incorporates key source information into the generation of the target sequence. The feedforward fully connected sublayer is used for nonlinear transformation to extract higher-level feature representations. The HSS-Transformer stacks the spatiotemporal Transformers, and performs spatiotemporal Transformer operations on the intermediate state spatiotemporal sequences after the convolution operation on the initial input SIT spatiotemporal sequence, and gradually incorporates the predicted values of each different spatial size in the process of generating the output.

5. The method for intelligent prediction of polar sea ice thickness based on multi-layered spatiotemporal Transformers according to claim 4 is characterized by: In step 3, the temporal multi-head attention mechanism is mainly used to capture the temporal correlation in the SIT spatiotemporal sequence and establish the dependency relationship between different time points. The calculation formula of the temporal attention mechanism of each attention head is as follows: Where Q, K and V represent query, key and value tensors respectively, d k is the dimension of the query vector Q; Q represents the input query information and is the input tensor of the attention mechanism; K is used to measure the correlation between Q and other information; V represents the actual information related to each Q; after calculating the attention of each attention head, the results of multiple heads are spliced together and a linear transformation is performed to obtain the final temporal multi-head attention output: MultiHead(Q,K,V)=Concat(head1,...,head p )W o Where p represents the number of attention heads, W o is a learnable weight matrix; in the prediction of SIT spatiotemporal sequences, the spatiotemporal Transformer structure uses both temporal multi-head attention and temporal multi-head self-attention; when only the dependencies within the sequence are considered, that is, when Q = K = V, the attention mechanism becomes a self-attention mechanism; the self-attention mechanism is mainly used to capture the dependencies between different positions in the sequence without involving interactions with external sequences; in addition, the model also uses the absolute position encoding in the original Transformer to indicate the position of each time point in the sequence, thereby assisting the model in understanding the relative positions of each time point in the time series.

6. The method for intelligent prediction of polar sea ice thickness based on multi-layered spatiotemporal Transformers according to claim 5 is characterized by: In step 4, the input is the SIT space-time sequence input is I=R n×l×c×h×w , first reshape the input tensor to I = R (nhw)×l×c , where n represents the batch size, l represents the time step, c is the number of channels, h and w are the spatial dimensions; next, the standard Transformer is used to perform temporal attention calculation on the reshaped sequence, and the obtained spatiotemporal features will be reshaped into I'=R n×l×c×h×w , for use.

7. The method for intelligent prediction of polar sea ice thickness based on multi-layered spatiotemporal Transformers according to claim 6 is characterized by: When processing the spatial multi-head attention mechanism, a unique code is generated for each spatial point, which can accurately represent the position of the point in the two-dimensional grid; each coordinate of the two-dimensional space (i.e., x and y coordinates) is encoded using sine and cosine functions respectively to obtain two independent encoding vectors, and these two vectors are spliced to form a unique spatial position code; let the spatiotemporal feature sequence calculated by the temporal multi-head self-attention mechanism be I'=R n ×l×c×h×w , first reshape it into I'=R (nl)×(hw)×c , making it similar to a time series; then, use the same transformation matrix to perform matrix multiplication with I′ to obtain the query Q, key K and value $V$; then, add the previously generated two-dimensional spatial position code to Q and K, and embed the spatial position information into the matrix for subsequent calculation of the spatial multi-head self-attention mechanism; next, the output feature sequence after the spatial multi-head attention mechanism operation is deformed to obtain the output, which is the spatiotemporal sequence of sea ice thickness forecast for the next 15 days I”=R n×l×c×h×w .

8. The method for intelligent prediction of polar sea ice thickness based on multi-layered spatiotemporal Transformers according to claim 1 is characterized by: In step 5, the prediction result of HSS-Transformer is denormalized and the land points are restored to obtain the final SIT prediction value; Denormalization and destandardization operations are the opposite of normalization and standardization operations.

Images