A method for repairing buoy meteorological data based on a space-time dual attention neural network

Abstract

The application relates to the field of data quality control of marine data buoys, and discloses a kind of buoy meteorological data repair method based on space-time double attention neural network, comprising the following steps: data acquisition and pretreatment;Establish a space-time double attention neural network model: the model includes time coding link, space coding link and feature fusion layer;Model training, the time characteristics of containing the target element to be predicted are input into the time coding link, the meteorological elements are input into the space coding link, and the model output is the target element to be predicted at the moment;After model training, the model is evaluated using the test set;The missing values or abnormal values in the real-time collected multi-dimensional meteorological element data are predicted using the qualified model, and the data is repaired.The method disclosed by the application effectively combines time and space constraints, realizes the prediction and replacement of abnormal values and missing values in meteorological data, and thus realizes the repair of meteorological data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data quality control of ocean data buoys, and particularly relates to a buoy meteorological data repairing method based on a space-time dual attention neural network. BACKGROUND

[0002] Meteorological research needs to use long-term observation data in the research area. Through these complete data, local meteorological characteristics can be analyzed to form inferences. Among them, the ocean buoy is an important platform for obtaining meteorological and hydrological data, with characteristics of long-term, continuity and stability, and can continuously measure various ocean and atmospheric parameters such as sea surface temperature, wind speed, wind direction and air pressure. Real-time data provided by the buoy can be used for verification and improvement of numerical weather prediction models, and is crucial for studying the interaction between the ocean and the atmosphere.

[0003] Ocean buoy meteorological sensors often have missing data and abnormalities due to external environmental factors (such as damage to components / sensors caused by exposure to sunlight, strong winds, high waves, and high salt mist) and internal problems (such as missing calibration and aging of components). These problems distort data characteristics and change statistical quantities, making analysis more difficult, so it is necessary to identify and repair abnormal data to provide reliable data.

[0004] For statistical methods, mean interpolation, median interpolation, and mode interpolation are extremely simple, but they severely underestimate the variability and correlation between variables, and are obviously not suitable for meteorological data, which contains time series characteristics.

[0005] In deep learning, RNN and its derivative models GRU and LSTM are effective and widely used for time series data, but have defects: RNN has gradient explosion / vanishing problem, and improved RNN still cannot get rid of the gradient problem; GRU alleviates this problem, but does not explore the importance of information, making it difficult to learn long sequences; LSTM is better at processing long sequences, but has high computational cost, and it is difficult to capture long-term dependencies when processing extremely long sequences, and it is prone to overfitting (especially when training data is scarce).

[0006] The Transformer's global self-attention mechanism demonstrates unique advantages in capturing complex temporal dependencies and long-term trends in time series, making it possible to achieve dependencies over longer distances. Furthermore, meteorological elements are not independent but interconnected. This correlation structure can be represented in computers using graph structures, but these graph structures are not regular Euclidean spatial models and therefore cannot be processed using traditional CNNs. Generative Neural Networks (GNNs) are specifically designed for learning and predicting such non-Euclidean spatial data. GNNs are mainly divided into GCN, GAT, and GAE. However, GNN models generally have some limitations, such as weak generalization ability, high computational cost, and limited expressive power. In addition, in meteorological data applications, a single meteorological element is often only slightly influenced by multiple related elements. Relying solely on the correlation between elements for prediction is insufficient to fully interpret the complex patterns of meteorological element changes.

[0007] Furthermore, both Transformer and GAT share a common flaw in their core attention mechanisms: they tend to assign unnecessary attention to unrelated or weakly correlated connections or sequences. This redundant attention not only wastes computational resources but can also introduce noise into the model, reducing prediction accuracy. For example, in meteorological data, wave period and temperature are variables with extremely weak physical correlations. If the model assigns significant attention weights to these two variables, it may interfere with the model's ability to capture the true causal relationship, thus affecting the accuracy of the judgment. In summary, while existing methods can repair and imputate buoy meteorological data from either the temporal or spatial dimensions, meteorological data itself possesses heterogeneous characteristics in both temporal and spatial aspects. Relying solely on a single property for completion often leads to the loss of another type of feature information, failing to truly achieve deep fusion and collaborative prediction of spatiotemporal features. Moreover, accurately capturing the physical correlations between different meteorological elements and constructing interpretable, strongly correlated influence relationship models are also key to improving the quality of data imputation. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a buoy meteorological data repair method based on a spatiotemporal dual attention neural network. This method effectively combines temporal and spatial constraints, enabling the prediction and replacement of outliers and missing values ​​in meteorological data, thereby achieving the goal of meteorological data repair.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows:

[0010] A method for restoring meteorological data from buoys based on a spatiotemporal dual-attention neural network includes the following steps:

[0011] Step 1, Data Acquisition and Preprocessing:

[0012] Collecting historical multi-dimensional meteorological element data obtained by the ocean data buoy, dividing a training set and a test set, wherein the data in the test set is preprocessed to obtain a test set containing missing values and abnormal values;

[0013] Step 2, establishing a spatio-temporal double-attention neural network model:

[0014] The model comprises a time encoding link, a space encoding link and a feature fusion layer;

[0015] The time encoding link comprises a linear embedding layer, a position encoder and a multi-layer Transformer encoder, each layer of the Transformer encoder comprising a multi-head cross-attention mechanism layer, a residual link and a layer normalization layer and a feedforward neural network layer;

[0016] The space encoding link comprises an adaptive physical constraint matrix, a graph convolution operation layer and a feature processing layer;

[0017] The feature fusion layer outputs after fusing the features output by the time encoding link and the space encoding link;

[0018] Step 3, model training and evaluation:

[0019] The data in the training set is used to train the model, first, the data is divided into time features containing a target element to be predicted and meteorological elements, the time features containing the target element to be predicted are input into the time encoding link, the meteorological elements are input into the space encoding link, and the model outputs the target element at the time to be predicted; after the model is trained, the test set is used to evaluate the model, the predicted value of the model is compared with the true value, and a qualified model is obtained;

[0020] Step 4, data repair:

[0021] The missing values or abnormal values in the real-time collected multi-dimensional meteorological element data are predicted by using the qualified model, so as to realize the repair of the data.

[0022] In the above scheme, the time feature containing the target element to be predicted is input into the sequence First, the linear embedding layer is used to map the discrete input symbol to a dense vector representation, and then the position encoding is added to obtain the fused sequence ; the sequence enters the multi-head cross-attention mechanism layer, and the calculated attention score is combined with the sequence to perform a residual link and layer normalization, and output the layer normalization result x; then x enters the feedforward neural network layer, which performs nonlinear transformation and deep feature abstraction on each position vector containing global context information, and outputs ; then Residual connection and layer normalization are performed again with x, and finally the time sequence feature matrix of the target element is obtained and output.

[0023] In a further technical solution, the position encoding is an encoding of time information by a position encoder, and the position encoder generates a fixed position encoding vector using a cosine function:

[0024] ;

[0025] ;

[0026] wherein, represents the position encoding of the even dimension index, represents the position encoding of the odd dimension index, pos is the time step position, and i is the dimension index. represents the total dimension of the input, including the dimension of the position vector and the dimension of the word embedding vector.

[0027] In a further technical solution, the sequence After entering the multi-head cross attention mechanism layer, first will be multiplied by the attention weight matrix to obtain Q, K, and V, which represent the query, key, and value matrices, respectively; then the attention score is calculated:

[0028] ;

[0029] wherein, is the attention score, N is the length of the input data, is the dimension of ;

[0030] After obtaining the attention score , a residual connection and layer normalization are performed with the input sequence , and the calculation process is as follows:

[0031] ;

[0032] wherein, x is the layer normalization result, and LayerNorm is the layer normalization function.

[0033] In a further technical solution, after x enters the feedforward neural network layer, the calculation formula is as follows:

[0034] ;

[0035] wherein, represents the abstraction of the features calculated by the feedforward fully connected layer after nonlinear transformation, is the output of the feedforward neural network layer. Function; , is a weight matrix of the feedforward neural network, , is a bias of the feedforward neural network;

[0036] Subsequently, with x again once a residual link and layer normalization, finally get the time series feature matrix of the target element and output, the calculation process is as follows:

[0037] ;

[0038] wherein, indicates a layer normalization function.

[0039] In the above scheme, the meteorological element input sequence After inputting the spatial encoding link, the adaptive physical constraint matrix is performed graph convolution operation, and the graph convolution calculation formula is as follows:

[0040] ;

[0041] wherein, indicates the meteorological element obtained after graph convolution calculation;

[0042] Then enter the feature processing layer, in the feature processing layer, the meteorological element is projected to a high bit space through a ReLU function for nonlinear transformation, and finally the spatial feature of the target element is obtained and output, the calculation process is as follows:

[0043] ;

[0044] wherein, , is a weight matrix of the spatial encoding link, , is a bias vector of the spatial encoding link.

[0045] In the further technical scheme, the adaptive physical constraint matrix is obtained after adaptive adjustment in the model training process, and the calculation formula is as follows:

[0046] ;

[0047] wherein, indicates a Sigmoid function, indicates a Hadamard product; is a physical prior matrix, which is a n*n Boolean matrix, and the row and column correspond to the meteorological element set wherein n is the number of meteorological elements; the row index in the matrix corresponds to the target element, i.e. the affected element, and the column index corresponds to the original meteorological element, i.e. the influencing element; for the row where the target element is located, the weight value of the column position corresponding to the original meteorological element influencing the target element is set to 0 or 1: wherein 0 indicates that the original meteorological element and the target element are irrelevant elements, and 1 indicates that the original meteorological element and the target element are relevant elements; is a physical learning matrix, which is a learnable parameter matrix, and the initial value of the matrix is the same as , the weight value in the matrix will be automatically adjusted according to the gradient descent process during training, and the range is kept in [0, 2].

[0048] In the above scheme, in the feature fusion layer, the output fusion feature is calculated as follows:

[0049] ;

[0050] wherein is the spatial feature output by the spatial encoding link, is the temporal feature output by the temporal encoding link, represents Hadamard product, is a gating vector:

[0051] ;

[0052] wherein represents Sigmoid function, , is the weight matrix of the feature fusion layer.

[0053] In the above scheme, in step 4, for the consecutive missing values, a bidirectional prediction method is used in the prediction stage, i.e. the input sequence is first predicted in the forward time direction, and then predicted in the reverse time direction, and the output value of the prediction is:

[0054] ;

[0055] wherein is the prediction output value, is the forward output fusion feature, is the output fusion feature, is the position weight.

[0056] In the further technical scheme, in step 4, according to the relative position of the missing value to the prediction direction, a dynamic weight is used for prediction, and the weight of the position closer to the prediction direction is higher, and the weight range is limited in [0.1, 0.9]; the function expression is:

[0057] ;

[0058] wherein, is the relative position of the consecutive missing values for this place, is the consecutive length of the missing values.

[0059] Through the technical scheme, the buoy meteorological data repairing method based on the space-time double attention neural network has the following beneficial effects:

[0060] 1. The space-time double attention neural network model combines the Transformer model and the GAT graph attention network with improved attention mechanism, constructs a space-time double link reasoning structure, and realizes time sequence and spatial feature separation extraction and fusion prediction.

[0061] 2. In the time sequence link, the time feature is first extracted and combined with the target meteorological element to form a time sequence input to avoid information leakage and pollution of attention calculation; then, the Transformer encoder structure is used to model the sequence to effectively capture the long-range dependence relationship.

[0062] 3. In the space link, an adaptive physical constraint matrix GAT model is innovatively designed to capture features between features, i.e., spatial dependence, the adaptive physical constraint matrix is used to replace the original GAT attention mechanism, which effectively suppresses the link between irrelevant features, strengthens the correlation between related features, and dynamically adjusts the weight relationship during training, thereby reducing the calculation amount and improving the prediction accuracy.

[0063] 4. In order to effectively handle the continuous abnormal values and missing values in the data, a bidirectional prediction method is adopted according to the error accumulation rule and the time sequence data characteristics. In order to ensure the rationality of the prediction result, a dynamic weight mechanism is introduced according to the position of the missing data relative to the prediction direction: the closer the position along the prediction direction, the higher the weight, and the weight range is limited to [0.1, 0.9]. Experimental results show that this method significantly improves the prediction accuracy.

[0064] Overall, the model effectively combines time and space constraints, realizes prediction and replacement of abnormal values and missing values in meteorological data, and realizes repair of meteorological data. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows.

[0066] Figure 1 A space-time double attention neural network ST-DAN structure schematic diagram disclosed by the embodiments of the present application;

[0067] Figure 2 Structure diagram for time encoding link;

[0068] Figure 3 Flow chart for attention mechanism calculation;

[0069] Figure 4 Structure diagram for adaptive physical mask matrix;

[0070] Figure 5 Repair effect diagram of air temperature data, (a) is Transformer, (b) is ST-DAN;

[0071] Figure 6 Figure 5 Enlarged contrast diagram in area A, (a) is Transformer, (b) is ST-DAN;

[0072] Figure 7 Figure 5 Enlarged contrast diagram in area B, (a) is Transformer, (b) is ST-DAN;

[0073] Figure 8 Repair effect diagram of wind speed data, (a) is Transformer, (b) is ST-DAN;

[0074] Figure 9 Figure 8 Enlarged contrast diagram in area A, (a) is Transformer, (b) is ST-DAN;

[0075] Figure 10 Figure 8 Enlarged contrast diagram in area B, (a) is Transformer, (b) is ST-DAN. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

[0077] The present application provides a kind of based on space-time double attention neural network's buoy meteorological data repair method, comprising the following steps:

[0078] Step 1, data acquisition and pretreatment:

[0079] ​​​​The historical multi-dimensional meteorological element data collected by the ocean data buoy includes but is not limited to pressure, temperature, wave period, wind speed, humidity and other multi-dimensional meteorological elements. The collected data has good time continuity, and the selected meteorological variables can be flexibly adjusted according to the actual available data types and contents. In this embodiment, the data set uses the complete data collected by the ocean data buoy located in the Qingdao wheat island region for experimental verification. The data set has a time resolution of ten minutes, contains five meteorological elements of temperature, wind speed, pressure, wave period and humidity, and divides the training set: test set into a training set and a test set-true value according to the ratio of 8:2.

[0080] The data in the test set is preprocessed to obtain a test set containing missing values and abnormal values.

[0081] The target element data is detected, abnormal value points and missing value points are found out by gradient detection, peak detection and continuity detection methods, and are marked. To verify the effectiveness of the model, this embodiment generates a test set Y` by inserting abnormal values into Y, and then repairs Y` through the model. The repair result is compared with Y to evaluate the model. First, replace 15% of the data in the test set-true value Y with randomly distributed missing values with a maximum continuous value of 4 to simulate the missing values in the actual collected data. Then, mark these data to obtain . Then, artificial noise is inserted, and 5% of the data is converted into abnormal values. For this purpose, an adaptive Gaussian noise injection method based on stratified sampling is designed. The noise injection process includes the following three steps:

[0082] First, randomly sort the candidate set , where V is the non-empty value point in the data , and then divide it into H layers, denoted as . The number of abnormal values m allocated to each layer is calculated by the following formula.

[0083] ;

[0084] where is the total number of noise points to be inserted. For the remainder , the first R layers are allocated one more sample to ensure that the sample allocation is as balanced as possible. After determining the number of abnormal values in each layer, m original data points are independently extracted from each layer of data.

[0085] Second, the generation of adaptive Gaussian noise. Noise intensity is crucial to the prediction effect of the model. The present application generates Gaussian noise adaptively according to the standard deviation of the original data. The noise generation function is wherein, is the standard deviation of the original data, by scaling the standard deviation by 0.5 times, the noise intensity is controlled within a reasonable range, and it is ensured that the noise can introduce data changes and will not excessively destroy the characteristics of the original data.

[0086] The third step is to inject noise, and the generated noise is injected to obtain the data points after noise injection , and the calculation formula is:

[0087] ;

[0088] The process accurately adds noise to the selected data points through index matching, and uniformly injects a specific proportion of noise in the time dimension, which can better simulate the real situation while preserving the distribution characteristics of the original data. After the above steps, the test set Y` containing 15% missing values and 5% abnormal values is obtained.

[0089] Step 2, establish a spatio-temporal dual attention neural network model:

[0090] The spatio-temporal dual attention neural network (ST-DAN) combining the Transformer model and the Graph Attention Network (GAT) deep learning model proposed by the application is used to realize high-precision time series data repair. The model has a time and space dual-link structure, the time link uses the Transformer encoder layer to capture the time series characteristics of a single element, and the space link captures the correlation between elements through GAT, and the algorithm is optimized and improved, a new physical adjacency matrix is introduced, and the weights of the influence between elements are dynamically adjusted to obtain a more realistic prediction result.

[0091] The overall architecture of ST-DAN is shown in Figure 1 , which is composed of a time encoding link, a space encoding link and a feature fusion layer. The original input data is calculated in parallel through the time encoding link and the space encoding link, and then the feature fusion is performed to finally output the prediction data. Before data input, the z-score method is used to standardize the data, and the input features are converted to a distribution with a mean of 0 and a standard deviation of 1, to improve the stability of model training. The input data format is m-dimensional time elements, such as year, month, day, hour, minute, etc.; and n-dimensional meteorological elements, such as temperature, pressure, wind speed, wave period, etc. Then the data will be split into m-dimensional time features plus 1-dimensional target meteorological elements, which enter the time encoding link; and all n-dimensional meteorological elements enter the space encoding link.

[0092] 1. Time encoding link

[0093] As shown in Figure 2 , the time encoding link includes a linear embedding layer, a position encoder, and a multi-layer Transformer encoder, each layer of the Transformer encoder including a multi-head cross-attention mechanism layer, a residual link, and a layer normalization layer, and a feedforward neural network layer.

[0094] The self-attention mechanism included in the conventional Transformer encoder can capture long-distance time sequence correlations in data and can more effectively fit the time evolution trend of the data, thereby obtaining time features. However, this mechanism performs global calculation on the attention scores between all elements in the input sequence, which may not only produce a huge amount of calculation when processing complex data, but also may produce attention scores that do not conform to objective reality. To this end, the present application first imposes strict constraints on the input of the Transformer encoder, so that it only receives time elements and target elements, thereby avoiding redundant associations between irrelevant elements, reducing computational complexity, and improving training efficiency and prediction accuracy. At the same time, the use of a multi-layer Transformer encoder further enhances the expression ability of the model through a hierarchical feature extraction mechanism: the bottom layer encoder can focus on capturing local time patterns and fine-grained features, and the high layer encoder can realize deep integration of global context based on low layer features, forming a progressive feature abstraction process from specific to abstract. This multi-layer stacking structure not only allows the model to capture multi-scale time sequence dependencies - both preserving local dynamics within short periods and depicting associated patterns in long periods, but also alleviates the gradient vanishing problem in deep networks through residual connections and layer normalization mechanisms, allowing the model to be stable when stacking more encoding layers and continuously improving feature representation capabilities, ultimately providing stronger fitting ability and generalization performance for modeling complex time series data.

[0095] The time encoder first encodes the observation time through a position encoding function, then establishes a multi-layer Transformer encoder, and uses its multi-head attention mechanism to learn and train m+1-dimensional data, fully capturing the change characteristics of the target element in terms of time regularity, such as diurnal and seasonal changes in temperature. Inputting only m-dimensional time elements and 1-dimensional target elements ensures that only the features of the target element are captured and learned during the model learning process, avoiding learning pollution caused by irrelevant links and reducing learning accuracy. After learning and training through the multi-layer Transformer encoder, the time encoding link outputs time features.

[0096] Input sequence First, the discrete input symbols are mapped to dense vector representations through a linear embedding layer, then added with position encoding. Position encoding is the encoding of time information by the position encoder. Since the Transformer model only focuses on the content similarity between the elements of interest, it lacks the ability to perceive the order of the elements, so position encoding is essential. The position encoder generates fixed position encoding vectors using the sine / cosine function:

[0097] ;

[0098] ;

[0099] where, represents the position encoding of even dimension index, represents the position encoding of odd dimension index, pos is the time step position, i is the dimension index; represents the total dimension of the input, including the dimension of the position vector and the dimension of the word embedding vector.

[0100] The input sequence after position encoding fusion into the multi-head cross attention mechanism layer, its structure and calculation process are shown in Figure 3 . First, will be multiplied by the attention weight matrix respectively, to get Q, K, V, which represent the query, key, and value matrices, respectively, then start calculating the attention score, the attention score calculation method is:

[0101] ;

[0102] where, is the attention score, N is the length of the input data, is the dimension of the input data , is the dimension of . Taking this experiment as an example, contains day, hour and temperature, then is 3, at this time is an N*3 matrix, and is set to 256, that is, is 256, shape is , that is, 3*256. After and matrix multiplication, Q, K, V matrices with shape N*256 are obtained.

[0103] After obtaining the attention score , it will be added with the input sequence The calculation process for performing a residual link and layer normalization is as follows:

[0104] ;

[0105] Where x is the layer normalization result, and LayerNorm is the layer normalization function.

[0106] Then, x is fed into a feedforward network (FFN). The feedforward network independently acts on the representation vector at each position in the sequence output by the self-attention layer. Its main function is to perform nonlinear transformations and deep feature abstraction on each position vector that already contains global context information. Its structure adopts an expansion-nonlinearity-contraction design. First, the input vector x is passed through a weight matrix... and bias Linear projection to a higher dimension d ff , ,here We set the value to 256. Then, we apply a non-linear activation function to the dimensionality-upgraded result; in this embodiment, we use the ReLU function. Finally, we pass the activated high-dimensional vector through a weight matrix. and bias Linear projection back to the original Dimension, its mathematical formula is:

[0107] ;

[0108] in, An abstraction representing the characteristics calculated after nonlinear transformation of the feedforward fully connected layer. , Here is the weight matrix of the feedforward neural network. , The bias of the feedforward neural network is used; the parameters (W1, W2, b1, b2) of the FFN are shared across all sequence positions within the same layer, but the computation is position-independent.

[0109] This layer significantly enhances the model's representational capabilities by introducing powerful non-linear activations and high-dimensional mappings, and is a key component in the model's learning of complex features and contribution to the majority of parameters. FFN complements the self-attention mechanism, which focuses on learning inter-location dependencies, together forming the powerful feature learning capability of the Transformer encoder layer.

[0110] Subsequently, The matrix will be re-linked with x using residual linking and layer normalization to obtain the temporal feature matrix of the target element. The calculation process is as follows: (The output is then presented.)

[0111] ;

[0112] wherein, denotes a layer normalization function.

[0113] 2. Spatial encoding link

[0114] The spatial encoding link is composed of a physics-aware GAT (Physics Aware GAT) designed by us as a spatial encoder, as shown in the following formula (1) : Figure 3 As shown in the formula (1), the spatial encoding link includes an adaptive physics constraint matrix, a graph convolution operation layer and a feature processing layer.

[0115] Input sequence of the spatial encoding link After entering the spatial encoding path, an adaptive physics constraint matrix is first obtained. The attention mechanism in the traditional graph attention network relies on data-driven implicit learning relationship, which can automatically capture the relationship between features and dynamically adjust the weight. It is extremely convenient and effective for data that cannot directly know the correlation between features, but there may be false correlation captured and lack of physical interpretation in the learning process. For meteorological data, we can directly know the relationship between features, such as temperature, wind speed and pressure, which are related to precipitation. The adaptive physics mask matrix mechanism proposed in the present application replaces the traditional attention mechanism, which can forcibly prohibit the connection that violates the domain knowledge through the predefined physics constraint matrix, so that the elements that exist in the correlation can be correctly connected, thereby enhancing the interpretability of the model, avoiding unnecessary calculation, and improving the model precision and training efficiency.

[0116] As shown in the following formula (2) : Figure 4 First, a physical prior matrix is preset, which is an n*n boolean matrix, and the row and column correspond to the meteorological element set , where n is the number of meteorological elements; the row index of the matrix corresponds to the target element, i.e. the affected element, and the column index corresponds to the original meteorological element, i.e. the influencing element; for the row where the target element is located, the weight value of the column position corresponding to the original meteorological element that exerts influence is set to 0 or 1: where 0 indicates that the original meteorological element and the target element are irrelevant elements, and 1 indicates that the original meteorological element and the target element are related elements; taking the of the present embodiment as an example, five meteorological features correspond to temperature, wind speed, pressure, wave period and humidity respectively, if the temperature is to be predicted, the weight value of the first row can be set to , if the wind speed is to be predicted, the weight value of the second row can be set to , so that the association between irrelevant elements can be ensured.

[0117] However, only the boolean matrix Since the influence weights between meteorological elements cannot be quantified, this invention designs an adaptive physical constraint matrix module, in which... It will be copied and labeled as a physics learning matrix. This matrix is ​​a learnable parameter matrix, and its initial values ​​are... Similarly, the weights within this matrix are automatically adjusted continuously during the gradient descent process, remaining within the range of [0, 2]. After gradient descent update... Will be with Boolean matrix Take the Hadamardi product during training. It is fixed and unchanging, thus effectively ensuring the matrix during training. Positions with a weight of 0 within the matrix are always set to 0, effectively prohibiting links between irrelevant elements. After obtaining the result, it undergoes Sigmoid regularization to control the range within [0,1]. Then, it is multiplied by 2 to restore the weight range to [0,2]. This ensures that the weights in the matrix do not exceed the limit due to cyclic accumulation during the update process, and by maintaining the weights within [0,2], the range is widened, increasing the gradient for weight optimization and making the changes smoother. After this calculation, the updated matrix is ​​finally obtained. As the output of the adaptive physical constraint matrix, the calculation formula is as follows:

[0118] ;

[0119] in, This represents the Sigmoid function. It represents the Hadamardi (or Hadama) stack.

[0120] This process organically combines physical priors with data-driven learning, preserving the physical constraints of feature relationships while allowing the model to adaptively adjust the influence weights. In actual calculations, the weights range from [0, 2], where 0 represents no correlation and 2 represents strong correlation. This is achieved by obtaining the adaptive physical constraint matrix. back, As a mask and input sequence The graph convolution operation will then begin. The formula for graph convolution is as follows:

[0121] ;

[0122] in, The meteorological elements obtained after graph convolution calculation are not yet the required spatial features and need to be further transformed through the feature processing layer.

[0123] In the feature processing layer, the resulting meteorological features The target element is projected to a high bit space through a ReLU function for nonlinear transformation, and finally the spatial features of the target element are obtained and output, and the calculation process is as follows:

[0124] ;

[0125] wherein, , is a weight matrix of the spatial encoding link, , is a bias vector of the spatial encoding link.

[0126] The spatial encoding link also learns and trains the input n-dimensional meteorological elements while the time encoding link learns and trains, wherein the n-dimensional meteorological elements already contain the target element, and the GAT neural network is used to focus on the feature association between meteorological elements. The GAT here is not a traditional graph attention neural network, but a GAT neural network optimized by using the learning physical constraint matrix proposed in the application instead of the attention mechanism. The matrix can strengthen the association between related meteorological elements and prohibit the connection between irrelevant variables, for example, when the temperature data is the repair object, the related variables are air pressure and wind speed, and the irrelevant variables are wave periods, thereby avoiding the poor prediction effect caused by the traditional attention mechanism capturing the association between irrelevant elements. At the same time, the parameters in the matrix are not fixed once set, but will change with the training process, and the changed matrix is continuously taken Hadamard product with the original set matrix to ensure that the parameters of the related elements are continuously optimized, and the parameters between the irrelevant elements are always 0. After learning and training through the spatial link, the spatial features are output.

[0127] 3, feature fusion layer

[0128] The feature fusion layer outputs the features output by the time encoding link and the spatial encoding link after fusion.

[0129] Subsequently, after obtaining the time features and the space features in the feature fusion module, a decoupling-fusion strategy is used to dynamically adjust the space-time weight by using linear transformation and sigmoid generation gating mechanism. The design follows: physical consistency, which ensures that the spatial features carry clear meteorological physical relationship; time sequence dynamics, which captures the periodicity and trend mode in the time features; adaptive interaction, which dynamically adjusts the contribution degree of the space-time features through the gating mechanism. Finally, the fusion features are generated, and the final prediction result is output based on the features.

[0130] Specifically, the spatial encoding link outputs a feature matrix through the improved graph attention network; the time encoding link generates a time sequence feature matrix through the Transformer encoder after splicing the target meteorological features and the time features Both are spliced in the hidden layer dimension, and adaptive weight allocation is realized through the gated fusion network: first, linear calculation and Sigmoid activation are used to generate a gating vector:

[0131] ;

[0132] Wherein, , is the learnable weight matrix of the linear fusion layer, and then:

[0133] ;

[0134] realize feature fusion, output fusion features The design constrains the physical consistency of spatial features through a physical mask matrix, and dynamically adjusts the contribution of space-time features using a gating mechanism.

[0135] In the prediction output stage, the sliding window method is used for prediction output, and experimental tests show that the sliding window parameters set by the model are the data amount of one day, if the minimum time unit of data collection is hour, then set to 24, if the minimum time unit of data collection is ten minutes, then the input window size is set to 144, and so on, the output window size is 1, and the step is 1. That is, the sliding window slides one unit of data each time, and one day of data is used to predict one unit of data. This method allows the model to capture the mapping relationship from the past one day to the future one time step.

[0136] Step 3, model training and evaluation:

[0137] The data in the training set is used to train the model, first, the data is divided into time characteristics containing the target elements to be predicted and meteorological elements, the time characteristics containing the target elements to be predicted are input into the time encoding link, and the meteorological elements are input into the space encoding link. The model output is the target element at the time to be predicted.

[0138] The data set used for training does not contain abnormal values and missing values, and all data is known to the model. The model will learn the trend of the target meteorological element in the data through unsupervised learning. After training is completed, the model will be packaged and saved for prediction and repair tasks of the data to be repaired. After the model is trained, the packaged model is used to start predicting the test set Y`, and the data marked as abnormal in Y` is predicted and replaced in the prediction output stage.

[0139] To effectively model time dependence and feature interaction, sliding window technology is used to convert the original sequence into an input-output pair set. Given a time series of length , d is the feature dimension, and the training samples are generated by sliding windows where the input sequence contains L time step features, and the target value is the target variable at the next time step. This method allows the model to learn the mapping relationship from the past L time steps to the future P time steps. In the data set collected in this embodiment, since the time resolution of the data is 1 hour, L = 24 and P = 1 are set, that is, when data labeled as an abnormal value or a missing value is identified, the data at this point is replaced with the data 24 hours before the data point to achieve data repair.

[0140] During the experiment, the set hyperparameters include model structure parameters and training optimization parameters. In the model structure parameters, the size of the hidden layer is set to 256, the number of attention heads in the multi-head cross-attention mechanism is 8, and the random inactivation rate is set to 0.1. The model structure parameters control the complexity of the model, and when the prediction result is overfitting or underfitting, the model structure parameters can be adjusted. In the training optimization parameters, the learning rate is set to 1e-4, the batch size is set to 64, and the gradient clipping size is set to 1.0. The training optimization parameters control the convergence speed and training speed of the model, and when the model training convergence process appears large amplitude oscillation or training speed is too slow, the training optimization parameters can be adjusted.

[0141] After the model is trained, the test set is used to evaluate the model, and the predicted value of the model is compared with the true value to obtain the qualified model.

[0142] The loss function uses the mean absolute error (MAE) as the training target, which is calculated as follows:

[0143] ;

[0144] where M is the number of samples, is the data in the test set - true value Y, is the predicted value. MAE has physical abnormal robustness, and when facing extreme values in meteorological data, MAE is less affected, which meets the fault tolerance demand of meteorological prediction on abnormal points. And MAE has the advantage of interpretability, the physical unit of MAE is consistent with the prediction target, which is convenient for direct evaluation of model performance.

[0145] The gradient clipping mechanism is introduced in the training process to prevent gradient explosion, and the gradient clipping is calculated as shown in the following formula:

[0146] ;

[0147] where θ is the gradient threshold, which is preset to 1, G is the current gradient, is the L2 norm of G, The updated gradient. During the training process, if the current gradient reaches the threshold value θ, the gradient clipping is triggered, and then the updated gradient is used to continue the gradient descent for prediction.

[0148] In addition to the MAE loss function, the mean square error MSE, the root mean square error RMSE, and the coefficient of determination R2 are also used as evaluation indicators, and the calculation formulas are as follows:

[0149] ;

[0150] ;

[0151] ;

[0152] The four data can reflect the difference between the predicted data and the true value, and the four data are usually between 0 and 1, among which MAE, MSE and RMSE are more effective as they approach 0, and R 2 is more effective as it approaches 1.

[0153] Step 4, data repair:

[0154] Use the qualified model to predict the missing values or abnormal values in the real-time collected multi-dimensional meteorological element data, and realize the repair of the data.

[0155] For consecutive missing values, a bidirectional prediction method is used in the prediction stage, that is, the input sequence is first predicted in the forward time direction, and then predicted in the reverse time direction, and the output value of the prediction is:

[0156] ;

[0157] wherein, is the predicted output value, is the fusion feature of the forward output, is the fusion feature of the output, is the position weight.

[0158] According to the relative position of the missing value to the prediction direction, dynamic weight is used for prediction, and the weight of the position closer to the prediction direction is higher, and the weight range is limited to [0.1, 0.9]; The function expression is:

[0159] ;

[0160] wherein, is the relative position of the missing value to the consecutive missing value at this position, is the consecutive length of the missing value.

[0161] Experimental verification

[0162] 1、Experimental data source: The experimental test is performed by selecting the measured data of the ocean data buoy arranged with wheat islands, five meteorological elements of air temperature, wind speed, pressure, wave period and humidity are selected, and the data set is divided for training and testing.

[0163] 2、Model parameter setting: The measured data of air temperature and wind speed are predicted and repaired in this embodiment, so different parameter settings are made according to different target elements, especially the physical constraint matrix part, the wind speed, pressure and humidity are selected as the related elements for air temperature, and the air temperature and wave period are selected as the related elements for wind speed.

[0164] 3、Comparison of experimental results: The existing Transformer is selected for result comparison, the repair effect of air temperature data is as shown in Figure 5 , Figure 5 The horizontal coordinate of each point is 00:00 of each date, the green area and the red area in the figure are two parts with obvious differences, and the local enlarged detail comparison is shown in Figure 6 and Figure 7 , Figure 6 The time of each point in the horizontal coordinate is each time of December 22, Figure 7 The time of each point in the horizontal coordinate is each time of December 28. The repair effect of wind speed data is as shown in Figure 8 , Figure 8 The horizontal coordinate of each point is 00:00 of each date, the green area and the red area in the figure are also two parts with obvious differences, and the local enlarged comparison chart is as shown in Figure 9 and Figure 10 , Figure 9 The time of each point in the horizontal coordinate is each time of March 22, Figure 10 The time of each point in the horizontal coordinate is each time of March 23. The experimental results show that the model proposed in the present application has good prediction accuracy and stronger robustness for meteorological data repair.

[0165] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for restoring buoy meteorological data based on a spatiotemporal dual-attention neural network, characterized in that, Includes the following steps: Step 1, Data Acquisition and Preprocessing: Historical multi-dimensional meteorological data obtained from ocean data buoys were collected and divided into training and test sets. The test set data was preprocessed to obtain a test set containing missing values ​​and outliers. Step 2, establish a spatiotemporal dual attention neural network model: The model includes a temporal coding link, a spatial coding link, and a feature fusion layer; The time-encoding link includes a linear embedding layer, a position encoder, and a multi-layer Transformer encoder. Each Transformer encoder includes a multi-head cross-attention mechanism layer, residual links, a layer normalization layer, and a feedforward neural network layer. The spatial coding link includes an adaptive physical constraint matrix, a graph convolution operation layer, and a feature processing layer; The feature fusion layer fuses the features output from the temporal coding link and the spatial coding link and then outputs them. Step 3, Model Training and Evaluation: The model is trained using data from the training set. First, the data is divided into temporal features and meteorological features containing the target elements to be predicted. The temporal features containing the target elements to be predicted are input into the temporal encoding link, and the meteorological features are input into the spatial encoding link. The model output is the target elements at the time to be predicted. After the model is trained, the model is evaluated using the test set. The model's predicted values ​​are compared with the true values ​​to obtain a qualified model. Step 4, Data Repair: By using qualified models, missing or outlier values ​​in real-time collected multi-dimensional meteorological element data can be predicted to achieve data repair. Meteorological element input sequence After inputting the spatial coding link, it is combined with the adaptive physical constraint matrix. The graph convolution operation is performed, and the formula for calculating graph convolution is as follows: ； in, The meteorological elements are obtained after the graph is convolved. Then it enters the feature processing layer, where meteorological elements... After a nonlinear transformation using the ReLU function and projection onto a higher-order space, the spatial features of the target elements are finally obtained. The calculation process is as follows: ； in, , This is the weight matrix of the spatial coding link. , This is the bias vector for the spatial coding link; The adaptive physical constraint matrix The formula for calculating this result is as follows: The model is adaptively adjusted during training. ； in, This represents the Sigmoid function. It represents the Hadamardi (or Hadama) stack; It is a physical prior matrix, which is an n*n Boolean matrix where each row and column corresponds to a set of meteorological elements. , where n is the number of meteorological elements; the row index in this matrix corresponds to the target element, i.e. the affected element, and the column index corresponds to the original meteorological element, i.e. the element that exerts the influence; for the row where the target element is located, the weight of the column position corresponding to the original meteorological element that exerts the influence is set to 0 or 1: where 0 indicates that the original meteorological element and the target element are unrelated elements, and 1 indicates that the original meteorological element and the target element are related elements. This is the physics learning matrix, which is a learnable parameter matrix with initial values ​​equal to... Similarly, the weights within this matrix will be automatically adjusted continuously according to the gradient descent process during training, and the range will remain within [0,2]. In the feature fusion layer, the output fused features The calculation is as follows: ； in, The spatial features output by the spatial coding link. The time characteristics output by the time-coded link. It represents the Hadamah accumulation. To form a gated vector: ； in, This represents the Sigmoid function. , This is the weight matrix of the feature fusion layer.

2. The buoy meteorological data repair method based on spatiotemporal dual attention neural network according to claim 1, characterized in that, Input sequence containing time features of the target element to be predicted First, a linear embedding layer maps discrete input symbols into a dense vector representation, which is then added to positional encoding to obtain the fused sequence. ;sequence Entering the multi-head cross-attention mechanism layer, the calculated attention score is compared with the sequence. Perform a residual linking and layer normalization, and output the layer normalization result x. Then x enters the feedforward neural network layer, where it performs nonlinear transformations and deep feature abstraction on each position vector that already contains global context information, and outputs... ; then Perform a residual link and layer normalization again with x to finally obtain the temporal feature matrix of the target element. And output it.

3. The buoy meteorological data repair method based on spatiotemporal dual attention neural network according to claim 2, characterized in that, The position encoding is the encoding of time information by the position encoder. The position encoder uses a sine / cosine function to generate a fixed position encoding vector: ； ； in, This indicates the positional encoding of even-numbered dimension indices. This represents the position encoding of odd-numbered dimension indices, where pos is the time step position and i is the dimension index. This represents the total dimension of the input, including the position vector dimension and the word embedding vector dimension.

4. The buoy meteorological data repair method based on spatiotemporal dual attention neural network according to claim 2, characterized in that, sequence After entering the multi-head cross-attention mechanism layer, first They will be respectively compared with the attention weight matrix Perform matrix multiplication to obtain Q, K, and V, representing the query, key, and value matrices, respectively; then calculate the attention score: ； in, The attention score is given by N, where N is the length of the input data. for The dimension; In obtaining attention scores Then it will be compared with the input sequence. The calculation process for performing a residual link and layer normalization is as follows: ； Where x is the layer normalization result, and LayerNorm is the layer normalization function.

5. The buoy meteorological data repair method based on spatiotemporal dual attention neural network according to claim 2, characterized in that, After x enters the feedforward neural network layer, its calculation formula is: ； in, An abstraction representing the characteristics calculated after nonlinear transformation of the feedforward fully connected layer. for function; , Here is the weight matrix of the feedforward neural network. , This is the bias of the feedforward neural network; Subsequently, Perform a residual link and layer normalization again with x to finally obtain the temporal feature matrix of the target element. The calculation process is as follows: (The output is then presented.) ； in, The representation layer normalization function.

6. The buoy meteorological data repair method based on spatiotemporal dual attention neural network according to claim 1, characterized in that, In step 4, for consecutive missing values, a bidirectional prediction method is used in the prediction stage. That is, the input sequence is first predicted in the forward time direction, and then predicted in the reverse time direction. The predicted output value is: ； in, To predict the output value, The fusion features are for positive output. The output fusion features, For positional weights.

7. The buoy meteorological data repair method based on spatiotemporal dual attention neural network according to claim 6, characterized in that, In step 4, based on the relative position of the missing value with respect to the prediction direction, dynamic weights are used for prediction. The closer the position is to the prediction direction, the higher the weight is assigned. The weight range is limited to [0.1, 0.9]. Its function expression is: ； in, This represents the relative position of the missing value with respect to the consecutive missing values ​​at that point. The length of the consecutive missing values.

Images