Marine environment prediction method based on principal component analysis-converter fusion model

Abstract

The invention discloses a marine environment element prediction method based on a principal component analysis-converter fusion model. The method comprises the following steps: performing quality control and data cleaning on a marine environment element data set to realize time-space dimension unification of heterogeneous data; building a prediction model on the basis of a converter network, taking day-by-day data after quality control and data cleaning as a target variable of the model, performing dynamics mechanism analysis based on an observation time sequence of the target variable, and selecting a dynamics prediction factor; dimensionality reduction processing is carried out on the high-dimensional forecasting factors through principal component analysis, and key feature information is effectively extracted; and inputting the optimized features into a converter network for modeling prediction. Compared with a traditional converter model, the average prediction relative error of the fusion model provided by the invention is reduced by 30%, and the prediction precision is remarkably improved. According to the invention, a new thought is provided for improving the marine environment element prediction precision through deep fusion, and more reliable decision support can be provided for the fields of marine navigation safety and the like.

Description

Technical Field

[0001] The present invention relates to the field of marine environmental factor prediction, and in particular to a method for predicting marine environmental factors by integrating principal component analysis (PCA) with a transformer model for the first time. The method involves using principal component analysis to reduce the dimension and construct prediction factors, which are input into the transformer model for predicting marine environmental factors, thereby achieving a dual improvement in the accuracy and efficiency of marine environmental factor prediction. Background Art

[0002] The prediction of marine environmental factors plays a vital role in the safety of ship navigation and production at sea. However, there are currently problems such as low accuracy and low efficiency in the prediction of marine environmental factors. Marine environmental factors include basic elements such as sea surface height, sea surface temperature, and chlorophyll concentration, which have a key impact on coastal urban engineering construction, disaster prevention and mitigation, and maritime defense. The prediction of marine environmental factors has important theoretical and practical significance for global and regional ocean-related research. At present, machine learning methods have been proven to be effective in predicting marine environmental factors. Long Short-Term Memory Neural Network (LSTM), Convolutional Neural Network (CNN), Convolutional Long Short-Term Memory Network (ConvLSTM) and Random Forest (RF) are machine learning models that are widely used in the prediction of marine environmental factors. However, the existing marine environmental factor prediction methods have certain limitations, which can be divided into two aspects. First, in terms of prediction factors, when too few prediction factors are selected, the prediction accuracy of marine environmental factors will be low, and when too many prediction factors are selected, the prediction efficiency and accuracy of marine environmental factors will be low; second, in terms of prediction methods, traditional machine learning models are difficult to capture the relationship between marine environmental factors. Therefore, this plan intends to improve the prediction factor processing and prediction methods, so as to further enhance the prediction ability of marine environmental factors.

[0003] In terms of prediction factor processing, the existing studies that consider multi-factor prediction have limitations in the selection of prediction factors. When too few prediction factors are selected, insufficient understanding of the dynamics will lead to low accuracy in environmental factor prediction; when the relationship between all marine elements is fully considered, there are too many prediction factors, which leads to low efficiency and low accuracy in the prediction of marine environmental elements.

[0004] In terms of prediction methods, traditional neural convolutional networks have problems such as difficulty in capturing long-distance spatial dependencies, easy gradient vanishing of RNN and LSTM, low training efficiency, and incomplete understanding of dynamic mechanisms of graph neural systems. The converter model can simultaneously focus on all position information in the input sequence through the self-attention mechanism, rather than just local information, which enables the converter model to better capture long-term dependencies in the sequence. Summary of the invention

[0005] In view of the above-mentioned prior art, the present invention provides a method for predicting the marine environment based on the principal component analysis-converter fusion model. The method of the present invention uses the dimension reduction method of principal component analysis before the machine learning model prediction to find the prediction factors that play a major role, and designs a machine learning prediction scheme with multiple factor inputs, so that the accuracy and efficiency of the prediction of marine environmental elements are double improved. The converter model is used to predict marine environmental elements, thereby improving the accuracy of the prediction and achieving long-term stable prediction of marine environmental elements.

[0006] In order to solve the above technical problems, the present invention proposes a method for predicting marine environment based on principal component analysis-converter fusion model, comprising the following steps:

[0007] Step 1: Collect multi-year daily time series data of marine environmental elements through satellite observation to construct a marine environmental element dataset; perform quality control and data cleaning on the marine environmental element dataset to unify the spatiotemporal dimensions of heterogeneous data, including: outlier identification and elimination and missing value repair;

[0008] Step 2: Build a prediction model based on the converter network.

[0009] Step 3: Use the daily data after quality control and data cleaning in step 1 as the target variable of the model, perform kinetic mechanism analysis based on the observed time series of the target variable, and select kinetic prediction factors; use the daily data of the selected kinetic prediction factors as the input feature matrix of the prediction model built in step 2; and normalize the input feature matrix;

[0010] Step 4: Use the principal component analysis method to reduce the dimension of the normalized input feature matrix, extract the feature information with the contribution rate of the first t items, and generate the optimized input data of the prediction model, including:

[0011] Step 4-1) Construct the principal component analysis sample matrix,

[0012] According to the number of selected dynamic prediction factors n and m days of time series data, the sample matrix is:

[0013]

[0014] In formula (1), X represents the sample matrix, and x represents a certain eigenvalue at a certain moment;

[0015] Step 4-2) Standardize the sample matrix.

[0016] First, calculate the column mean of the sample matrix:

[0017]

[0018] In formula (2), Represents the mean of the jth column of the sample matrix;

[0019] Then, calculate the standard deviation of the sample matrix:

[0020]

[0021] In formula (3), S j represents the standard deviation of the jth column;

[0022] Normalize the data:

[0023]

[0024] In formula (4), a represents the result of normalization of a certain feature value at a certain moment;

[0025] Get the standardized sample matrix A:

[0026]

[0027] In formula (5), represents the jth column of the normalized matrix A;

[0028] Step 4-3) Calculate the covariance matrix B of the standardized sample matrix A,

[0029]

[0030] Step 4-4) Calculate the eigenvalues ​​and eigenvectors of B,

[0031] Eigenvalue λ 1 ≥λ 2 ≥λ 3 ≥…≥λ n ≥0 (8) Eigenvector

[0032] Step 4-5) Calculate the principal component contribution rate and select the principal component.

[0033]

[0034] The i-th principal component:

[0035] Select to reduce the multidimensional prediction factor into a t-dimensional feature data set, i = t, calculate formula (11), and obtain the optimized input data of the prediction model;

[0036] Step 5: Divide the obtained t-dimensional feature data set into training set, validation set and test set according to the ratio of 7:2:1;

[0037] In the model training stage, based on the mapping relationship between the optimized input data and the target variable obtained in step 4 and the preset iteration number threshold, the model parameters are dynamically adjusted through the adaptive optimization algorithm to ensure the generalization performance and prediction accuracy of the model;

[0038] Perform system performance evaluation on the trained model based on the validation set data. Define the hyperparameter space and optimization algorithm based on the evaluation index results, adaptively tune the model hyperparameters, and obtain a prediction model with optimal prediction performance.

[0039] Step 6: Denormalize the target matrix output by the prediction model to finally obtain the forecast field of marine environmental elements within the target sea area and time range, and realize the prediction of marine environmental elements.

[0040] Furthermore, the marine environment prediction method of the present invention comprises:

[0041] In step 1, the marine environmental factors refer to one or more of sea surface temperature, sea surface height, significant wave height, water pressure and tide. The method for identifying and removing outliers is a combination of one or more of statistical methods, box plot analysis and empirical removal methods. The method for repairing missing values ​​is a combination of one or more of the mean or median filling, regression analysis, spatial interpolation, K-nearest neighbor-based machine learning algorithm and random forest method selectively applied according to data characteristics.

[0042] In step 3, the method of kinetic mechanism analysis includes: a combination of one or more of numerical simulation and model analysis, data analysis and statistical methods, spectrum analysis, feature analysis, mathematical modeling and theoretical analysis.

[0043] In step 5, the hyperparameter space is defined as follows: the parameters that affect the performance and generalization ability of the model are the hyperparameter space, including: learning rate (learn_rate), dropout rate (dropout_rate), fully connected layer dimension (dense_dim), number of blocks (num_blocks), number of attention heads (num_heads) and embedding dimension (embed_dim); the range and value of the optimized hyperparameters are as follows:

[0044] For learning rate, the range is between 0.0001 and 0.01; for dropout rate, the range is between 0.01 and 0.2; for fully connected layer dimension, the range is between 32 and 256; for number of blocks, the range is between 3 and 8; for number of attention heads, choose from 4, 8, 16, and 32; for embedding dimension, choose from 32, 64, 128, and 256.

[0045] In step 5, the hyperparameter optimization method uses any of empirical parameter adjustment, grid search, random search, gradient optimization, and Bayesian optimization to optimize the model parameters.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] Compared with relative importance sorting, dimension reduction using principal component analysis can significantly reduce the time for predicting marine environmental elements. While improving prediction efficiency, it removes redundant information in the process of capturing the principal component, thereby improving the accuracy of prediction. At the same time, the converter model is used to solve the problem of long-distance spatial dependence of marine environmental elements. Compared with existing methods, the marine environmental element prediction method based on the principal component analysis-converter fusion model fusion concept proposed in the present invention more comprehensively considers and effectively analyzes the dynamic relationship between marine environmental elements. At the same time, principal component analysis is used to reduce the dimension of input factors, thereby improving the accuracy and efficiency of marine environmental element prediction. In terms of prediction results, compared with the prior art, the average relative error of the present invention is greatly reduced, which significantly improves the prediction accuracy of marine environmental elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the marine environment prediction method based on the principal component analysis-converter fusion model of the present invention;

[0049] Figure 2 It is a flow chart of the traditional conversion model algorithm in the prior art;

[0050] Figure 3 The figure is a comparison between the prediction results of the present invention and those of the prior art. DETAILED DESCRIPTION

[0051] The design idea of ​​the marine environmental factor prediction method based on the principal component analysis-converter fusion model proposed in the present invention is: in order to solve the problems of low prediction efficiency and insufficient precision in the existing marine environmental factor prediction methods, the marine environmental factor prediction method of the present invention integrates the principal component analysis method with the converter network, and reduces the dimension of high-dimensional prediction factors through principal component analysis, effectively extracts key feature information, reduces data redundancy, and significantly improves computing efficiency; the optimized features are input into the converter network for modeling and prediction, giving full play to the advantages of the converter network in time series data, and realizing high-efficiency and high-precision prediction of marine environmental factors. It has been verified by experiments that Figure 2 Compared with the traditional converter model shown in FIG. 1 , the principal component analysis-converter fusion model proposed in the present invention reduces the average prediction relative error by 30%, and the prediction accuracy is significantly improved. Figure 3 As shown, at the same time, the fusion model has strong practicality and scalability. In the marine environmental factor prediction method of the present invention, by deeply integrating dynamic oceanography and machine learning technology, the high-dimensional feature processing and nonlinear modeling problems in the prediction of marine environmental factors are systematically solved. This method not only provides a new idea for improving the prediction accuracy of marine environmental factors, but also has important value in practical applications, and can provide more reliable decision-making support for the fields of marine navigation safety, marine resource development, and marine environmental protection.

[0052] In order to further understand the content, characteristics and effects of the present invention, the sea surface height (SSH) prediction of a part of the northwest Pacific Ocean (139°E-149°E, 16°N-26°N) is taken as a specific example and described in detail with reference to the accompanying drawings.

[0053] The source, year, temporal resolution, airborne resolution and spatial range of the relevant data collected in this embodiment are:

[0054] Sea surface height (SSH), data source: Archiving, Validation and Interpretation of Satellite Oceanographic data (AVISO), years 1993-2021, temporal resolution is daily, spatial resolution is 1°×1°, spatial range is the northwestern Pacific (5°N-40°N, 108°E-180°E).

[0055] Sea surface temperature (SST), data source: The Operational Sea Surface Temperature and Ice Analysis (OSTIA), years 1993-2021, time resolution daily, spatial resolution 1°×1°, spatial range: Northwest Pacific (5°N-40°N, 108°E-180°E).

[0056] Wind stress curl, spatial gradient of sea surface height, total precipitation and 2m atmospheric temperature, data source: The fifthgeneration ECMWF reanalysis (ERA5), years 1993-2021, time resolution daily, spatial resolution 1°×1°, spatial range Northwest Pacific (5°N-40°N, 108°E-180°E).

[0057] like Figure 1 As shown, the present invention proposes a method for predicting marine environmental elements based on principal component analysis-converter fusion model. First, multi-year daily time series data of sea surface temperature (SST) and sea surface height (SSH) are obtained through the above-mentioned relevant public database, and the obtained data are data that have achieved the unification of the spatiotemporal dimensions of heterogeneous data; a marine environmental element data set is constructed; then, a prediction model is built based on the converter network, and the daily sea surface height data is used as the target variable of the model. A dynamic mechanism analysis (including numerical simulation and model analysis, feature analysis and theoretical analysis) is performed based on the observed time series of the target variable. The dynamic prediction factors selected in this embodiment include: six elements of sea surface height, sea surface temperature, total precipitation, 2m atmospheric temperature, wind stress curl and sea surface height spatial gradient as prediction factors. The daily data of the selected dynamic prediction factors are used as the input feature matrix for building the prediction model; the input feature matrix is ​​normalized; and the prediction factor dimension is reduced according to the principal component analysis mentioned in the present invention, thereby obtaining a two-dimensional prediction factor data set (in the specific example, the prediction factor is reduced to two dimensions), and then the obtained two-dimensional prediction factor data set is used as the input feature element, and the sea surface height is used as the output response element. After that, the Bayesian network is used to automatically adjust the parameters to obtain the best hyperparameter combination, and 70% of the sample size is selected as the principal component analysis-converter fusion model training set of the present invention, and finally the prediction result corresponding to the remaining 30% test set is selected as the sea surface height prediction value. The specific operation is as follows:

[0058] (I) Selection of prediction factors based on knowledge of ocean dynamics.

[0059] In addition to the sea surface height itself, the following prediction factors are selected based on the ocean dynamics mechanism:

[0060] 1) Sea surface temperature. Historical studies have shown that the sea surface height at mid- and low-latitudes has a good temporal and spatial correlation with the sea surface temperature. The spatial height caused by changes in the ocean temperature water column is also the cause of sea level changes. Since it is difficult to obtain temperature data at full depth, and the sea surface temperature can characterize temperature changes to a certain extent, the sea surface temperature is used as a prediction factor.

[0061] 2) Total precipitation. Regional sea level changes are affected by freshwater flux from evaporation and precipitation, so total precipitation is considered as a predictor of sea surface height.

[0062] 3) 2m atmospheric temperature. 2m atmospheric temperature refers to the atmospheric temperature 2m above the surface of land, ocean or land water. Here it refers to the atmospheric temperature 2m above the ocean. The atmosphere and ocean are two important components of the climate system. The two influence each other. The specific heat capacity of the atmosphere is much smaller than that of the ocean, and it is more sensitive to climate change. Changes in 2m atmospheric temperature will affect the changes in sea surface temperature accordingly, and then affect the sea surface height. Therefore, the 2m atmospheric temperature is considered as a predictor of sea surface height.

[0063] 4) Wind stress curl. The upper circulation in the North Pacific is mainly driven by wind, and wind stress can have an important impact on the sea surface height through the mechanisms of Ekman suction and Sverdrup equilibrium. Regionally variable Ekman transport leads to upwelling or downwelling, causing seawater mass transport in the open ocean, which in turn affects the sea surface height. Due to the Ekman suction rate:

[0064]

[0065] Among them, w e represents the Ekman suction rate, curl represents the curl, τ represents the wind stress, ρ represents the sea water density, and represents the Coriolis parameter.

[0066] If the wind stress curl is negative, transport convergence produces downwelling, causing sea level rise; conversely, transport divergence will cause sea level drop. The Sverdrup equation shows that the north-south mass transport caused by ocean circulation depends on the wind stress curl:

[0067] βM y =curl z (τ)

[0068] Among them, M y The wind stress curl is innovatively considered as a predictor of sea surface height based on the above dynamic theory.

[0069] 5) Spatial gradient of sea surface height. Factors such as wind stress can cause spatial variation in sea surface height, i.e., spatial gradient of sea surface height. Wind stress curl is related to Ekman suction and the latitudinal gradient of sea surface height, while the latitudinal and longitudinal gradients of sea surface height can obtain the frontal information of sea surface current. Introducing the latitudinal and longitudinal gradients of sea surface height is equivalent to considering the geostrophic flow velocity, and the distribution of geostrophic flow can reflect the distribution of sea surface height to a certain extent. Therefore, the longitudinal and latitudinal gradients of sea surface height are introduced as prediction factors to predict sea surface height.

[0070] (II) Constructing the principal component analysis sample matrix. Taking the six prediction factors selected for sea surface height prediction as an example, in this embodiment, there are 3000 samples and 6 features, which can form a 3000×6 sample matrix X:

[0071]

[0072] In formula (1), X represents the sample matrix, and x represents a certain eigenvalue at a certain moment;

[0073] (iii) Standardize the sample matrix.

[0074] First, calculate the column mean of the sample matrix: Represents the mean of the jth column of the sample matrix.

[0075] Then, calculate the standard deviation of the sample matrix: S j represents the standard deviation of the jth column.

[0076] Normalize the data: a represents the result of normalization of a certain eigenvalue at a certain moment, and the standardized sample matrix A is obtained:

[0077]

[0078] Represents the j-th column of the normalized matrix A.

[0079] (IV) Calculate the covariance matrix B of the standardized sample matrix A

[0080]

[0081] (V) Calculate the eigenvalues ​​and eigenvectors of B

[0082] Eigenvalue λ 1 ≥λ 2 ≥λ 3 ≥…≥λ 6 ≥0

[0083] Eigenvector

[0084] Select to reduce the multidimensional predictor to a t-dimensional feature dataset, i = t,

[0085] (VI) Calculate the principal component contribution rate and select the principal component

[0086]

[0087] The i-th principal component:

[0088] In the present invention, n contribution rates are sorted according to size, and the first i (i from 1 to n) with the largest contribution rates are selected as principal components. In this embodiment, n = 6, i = 2, and the first two with the largest contribution rates are selected as principal components, that is, the multidimensional prediction factor is reduced to a two-dimensional feature data set. Therefore, the calculation is performed according to i = 2 to obtain the optimized input data of the prediction model.

[0089] (VII) Principal Component Analysis-Converter Fusion Model Prediction

[0090] The original sample set is composed of a two-dimensional feature data set after dimensionality reduction by principal component analysis as the input sample set and the output sample set (sea surface height), which is divided into a training set, a validation set and a test set according to a ratio of 7:2:1. In order to determine the prediction results of this model, all samples in the original sample set are divided into two non-overlapping parts: a training sample set and a test sample set. 70% of the original samples are selected for training. In the model training stage, based on the mapping relationship between the optimized input data and the target variable, combined with the preset iteration number threshold, the Bayesian network hyperparameter optimization method is used to dynamically adjust the model parameters through an adaptive optimization algorithm to optimize the model parameters and ensure the generalization performance and prediction accuracy of the model; the hyperparameter space is defined as: the parameters that affect the performance and generalization ability of the model are the hyperparameter space, and the following parameters are adjusted:

[0091] Learning rate learn_rate(0.0001, 0.01), fully connected layer dimension dense_dim(32, 256), number of blocks num_blocks(3, 8), dropout rate dropout_rate(0.01, 0.2), embedding dimension embed_dim(32, 64, 128, 256), number of attention heads num_heads(4, 8, 16, 32).

[0092] The optimal hyperparameters are obtained, and the prediction model with the best prediction performance is obtained. The remaining 30% of the two-dimensional feature samples are input into this model to obtain the target matrix output by the model. The target matrix output by the prediction model is denormalized, and finally the forecast field of marine environmental elements within the target sea area and time range is obtained to realize the prediction of marine environmental elements.

[0093] The root mean square error (RMSE) and the mean relative error (MAPE) can be calculated based on the predicted values ​​and test values ​​of the marine environmental elements. Taking the two into consideration, the prediction effect of the principal component analysis-converter fusion model can be evaluated and analyzed.

[0094] Compared with the traditional converter model in the prior art that does not use PCA dimensionality reduction, the prediction results of the PCA-converter fusion model are comparable to those of the traditional converter model ( Figure 2 ) The prediction results are compared with Figure 3 As shown. The average relative error of the prediction results of the principal component analysis-converter fusion model is reduced by 30% compared with the prior art, which improves the prediction efficiency and greatly improves the prediction accuracy. The marine environmental element prediction method based on the principal component analysis-converter fusion model of the present invention can accurately predict the marine environmental elements by analyzing the prediction factors and performing dimensionality reduction.

[0095] Although the present invention has been described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many improvements and changes without departing from the purpose of the present invention, which are all within the protection of the present invention.

Claims

1. A method for predicting marine environment based on principal component analysis-converter fusion model, characterized in that: The steps include: Step 1: Collect multi-year daily time series data of marine environmental elements through satellite observation to construct a marine environmental element dataset; perform quality control and data cleaning on the marine environmental element dataset to unify the spatiotemporal dimensions of heterogeneous data, including: outlier identification and elimination and missing value repair; Step 2: Build a prediction model based on the converter network. Step 3: Use the daily data after quality control and data cleaning in step 1 as the target variable of the model, perform kinetic mechanism analysis based on the observed time series of the target variable, and select kinetic prediction factors; use the daily data of the selected kinetic prediction factors as the input feature matrix of the prediction model built in step 2; and normalize the input feature matrix; Step 4: Use the principal component analysis method to reduce the dimension of the normalized input feature matrix, extract the feature information with the contribution rate of the first t items, and generate the optimized input data of the prediction model, including: Step 4-1) Construct the principal component analysis sample matrix, According to the number of selected dynamic prediction factors n and m days of time series data, the sample matrix is: In formula (1), X represents the sample matrix, and x represents a certain eigenvalue at a certain moment; Step 4-2) Standardize the sample matrix. First, calculate the column mean of the sample matrix: (2) In formula (2), Represents the mean of the jth column of the sample matrix; Then, calculate the standard deviation of the sample matrix: In formula (3), S j represents the standard deviation of the jth column; Normalize the data: In formula (4), a represents the result of normalization of a certain feature value at a certain moment; Get the standardized sample matrix A: In formula (5), represents the jth column of the normalized matrix A; Step 4-3) Calculate the covariance matrix B of the standardized sample matrix A, Step 4-4) Calculate the eigenvalues ​​and eigenvectors of B, Eigenvalue λ1≥λ2≥λ3≥…≥λ n ≥0 (8) Step 4-5) Calculate the principal component contribution rate and select the principal component. The i-th principal component: Select to reduce the multidimensional prediction factor into a t-dimensional feature data set, i = t, calculate formula (11), and obtain the optimized input data of the prediction model; Step 5: Divide the obtained t-dimensional feature data set into training set, validation set and test set according to the ratio of 7:2:1; In the model training stage, based on the mapping relationship between the optimized input data and the target variable obtained in step 4 and the preset iteration number threshold, the model parameters are dynamically adjusted through the adaptive optimization algorithm to ensure the generalization performance and prediction accuracy of the model; Perform system performance evaluation on the trained model based on the validation set data. Define the hyperparameter space and optimization algorithm based on the evaluation index results, adaptively tune the model hyperparameters, and obtain a prediction model with optimal prediction performance. Step 6: Denormalize the target matrix output by the prediction model to finally obtain the forecast field of marine environmental elements within the target sea area and time range, and realize the prediction of marine environmental elements.

2. The method for predicting the marine environment according to claim 1, characterized in that: In step 1, the marine environmental factor refers to one or more of sea surface temperature, sea surface height, significant wave height, water pressure and tide.

3. The method for predicting the marine environment according to claim 1, characterized in that: In step 1, the method for identifying and eliminating outliers is a combination of one or more of a statistical method, a box plot analysis, and an empirical removal method; In step 1, the missing value repair method is to selectively use a combination of one or more of mean or median filling, regression analysis, spatial interpolation, K-nearest neighbor-based machine learning algorithm and random forest method according to data characteristics.

4. The method for predicting the marine environment according to claim 1, characterized in that: In step 3, the method of kinetic mechanism analysis includes: a combination of one or more of numerical simulation and model analysis, data analysis and statistical methods, spectrum analysis, feature analysis, mathematical modeling and theoretical analysis.

5. The method for predicting the marine environment according to claim 1, characterized in that: In step 5, the hyperparameter space is defined as follows: the parameters that affect the performance and generalization ability of the model are the hyperparameter space, including: learning rate (learn_rate), dropout rate (dropout_rate), fully connected layer dimension (dense_dim), number of blocks (num_blocks), number of attention heads (num_heads) and embedding dimension (embed_dim); the range and value of the optimized hyperparameters are as follows: For the learning rate, the range is between 0.0001 and 0.01; For the discard rate, the range is between 0.01 and 0.2; For the fully connected layer dimension, the range is between 32 and 256; For the number of blocks, the range is between 3 and 8; For the number of attention heads, choose from 4, 8, 16, and 32; For the embedding dimension, select from four values: 32, 64, 128, and 256.

6. The method for predicting the marine environment according to claim 1, characterized in that: In step 5, the hyperparameter optimization method uses any of empirical parameter adjustment, grid search, random search, gradient optimization, and Bayesian optimization to optimize the model parameters.

Images