Sea observation data-oriented base model construction method and system

Abstract

The invention belongs to the technical field of marine science and data processing, and provides a base model construction method and system for marine observation data, and the method comprises the steps: constructing a marine base model; inputting the acquired ocean time series data into an ocean foundation model, constructing a Token sequence of a timestamp granularity level of the ocean time series data, and connecting an offset delay feature vector and a covariable vector before a timestamp into a timestamp Token vector; in combination with a causal self-attention mechanism and rotation position embedding, the features of the Token sequence are mapped to a decoder of a Transform, and a next timestamp sequence is generated; and defining a loss function, and optimizing model parameters to obtain a trained ocean foundation model. According to the method, universal feature representation and time sequence modes are learned on various ocean data sets, so that an ocean universal model with high generalization ability is constructed, large-scale and multi-source ocean data can be processed, time sequence characteristics of the data can be captured, and real-time data processing and prediction ability can be provided.

Description

Technical Field

[0001] The present disclosure relates to the field of marine science and data processing technology, and in particular to a method and system for constructing a base model for marine observation data. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] With global climate change and the increasing development of marine resources, the analysis and understanding of marine observation data has become particularly important. Marine observation time series data contains rich marine environmental information, such as ocean temperature, salinity, flow rate, biochemical information, etc., which is of great significance for studying ocean circulation, climate change, biodiversity and other fields. However, traditional marine observation data analysis methods often rely on simple statistical models or physical models, which cannot fully utilize the complexity and nonlinear characteristics of time series data. Therefore, in order to deeply understand and model and analyze marine observation data, it is necessary to explore new intelligent analysis methods.

[0004] At present, a data foundation that can support the construction of a marine observation data base model has been formed at home and abroad. Among them, the Copernicus program is an earth observation project led by the European Union, which aims to provide continuous global environmental monitoring services through satellites and ground facilities. The Copernicus Marine Environment Monitoring Service (CMEMS) under the program is responsible for providing global and regional marine observation data and products. CMEMS uses satellite remote sensing technology and model simulation to provide a variety of time series data such as ocean temperature, salinity, sea level, ocean currents, and sea ice cover. The ARGO program is an international cooperation global ocean observation project. Through autonomous profiling buoys deployed in the global ocean, it regularly measures parameters such as temperature, salinity, and pressure within the upper 2,000 meters of the ocean. In addition, the ARGO buoy floats to the sea surface at regular intervals (usually 10 days) and transmits the observation data to the shore-based receiving station via satellite, thereby forming time series data of marine environmental parameters.

[0005] Ocean base models play a vital role in marine science research, climate change prediction and environmental protection. The research and development of ocean base models is a hot issue in the current field of marine science. However, there are relatively few studies on base models of marine observation data. There are mainly the following difficulties: First, the data volume is large and complex. Marine observation time series data usually contains a large number of spatiotemporal data points, and the relationship between the data is complex. Processing such large-scale and complex data requires efficient data processing and analysis methods. Secondly, there is often noise and incompleteness in the observation data, which has an adverse effect on the accuracy of the model. Finally, marine environmental changes are usually diverse and complex. The developed model needs to have strong generalization ability, be able to adapt to different sea areas and changing environmental conditions, while maintaining the accuracy and reliability of the prediction. Therefore, it has become an urgent need to develop a base model for marine observation data and use it to improve the reliability and accuracy of prediction. Summary of the invention

[0006] In response to the above-mentioned defects, the present invention proposes a base model construction method and system for ocean observation data. Through pre-training, common feature representations and time series patterns are learned on a variety of ocean data sets, so as to construct a general ocean model with high generalization ability. It can not only process large-scale, multi-source ocean data, but also capture the time series characteristics of the data, and provide real-time data processing and prediction capabilities.

[0007] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0008] The first aspect of the present disclosure provides a method for constructing a base model for ocean observation data, comprising the following steps:

[0009] Constructing ocean foundation models;

[0010] Input the acquired ocean time series data into the ocean base model, construct a token sequence at the timestamp granularity level of the ocean time series data, and connect the offset delay feature vector and covariate vector before the timestamp to the timestamp token vector;

[0011] Combining the causal self-attention mechanism and rotation position embedding, the features of the Token sequence are mapped to the Transformer decoder to generate the next timestamp sequence;

[0012] Define the loss function, optimize the model parameters, and obtain the trained ocean base model.

[0013] As a further implementation method, before constructing the Token sequence at the timestamp granularity level of the ocean time series data, it also includes standardization preprocessing of the ocean time series data.

[0014] As a further implementation method, a Token sequence at the timestamp granularity level of the ocean time series data is constructed, specifically:

[0015] The leading sequence offset delay features of the ocean time series data timestamp and the covariate features of the ocean time series data are extracted, the two types of features are concatenated and merged to obtain the timestamp Token vector, and the Token sequence at the timestamp granularity level is constructed based on the timestamp Token vector.

[0016] As a further implementation, covariates are variables other than time itself, including static covariates and dynamic covariates.

[0017] As a further implementation method, multiple decoders of the Transformer are stacked in the model, and a probability distribution head is used to predict the parameters of a selected distribution, so that the model has the ability to predict intervals of future ocean observation data.

[0018] As a further implementation, the probability distribution head adopts Student's T distribution.

[0019] As a further implementation, a masked self-supervised learning method is used to train the base model, and the loss function used in the self-supervised training is the negative log-likelihood function based on the predicted distribution.

[0020] A second aspect of the present disclosure provides a base model construction system for ocean observation data, comprising:

[0021] The model building module is configured to: build an ocean base model;

[0022] The Token sequence construction module is configured to: input the acquired ocean time series data into the ocean base model, construct a Token sequence at the timestamp granularity level of the ocean time series data, and connect the offset delay feature vector and covariate vector before the timestamp to the timestamp Token vector;

[0023] The model prediction module is configured to: combine the causal self-attention mechanism and the rotation position embedding to map the features of the token sequence to the Transformer decoder to generate the next timestamp sequence;

[0024] The model training module is configured to: define the loss function, optimize the model parameters, and obtain the trained ocean base model.

[0025] The third aspect of the present disclosure provides a medium on which a program is stored. When the program is executed by a processor, the steps in the method for constructing a base model for ocean observation data described in the first aspect of the present disclosure are implemented.

[0026] The fourth aspect of the present disclosure provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for constructing a base model for ocean observation data described in the first aspect of the present disclosure are implemented.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present disclosure proposes a method and system for constructing a base model for ocean observation data, which combines ocean observation data with a large model for the first time to construct a base large model for ocean observation data. Through pre-training, common feature representations and time series patterns are learned on a variety of ocean data sets to build a general ocean model with high generalization capabilities. Not only does it have excellent predictive capabilities and high accuracy, but it can also use pre-trained large models to achieve few-sample or zero-sample reasoning. By training and fine-tuning with task-specific data sets based on the pre-trained model, the model can better adapt to specific downstream tasks. The base model proposed in the present disclosure can not only process large-scale, multi-source ocean data, but also capture the time series characteristics of the data, providing real-time data processing and prediction capabilities.

[0029] The present disclosure proposes a method for constructing a timestamp granularity-level Token sequence for ocean observation data that integrates offset delay features and dynamic and static covariate information. By independently constructing a Token for each timestamp, the model's ability to parse the real-time dynamics of ocean observation data is significantly improved, while being able to adapt to the irregularity of timestamp frequency. By integrating the delay offset features of the preceding sequence and the dynamic and static covariate information, the model's ability to capture long-term trends is enhanced, and its adaptability to short-term rapid changes is also improved. In addition, the constructed Token sequence contains rich historical features and additional information, providing an efficient learning and reasoning basis for the model. It significantly improves the prediction accuracy of the base model and provides a more accurate, flexible and robust solution for time series analysis of ocean observation data.

[0030] The present invention discloses a base model for ocean observation data, which introduces multiple data standardization methods in the input layer, can effectively reduce the impact of outliers on time series analysis and improve the robustness and stability of the model. A timestamp granularity Token embedding layer is designed in the model, and the offset delay feature vector and covariate vector before the timestamp are connected to the original timestamp Token vector, so that the model can not only learn the instant changes of the timestamp data, but also make full use of the historical information and external information of the time series, so as to learn richer data rules and provide more accurate prediction results. In addition, by stacking multiple Transformer processing layers, our model can capture the multi-scale characteristics of time series data layer by layer, and improve the accuracy and reliability of prediction. And as the number of layers increases, large models can handle larger data sets, and it is also convenient to improve the model's ability to process and understand large amounts of data. Finally, the model's features are projected onto the parameters of the probability distribution through the distribution head layer, and the model learns to output the distribution of the value of the next time step based on historical features, so as to better capture the diversity and complexity of the data, thereby improving the model's ability to fit the data distribution, and thus improving the accuracy and generalization of the prediction. In addition, the introduction of the probability distribution head ensures that the model has the ability of "interval prediction" rather than "point prediction" by sampling on the distribution.

[0031] The present invention proposes a method and system for constructing a base model for ocean observation data. By processing the original ocean data files, a general ocean in-situ observation pre-training data set is constructed, which greatly facilitates further data analysis, model training and prediction, and also improves the availability and access efficiency of data. In addition, our method for processing data is highly scalable and compatible. With simple fine-tuning, efficient prediction of any type of ocean time series data in any sea area can be achieved, and the prediction results are excellent, which significantly improves the application value of ocean data analysis and prediction.

[0032] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0034] Figure 1 This is a schematic diagram of constructing the timestamp Token of the ocean observation time series data disclosed in the present invention;

[0035] Figure 2The process of the base model construction method for ocean observation data disclosed in this disclosure is Figure 1 ;

[0036] Figure 3 It is a schematic diagram of the overall architecture of the base model for ocean observation data disclosed in the present invention;

[0037] Figure 4 is a flow chart for constructing an ocean in-situ observation data set for base model construction disclosed in the present invention;

[0038] Figure 5 The process of the base model construction method for ocean observation data disclosed in this disclosure is Figure 2 ;

[0039] Figure 6 is the training loss of the disclosed pedestal model on Argo profile data;

[0040] Figure 7 are the training loss and evaluation loss of the disclosed base model on the Copernicus data;

[0041] Figure 8 is a diagram showing the predicted effect of the base model disclosed in the present invention on Argo temperature profile data;

[0042] Fig. 9 This is a diagram showing the predicted effect of the base model disclosed in the present invention on the temperature data of a certain Copernicus site;

[0043] Fig.10 This is a diagram showing the predicted effect of the base model disclosed in the present invention on the salinity data of a certain Copernicus station;

[0044] Fig.11 This is a diagram showing the predicted effect of the base model disclosed in the present invention on the oxygen saturation data of a certain Copernicus station;

[0045] Fig.12 It is a diagram showing the prediction effect of the disclosed pedestal model on the dissolved oxygen data of a certain Copernicus site;

[0046] Fig.13 This is a diagram showing the predicted effect of the base model disclosed in the present invention on the conductivity data of a certain Copernicus site;

[0047] Fig.14 This is a diagram showing the prediction effect of the disclosed base model on the chlorophyll data of a certain Copernicus site;

[0048] Fig.15 This is a diagram showing the prediction effect of the disclosed base model on the sea pressure data of a certain Copernicus station;

[0049] Fig.16 This is a diagram showing the predicted effect of the base model disclosed in the present invention on the surface pressure data of a Copernicus station;

[0050] Fig.17 This is a diagram showing the prediction effect of the base model disclosed in the present invention on the horizontal wind speed data of a Copernicus site. DETAILED DESCRIPTION

[0051] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0052] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0053] In the absence of conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.

[0054] Embodiment 1

[0055] like Figure 2 and Figure 5 As shown, the first embodiment of the present disclosure provides a method for constructing a base model for ocean observation data, which mainly includes the following steps:

[0056] S1 builds the ocean base model;

[0057] S2 inputs the acquired ocean time series data into the ocean base model, constructs a Token sequence at the timestamp granularity level of the ocean time series data, and connects the offset delay feature vector and covariate vector before the timestamp to the timestamp Token vector;

[0058] S3 combines the causal self-attention mechanism and rotation position embedding to map the features of the Token sequence to the Transformer decoder to generate the next timestamp sequence;

[0059] S4 defines the loss function, optimizes the model parameters, and obtains the trained ocean base model.

[0060] Our model is inspired by the text language model architecture. However, unlike traditional text language models, we innovated the model architecture to enable it to process ocean time series data, and developed a large-scale model focusing on ocean observation time series data on this basis. Specifically, Figure 2 shows our ocean model building process, and Figure 3 It shows the overall architecture of the model.

[0061] In traditional time series data processing and analysis, a common practice is to divide the data into multiple continuous patches, each patch contains multiple data points within a certain range, and the data points in a patch are usually processed uniformly. Although this method simplifies the data processing process and helps capture the overall trend, it inevitably ignores subtle changes and short-term fluctuations in the data, so this method may not accurately reflect the real-time dynamics of time series data. In addition, this method usually assumes that the frequency of timestamps is uniform, but the frequency of timestamps of ocean time series data is often irregular. In addition, existing time series prediction models usually only consider the data points of the time series itself, while ignoring other external factors that may affect the prediction results. For this reason, it is necessary to introduce covariate information in the Token vector.

[0062] This paper proposes a method for constructing a Token sequence at the timestamp granularity of ocean observation data by integrating offset delay characteristics and dynamic and static covariate information. An independent Token will be constructed for each timestamp, which significantly improves the parsing ability and prediction accuracy of the model. The schematic diagram of the construction of the Token sequence of ocean observation data is shown in the figure. Figure 1 As shown, it includes the following key processes:

[0063] 1. Extraction of pre-sequence delay offset features: Time series data is context-dependent. The pre-sequence of the timestamp contains important information that affects the value of the current timestamp and also reflects the state of the ocean observation variables in the past. Therefore, the pre-sequence cannot be ignored and is particularly important. Therefore, for timestamp point t, we select the pre-sequence with a length of L before the timestamp t to extract the pre-offset delay feature vector.

[0064] 2. Covariate information feature extraction: We continue to extract the current timestamp Toke n Introducing covariate information. Covariates are variables other than time itself that may affect the value or pattern of the time series. Covariates can provide additional information to help us better understand the behavior and trends of marine time series data. Therefore, we introduce a method for embedding marine data covariates.

[0065] First, the information such as year, month, day, hour, and minute extracted from the current timestamp is embedded into the Token as dynamic covariates as additional information to help the model capture the periodic changes in the time series.

[0066] Secondly, embedding the latitude and longitude of the source of the observation data, sea area information, site type, time range, and collection frequency into the Token as static covariates can provide the model with key spatial, temporal, and observational condition backgrounds.

[0067] By introducing static covariates and dynamic covariates, the ocean base model of the present invention can not only capture the long-term trend of ocean time series data, but also adapt to rapid changes in the short term. This covariate method that comprehensively utilizes static and dynamic information helps the model better understand the background characteristics of the ocean environment and significantly improves the accuracy and robustness of predictions, especially in a changeable ocean environment.

[0068] 3. Timestamp Toke n Sequence construction: After obtaining the preceding sequence delay offset features and covariate features, we can construct the final timestamp Toke n .

[0069] First, the two types of features are concatenated and merged. Here, the generated pre-sequence delay offset feature vector is defined as h t , and define the resulting covariate eigenvector as v t , then the Token vector at timestamp t can be expressed as x t =h t +v t .

[0070] Secondly, since we are performing a prediction task, that is, to predict a certain length of future window based on historical sequence information, we need to construct a sample sequence for model training and reasoning, namely, a Token sequence. Here it is defined as Sequence (here C is the context window that has a greater impact on the prediction of future values, and P is the future prediction window). n The vector sequence is used as the input of the ocean model. Since each token includes the current and past historical features of the timestamp, it is more conducive to model learning and capturing the changing rules in the time series.

[0071] Toke at the timestamp granularity level when building ocean time series data n Before the sequence is generated, the ocean time series data are also preprocessed by standardization.

[0072] The model supports multiple data standardization methods such as robust standardization, standard deviation standardization, and mean standardization when processing input data, which enhances the adaptability and generalization ability of different types of time series data. Among them, robust standardization is to remove the median and scale it according to the interquartile range (IQR), which reduces the impact of outliers on the model. For a sequence x with a context window size of 1：C ={x1, x2, ..., x C We normalize each time point to:

[0073]

[0074] in, xt is the time series value corresponding to time t, and Med() is the median of the included time series data. In this way, this model can process ocean time series data more stably and improve the accuracy of prediction.

[0075] Construct a Toke at the timestamp granularity level for the standardized preprocessed ocean time series data n For a sequence of length (Here L is the offset delay feature relative to the current timestamp extracted from a sequence before the current timestamp, C is the context window that has a greater impact on the prediction of future values, and P is the future prediction window) univariate sequence and its covariate, we use a special Token construction method to connect the offset delay feature vector and covariate vector before the timestamp to the original timestamp Toke n In the vector, a novel timestamp Toke is constructed n Vector representation, which in turn forms a Toke for large ocean model learning n By combining static and dynamic covariates, our base model can make full use of the historical information and external information of the time series, thereby learning richer data patterns and providing more accurate prediction results.

[0076] We pass this token sequence through a shared linear projection layer to map the features to the decoder part of the Transformer. The causal self-attention mechanism and the rotation position embedding are combined here to significantly improve the processing capability and prediction accuracy of long sequence data. The causal self-attention mechanism introduces causal constraints to ensure that the model only considers past information when predicting, effectively preventing information leakage and improving the accuracy of predictions. The rotation position embedding uses scaled rotation embedding technology, which can flexibly handle contextual data of different lengths and enhance the performance of the model when processing long sequence data. In addition, each layer of the Transformer decoder is pre-normalized through root mean square layer normalization (RMSNorm). Finally, as a large model, in order to be able to extract massive knowledge and extract deeper features and information, we stacked multiple Transformer processing layers in the model.

[0077] After M masked Transformer layers, the model uses a probability distribution head to predict the parameters of a selected distribution, which enables the model to have the ability to predict the interval of future ocean observation values, ensuring that our model can not only do traditional "data point prediction", but also "data interval range prediction", which can adapt to a variety of downstream decision-making needs. The interval probability distribution of the predicted value is more scientific than predicting a single value, and can guide the construction of flexible decisions. In the selection of the distribution head, we adopted the Student's T distribution, a specific parameter probability distribution, and output three parameters corresponding to this distribution: degrees of freedom, mean, and scale. These parameters are ensured to be positive through nonlinear activation functions.

[0078] During the training process, we adopted a masked self-supervised learning method. The loss function used in self-supervised training is based on the negative log-likelihood (NLL) of the predicted distribution, which is a commonly used loss function in probability prediction and is used to measure the consistency between the probability distribution predicted by the model and the true observation value.

[0079] Based on the proposed base model, we need to build a targeted ocean pre-training dataset. Before building the pre-training dataset, we need to process the original ocean data files and build a general ocean in-situ observation dataset suitable for the model. The process of building an ocean in-situ observation dataset is as follows: Figure 4As shown in the figure. We first read and parse the contents of different types of source files, process abnormal or redundant data in the files by traditional methods or artificial intelligence methods, then extract the data of each observation point according to the ocean elements and extract the basic attributes of the data such as longitude and latitude, start and end time, observation method and sampling frequency to form a single file. Finally, we name and store the processed data files according to unified rules to form our ocean observation dataset. In addition, in order to meet the needs of different ocean datasets, including Argo and Copernicus data, we proposed and applied a number of advanced data preprocessing techniques. For the Argo dataset, we built an efficient data parsing framework that can automatically process each profile data and parse it into a format suitable for the GluonTS framework; and adopt innovative conversion algorithms to smoothly convert the ocean in-situ observation data into time series pre-training data to ensure data integrity and consistency. For the Copernicus dataset, we designed a strict data cleaning process, including removing outliers, processing missing values, ensuring data quality, and accurately calculating the sampling frequency of timestamps to ensure the accuracy of time series data. In addition, in addition to the Argo and Copernicus datasets, our method is highly extensible and compatible and can receive and process other types of datasets. It can adapt to new dataset requirements with simple modifications, which fully reflects the high innovation and flexibility of our method.

[0080] The base model based on ocean observation data we proposed not only has excellent predictive ability and high accuracy, but also can use pre-trained large models to achieve few-shot or zero-shot reasoning. By using task-specific datasets for training and fine-tuning based on the pre-trained model, the model can be better adapted to specific downstream tasks. The traditional ocean data reasoning model needs to be trained on a large amount of historical data from the target sea area, while this reasoning model only requires little or no historical data from the target sea area to start effective reasoning. In addition, after the base model we proposed is pre-trained on a large amount of source domain data, it can be migrated and applied to various downstream tasks such as prediction, anomaly detection, and classification, which can achieve good migration applications.

[0081] Model training, evaluation and testing: Both the training set and the test set use the global ocean Argo scattered data set and the Copernicus global ocean near-real-time in-situ quality control observation data set. The data of the training set and the test set are non-overlapping.

[0082] The Argo dataset contains more than 2.3 million core profiles of temperature and salinity and some observation profiles involving biogeochemical elements (such as dissolved oxygen, chlorophyll, yellow matter, particulate backscatter coefficient, irradiance, nitrate and pH). In 2019, the China Argo Real-time Data Center began to collect and organize all Argo float observation data deployed in the global ocean, and re-control their quality. The resulting global ocean Argo temperature and salinity profile scatter data set is stored in .dat format. The dataset consists of 2,244,712 data files with an original data volume of 41.1GB and 7.56GB after compression. The time range of the dataset is from 1997 to 2020, and the spatial range is wide, including the Pacific Ocean, Indian Ocean, Atlantic Ocean, Arctic Ocean and major marginal sea areas in the world.

[0083] The Copernicus Global Ocean Near Real-time In-situ Quality Control Observation Dataset contains 22 data subsets, of which the moored buoy data subset has 2,800 points and more than 7 billion records, and the vertical profile buoy data subset contains more than 2.79 million profiles and more than 1.1 billion records. The dataset covers meteorological (wind speed, wind direction, atmospheric pressure, etc.), biochemistry (dissolved oxygen, chlorophyll, turbidity, blood oxygen saturation, etc.), waves (wave height, wave period, wave direction, etc.), and ocean physics (temperature, salinity, conductivity, current velocity, current direction, etc.). This experiment uses the core Argo profile data and the Copernicus moored buoy data subset MO.

[0084] The parameters that need to be set for this model include: the length of the input sequence, the length of the predicted sequence, the maximum predicted sequence length, the number of lags, the probability of data enhancement, the ratio of frequency masks, the ratio of frequency mixing, the maximum number of cycles during training, the number of batches per cycle, the number of samples per batch, the learning rate, etc.

[0085] Training loss and evaluation loss are two important indicators for measuring model performance. Training loss reflects the error of the model on the training data and is used to guide the model training process. Evaluation loss reflects the performance of the model on the validation set and is used to evaluate the generalization ability of the model and adjust hyperparameters. Figure 6 and Figure 7 The training loss and evaluation loss of the model on the core Argo profile data and MO data are shown respectively. It can be seen from the figure that the model has good performance and strong generalization ability.

[0086] The purpose of model testing is to test the final performance of the model and ensure the effectiveness and reliability of the model in practical applications. The model was tested on Argo temperature profile data and MO data, and the test indicators included mean absolute error (MAE), mean square error (MSE), mean absolute percentage error (MAPE), and determination coefficient (R2). Figure 8The test results of the model on Argo temperature profile data are shown. It can be seen from the results that the model has good generalization ability and accurate prediction ability. Figure 9-17 The temperature test results, salinity test results, oxygen saturation test results, dissolved oxygen test results, conductivity test results, chlorophyll test results, sea pressure test results, surface air pressure test results, and horizontal wind speed test results of the model at different MO data stations are displayed respectively. From the results, it can be seen that the model has demonstrated good prediction capabilities in different marine environmental elements.

[0087] Embodiment 2

[0088] The second embodiment of the present disclosure provides a base model construction system for ocean observation data, including:

[0089] The model building module is configured to: build an ocean base model;

[0090] The Token sequence construction module is configured to: input the acquired ocean time series data into the ocean base model, construct a Token sequence at the timestamp granularity level of the ocean time series data, and connect the offset delay feature vector and covariate vector before the timestamp to the timestamp Token vector;

[0091] The model prediction module is configured to: combine the causal self-attention mechanism and the rotation position embedding to map the features of the token sequence to the Transformer decoder to generate the next timestamp sequence;

[0092] The model training module is configured to: define the loss function, optimize the model parameters, and obtain the trained ocean base model.

[0093] The more detailed steps are the same as those in the first embodiment and will not be repeated here.

[0094] Embodiment 3

[0095] Embodiment 3 of the present disclosure provides a medium on which a program is stored. When the program is executed by a processor, the steps in the method for constructing a base model for ocean observation data as described in Embodiment 1 of the present disclosure are implemented.

[0096] The more detailed steps are the same as those in the first embodiment and will not be repeated here.

[0097] Embodiment 4

[0098] Embodiment 4 of the present disclosure provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a method for constructing a base model for ocean observation data as described in Embodiment 1 of the present disclosure are implemented.

[0099] The more detailed steps are the same as those in the first embodiment and will not be repeated here.

[0100] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for constructing a base model for ocean observation data, characterized in that: The following steps are involved: Constructing ocean foundation models; Input the acquired ocean time series data into the ocean base model, construct a token sequence at the timestamp granularity level of the ocean time series data, and connect the offset delay feature vector and covariate vector before the timestamp to the timestamp token vector; Combining the causal self-attention mechanism and rotation position embedding, the features of the Token sequence are mapped to the Transformer decoder to generate the next timestamp sequence; Define the loss function, optimize the model parameters, and obtain the trained ocean base model.

2. A method for constructing a base model for ocean observation data as claimed in claim 1, characterized in that: Before constructing the Token sequence at the timestamp granularity level of the ocean time series data, the ocean time series data is also subjected to standardized preprocessing.

3. A method for constructing a base model for ocean observation data as claimed in claim 1, characterized in that: Create a token sequence at the timestamp granularity level for ocean time series data, specifically: The leading sequence offset delay features of the ocean time series data timestamp and the covariate features of the ocean time series data are extracted, the two types of features are concatenated and merged to obtain the timestamp Token vector, and the Token sequence at the timestamp granularity level is constructed based on the timestamp Token vector.

4. A method for constructing a base model for ocean observation data as claimed in claim 3, characterized in that: Covariates are variables other than time itself, including static covariates and dynamic covariates.

5. A method for constructing a base model for ocean observation data as claimed in claim 1, characterized in that: The Transformer decoders are stacked in multiple layers in the model, and the probability distribution head is used to predict the parameters of a selected distribution, so that the model has the ability to predict the interval of future ocean observation data.

6. A method for constructing a base model for ocean observation data as claimed in claim 5, characterized in that: The probability distribution head adopts Student's T distribution.

7. A method for constructing a base model for ocean observation data as claimed in claim 1, characterized in that: The base model is trained using a masked self-supervised learning method, and the loss function used in the self-supervised training is the negative log-likelihood function based on the predicted distribution.

8. A base model construction system for ocean observation data, characterized in that: include: The model building module is configured to: build an ocean base model; The Token sequence construction module is configured to: input the acquired ocean time series data into the ocean base model, construct a Token sequence at the timestamp granularity level of the ocean time series data, and connect the offset delay feature vector and covariate vector before the timestamp to the timestamp Token vector; The model prediction module is configured to: combine the causal self-attention mechanism and the rotation position embedding to map the features of the token sequence to the Transformer decoder to generate the next timestamp sequence; The model training module is configured to: define the loss function, optimize the model parameters, and obtain the trained ocean base model.

9. A medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for constructing a base model for ocean observation data as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the method for constructing a base model for ocean observation data as described in any one of claims 1 to 7 are implemented.

Images