A weather forecasting method and system based on self-supervised pre-training

Abstract

The present invention relates to the field of weather forecasting technology and provides a weather forecasting method and system based on self-supervised pre-training, comprising: obtaining meteorological element observation values at multiple weather stations within multiple time periods, and using a weather forecast model to obtain weather forecast values for each weather station; wherein the weather forecast model is trained using weather forecast task data containing three dimensions: time, station, and weather variable characteristics. Specifically, based on the weather similarity between different stations and the weather similarity between different times, a spatial encoder and a temporal encoder are obtained through comparative self-supervised learning; based on the relationship between different weather variable characteristics, constructed missing weather forecast task data are reconstructed through generative self-supervised learning to obtain a context reconstruction encoder; and the pre-trained spatial encoder, temporal encoder, and context reconstruction encoder are transferred to the weather forecast model and then fine-tuned, thereby improving weather forecast accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of weather forecasting, and in particular relates to a weather forecasting method and system based on self-supervised pre-training. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Weather is closely intertwined with our lives, impacting every aspect of life, including transportation, agricultural production, and energy generation. Accurate weather forecasts can provide early warning of meteorological disasters like floods and hurricanes, safeguarding life and property. Furthermore, in the field of new energy, providing forecast data such as wind speed, irradiance, and rainfall, essential for renewable energy prediction, allows for efficient and effective energy allocation. Furthermore, they greatly facilitate everyday travel.

[0004] Current weather forecasting methods are mainly divided into numerical weather forecasting methods based on physical models and data-driven machine learning methods.

[0005] The numerical weather prediction method based on physical models is currently the main weather prediction method. It uses supercomputers to solve nonlinear differential equations related to atmospheric dynamics. At the same time, it has obvious disadvantages, such as the consumption of a large amount of computing resources and the uncertainty of initial condition settings.

[0006] With the development of artificial intelligence (AI), data-driven machine learning methods are increasingly gaining attention, particularly deep learning methods, which have garnered widespread attention from researchers. Compared to numerical weather prediction methods, data-driven models consume less computing resources and do not require the configuration of initial conditions.

[0007] However, most existing data-driven machine learning weather forecasting methods still have some shortcomings: they simply treat a weather station as a whole, without exploring the relationship between weather variables belonging to multiple stations in a more fine-grained manner, or use complex network structures. Summary of the Invention

[0008] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a weather forecasting method and system based on self-supervised pre-training, which uses the idea of comparative self-supervision to pre-train to obtain a spatial encoder and a temporal encoder, and uses the idea of generative self-supervision to pre-train to obtain a context reconstruction encoder. It fully considers the similarity of weather conditions between multiple weather stations in the spatial dimension, the similarity brought about by periodic changes in weather in the temporal dimension, and the spatiotemporal dependence between weather variables of multiple weather stations in the context perspective, thereby improving the weather forecast accuracy.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A first aspect of the present invention provides a weather forecasting method based on self-supervised pre-training, comprising:

[0011] Obtain meteorological element observation values for multiple weather stations over multiple time periods;

[0012] Based on the obtained meteorological element observation values, the weather forecast model is used to obtain the weather forecast value for each weather station;

[0013] Among them, the weather prediction model is trained using weather prediction task data with three dimensions: time, station and weather variable characteristics. Specifically: based on the weather similarity between different stations and the weather similarity between different times, the spatial encoder and temporal encoder are obtained through comparative self-supervised learning; based on the relationship between different weather variable characteristics, the constructed missing weather prediction task data is reconstructed through generative self-supervised learning to obtain the context reconstruction encoder; after migrating the pre-trained spatial encoder, temporal encoder and context reconstruction encoder to the weather prediction model, they are fine-tuned based on the weather prediction task data.

[0014] Furthermore, the spatial encoder is pre-trained using the contrastive loss function;

[0015] The formula of the contrast loss function is as follows:

[0016]

[0017] Among them, z a 、 and Represent the feature representation of anchor points, positive samples and negative samples respectively; take a weather station as the anchor point X a , and set a distance threshold. When the weather similarity between a weather station and the anchor point is less than the distance threshold, the weather station is divided into a positive sample, otherwise it is divided into a negative sample, and the positive sample set is obtained. and negative sample set

[0018] Furthermore, triplet loss is used to optimize pre-training of the temporal encoder;

[0019] The formula for the triplet loss is as follows:

[0020]

[0021] Among them, β is the interval parameter; taking a certain moment as the anchor point X *, calculate the weather similarity between the remaining moments and the anchor moment along the time axis, and take the moment with the highest similarity as the positive sample X + , the moment with the lowest similarity is taken as the negative sample X - , for anchor point X * , positive sample X + and negative samples X - , respectively, obtain high-level semantic representations e and e after temporal self-supervised pre-training models. + and e - .

[0022] Furthermore, after processing missing values, numerical normalization and input dimension of weather data, weather forecast task data with three dimensions of time, station and weather variable characteristics are obtained.

[0023] Furthermore, the weather forecast model further includes a splicing layer;

[0024] The concatenation layer combines the feature representations obtained by the spatial encoder, temporal encoder, and context reconstruction encoder to obtain the concatenated features:

[0025] H=concat(H s ,H t ,H c )

[0026] Among them, H s 、H t and H c These are the feature representations obtained by the spatial encoder, temporal encoder, and context reconstruction encoder, respectively.

[0027] Furthermore, the weather prediction model further includes a decoder;

[0028] The decoder consists of three sequentially connected convolutional layers, and the outputs of the first two convolutional layers are subjected to a ReLU activation function.

[0029] Furthermore, the spatial encoder consists of two layers of convolution and a position attention module;

[0030] The output of the position attention module is:

[0031] P′=αB+P

[0032] B=eshape(AV)

[0033] A=Softmax(QK)

[0034] Among them, Q, K and V are three parallel convolutional layers of the position attention module to obtain three feature matrices, P represents the input of the position attention module, and α is a learnable scale factor.

[0035] A second aspect of the present invention provides a weather forecasting system based on self-supervised pre-training, comprising:

[0036] A data acquisition module is configured to: acquire meteorological element observation values of multiple weather stations within multiple time periods;

[0037] A weather forecast module is configured to: obtain a weather forecast value for each weather station based on the obtained meteorological element observation values using a weather forecast model;

[0038] Among them, the weather prediction model is trained using weather prediction task data with three dimensions: time, station and weather variable characteristics. Specifically: based on the weather similarity between different stations and the weather similarity between different times, the spatial encoder and temporal encoder are obtained through comparative self-supervised learning; based on the relationship between different weather variable characteristics, the constructed missing weather prediction task data is reconstructed through generative self-supervised learning to obtain the context reconstruction encoder; after migrating the pre-trained spatial encoder, temporal encoder and context reconstruction encoder to the weather prediction model, they are fine-tuned based on the weather prediction task data.

[0039] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the weather forecasting method based on self-supervised pre-training as described above.

[0040] The fourth aspect of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a weather forecasting method based on self-supervised pre-training as described above are implemented.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention provides a weather forecasting method based on self-supervised pre-training, which uses the idea of contrastive self-supervision to pre-train a spatial encoder and a temporal encoder, and uses the idea of generative self-supervision to pre-train a context reconstruction encoder. It fully considers the similarity of weather conditions between multiple weather stations in the spatial dimension, the similarity brought about by periodic changes in weather in the temporal dimension, and the spatiotemporal dependence between weather variables of multiple weather stations in the context perspective, so that the model expression ability is effectively improved, thereby further improving the prediction ability of the weather forecast model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0044] Figure 1 is a flow chart of a weather forecasting method based on self-supervised pre-training according to the first embodiment of the present invention;

[0045] FIG2( a ) is a schematic diagram of deleting variables from a supervised pre-training module in context reconstruction according to the first embodiment of the present invention;

[0046] FIG2( b ) is a schematic diagram of the restored variables in the context reconstruction self-supervised pre-training module according to the first embodiment of the present invention;

[0047] Figure 3 1 is a structural diagram of a weather forecast model according to a first embodiment of the present invention;

[0048] Figure 4 This is a structural diagram of the position attention module in the spatial encoder of Example 1 of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0050] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0051] Example 1

[0052] This embodiment provides a weather forecasting method based on self-supervised pre-training. It uses the idea of contrastive self-supervision to pre-train a spatial encoder and a temporal encoder, and uses the idea of generative self-supervision to pre-train a context reconstruction encoder. It fully considers the similarity of weather conditions between multiple weather stations in the spatial dimension, the similarity brought about by periodic changes in weather in the temporal dimension, and the spatiotemporal dependence between weather variables at multiple weather stations in the context perspective, thereby improving the accuracy of weather forecasting. Figure 1 As shown, the following steps are included:

[0053] Step 1: Obtain historical weather data, including meteorological element observation values (such as temperature, air pressure, humidity, etc.) from multiple weather stations over a period of time.

[0054] Step 2: Preprocess the historical weather data, including processing missing values, numerical normalization, and input dimension processing, to obtain weather forecast task data with three dimensions: time, space, and weather variable characteristics.

[0055] Step 201: Use linear interpolation to process missing values. The formula is as follows:

[0056]

[0057] Among them, x t represents the missing weather data at the current time t, x a represents the weather data at time a, and is the most recent weather data with a value greater than time t, x b Represents the weather data at time b, which is also the most recent weather data with a value less than time t.

[0058] Step 202: Use the min-max normalization method to scale all data to the range of [0, 1]. The conversion formula is as follows:

[0059]

[0060] Among them, x ori Represents the original weather data, x min Indicates the minimum value of historical weather data, x max represents the maximum value of historical weather data, and x represents the standardized weather data.

[0061] Step 203: Process the historical weather data into a three-dimensional format, i.e., input sample Where T represents the time window, C represents the number of weather stations, and F represents the dimension of weather features.

[0062] Step 3: Build self-supervised pre-training data and models. The specific method is as follows:

[0063] In the pre-training stage, two self-supervised learning methods are used, namely contrastive self-supervision and generative self-supervision. The self-supervised method makes full use of the characteristics of the data itself to guide learning, and does not rely on manually annotated labels. Among them, the core idea of contrastive self-supervision is to perform representation learning through comparison between data, so that the representation differences of similar samples are smaller and the representation differences of dissimilar samples are larger. It focuses more on learning data differences from the feature space of the abstract semantic level. Generative self-supervision is a reconstruction based on the element level, that is, the pre-processed raw data is mapped to the latent space through the encoder, and then the feature embedding in the latent space is restored to the initial data through the decoder. Unlike contrastive self-supervised learning, generative self-supervised learning pays more attention to the detailed features of the instance. In the present invention, the two are combined to learn the feature extractor from both macro and micro granularity.

[0064] Step 301: In the spatial dimension, considering factors such as topography and climate zones, even if the distance between regions is far, their weather conditions still have certain similarities. Therefore, the idea of contrastive self-supervision is used to construct a spatial encoder. Specifically, given the weather data of multiple weather stations in a time period T, a weather station is used as an anchor point. The Euclidean distance between the remaining weather stations and the anchor point is calculated using the preprocessed raw weather data to obtain the similarity between the weather stations. That is, the greater the distance, the higher the similarity, and vice versa. A distance threshold γ is set here. When the distance between a weather station and the anchor point is less than the threshold γ, the weather station is classified as a positive sample, otherwise it is classified as a negative sample, and then a positive sample set is obtained. and negative sample set

[0065] Through contrastive self-supervised learning, the distance between positive sample pairs in the latent space is made closer, and the distance between negative sample pairs is made farther.

[0066] For the spatial self-supervised pre-training model G s , which includes a spatial encoder and a nonlinear projection head. Among them, the spatial encoder is as follows Figure 3 As shown in the figure, the input data passes through the first convolutional layer with a kernel size of 1×1 (Conv 1×1), the nonlinear activation function Relu, the position attention module (PAM), and the second convolutional layer with a kernel size of 1×1 (Conv 1×1) and the nonlinear activation function Relu, thereby obtaining a preliminary spatial feature representation of the weather station. A nonlinear projection head, namely a fully connected layer with the nonlinear activation function Relu, is then used to obtain a more advanced semantic representation. The addition of the position attention module to the spatial encoder aims to more deeply and comprehensively explore the relationships between weather stations.

[0067] like Figure 4 As shown, the position attention module is mainly composed of three parallel convolutional layers, through which three feature matrices, Q, K, and V, can be obtained, and then the attention weight matrix is calculated.

[0068] A=Softmax(QK)

[0069] We can further get the output of the attention module

[0070]

[0071] P′=αB+P

[0072] in, represents the input of the position attention module, T′ represents the channel dimension after the first convolution layer of the spatial encoder, and α is a learnable scale factor.

[0073] Input Data After the spatial self-supervised pre-training model, high-level semantic representation is obtained where z irepresents the high-level semantic feature representation of weather station i, d represents the feature dimension of weather station i after being mapped by the spatial self-supervised pre-training model, and finally the contrastive loss function is used to optimize the spatial encoder. The formula is as follows:

[0074]

[0075] Among them, z a 、 and The feature representations obtained by the spatial self-supervised pre-training model represent anchor points, positive samples, and negative samples, respectively, and the dot product is used to calculate the similarity between the vectors. This loss function makes the feature representation distance between the anchor point and the positive sample closer and the feature representation distance between the anchor point and the negative sample farther, thus making the spatial encoder have stronger representation and discrimination capabilities.

[0076] Step 302: In the time dimension, considering the periodic changes of weather, the idea of comparative self-supervision is also used to mine the similarities of weather at different times. Specifically, the weather data of multiple weather stations at a certain time t are used as anchor points. Calculate the Euclidean distance between the weather data of the remaining moments and the anchor moment along the time axis to obtain their similarity, and take the moment with the highest similarity as the positive sample The moment with the lowest similarity is used as a negative sample Then, given the weather data of multiple weather stations within the T time period The corresponding positive sample can be obtained and negative samples It should be noted that because the input anchor weather data is from multiple moments, it also corresponds to positive samples and negative samples at each moment. Finally, the positive samples (negative samples) of each moment are spliced together to obtain positive samples (negative samples) of T dimensions.

[0077] Temporal self-supervised pre-training model G T It is composed of a time encoder and a nonlinear projection head (multi-layer perceptron). The time encoder mainly uses a shared recurrent neural network RNN to extract the weather data of each station (X1, X2, ..., ) time series features, then concatenate the time series features of each weather station and pass them through a fully connected layer (Linear), and expand the dimension (Expand dimension) to generate three-dimensional feature data, and then continue to pass through a convolution layer (Conv 1×1) with a convolution kernel size of 1×1 and a nonlinear activation function Relu to obtain low-level time features. Here, a nonlinear projection head is also used, that is, a fully connected layer with a nonlinear activation function Relu to obtain high-level semantic features in the time dimension. Specifically, for the anchor point X * , positive sample X+ and negative samples X - , which respectively obtain high-level semantic representations e and e after time self-supervised pre-training models. + and Here, triplet loss is used to optimize the temporal encoder, and the formula is as follows:

[0078]

[0079] Here, β is the margin parameter. This loss function reduces the distance between the anchor point and the positive samples and increases the distance between the anchor point and the negative samples, thereby providing a better representation of the data. The difference between this optimization and the spatial self-supervised pre-training model lies in the number of positive and negative samples. In the temporal dimension, one positive and one negative sample are selected at each moment.

[0080] Through contrastive self-supervised learning, the distance between positive sample pairs in the latent space is made closer, and the distance between negative sample pairs is made farther.

[0081] Step 303: In the context dimension, since no weather variable is isolated and there are certain correlations between them (for example, temperature and humidity are negatively correlated), a generative self-supervised approach is used to reconstruct the missing weather data based on the relationships between different weather variable features. Context refers to the relationship between a weather variable and other variables at the same site and across sites.

[0082] In the context dimension, given weather data from multiple weather stations within a time period T, multiple weather variable sequences are randomly deleted and the position index of each variable sequence is recorded. Through generative self-supervised learning, the reconstructed weather data is closer to the original data.

[0083] Context reconstruction from the supervised pre-training model G C It consists of a context reconstruction encoder and a context reconstruction decoder. Both the context reconstruction encoder and decoder use the structure of Vision Transformer, such as Figure 3As shown in the figure, its structure includes layer normalization (LayerNorm), multi-head attention mechanism (Multi-Head Attention), residual connection and multi-layer perceptron (MLP), and the context reconstruction encoder contains 3 layers of this structure, while the context reconstruction decoder contains 1 layer. Specifically, for this structure, the input data is sequentially normalized (LayerNorm) and multi-head attention mechanism (Multi-HeadAttention), and the output is connected to the input data for the first residual connection. The feature data after the first residual connection is then sequentially normalized (LayerNorm) and multi-layer perceptron (MLP). The output is then connected to the feature data after the first residual connection for the second residual connection, thereby obtaining the contextual semantic features of the weather data. For weather data from multiple weather stations within a time period T, as shown in Figure 2(a), multiple weather variable sequences are randomly deleted in a certain proportion (assuming that Figure 2(a) represents the temperature, humidity, and air pressure at three weather stations within the time period T, only the humidity variable sequence for station 1, the temperature variable sequence for station 2, and the air pressure variable sequence for station 3 within the time period T are retained, and the remaining variable sequences are deleted). The remaining variable sequences are then densely rearranged and fed into the context reconstruction encoder to obtain feature representations in the latent space. Subsequently, as shown in Figure 2(b), a shared feature embedding is used as the representation of the removed variable sequence, which is inserted into the feature representation of the latent space in its original order (for example, the temperature and pressure variable sequences for station 1, the humidity and pressure variable sequences for station 2, and the temperature and humidity variable sequences for station 3 in the above assumption are sequentially inserted into the feature representation of the latent space in the form of a shared feature embedding). This is then fed into the context reconstruction decoder to reconstruct the data. The selected deletion ratio is greater than 50%. A higher deletion ratio can accelerate model training and increase the difficulty of pre-training tasks, thereby obtaining a more robust and generalizable context reconstruction encoder. In addition, the context reconstruction encoder has more layers than the context reconstruction decoder, which gives the context encoder stronger feature representation capabilities, which is beneficial to improving the final weather forecast accuracy.

[0084] In the data reconstruction task, the mean square error loss function is used to optimize the context reconstruction model. The formula is as follows:

[0085]

[0086] in, To reconstruct weather data.

[0087] Step 4: Fine-tune the model. The specific method is as follows:

[0088] like Figure 3As shown, the final weather forecast model framework consists of the spatial encoder f s , time encoder f t , context reconstruction encoder f c , concatenation layer and decoder f d composition.

[0089] Specifically, the spatial encoder, temporal encoder, and context reconstruction encoder obtained in the above self-supervised pre-training stage are migrated to the final weather prediction model, and then the final weather prediction model is fine-tuned.

[0090] The feature representations obtained by the three encoders are combined through the concatenation layer to obtain the concatenated feature H:

[0091]

[0092]

[0093]

[0094]

[0095] Among them, H s 、H t and H c These are the feature representations obtained by the spatial encoder, temporal encoder, and context reconstruction encoder, respectively.

[0096] The decoder consists of three sequentially connected convolutional layers with kernel sizes of 1×K (K=F-M+1), 1×1, and 1×1, respectively. The outputs of the first two convolutional layers are activated by the ReLU function to enhance nonlinear expression capabilities. After the concatenated feature H passes through the decoder, the weather forecast value for the next time period is obtained:

[0097]

[0098] Where τ represents the forecast time window, C represents the number of weather stations, and M represents the number of target weather variables.

[0099] In the fine-tuning stage of the weather forecast model, the mean absolute error loss function is used as the objective function:

[0100]

[0101] Where N represents the number of samples, and Denote the actual value and predicted value of the weather variable at time t+j, respectively. During the training process, the Adam optimizer is used to optimize the weather forecast model until the loss function converges.

[0102] Step 5: During the testing phase, each predicted value Perform anti-normalization processing, that is, (y min Indicates the historical minimum value of the corresponding weather variable, y max Indicates the historical maximum value of the corresponding weather variable), and finally the weather forecast model is evaluated, and its evaluation indicators are the root mean square error (RMSE) and mean absolute error (MAE):

[0103]

[0104]

[0105] Where n represents the number of all predicted values, y i and represent the true value and the predicted value respectively.

[0106] Step 6: During the use phase, meteorological element observation values of multiple weather stations are obtained over multiple time periods, and weather forecast values of multiple weather stations in the next time period are obtained using a weather forecast model.

[0107] This embodiment provides a weather prediction method based on self-supervised pre-training, which fully considers the similarity of weather conditions between multiple weather stations in the spatial dimension, the similarity brought about by periodic changes in weather in the temporal dimension, and the spatiotemporal dependence between weather variables of multiple weather stations in the contextual perspective, thereby effectively improving the model's expression ability, thereby further improving the model's prediction ability.

[0108] This embodiment provides a weather prediction method based on self-supervised pre-training. The method obtains historical weather data from multiple weather stations, fills missing values in the data, normalizes the data, and processes it into a three-dimensional form, so that it contains three dimensions: time, station, and weather variable characteristics. The method constructs self-supervised pre-training tasks from the spatial, temporal, and context perspectives. Based on the similar weather conditions between different stations and the weather similarities at different times caused by periodic weather changes, a spatial encoder and a temporal encoder are obtained through comparative self-supervised learning. In addition, based on the dependencies between weather variables, some elements in the original input are deleted, and the weather data is reconstructed through generative self-supervised learning to obtain a context reconstruction encoder. The three encoders obtained from pre-training are transferred to the final weather prediction model framework, and the overall model is then fine-tuned based on the final weather prediction task data. Compared with the existing technology, the present invention integrates the concept of self-supervised pre-training into the weather prediction task, fully utilizing the spatial, temporal, and contextual characteristics of weather data, thereby further improving the accuracy of weather prediction.

[0109] Example 2

[0110] This embodiment provides a weather forecasting system based on self-supervised pre-training, which specifically includes:

[0111] A data acquisition module is configured to: acquire meteorological element observation values of multiple weather stations within multiple time periods;

[0112] A weather forecast module is configured to: obtain a weather forecast value for each weather station based on the obtained meteorological element observation values using a weather forecast model;

[0113] Among them, the weather prediction model is trained using weather prediction task data with three dimensions: time, station and weather variable characteristics. Specifically: based on the weather similarity between different stations and the weather similarity between different times, the spatial encoder and temporal encoder are obtained through comparative self-supervised learning; based on the relationship between different weather variable characteristics, the constructed missing weather prediction task data is reconstructed through generative self-supervised learning to obtain the context reconstruction encoder; after migrating the pre-trained spatial encoder, temporal encoder and context reconstruction encoder to the weather prediction model, they are fine-tuned based on the weather prediction task data.

[0114] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation processes are the same, which will not be repeated here.

[0115] Example 3

[0116] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the weather prediction method based on self-supervised pre-training as described in the first embodiment above are implemented.

[0117] Example 4

[0118] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a weather prediction method based on self-supervised pre-training as described in the first embodiment above are implemented.

[0119] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0120] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0123] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0124] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A weather forecasting method based on self-supervised pre-training, characterized in that: include: Obtain meteorological element observation values for multiple weather stations over multiple time periods; Based on the obtained meteorological element observation values, the weather forecast model is used to obtain the weather forecast value for each weather station; The weather forecast model is trained using weather forecast task data with three dimensions: time, station, and weather variable features. Specifically, based on the weather similarity between different stations and between different times, a spatial encoder and a temporal encoder are obtained through comparative self-supervised learning. Based on the relationship between different weather variable features, a context reconstruction encoder is obtained by reconstructing the constructed missing weather forecast task data through generative self-supervised learning. After migrating the pre-trained spatial encoder, temporal encoder, and context reconstruction encoder to the weather forecast model, they are fine-tuned based on the weather forecast task data. Among them, the idea of contrastive self-supervision is used to construct a spatial encoder. Specifically, a weather station is used as an anchor point. , and set a distance threshold. When the weather similarity between a weather station and the anchor point is less than the distance threshold, the weather station is divided into a positive sample, otherwise it is divided into a negative sample, and the positive sample set is obtained. and negative sample set ; Among them, the idea of contrastive self-supervision is used to construct a time encoder. Specifically, a certain moment is used as an anchor point. , calculate the weather similarity between the remaining moments and the anchor moment along the time axis, and take the moment with the highest similarity as the positive sample , the moment with the lowest similarity is taken as a negative sample ; Among them, context refers to the relationship between a weather variable and other variables at the same site and different sites. In the context dimension, given the weather data of multiple weather sites within a time period of T, multiple weather variable sequences are randomly deleted, and the position index of each variable sequence is recorded.

2. A weather forecasting method based on self-supervised pre-training according to claim 1, characterized in that: Pre-train the spatial encoder using contrastive loss function; The formula of the contrast loss function is as follows: in, 、 and Represent the feature representations of anchor points, positive samples, and negative samples respectively.

3. A weather forecasting method based on self-supervised pre-training according to claim 1, characterized in that: Use triplet loss to optimize pre-training of temporal encoders; The formula for the triplet loss is as follows: in, is the interval parameter; for anchor points , positive samples and negative samples , respectively, obtain high-level semantic representations through temporal self-supervised pre-training models 、 and .

4. A weather forecasting method based on self-supervised pre-training according to claim 1, characterized in that: After processing missing values, numerical normalization and input dimension processing of weather data, weather forecast task data with three dimensions of time, station and weather variable characteristics are obtained.

5. The weather forecasting method based on self-supervised pre-training according to claim 1, characterized in that: The weather prediction model also includes a splicing layer; The concatenation layer combines the feature representations obtained by the spatial encoder, temporal encoder, and context reconstruction encoder to obtain the concatenated features: in, These are the feature representations obtained by the spatial encoder, temporal encoder, and context reconstruction encoder, respectively.

6. A weather forecasting method based on self-supervised pre-training according to claim 1, characterized in that: The weather prediction model also includes a decoder; The decoder consists of three sequentially connected convolutional layers, and the outputs of the first two convolutional layers are subjected to a ReLU activation function.

7. A weather forecasting method based on self-supervised pre-training according to claim 1, characterized in that: The spatial encoder consists of two convolutional layers and a position attention module; The output of the position attention module is: Among them, Q, K and V are three parallel convolutional layers of the position attention module to obtain three feature matrices. represents the input of the position attention module, is the learnable scaling factor.

8. A weather forecast system based on self-supervised pre-training, characterized in that: include: A data acquisition module is configured to: acquire meteorological element observation values of multiple weather stations within multiple time periods; A weather forecast module is configured to: obtain a weather forecast value for each weather station based on the obtained meteorological element observation values using a weather forecast model; The weather forecast model is trained using weather forecast task data with three dimensions: time, station, and weather variable features. Specifically, based on the weather similarity between different stations and between different times, a spatial encoder and a temporal encoder are obtained through comparative self-supervised learning. Based on the relationship between different weather variable features, a context reconstruction encoder is obtained by reconstructing the constructed missing weather forecast task data through generative self-supervised learning. After migrating the pre-trained spatial encoder, temporal encoder, and context reconstruction encoder to the weather forecast model, they are fine-tuned based on the weather forecast task data. Among them, the idea of contrastive self-supervision is used to construct a spatial encoder. Specifically, a weather station is used as an anchor point. , and set a distance threshold. When the weather similarity between a weather station and the anchor point is less than the distance threshold, the weather station is divided into a positive sample, otherwise it is divided into a negative sample, and the positive sample set is obtained. and negative sample set ; Among them, the idea of contrastive self-supervision is used to construct a time encoder. Specifically, a certain moment is used as an anchor point. , calculate the weather similarity between the remaining moments and the anchor moment along the time axis, and take the moment with the highest similarity as the positive sample , the moment with the lowest similarity is taken as a negative sample ; Among them, context refers to the relationship between a weather variable and other variables at the same site and different sites. In the context dimension, given the weather data of multiple weather sites within a time period of T, multiple weather variable sequences are randomly deleted, and the position index of each variable sequence is recorded.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the weather forecasting method based on self-supervised pre-training as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the weather forecasting method based on self-supervised pre-training as described in any one of claims 1 to 7 are implemented.

Images