Weather prediction method, device and system based on federated learning

Abstract

The invention discloses a weather prediction method, device and system based on federated learning, and belongs to the technical field of weather prediction, the weather prediction method based on federated learning combines a Transform model with a KAN model, and uses the advantages of the Transform model, the KAN model and a federated learning framework to predict the weather prediction. The provided first weather prediction model constructed on the basis of the Transform-KAN can have good data fitting performance. According to the method, the weather data are classified, then the weather data are placed under a federated learning framework, the automatic weather stations distributed at all places can carry out distributed collaborative learning, global features and space-time dependency relationships in the weather data can be well learned, and better data fitting ability and generalization performance can be achieved while data privacy is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of meteorological forecasting, and more specifically, relates to a weather forecasting method, device and system based on federated learning. Background Art

[0002] In today's climate of dramatic change, the frequent occurrence of natural disasters and the resulting losses cannot be underestimated. This makes weather forecasting increasingly important in all aspects of life, from human life to socioeconomic development. Accurate and robust weather forecasts help people make better plans, effectively reducing the loss of life and property caused by climate change.

[0003] The vast majority of existing weather forecasting methods can be divided into two categories: physics-based numerical forecasting methods and data-driven forecasting methods. Physics-based numerical forecasting methods use a series of complex mathematical and physical equations to describe the changing patterns of weather variables. However, these mathematical and physical equations are usually very complex and require a lot of computing resources and time, which puts a huge burden on hardware facilities. In contrast, data-driven forecasting methods learn the changing patterns of weather variables from historical weather data without solving complex mathematical and physical equations, which makes them simpler and more cost-effective. In recent years, many data-driven forecasting methods have been proposed and have achieved good results in accurate weather forecasting.

[0004] Among various data-driven methods, the Transformer, with its unique multi-head attention structure, is widely recognized for its promising application in time series forecasting tasks such as weather forecasting. Although the Transformer demonstrates excellent capabilities in global feature extraction, it has been questioned by some researchers due to its performance degradation and computational overhead when forecasting sequences with large look-back windows. Furthermore, due to its high complexity, the Transformer is prone to overfitting, which results in high variance errors in the forecast results. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides a weather forecasting method, device and system based on federated learning, which aims to solve the technical problem that Transformer is prone to overfitting and leads to high prediction errors in the weather forecast process.

[0006] To achieve the above objectives, according to one aspect of the present invention, a weather forecasting method based on federated learning is provided, comprising:

[0007] S1: The central server sends the first weather forecast model built based on Transformer and KAN to each automatic weather station;

[0008] S2: Each of the automatic weather stations updates the received first weather forecast model using the locally collected historical weather data set to obtain a second weather forecast model, and uploads the second weather forecast model to the central server;

[0009] S3: The central server aggregates the second weather prediction models from each of the automatic weather stations to obtain a third weather prediction model;

[0010] S4: Determine whether the third weather forecast model meets a preset condition. If not, send the third weather forecast model as the first weather forecast model to each of the second automatic weather stations and proceed to S2. If the third weather forecast model meets the preset condition, send the third weather forecast model as the target weather forecast model to each of the automatic weather stations.

[0011] S5: Each of the automatic weather stations pre-processes the local weather data collected in the current period, and inputs the pre-processed data into the received target weather forecast model to obtain weather forecast results for the future period.

[0012] Furthermore, the target weather prediction model includes: an input layer, a Transformer layer, a KAN layer, a flattening layer, two linear layers and an output layer connected in sequence; the input layer is used to receive the preprocessed local weather data and pass it to the subsequent Transformer layer; the Transformer layer is used to establish a dependency relationship between any two elements of the local weather data through a self-attention mechanism to extract comprehensive global features in the local weather data and transmit them to the KAN layer; the KAN layer is used to optimize the global features by placing a learnable activation function on the weights; the flattening layer is used to convert the optimized global features from multidimensional data into a one-dimensional vector; the two linear layers are used to perform linear transformation on the one-dimensional vector to improve the expression ability, and then generate the prediction results corresponding to the local weather data through the output layer.

[0013] Furthermore, the S2 includes: the automatic weather station uses the historical weather data set to calculate the first weather forecast model w sent from the central server. t Perform local training to obtain the updated second weather forecast model The second weather forecast model Upload to the central server; where i is the number of the automatic weather station, g i The average gradient is calculated by stochastic gradient descent for AWS i, and η is the learning rate.

[0014] Furthermore, the S3 includes: the central server uses the formula The second weather prediction model from each automatic weather station is aggregated to obtain the third weather prediction model; where S is the set of all automatic weather stations participating in federated learning, n i is the data volume of the historical weather dataset of the i-th automatic weather station, and n is the total data volume of all automatic weather stations.

[0015] Furthermore, before S2, the method further includes: S01: each meteorological station in the region collects local weather data within a historical period; S02: pre-processing the received local weather data to obtain a historical weather data set.

[0016] Furthermore, the S02 includes: performing missing value processing, outlier processing and data normalization on the local weather data to construct the historical weather data set.

[0017] Furthermore, the missing value processing includes: for missing values ​​in the local weather data, if they are single, they are filled with the average of two adjacent data; if the missing values ​​are continuous and located at the beginning of the entire data set, they are filled with the next available data; if they are located in the middle or end of the data set, they are filled with the previous available data.

[0018] According to another aspect of the present invention, a weather forecasting device based on federated learning is provided, comprising:

[0019] The distribution module is used by the central server to distribute the first weather forecast model built based on Transformer and KAN to each automatic weather station;

[0020] a training module configured for each of the automatic weather stations to update the received first weather prediction model using a locally collected historical weather data set to obtain a second weather prediction model, and to upload the second weather prediction model to the central server;

[0021] an aggregation module, configured for the central server to aggregate the second weather forecast models from each of the automatic weather stations to obtain a third weather forecast model;

[0022] a judgment module, configured to judge whether the third weather prediction model satisfies a preset condition; if not, sending the third weather prediction model as the first weather prediction model to each of the second automatic weather stations and entering the training module for further training; if not, sending the third weather prediction model as the target weather prediction model to each of the automatic weather stations;

[0023] The prediction module is used for each of the automatic weather stations to pre-process the local weather data collected in the current period and input the data into the received target weather prediction model to obtain the weather prediction results for the future period.

[0024] According to another aspect of the present invention, a weather forecasting system based on federated learning is provided, comprising a memory, a processor, a central server, and a plurality of automatic weather stations, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned weather forecasting method based on federated learning when executing the computer program.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned weather forecasting method based on federated learning are implemented.

[0026] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0027] (1) The present invention provides a weather forecasting method based on federated learning. By combining the Transformer model with the KAN model and leveraging the advantages of the Transformer model, the KAN model, and the federated learning framework, the proposed first weather forecasting model based on Transformer-KAN can have good data fitting performance. By placing it under the framework of federated learning, distributed automatic weather stations in various locations can perform distributed collaborative learning, which can effectively learn the global features and spatiotemporal dependencies in weather data, and can achieve better data fitting capabilities and generalization performance while ensuring data privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart of the weather forecasting method based on federated learning provided in Example 1 of the present invention;

[0029] Figure 2 A schematic diagram of a weather forecasting method based on federated learning provided in Example 1 of the present invention;

[0030] Figure 3 A schematic diagram of a target weather forecast model provided in Example 1 of the present invention;

[0031] Figure 4 A simulation diagram of the weather forecasting method based on federated learning provided in Example 1 of the present invention at the automatic weather station numbered 001;

[0032] Figure 5 A simulation diagram of the weather forecasting method based on federated learning in an automatic weather station numbered 002 is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0034] First, the terms involved in the present invention are explained as follows:

[0035] Transformer refers to a deep learning model architecture based on the self-attention mechanism.

[0036] KAN refers to a neural network architecture inspired by the Kolmogorov-Arnold representation theorem in mathematics.

[0037] Federated learning refers to a distributed machine learning framework that allows multiple participants to collaboratively train shared models without sharing the original data.

[0038] A central server refers to a master server that serves as a core node in a network architecture and is responsible for centralized management, coordination, and storage of data.

[0039] Example 1

[0040] like Figure 1 and Figure 2 As shown, this embodiment provides a weather prediction method based on federated learning, including: S1: the central server sends the first weather prediction model constructed based on Transformer and KAN to each automatic weather station. S2: each automatic weather station uses the historical weather data set collected locally to update the received first weather prediction model to obtain a second weather prediction model, and uploads the second weather prediction model to the central server. S3: the central server aggregates the second weather prediction models from each automatic weather station to obtain a third weather prediction model. S4: Determine whether the third weather prediction model meets the preset conditions. If not, the third weather prediction model is sent as the first weather prediction model to each second automatic weather station and enters S2; if it is satisfied, the third weather prediction model is sent as the target weather prediction model to each automatic weather station; the target weather prediction model is as follows: Figure 3 S5: Each automatic weather station pre-processes the local weather data collected in the current period and inputs it into the received target weather forecast model to obtain the weather forecast results for the future period.

[0041] Furthermore, the target weather prediction model includes: an input layer, a Transformer layer, a KAN layer, a flattening layer, two linear layers and an output layer connected in sequence; the input layer is used to receive the preprocessed local weather data and pass it to the subsequent Transformer layer; the Transformer layer is used to establish a dependency relationship between any two elements of the local weather data through a self-attention mechanism to extract comprehensive global features in the local weather data and transmit them to the KAN layer; the KAN layer is used to optimize the global features by placing a learnable activation function on the weights; the flattening layer is used to convert the optimized global features from multidimensional data into a one-dimensional vector; two linear layers are used to perform linear transformation on the one-dimensional vector to improve the expression ability, and then generate the prediction results corresponding to the local weather data through the output layer.

[0042] Specifically, the target weather prediction model is a seven-layer network model, which consists of an input layer, a Transformer layer, a KAN layer, a flattening layer, two linear layers, and an output layer. The KAN layer is placed after the Transformer layer to combine the advantages of the Transformer model and the KAN model. The Transformer layer is used to extract global features from weather data and pass them to the KAN layer. The KAN layer is used to learn the extracted features and perform high-precision optimization on them. This target weather prediction model can well combine the advantages of the Transformer model and the KAN model, and at the same time has good global feature extraction capabilities and data fitting capabilities.

[0043] Furthermore, S2 includes: the automatic weather station uses the historical weather data set to predict the first weather forecast model w issued by the central server. t Perform local training to obtain the updated second weather forecast model Then the second weather forecast model Upload to the central server; where i is the number of the automatic weather station, g i The average gradient is calculated by stochastic gradient descent for AWS i, and η is the learning rate.

[0044] Furthermore, S3 includes: the central server utilizes the formula The second weather prediction model from each automatic weather station is aggregated to obtain the third weather prediction model; where S is the set of all automatic weather stations participating in federated learning, n i is the data volume of the historical weather dataset of the i-th automatic weather station, and n is the total data volume of all automatic weather stations.

[0045] Each automatic weather station uses the locally acquired weather data to independently train the received Transformer-KAN model and update the model parameters. Specifically, in the tth round, the automatic weather station i uses stochastic gradient descent to calculate the average gradient g i Then, the automatic weather station uses the average gradient g i and the learning rate η for the local model To update: After this, each automatic weather station sends the updated model parameters back to the central server.

[0046] Furthermore, before S2, the following steps are also included: S01: each meteorological station in the region collects local weather data within a historical period; S02: pre-processes the received local weather data to obtain a historical weather data set.

[0047] Specifically, automatic weather stations equipped with a variety of smart sensors are used to collect local weather data sets in multiple locations. In order to better learn the time dependency in weather data, these automatic weather stations can collect various local weather-related variables every 15 minutes. At the same time, these automatic weather stations support the Internet of Things protocol, and each time weather data is collected, the weather data is uploaded to the central server. Specifically, these smart sensors include barometers, anemometers, etc., which can measure various weather variables every 15 minutes, including atmospheric pressure, wind speed, ambient temperature, etc., and record them in a CSV file, which is then uploaded to the central server.

[0048] Furthermore, S02 includes: performing missing value processing, outlier processing, and data normalization on the local weather data to construct a historical weather dataset. Preferably, the missing value processing includes: for missing values ​​in the local weather data, if they are single, filling them with the average of two adjacent data; if the missing values ​​are continuous and located at the beginning of the entire dataset, filling them with the next available data; if they are located in the middle or end of the dataset, filling them with the previous available data.

[0049] Specifically, data preprocessing is performed on the weather data from various weather stations, including missing value processing, outlier processing, data normalization, and dataset construction. For example, multiple CSV files are first merged into a single CSV file by year, and then the weather data is processed for missing values. Weather data collected by automatic weather stations has certain missing values ​​due to sensor failures and other reasons. For these missing values ​​in weather data, if they are single, they are filled with the average of the two adjacent data. If the missing values ​​are continuous and located at the beginning of the entire dataset, they are filled with the next available data. If they are located in the middle or end of the dataset, they are filled with the previous available data.

[0050] After processing missing values ​​in the weather data, outliers are processed. Due to sensor errors, some outliers exist in the weather data. These outliers are processed using a sliding window of appropriate size. The 3σ principle is used to detect these outliers. Specifically, data falling outside the range (μ-3σ, μ+3σ) is considered an outlier and replaced by the mean value within the sliding window. u represents the mean value of the data within the sliding window, and σ represents the standard deviation of the data within the sliding window.

[0051] After processing the outliers in the weather data, the weather data is normalized. In order to speed up the training and convergence speed, and to improve the prediction accuracy of the model, this study uses minimum-maximum normalization to normalize the weather data. The maximum-minimum normalization is: x is the data to be processed for maximum-minimum normalization, x max is the maximum value in the data, x min is the minimum value in the data, xno rm The data are normalized to the maximum and minimum values.

[0052] After normalizing the weather data, the training set is constructed. Of the processed weather data, the first 60% is used as the training set, the next 20% is used as the validation set, and the last 20% is used as the test set.

[0053] The present invention is described in detail below with reference to simulation:

[0054] The simulation parameters are set as follows: the total number of automatic weather stations participating in the training N = 20, the maximum number of iterations T = 10, the training rounds of each automatic weather station E = 35 in each iteration, the learning rate η = 0.0001, the batch size B = 64, the width of the KAN W = [8,160], the spline order of the KAN k = 3, the grid size of the KAN G = 3, the depth of the KAN L = 1, and the sliding window size S = 500 for handling outliers.

[0055] The simulation uses relevant weather variables from the past 20 steps to predict the ambient temperature 5 steps into the future.

[0056] like Figure 4 As shown in the figure, in the automatic weather station numbered 001, the proposed weather forecasting method based on federated learning is compared with other benchmark methods. Other benchmark methods include the Transformer-KAN method without federated learning, the Transformer method, the KAN method, the LSTM method, the TCN method, the Random Forest method, and the XGBoost method. It can be seen that the mean square error of the proposed method is lower than that of the other benchmark methods, which means that the proposed method has a lower prediction error, that is, a higher prediction accuracy. Figure 5 As shown, the proposed federated learning-based weather forecasting method also achieves lower mean squared error (MSE) than other baseline methods for the automatic weather station 002. The proposed method also performs similarly for other different automatic weather stations. The proposed method consistently achieves lower prediction errors than the baseline methods across multiple automatic weather station datasets, demonstrating its good generalization and robustness.

[0057] Through the above simulations, it can be seen that the proposed weather forecasting method based on federated learning has stronger data fitting ability and better generalization performance than other benchmark methods, while also being able to better protect data privacy.

[0058] Example 2

[0059] This embodiment provides a weather prediction device based on federated learning, including: a sending module, a training module, an aggregation module, a judgment module and a prediction module.

[0060] Among them, the sending module is used for the central server to send the first weather prediction model built based on Transformer and KAN to each automatic weather station. The training module is used for each automatic weather station to use the locally collected historical weather data set to update the received first weather prediction model to obtain the second weather prediction model, and upload the second weather prediction model to the central server. The aggregation module is used for the central server to aggregate the second weather prediction models from each automatic weather station to obtain the third weather prediction model. The judgment module is used to judge whether the third weather prediction model meets the preset conditions. If not, the third weather prediction model is sent as the first weather prediction model to each second automatic weather station, and enters the training module for training again; if it meets the conditions, the third weather prediction model is sent as the target weather prediction model to each automatic weather station. The prediction module is used for each automatic weather station to pre-process the local weather data collected in the current time period, and input it into the received target weather prediction model to obtain the weather forecast results for the future time period.

[0061] Example 3

[0062] This embodiment provides a weather forecasting system based on federated learning, including a memory, a processor, a central server and multiple automatic weather stations. The memory stores a computer program, and when the processor executes the computer program, the steps of the weather forecasting method based on federated learning provided in Example 1 are implemented.

[0063] Example 4

[0064] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the weather forecasting method based on federated learning provided in Example 1 are implemented.

[0065] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A weather forecasting method based on federated learning, characterized in that: include: S1: The central server sends the first weather forecast model built based on Transformer and KAN to each automatic weather station; S2: Each of the automatic weather stations updates the received first weather forecast model using the locally collected historical weather data set to obtain a second weather forecast model, and uploads the second weather forecast model to the central server; S3: The central server aggregates the second weather prediction models from each of the automatic weather stations to obtain a third weather prediction model; S4: Determine whether the third weather forecast model meets a preset condition. If not, send the third weather forecast model as the first weather forecast model to each of the second automatic weather stations and proceed to S2. If the third weather forecast model meets the preset condition, send the third weather forecast model as the target weather forecast model to each of the automatic weather stations. S5: Each of the automatic weather stations pre-processes the local weather data collected in the current period, and inputs the pre-processed data into the received target weather forecast model to obtain weather forecast results for the future period.

2. The weather forecasting method based on federated learning according to claim 1, characterized in that: The target weather prediction model includes: an input layer, a Transformer layer, a KAN layer, a flattening layer, two linear layers, and an output layer connected in sequence; The input layer is used to receive the preprocessed local weather data and pass it to the subsequent Transformer layer; The Transformer layer is used to establish a dependency relationship between any two elements of the local weather data through a self-attention mechanism, so as to extract comprehensive global features from the local weather data and transmit them to the KAN layer; The KAN layer is used to optimize the global features by placing a learnable activation function on the weights; The flattening layer is used to convert the optimized global features from multi-dimensional data into a one-dimensional vector; The two linear layers are used to perform linear transformation on the one-dimensional vector to improve the expression ability, and then generate the prediction result corresponding to the local weather data through the output layer.

3. The weather forecasting method based on federated learning according to claim 2, characterized in that: The S2 includes: the automatic weather station uses the historical weather data set to calculate the first weather forecast model w issued from the central server. t Perform local training to obtain the updated second weather forecast model The second weather forecast model Upload to the central server; where i is the number of the automatic weather station, g i The average gradient is calculated by stochastic gradient descent for AWS i, and η is the learning rate.

4. The weather forecasting method based on federated learning according to claim 3, wherein: The S3 includes: the central server uses the formula The second weather prediction model from each automatic weather station is aggregated to obtain the third weather prediction model; where S is the set of all automatic weather stations participating in federated learning, n i is the data volume of the historical weather dataset of the i-th automatic weather station, and n is the total data volume of all automatic weather stations.

5. The weather forecasting method based on federated learning according to claim 1, wherein: The S2 also includes: S01: Each weather station in the region collects local weather data for the historical period; S02: Preprocess the received local weather data to obtain a historical weather dataset.

6. The weather forecasting method based on federated learning according to claim 5, characterized in that: The S02 includes: performing missing value processing, outlier processing and data normalization on the local weather data to construct the historical weather data set.

7. The weather forecasting method based on federated learning according to claim 6, characterized in that: The missing value processing includes: for missing values ​​in the local weather data, if they are single, they are filled with the average of two adjacent data; if the missing values ​​are continuous and located at the beginning of the entire data set, they are filled with the next available data; if they are located in the middle or end of the data set, they are filled with the previous available data.

8. A weather forecasting device based on federated learning, characterized in that: include: The distribution module is used by the central server to distribute the first weather forecast model built based on Transformer and KAN to each automatic weather station; a training module configured for each of the automatic weather stations to update the received first weather prediction model using a locally collected historical weather data set to obtain a second weather prediction model, and to upload the second weather prediction model to the central server; an aggregation module, configured for the central server to aggregate the second weather forecast models from each of the automatic weather stations to obtain a third weather forecast model; a judgment module, configured to judge whether the third weather prediction model satisfies a preset condition; if not, sending the third weather prediction model as the first weather prediction model to each of the second automatic weather stations and entering the training module for further training; if not, sending the third weather prediction model as the target weather prediction model to each of the automatic weather stations; The prediction module is used for each of the automatic weather stations to pre-process the local weather data collected in the current period and input the data into the received target weather prediction model to obtain the weather prediction results for the future period.

9. A weather forecasting system based on federated learning, comprising a memory, a processor, a central server, and a plurality of automatic weather stations, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images