A multi-source fusion meteorological real-time analysis method based on CNN-GCN neural network
Through the CNN-GCN neural network integrating multi-source meteorological data and learning the relationship between sites, the problems of insufficient utilization of multi-source data and limitations in the existing technology are solved, and the generation of high-resolution meteorological fields and refined meteorological real-time analysis are realized. .
Patent Information
- Application Number
- CN202411162854.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-23
AI Technical Summary
Existing meteorological real-time analysis technologies cannot fully utilize the potential of multi-source data, especially in complex terrain areas, and are not very accurate and cannot effectively reflect subtle changes in small-scale meteorological real-time, resulting in challenges in refined meteorological disaster response.
Using a multi-source fusion meteorological real-time analysis method based on CNN-GCN neural network, a variety of meteorological data are integrated through a convolutional neural network, and a graph convolutional network is used to learn the spatial relationship between sites, combining attention mechanisms to fusion characteristics to generate high Resolution weather field.
It has achieved the integration of multi-source meteorological data to provide more comprehensive and continuous meteorological analysis, and can extract more details from low-resolution data, generate more refined meteorological results, ensuring accurate meteorological information is obtained in remote or undercover areas.
Smart Images

Figure CN118690253B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of meteorological models, and in particular to a multi-source fusion meteorological real-time analysis method based on a CNN-GCN neural network. Background Art
[0002] In the field of meteorology, real-time and accurate acquisition of live meteorological data plays an important role in weather forecasting, meteorological warning, disaster prevention, mitigation and relief, etc. At present, the acquisition of live meteorological data depends on observation equipment such as ground meteorological stations, radars and satellites. However, due to the limitations of the distribution of observation equipment, many areas are still blind spots for detection. Therefore, it is particularly important to infer the live meteorological information of those blind spots from the existing meteorological observation data.
[0003] Currently, the most widely used meteorological real-time analysis technology is mainly the interpolation method, which can estimate the meteorological conditions of unknown areas based on the data of known observation points. Common interpolation algorithms include bilinear interpolation, nearest neighbor interpolation, and Kriging interpolation. Bilinear interpolation calculates the value of the target point based on the four nearest data points around it, which is suitable for regular grid data processing; nearest neighbor interpolation estimates the meteorological state of the target point based on the value of the closest observation point; Kriging interpolation is an advanced interpolation method based on statistical models.
[0004] Although these methods are widely used today, they have certain limitations, such as the inability to fully utilize the potential of multi-source data and the lack of an effective mechanism to comprehensively analyze ground observations, satellite remote sensing and other meteorological data sources, which limits the multi-source fusion of data. Traditional interpolation methods such as nearest neighbor interpolation or linear interpolation cannot fully utilize the relationship between stations, especially in areas with complex terrain. They are often not accurate and cannot effectively reflect subtle changes in small-scale meteorological conditions. In addition, when dealing with data sources with lower resolution, traditional methods cannot effectively fill in data gaps, resulting in reduced prediction accuracy, which poses a challenge to refined meteorological disaster response. Summary of the invention
[0005] The purpose of the present invention is to provide a multi-source fusion meteorological real-time analysis method based on CNN-GCN neural network to solve the following technical problems:
[0006] It is impossible to fully utilize the potential of multi-source data and it is difficult to process large-scale data sets in real time. Especially in areas with complex terrain, the accuracy is often not high and it is unable to effectively reflect the subtle changes in small-scale meteorological conditions, which poses a challenge to the refined response to meteorological disasters.
[0007] The purpose of the present invention can be achieved through the following technical solutions:
[0008] A multi-source fusion meteorological real-time analysis method based on CNN-GCN neural network includes the following steps:
[0009] S1: Collect 5km grid meteorological data from different data sources every hour; the meteorological data includes temperature, rainfall, wind direction and speed, and humidity, and preprocess and standardize the meteorological data;
[0010] S2: Input the grid datasets from various data sources into the CNN network with multiple channels, design the convolution kernel size and step size, determine the number and depth of convolution layers, build the model convolution layer, extract the key features in the meteorological data, perform nonlinear mapping through the activation function ReLU, reduce the feature dimension through the pooling layer, build the fully connected layer to summarize the features and generate the downscaled output;
[0011] S3: Input the data of the meteorological observation sites into the GCN network, take the sites as nodes, the spatial distance between sites as edges, and the site data as node features to build a graph structure; design the graph convolution layer to capture the spatial relationship and feature dependency of the graph structure; determine the depth of the graph convolution network, build a multi-layer graph convolution structure, and use residual connections / skip connections to optimize the inter-layer connections. Use the DA-RE dataset as the training label, and calculate the root mean square error between the grid data output by GCN and the training label as the loss function;
[0012] S4: Based on the weighted sum method, the grid data features processed by CNN and the site data features processed by GCN are fused to build a CNN-GCN model. The fused features are processed through an additional convolutional layer, and the attention mechanism is introduced to dynamically adjust the weights of the output features of CNN and GCN.
[0013] S5: Construct training set and validation set, train the CNN-GCN model through the training set, calculate the gradient through back propagation, and update the parameters to minimize the loss function, evaluate the model performance and adjust it through the validation set.
[0014] As a further solution of the present invention: in S1, the data source includes real-time analysis of 5km resolution products, intelligent grid precipitation fusion CMPAS products, radar network hourly precipitation real-time estimation data, and meteorological automatic station hourly real-time data.
[0015] As a further solution of the present invention: In S1, the preprocessing includes converting the time zone of the meteorological data from universal time to Beijing time, adjusting all meteorological data sets to the same coordinate system, converting the Kelvin temperature into Celsius, and converting the horizontal wind speed and the vertical wind speed into average wind speed.
[0016] As a further solution of the present invention: in S1, the standardization process is to complete the missing files and elements and normalize the data from different data sources, and the formula is:
[0017] ;
[0018] Z represents any meteorological data, norm Represents the normalized expression of the meteorological data, Z max and Z min Indicates the maximum and minimum values of the same type of meteorological data.
[0019] As a further solution of the present invention: in S2, the specific training process of the CNN network is:
[0020] The formula for extracting key features from meteorological data is:
[0021] ;
[0022] Among them, K is the convolution kernel, B is the bias term, σ is the activation function, and F ij is the feature map obtained after convolution, i and j represent the position index in the feature map, where i represents the row index, j represents the column index, X represents the input data matrix, and m and n represent the elements in the input data that are offset by m rows and n columns relative to the current position (i, j);
[0023] The activation function is ReLU, which performs nonlinear mapping on the output of the convolutional layer. The formula is: ;
[0024] Where x represents the output value of the convolutional layer, and ReLU(x) represents the output value of the activation function;
[0025] The pooling layer is designed to reduce the feature dimension of the model, and the fully connected layer is constructed to summarize the pooled features to generate the downscaled output:
[0026] ;
[0027] Where W and b are the weights and biases of the fully connected layer, and P represents the output value of the pooling layer.
[0028] As a further solution of the present invention: in S3, the updating formula of the graph convolution layer is:
[0029] ;
[0030] Among them, H (l) represents the node features of the lth layer, H (l+1) represents the updated node features, , represents the adjacency matrix A plus self-connection, I N represents the identity matrix, D is the diagonal element of the degree matrix, W (l) is the weight of the layer and σ is the nonlinear activation function.
[0031] As a further solution of the present invention: in S5, the formula for updating the parameters is:
[0032] ; ;
[0033] Where η represents the learning rate, and They represent the partial derivatives of the loss function L with respect to the weight W and bias b respectively.
[0034] As a further solution of the present invention, it also includes visual display of meteorological data. When the user inputs the longitude and latitude information, the meteorological data of the corresponding location is displayed.
[0035] Beneficial effects of the present invention:
[0036] The present invention utilizes the CNN-GCN neural network to integrate multi-source meteorological data and provide more comprehensive and continuous meteorological analysis; the CNN-GCN model can partially learn the complex relationship between sites through the graph convolutional network, and utilize the powerful feature extraction ability of the deep learning model to extract more details from lower-resolution data and generate a high-resolution meteorological field, thereby providing more refined meteorological real-time results; the CNN-GCN neural network can receive any longitude and latitude input, predict the meteorological real-time data of the corresponding location, so that remote or insufficiently covered areas can also obtain accurate meteorological information; this is especially important for areas with complex terrain or sparse sites, ensuring that users can get the necessary meteorological support at any location. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The present invention will be further described below in conjunction with the accompanying drawings.
[0038] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION
[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] See also Figure 1As shown, the present invention is a multi-source fusion meteorological real-time analysis method based on CNN-GCN neural network, which aims to improve the downscaling processing accuracy and speed of offshore wind field data. Through this model, low-resolution meteorological wind field grid data is downscaled to high-resolution precise grids. This technical solution covers the entire process from data collection, model training, and system implementation, and introduces innovative technical improvements in each key step. The following is a preferred embodiment of the present invention, comprising the following steps:
[0041] 1. Data Collection:
[0042] Real-time analysis of 5km resolution products: hourly 5km grid data, such as temperature, rainfall, wind direction and speed, humidity and other elements.
[0043] Smart grid precipitation fusion CMPAS product: hourly 5km grid data rainfall data.
[0044] Radar network 1-hour precipitation real-time estimation data: hourly 5km grid data rainfall data.
[0045] Hourly real-time data of automatic meteorological stations: hourly station data, recording temperature, rainfall, wind direction and speed, humidity, etc.
[0046] 2. Data preprocessing:
[0047] Time alignment: Some data are uniformly converted from Universal Time to Beijing Time.
[0048] Latitude and longitude alignment: Ensure that all datasets are analyzed in the same coordinate system.
[0049] Temperature Unit Alignment: Convert Kelvin temperatures to Celsius.
[0050] Wind speed processing: Convert horizontal wind speed and vertical wind speed into average wind speed.
[0051] 3. Data cleaning and standardization:
[0052] Process missing files and elements to ensure data integrity and consistency. Standardize data to eliminate scale differences between different data sources and facilitate model processing. The formula for normalizing data from different data sources is:
[0053] ;
[0054] Z represents any meteorological data, norm Represents the normalized expression of the meteorological data, Z max and Z min Indicates the maximum and minimum values of the same type of meteorological data.
[0055] 4. CNN model training:
[0056] 1) Using the convolutional neural network (CNN) method to fuse multi-channel inputs of various types of grid datasets;
[0057] 2) Design the convolution kernel size and step size, determine the number and depth of convolution layers, build the model convolution layer, and extract the key features in the meteorological data. The specific formula is:
[0058] ;
[0059] Among them, K is the convolution kernel, B is the bias term, σ is the activation function, and F ij is the feature map obtained after convolution, i and j represent the position index in the feature map, where i represents the row index, j represents the column index, X represents the input data matrix, and m and n represent the elements in the input data that are offset by m rows and n columns relative to the current position (i, j);
[0060] 3) Select an appropriate activation function such as ReLU to perform nonlinear mapping on the output of the convolutional layer. The formula is:
[0061] ;
[0062] Here x represents the output value of the convolutional layer, and ReLU(x) represents the output value of the activation function.
[0063] 4) By designing the pooling layer, the feature dimension of the model is reduced, the abstract ability of the model is improved, and important meteorological feature information is retained;
[0064] 5) Construct a fully connected layer to summarize the features after convolution-activation-pooling and generate the downscaled output. The formula is:
[0065] ;
[0066] Where W and b are the weights and biases of the fully connected layer, and P represents the output value of the pooling layer.
[0067] 5. GCN model training:
[0068] 1) Use the data from the meteorological observation site as the input of the GCN model;
[0069] 2) Considering sites as nodes, the spatial distances between sites as edges, and site data as node features, a graph structure is constructed to describe the spatial features of grid data;
[0070] 3) Design the graph convolution layer to learn and update the feature representation of the node, and use the node features and adjacent nodes to update the node status, thereby capturing spatial relationships and feature dependencies. The formula is:
[0071] ;
[0072] Among them, H (l) represents the node features of the lth layer, H (l+1) represents the updated node features, , represents the adjacency matrix A plus self-connection, I N represents the identity matrix, D is the diagonal element of the degree matrix, W (l) is the weight of the layer and σ is the nonlinear activation function.
[0073] 4) Determine the depth of the graph convolutional network, build a multi-layer graph convolution structure, use residual connections or skip connections to optimize inter-layer connections, and avoid the gradient vanishing problem in deep networks.
[0074] 5) Using the DA-RE dataset as the training label, the root mean square error between the grid data output by the GCN model and DA-RE is calculated as the loss function.
[0075] 6. CNN-GCN connection:
[0076] 1) Feature fusion: The grid data features processed by the CNN model are fused with the site data features processed by the GCN model. This step is implemented through a feature fusion layer, in which the output features of the two models are combined using a weighted sum method.
[0077] 2) Feature transfer: The fused features are further processed through additional convolutional layers to enhance the correlation between features and reduce information loss.
[0078] 3) Dynamic adjustment: Introduce the attention mechanism to dynamically adjust the weights of the output features of CNN and GCN to optimize the quality and relevance of the final fused features.
[0079] 7. Model training and tuning:
[0080] According to the constructed CNN-GCN model, a large number of training sets are used for training. Based on the back propagation process, information is passed to the input layer layer by layer, the gradient of each layer parameter is calculated, and the parameters are updated. The formula is:
[0081] ; ;
[0082] Where η represents the learning rate, and Represent the partial derivatives of the loss function L with respect to the weight W and bias b, respectively. Adjust the model parameters to minimize the loss function. Use the validation set to evaluate the model performance and further adjust the model to improve the effect.
[0083] 8. System design:
[0084] System architecture: Design an efficient system architecture to support real-time data processing and model updates. The system should support high concurrent requests and be able to quickly respond to users' inquiries about weather conditions at any location.
[0085] User Interface: Develop a user-friendly user interface that allows users to easily input latitude and longitude information and quickly obtain relevant meteorological data.
[0086] Data security and privacy: Ensure that the system complies with data security and privacy protection standards when processing and storing data.
[0087] The problems of existing meteorological real-time analysis technology are mainly reflected in the following aspects:
[0088] Difficulty in data fusion: Traditional interpolation methods usually only target a single type of data source and lack an effective mechanism to comprehensively analyze ground observations, satellite remote sensing and other meteorological data sources, which limits the multi-source fusion of data.
[0089] Insufficient correlation between stations: Traditional interpolation methods such as nearest neighbor interpolation or linear interpolation cannot fully utilize the relationship between stations, especially in complex weather systems, and cannot accurately reflect the dynamic correlation between stations caused by changes in weather systems.
[0090] Resolution limitation: Traditional methods cannot effectively fill in the data gaps when processing data sources with lower resolution, resulting in reduced prediction accuracy. For example, bilinear interpolation works well in regular grids, but it is not ideal for situations where the distance between meteorological stations is uneven or data is missing.
[0091] The present invention aims to solve the above problems through the following aspects:
[0092] Improve data fusion capabilities: The CNN-GCN neural network can be used to integrate multi-source meteorological data, such as ground meteorological data, radar QPF data, etc., to provide more comprehensive and continuous meteorological analysis.
[0093] Enhance the correlation between sites: Traditional methods lack modeling of dynamic relationships between sites. The CNN-GCN model can partially learn the complex relationships between sites, such as the spatial dependence of wind direction and wind speed, through graph convolutional networks.
[0094] Breaking through the limitations of resolution: By leveraging the powerful feature extraction capabilities of deep learning models, more details can be extracted from lower-resolution data to generate high-resolution meteorological fields, thereby providing more refined meteorological real-time results.
[0095] Ability to obtain real-time information at any location: The CNN-GCN neural network can receive any longitude and latitude input and predict the real-time weather data at the corresponding location, so that remote or under-covered areas can also obtain accurate weather information. This is especially important for areas with complex terrain or sparse stations, ensuring that users can get the necessary meteorological support at any location.
[0096] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.
Claims
1. A multi-source fusion meteorological real-time analysis method based on CNN-GCN neural network, characterized in that: The following steps are involved: S1: Collect 5km grid meteorological data from different data sources every hour; the meteorological data includes temperature, rainfall, wind direction and speed, and humidity, and preprocess and standardize the meteorological data; S2: Input the grid datasets from various data sources into the CNN network with multiple channels, design the convolution kernel size and step size, determine the number and depth of convolution layers, build the model convolution layer, extract the key features in the meteorological data, perform nonlinear mapping through the activation function ReLU, reduce the feature dimension through the pooling layer, build the fully connected layer to summarize the features and generate the downscaled output; S3: Input the data of the meteorological real-time observation stations into the GCN network, take the stations as nodes, the spatial distances between stations as edges, and the station data as node features to build a graph structure; Design graph convolutional layers to capture the spatial relationships and feature dependencies of graph structures; Determine the depth of the graph convolutional network, build a multi-layer graph convolution structure, and use residual connections / skip connections to optimize inter-layer connections. Use the DA-RE dataset as the training label and calculate the root mean square error between the grid data output by GCN and the training label as the loss function. S4: Based on the weighted sum method, the grid data features processed by CNN and the site data features processed by GCN are fused to build a CNN-GCN model. The fused features are processed through an additional convolutional layer, and the attention mechanism is introduced to dynamically adjust the weights of the output features of CNN and GCN. S5: Construct training set and validation set, train the CNN-GCN model through the training set, calculate the gradient through back propagation, and update the parameters to minimize the loss function, evaluate the model performance and adjust it through the validation set.
2. According to claim 1, a multi-source fusion meteorological real-time analysis method based on CNN-GCN neural network is characterized in that: In S1, the data sources include real-time analysis products with a resolution of 5 km, intelligent grid precipitation fusion CMPAS products, radar network hourly precipitation real-time estimation data, and meteorological automatic station hourly real-time data.
3. The method for analyzing meteorological conditions based on a multi-source fusion CNN-GCN neural network according to claim 1 is characterized in that: In S1, the preprocessing includes converting the time zone of meteorological data from universal time to Beijing time, adjusting all meteorological data sets to the same coordinate system, converting Kelvin temperature to Celsius, and converting horizontal wind speed and vertical wind speed into average wind speed.
4. The method for analyzing meteorological conditions based on a multi-source fusion CNN-GCN neural network according to claim 1 is characterized in that: In S1, the standardization process is to complete the missing files and elements and normalize the data from different data sources. The formula is: ; Z represents any meteorological data, norm Represents the normalized expression of the meteorological data, Z max and Z min Indicates the maximum and minimum values of the same type of meteorological data.
5. The method for analyzing meteorological conditions based on a multi-source fusion CNN-GCN neural network according to claim 1 is characterized in that: In S2, the specific training process of the CNN network is: The formula for extracting key features from meteorological data is: ; Among them, K is the convolution kernel, B is the bias term, σ is the activation function, and F ij is the feature map obtained after convolution, i and j represent the position index in the feature map, where i represents the row index, j represents the column index, X represents the input data matrix, and m and n represent the elements in the input data that are offset by m rows and n columns relative to the current position (i, j); The activation function is ReLU, which performs nonlinear mapping on the output of the convolutional layer. The formula is: ; Where x represents the output value of the convolutional layer, and ReLU(x) represents the output value of the activation function; The pooling layer is designed to reduce the feature dimension of the model, and the fully connected layer is constructed to summarize the pooled features to generate the downscaled output: ; Where W and b are the weights and biases of the fully connected layer, and P represents the output value of the pooling layer.
6. The method for analyzing meteorological conditions based on a multi-source fusion CNN-GCN neural network according to claim 1 is characterized in that: In S3, the update formula of the graph convolution layer is: ; Among them, H (l) represents the node features of the lth layer, H (l+1) represents the updated node features, , represents the adjacency matrix A plus self-connection, I N represents the identity matrix, D is the diagonal element of the degree matrix, W (l) is the weight of the layer and σ is the nonlinear activation function.
7. The method for analyzing meteorological conditions based on a multi-source fusion CNN-GCN neural network according to claim 5 is characterized in that: In S5, the formula for updating the parameters is: ; ; Where η represents the learning rate, and They represent the partial derivatives of the loss function L with respect to the weight W and bias b respectively.
8. The method for analyzing meteorological conditions based on a multi-source fusion CNN-GCN neural network according to claim 1 is characterized in that: It also includes a visual display of meteorological data. When the user enters the longitude and latitude information, the meteorological data of the corresponding location will be displayed.
