End-to-end meteorological prediction large model
Through an end-to-end weather forecasting model, multimodal data fusion and edge computing are used to solve the problems of high computational complexity and lack of real-time performance of traditional weather forecasting models, and achieve high-precision and low-cost micro-regional weather forecasting.
Patent Information
- Application Number
- CN202510648845.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-20
- Publication Date
- 2025-09-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional meteorological forecasting models are computationally complex and time-consuming, making it difficult to meet real-time requirements. They also lack accuracy in micro-regional forecasts, especially for urban heat island effects and local severe convection phenomena, and rely on expensive supercomputer hardware.
It adopts an end-to-end meteorological forecasting model, directly accesses multi-source observation data through a multimodal data fusion module, a hierarchical physical constraint network and a lightweight inference engine, and combines edge computing nodes to achieve high spatiotemporal resolution forecasts, supporting 100-meter spatial grids and 5-minute updates.
The root mean square error of factors such as temperature and wind speed is reduced by 15%-30% under the 100-meter grid, and 5-minute updates are achieved, supporting real-time applications of power load scheduling and low-altitude logistics route planning, and reducing hardware costs.
Smart Images
Figure BDA0005410483570000091
Abstract
Description
Technical Field
[0001] The present invention relates to the field of grid node technology, and in particular to an end-to-end weather forecasting model. Background Art
[0002] Traditional weather forecasting relies on numerical models (such as WRF and ECMWF) and data assimilation techniques. This requires fusing observational data with model predictions to generate an initial field, which is computationally complex and time-consuming. Existing large-scale meteorological models (such as Pangu and Fengqing) perform well at the macroscale, but their prediction accuracy for microscopic phenomena such as the urban heat island effect and localized severe convection is insufficient, and they struggle to maintain a 100-meter spatial resolution and a 5-minute update frequency. Furthermore, traditional models rely on supercomputers, which limits their real-time and economical performance.
[0003] How to invent an end-to-end weather forecasting model to improve these problems has become an urgent problem to be solved by those skilled in the art. Summary of the Invention
[0004] In order to make up for the above shortcomings, the present invention provides an end-to-end meteorological forecasting model, which aims to improve the problems of high computational cost of traditional numerical models, inability to meet real-time requirements, difficulty in capturing micro-regional meteorological characteristics with macro-scale large models, reliance on expensive hardware for high spatiotemporal resolution forecasts, and high deployment costs.
[0005] The present invention is achieved in that:
[0006] The present invention provides an end-to-end weather forecasting model, comprising:
[0007] Multimodal data fusion module: Directly accesses raw observation data from satellites, radars, ground stations, etc., and realizes multi-source data feature fusion through the spatiotemporal attention mechanism, without the need for traditional data assimilation process;
[0008] Hierarchical physical constraint network: This includes a macro layer, a micro layer, and a physical constraint module. The macro layer uses a three-dimensional convolutional neural network to capture global atmospheric circulation characteristics, while the micro layer introduces a graph neural network to process complex boundary conditions such as terrain and urban underlying surfaces. The physical constraint module embeds a Hamiltonian conservation system and uses iterative energy conservation training to improve long-term forecast stability.
[0009] Lightweight inference engine: Based on model pruning and quantization technology, combined with edge computing nodes, it achieves 5-minute forecast updates and supports 100m×100m spatial grid output.
[0010] Preferably, the multimodal data fusion module preprocesses the original data through a self-supervised learning architecture, including reconstruction of missing data and noise suppression, wherein 30%-50% of the input data is randomly masked during the self-supervised training process to optimize model robustness.
[0011] Preferably, the graph neural network at the micro level constructs a graph structure including terrain grid nodes and urban building complex nodes. The node features include altitude, vegetation coverage, and building density, and the edge features include the distance between nodes and the thermal conductivity coefficient, which is used to model small-scale meteorological phenomena such as valley winds and urban heat island effects.
[0012] Preferably, the physical constraint module converts the atmospheric motion equation into an energy conservation loss function, and embeds the physical constraints into the model training through the Lagrange multiplier method to ensure that the prediction results meet the conservation of mass, momentum and energy.
[0013] Preferably, structured pruning and mixed precision quantization techniques are used to compress model parameters, combined with the distributed architecture of edge computing nodes to achieve a single-node inference delay of less than 30 seconds.
[0014] Preferably, it supports the input of multi-temporal observation data, captures the temporal evolution characteristics of meteorological elements through the temporal attention mechanism, and outputs high spatiotemporal resolution forecasts for the next 1-12 hours.
[0015] Preferably, in the low-altitude meteorological prediction scenario, the input data includes low-altitude sounding data with a vertical resolution of 100 meters and radar radial velocity data with a resolution of 500 meters, and the output includes a three-dimensional wind field prediction of the wind shear index at an altitude of 100 meters.
[0016] Preferably, in the power meteorological forecasting scenario, a correlation model between transmission line tower nodes and micro-meteorological elements is established through a graph neural network. The node features include tower height and insulator type, and the edge features include line span and conductor tension. The probability prediction of ice thickness and dancing amplitude is output.
[0017] Preferably, a multi-task learning architecture is adopted to simultaneously output forecasts of multiple factors such as temperature, wind speed, and precipitation. Each task branch shares the underlying feature extraction network, and the accuracy optimization of each factor is achieved through task-specific heads.
[0018] Preferably, the inference stage supports the collaborative deployment of edge computing nodes and the cloud. The edge nodes are responsible for real-time data processing and short-term forecasting, and the cloud is responsible for periodic model retraining and parameter updates, forming an "edge-cloud" closed-loop optimization system.
[0019] The beneficial effects of the present invention are:
[0020] 1. Improved accuracy: On a 100-meter grid, the root mean square error (RMSE) of factors such as temperature and wind speed is reduced by 15%-30% compared to traditional models.
[0021] 2. Enhanced real-time performance: 5-minute updates support real-time applications such as power load scheduling and low-altitude logistics route planning.
[0022] 3. Economic optimization: Based on edge computing node deployment, no supercomputer is required, reducing hardware costs. DETAILED DESCRIPTION
[0023] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0024] Example 1
[0025] Low-altitude weather forecast:
[0026] 1. Data input and preprocessing,
[0027] Satellite data: Sunflower-8 visible / infrared cloud image (1 km resolution), cloud top height and water vapor content inverted through radiation transfer model.
[0028] Radar data: C-band weather radar echoes (500-m resolution). After deblurring, radial velocity and reflectivity factors were extracted, and echo sequences were generated using a spatiotemporal sliding window (window size: 5 × 5 × 3, time step size: 3 minutes).
[0029] Low-altitude sounding data: 100-meter vertical layered data (0-1000 meters, 10 layers in total) detected by tethered balloons, including temperature, humidity, wind speed, and wind direction. Random noise is removed through Kalman filtering.
[0030] Preprocessing process: Time alignment (unified to UTC hour), spatial projection (WGS84 coordinate system to UTM projection) of multi-source data, and normalization to the [-1, 1] interval.
[0031] 2. Model training details,
[0032] Twin MAE structure: The encoder uses a 12-layer Transformer, the decoder is a lightweight CNN, the masking strategy is to randomly select 20% of the spatial area and 10% of the time steps for masking, the reconstruction target is the normalized value of the original data, and the loss function is L1 loss.
[0033] Physical constraint module: In the late stage of training (after epoch 50), Navier-Stokes equation constraints are introduced to construct penalty terms by calculating the divergence and vorticity of the predicted wind field. Where λ = 0.01 is the constraint weight.
[0034] Distributed training: A cluster of 8 NVIDIA A100 GPUs was used, using the PyTorch Distributed Data Parallel (DDP) framework. The batch size was set to 32, the training period was 100 epochs, the learning rate was initialized to 1e-4, and the cosine annealing decay was set to 1e-6.
[0035] 3. Reasoning and application,
[0036] Output products:
[0037] 3D wind field (100m x 100m x 100m grid, 0-1000m vertical stratification). Wind shear index (areas with vertical wind shear > 0.1s-1 are marked).
[0038] Temperature field (100m altitude layer, accuracy ±0.5℃).
[0039] Real-time deployment: The edge computing node uses NVIDIA Jetson AGX Orin (64GB memory, 48TOPS computing power), which receives the latest 30 minutes of observation data every 5 minutes. The inference takes 22 seconds, and the results are pushed to the airport air traffic control system via the MQTT protocol.
[0040] Actual results: In tests at Shenzhen Bao'an Airport, the accuracy of 2-hour forecasts for low-level jet streams (below 500 meters in altitude, wind speed >12m / s) reached 89%, a 35% improvement over traditional LIDAR radar inversion methods.
[0041] Example 2
[0042] Power weather forecast:
[0043] 1. Data input and feature engineering,
[0044] Basic meteorological data:
[0045] Ground weather station data: temperature (±0.1°C), humidity (±1%RH), wind speed (±0.2m / s), with an interval of 10 minutes.
[0046] Millimeter-wave radar rainfall data: 500-meter resolution precipitation intensity field, and three-dimensional wind field inversion using the VDRAS algorithm.
[0047] Transmission line data:
[0048] Tower inventory data: longitude, latitude, altitude, nominal height, insulator string type (porcelain insulator / composite insulator).
[0049] Online monitoring data: conductor tension (accuracy ±1%), insulator leakage current (resolution 1mA), ice coverage image (captured every 5 minutes).
[0050] Topographic data: SRTM 1-second DEM data (about 30-meter resolution), which is enhanced to 100-meter resolution through bilinear interpolation to extract slope, aspect, and roughness parameters.
[0051] 2. Graph Neural Network (GNN) modeling,
[0052] Graph structure construction:
[0053] Node: Each tower is a node, and the feature vector contains [altitude, slope, temperature, humidity, wind speed, tension, leakage current].
[0054] Edge: Undirected edges are established between adjacent towers, with features including [span, height difference, conductor type, and historical ice coverage times].
[0055] GNN layer design: A two-layer Graph Convolutional Network (GCN) is used, each layer is followed by a ReLU activation function and Dropout (ratio 0.2), and the output layer is mapped to the probability distribution of ice thickness (mm) and dancing amplitude (cm) through a fully connected layer.
[0056] 3. Training and Validation
[0057] Sample labeling: The ice thickness is manually annotated through drone inspection images (threshold: >5mm is a warning state), and positive and negative samples are generated based on meteorological conditions (positive samples: data 12 hours before ice cover occurs, negative samples: data during non-ice cover periods).
[0058] Loss function: multi-task loss L = L_regression (MSE) + L_classification (cross entropy), with a weight ratio of 1:1.5.
[0059] Transfer learning: The model was pre-trained using ice cover data from high-latitude regions around the world (such as Northern Europe and Canada), and then fine-tuned for data from North China (such as the Tianjin power grid), increasing convergence speed by 40%.
[0060] 4. Business applications,
[0061] Early warning mechanism: When the model predicts that the ice thickness is greater than 10mm and the probability is greater than 90%, a level 1 early warning is triggered and the DC ice melting device is activated.
[0062] Economic benefits: In the Tianjin 220kV Jibao Line, the accuracy of ice overlay warnings has increased from 72% using the traditional threshold method (based on temperature, humidity, and wind speed) to 95%, reducing unnecessary ice melting by approximately 15 times per year and saving approximately 3 million kWh of electricity.
[0063] Example 3
[0064] Agricultural weather forecast:
[0065] 1. Data input,
[0066] Doppler radar basic data (1km resolution): radial velocity, spectral width, reflectivity factor.
[0067] Soil moisture data: 0-20cm soil layer moisture (TDR sensor, accuracy ±3%).
[0068] Crop growth data: Crop coverage is inverted using the Sentinel-2 satellite NDVI index.
[0069] 2. Model optimization,
[0070] The micro-layer GNN introduces farmland grid nodes, and the edge features include crop type (corn / wheat) and irrigation method (sprinkler / drip irrigation).
[0071] The physical constraint module adds boundary layer equation constraints to simulate the impact of farmland thermal flux on convection triggering.
[0072] 3. Prediction results,
[0073] Output the movement path of thunderstorm cells and the falling area of short-term heavy rainfall (>20mm / h) in a 100-meter grid.
[0074] In the Zhoukou farmland experimental area in Henan Province, the accuracy of the one-hour advance warning for local short-term severe convection reached 81%, an increase of 28% compared with the traditional numerical model.
[0075] Experimental verification
[0076] 1. Datasets: Global Reanalysis Data (ERA5), China Meteorological Administration intensive observation data, and industry measured data;
[0077] 2. Evaluation indicators:
[0078] Spatial resolution: 100 m × 100 m (traditional model 1 km × 1 km);
[0079] Update frequency: 5 minutes (1 hour for traditional models);
[0080] 3. Accuracy comparison:
[0081]
[0082] The present invention adopts end-to-end data processing: skipping the data assimilation link, reducing the accumulation of intermediate errors, and directly improving the robustness of data processing through self-supervised learning. High-precision prediction of micro-regions: GNN is combined with terrain data to improve the prediction accuracy of low-altitude meteorology (such as airport wind shear) and power meteorology (such as icing on transmission lines) by more than 20%. Ultra-high spatiotemporal resolution: supports 100-meter spatial grids and 5-minute updates to meet real-time dynamic scheduling needs. That is, through multimodal data fusion, hierarchical physical constraints and lightweight inference engines, it breaks through the bottlenecks of traditional models in micro-regional accuracy and real-time performance, and provides efficient solutions for low-altitude economy, power scheduling and other fields.
[0083] The above is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.
Claims
1. An end-to-end weather forecasting model, characterized by: include: Multimodal data fusion module: directly accesses raw observation data from satellites, radars, ground stations, etc., and realizes multi-source data feature fusion through the spatiotemporal attention mechanism, without the need for traditional data assimilation process; Hierarchical physical constraint network: This includes a macro layer, a micro layer, and a physical constraint module. The macro layer uses a three-dimensional convolutional neural network to capture global atmospheric circulation characteristics, while the micro layer introduces a graph neural network to process complex boundary conditions such as terrain and urban underlying surfaces. The physical constraint module embeds a Hamiltonian conservation system and uses iterative energy conservation training to improve long-term forecast stability. Lightweight inference engine: Based on model pruning and quantization technology, combined with edge computing nodes, it achieves 5-minute forecast updates and supports 100m×100m spatial grid output.
2. The end-to-end weather forecasting model according to claim 1, characterized in that: The multimodal data fusion module preprocesses the raw data through a self-supervised learning architecture, including reconstruction of missing data and noise suppression, wherein 30%-50% of the input data is randomly masked during the self-supervised training process to optimize model robustness.
3. The end-to-end weather forecasting model according to claim 1, characterized in that: The micro-layer graph neural network constructs a graph structure including terrain grid nodes and urban building complex nodes. Node features include altitude, vegetation coverage, and building density, and edge features include inter-node distance and thermal conductivity coefficient, which is used to model small-scale meteorological phenomena such as valley winds and urban heat island effects.
4. The end-to-end weather forecasting model according to claim 1, characterized in that: The physical constraint module converts the atmospheric motion equation into an energy conservation loss function, and embeds the physical constraints into the model training through the Lagrange multiplier method to ensure that the prediction results meet the conservation of mass, momentum and energy.
5. The end-to-end weather forecasting model according to claim 1, characterized in that: Structured pruning and mixed precision quantization technologies are used to compress model parameters, and combined with the distributed architecture of edge computing nodes, single-node inference latency can be reduced to less than 30 seconds.
6. The end-to-end weather forecasting model according to claim 1, characterized in that: It supports the input of multi-temporal observation data, captures the temporal evolution characteristics of meteorological elements through the temporal attention mechanism, and outputs high spatiotemporal resolution forecasts for the next 1-12 hours.
7. The end-to-end weather forecasting model according to claim 1, characterized in that: In the low-altitude meteorological forecast scenario, the input data includes low-altitude sounding data with a vertical resolution of 100 meters and radar radial velocity data with a resolution of 500 meters, and the output includes a three-dimensional wind field forecast containing the wind shear index at an altitude of 100 meters.
8. The end-to-end weather forecasting model according to claim 1, characterized in that: In the power meteorological forecasting scenario, a correlation model between transmission line tower nodes and micro-meteorological elements is established through graph neural networks. Node features include tower height and insulator type, and edge features include line span and conductor tension. The output is the probabilistic prediction of ice thickness and dancing amplitude.
9. The end-to-end weather forecasting model according to claim 1, characterized in that: A multi-task learning architecture is adopted to simultaneously output forecasts of multiple factors such as temperature, wind speed, and precipitation. Each task branch shares the underlying feature extraction network and achieves factor-by-factor accuracy optimization through task-specific heads.
10. The end-to-end weather forecasting model according to claim 1, characterized in that: The inference stage supports the coordinated deployment of edge computing nodes and the cloud. The edge nodes are responsible for real-time data processing and short-term forecasts, while the cloud is responsible for periodic model retraining and parameter updates, forming an "edge-cloud" closed-loop optimization system.