Three-dimensional atmosphere live field fusion product generation method, device and equipment based on machine learning

Abstract

The invention discloses a three-dimensional atmosphere live field fusion product generation method, device and equipment based on machine learning. The method comprises the steps of background field downscaling, observation data preprocessing, observation data correction, three-dimensional live analysis and product generation. According to the method, the space texture generation capability of the GAN and the long-distance dependence modeling of the Transform are fused, hectometer-level resolution reconstruction of three-dimensional meteorological elements is realized, and system errors of different observation systems are eliminated based on an MLP correction model of a sounding benchmark. According to the method, three-dimensional space lattice point visualization of multiple meteorological elements such as temperature, humidity, wind field and radar reflectivity factors is achieved in a breakthrough mode, comprehensive deployment is achieved, deep fusion with an existing service system is achieved, the meteorological data visualization capability is remarkably improved, emerging application scenes such as disaster prevention and reduction and low-altitude economy are emphatically expanded, and the method has a wide application prospect. Value potential of meteorological data in the fields of industry enabling, disaster early warning and the like is deeply mined, and innovative kinetic energy is continuously injected for high-quality development of new-quality productivity enabling meteorological undertakings.

Description

Technical Field

[0001] This invention relates to the field of meteorological data processing technology, and in particular to a method, apparatus and equipment for generating three-dimensional atmospheric field fusion products based on machine learning. Background Technology

[0002] Currently, the mainstream approach to 3D atmospheric data still relies on numerical model assimilation and inversion. Numerical forecasting technology has laid the cornerstone of modern atmospheric science. However, constrained by factors such as models, initial values, and the instability of nonlinear systems like the atmosphere and ocean, numerical forecasting has always struggled to overcome the limitation of "predictability." At present, artificial intelligence is developing rapidly. Using machine learning, parameterization schemes are trained based on specific geographical locations and environmental conditions. The parameterization process is trained according to different terrains, locations, and structures, which can quickly replace the current experience-based and semi-experience-based schemes of numerical models. This study utilizes high-precision observation network data from Guizhou Province to construct a refined 3D fusion field with 10-minute intervals at the 100-meter level using a data-driven approach. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] To address the shortcomings of existing technologies, the method of this invention overcomes the predictability limitations of numerical weather prediction, enabling the generation of three-dimensional atmospheric real-time fields at the 10-minute interval (hundred meters per minute), supporting disaster early warning and low-altitude economic applications.

[0005] (II) Technical Solution

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for generating three-dimensional atmospheric field fusion products based on machine learning, comprising the following steps:

[0007] Step S1, Background Field Downscaling: Obtain the grib2 format grid background field and high-precision terrain data, and match the coarse-resolution model forecast field with the terrain data through bilinear or cubic spline interpolation to generate a high spatiotemporal resolution initial guess field.

[0008] Step S2, observation data preprocessing: Perform extreme value checks, time-varying checks, and consistency checks on ground, radiosonde, wind profiler, radar single-station, satellite, and microwave radiometer data to construct a unified data structure;

[0009] Step S3, Observational data correction: Based on the radiosonde data, a machine learning model is used to normalize and correct the microwave radiometer and wind profiler radar data.

[0010] Step S4, 3D Real-Time Analysis: Model and train relevant basic elements such as temperature, humidity, and wind, analyze and optimize the results to form the optimal model;

[0011] Step S5, Product Generation: Encode the analysis results according to NetCDF's CF convention and grib2 format, and output three-dimensional temperature, humidity, wind field and radar reflectivity factor products, and generate diagnostic data simultaneously.

[0012] As a preferred embodiment, the terrain matching in step S1 specifically includes: parsing the terrain data with the highest precision, converting it into a static format of the fusion system, and aligning the background field with the vertical structure of the terrain through interpolation.

[0013] As a preferred embodiment, the correction in step S3 includes: temperature and humidity profile correction using a neural network / random forest / support vector regression model; wind speed and direction correction using a gradient boosting tree classification model; specifically:

[0014] S31. Divide microwave radiometer and wind profiler radar data with co-located sounding data into a 70% training set, a 15% validation set, and a 15% test set.

[0015] S32. Optimize model hyperparameters based on root mean square error (RMSE), mean bias (Bias), and correlation coefficient (R) of the validation set.

[0016] As a preferred embodiment, the model in step S4 includes:

[0017] The super-resolution downscaling model based on GAN consists of two main parts: a generator and a discriminator.

[0018] Based on the Transformer super-resolution model, for meteorological data, the Transformer can capture the long-distance dependencies between different meteorological elements;

[0019] The MLP-based fusion correction model further fuses observation data on the basis of previous super-resolution downscaling to generate more accurate high-resolution feature data.

[0020] As a preferred option, in step S4, the Transformer model processes meteorological data at different altitudes through a multi-head attention mechanism and uses location coding to accurately resolve spatial relationships in latitude, longitude, and vertical direction.

[0021] As a preferred embodiment, the product in step S5 includes:

[0022] A 3D live field with 100-meter resolution and updates every 10 minutes;

[0023] Products with hourly temperature / humidity / wind field and minute-level radar reflectivity factor.

[0024] A machine learning-based 3D atmospheric field fusion product generation device includes:

[0025] Background field downscaling module, used for terrain matching and interpolation processing;

[0026] The observation preprocessing module enables quality control and structured storage of multi-source data;

[0027] The data correction module performs vertical observation correction based on machine learning.

[0028] The 3D analysis module deploys GAN, Transformer, and MLP models.

[0029] The product creation module generates standardized grid products and diagnostic data.

[0030] As a preferred option, the device employs a hybrid storage system of Virtual Valley Database and Gbase for structured / semi-structured data, and manages data streams through an MTV architecture, which includes:

[0031] The model layer handles database modeling;

[0032] The view layer manages the user interface display;

[0033] The controller layer coordinates user interaction responses.

[0034] The machine learning-based 3D atmospheric field fusion product generation device includes a computer device storing program instructions, which, when executed, implement the machine learning-based 3D atmospheric field fusion product generation method.

[0035] As a preferred option, it also includes a computer-readable storage medium storing a three-dimensional atmospheric field fusion product, including temperature, humidity, wind speed, wind direction, and radar reflectivity factor data in a gridded manner.

[0036] (III) Beneficial Effects

[0037] Compared with existing technologies, this invention provides a method, apparatus, and equipment for generating three-dimensional atmospheric field fusion products based on machine learning, which has the following beneficial effects:

[0038] I. This invention achieves 100-meter-level resolution reconstruction of 3D meteorological elements by integrating the spatial texture generation capabilities of GANs with the long-range dependency modeling of Transformers. Furthermore, it eliminates systematic errors from different observation systems using an MLP correction model based on radiosonde benchmarks. This method achieves breakthrough visualization of multiple meteorological elements, including temperature, humidity, wind field, and radar reflectivity factor, in 3D spatial grid format. It has been fully deployed and deeply integrated with existing operational systems, significantly enhancing meteorological data visualization capabilities. It focuses on expanding into emerging application scenarios such as disaster prevention and mitigation, and the low-altitude economy, deeply exploring the value potential of meteorological data in areas such as industrial empowerment and disaster early warning, and continuously injecting innovative momentum into the high-quality development of meteorological services. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the method flow and equipment structure modules of the present invention;

[0040] Figure 2 This is a schematic diagram of the technical process of the method of the present invention;

[0041] Figure 3 This is a schematic diagram of the data processing flow of the present invention. Detailed Implementation

[0042] To better understand the purpose, structure, and function of this invention, the following description, in conjunction with the accompanying drawings and specific embodiments, will further illustrate the method, apparatus, and equipment for generating three-dimensional atmospheric real-time field fusion products based on machine learning.

[0043] Example 1

[0044] refer to Figure 1-3 The present invention provides a method for generating three-dimensional atmospheric field fusion products based on machine learning, comprising the following steps:

[0045] Step S1, Background Field Downscaling: Obtain the grib2 format grid background field and high-precision terrain data, and match the coarse-resolution model forecast field with the terrain data through bilinear or cubic spline interpolation to generate a high spatiotemporal resolution initial guess field.

[0046] Step S2, observation data preprocessing: Perform extreme value checks, time-varying checks, and consistency checks on ground, radiosonde, wind profiler, radar single-station, satellite, and microwave radiometer data to construct a unified data structure;

[0047] Step S3, Observational data correction: Based on the radiosonde data, a machine learning model is used to normalize and correct the microwave radiometer and wind profiler radar data.

[0048] Step S4, 3D Real-Time Analysis: Model and train relevant basic elements such as temperature, humidity, and wind, analyze and optimize the results to form the optimal model;

[0049] Step S5, Product Generation: Encode the analysis results according to NetCDF's CF convention and grib2 format, and output three-dimensional temperature, humidity, wind field and radar reflectivity factor products, and generate diagnostic data simultaneously.

[0050] Specifically, in step S1 of this invention, terrain matching includes: parsing the terrain data with maximum accuracy, converting it into a static format of the fusion system, and aligning the background field with the vertical structure of the terrain through interpolation; step S3, correction, includes: temperature and humidity profile correction using a neural network / random forest / support vector regression model; wind speed and direction correction using a gradient boosting tree classification model; specifically:

[0051] S31. Divide microwave radiometer and wind profiler radar data with co-located sounding data into a 70% training set, a 15% validation set, and a 15% test set.

[0052] S32. Optimize model hyperparameters based on root mean square error (RMSE), mean bias (Bias), and correlation coefficient (R) of the validation set.

[0053] Furthermore, the model in step S4 of this invention includes:

[0054] The super-resolution downscaling model based on GAN consists of two main parts: a generator and a discriminator.

[0055] Based on the Transformer super-resolution model, for meteorological data, the Transformer can capture the long-distance dependencies between different meteorological elements;

[0056] The MLP-based fusion correction model, building upon previous super-resolution downscaling, further fuses observational data to generate more accurate high-resolution feature data. In step S4, the Transformer model processes meteorological data at different altitudes using a multi-head attention mechanism and employs location coding to accurately resolve spatial relationships in latitude, longitude, and vertical directions.

[0057] This invention fully utilizes data from the high-precision observation network of Guizhou Province to construct a refined 3D fusion field at the 10-minute interval (hundred meters per minute). A real-time quality control process for multi-source data is established. The core algorithm employs artificial intelligence to fuse the quality-controlled observations with the background field, obtaining an analysis field at a specified resolution. Real-time 3D reconstruction technology is implemented, dynamically generating spatial meteorological element products through algorithms. On-demand services are generated, including hourly-level temperature, humidity, wind speed, wind direction, and minute-level reflectivity factors. Real-time tracking and 3D visualization analysis of weather processes' occurrence, development, and dissipation are conducted, providing AI-powered data services based on reinforcement learning. The fused analysis field or dynamic downscaling field is interpolated horizontally and vertically to be processed onto a consistent latitude and longitude grid, providing users with standardized product files. The products in step S5 include:

[0058] A 3D live field with 100-meter resolution and updates every 10 minutes;

[0059] Products with hourly temperature / humidity / wind field and minute-level radar reflectivity factor.

[0060] This invention employs a combination of the Xugu database and a portion of Gbase for storage, achieving an efficient hybrid storage model for structured, semi-structured, and unstructured data. Collected and processed data is stored according to a unified information database object model and storage management specifications. Accurate and comprehensive basic data is pushed or synchronized to users for further applications, ensuring the reliability and consistency of basic data across modules. This meets the requirement of a robust and mature database management system for the large amount of management data generated in multi-source fusion real-time field analysis systems.

[0061] Example 2

[0062] The present invention also includes a machine learning-based three-dimensional atmospheric field fusion product generation device, which includes:

[0063] Background field downscaling module, used for terrain matching and interpolation processing;

[0064] The observation preprocessing module enables quality control and structured storage of multi-source data;

[0065] The data correction module performs vertical observation correction based on machine learning.

[0066] The 3D analysis module deploys GAN, Transformer, and MLP models.

[0067] The product creation module generates standardized grid products and diagnostic data.

[0068] Specifically, in this embodiment:

[0069] Background field downscaling module

[0070] Background field downscaling primarily considers the vertical structure of near-surface meteorological elements and their matching with actual terrain to obtain a more refined preliminary field that more closely approximates reality. The module collects gridded background field data in grib2 format, performs analysis and preprocessing. It collects the highest-precision terrain data available from within Guizhou Province, decodes it, and writes it into the static data format required by the fusion system. Using bilinear or cubic spline methods, the read high-resolution terrain and coarse-resolution model forecast fields are interpolated to a higher spatiotemporal resolution model background field.

[0071] Observation preprocessing module

[0072] The observation data preprocessing subsystem primarily organizes different observation data, designs appropriate data structures to store the data, avoids generating intermediate files, improves processing efficiency, simplifies program interfaces, and performs quality control before fusion. The module addresses issues such as missing measurements, errors, consistency errors, abrupt disturbances, and measurement errors in collected observation data from ground, radiosonde, wind profilers, radar single stations, radar mosaics, satellites, and microwave radiometers by performing extreme value checks, time-varying checks, and consistency checks.

[0073] Data correction module

[0074] Considering the systematic bias of the new ground-based vertical remote sensing system, it is necessary to correct the vertical observation data in advance. It is proposed to use radiosonde data as a benchmark to correct the microwave radiometer and wind profiler radar to ensure the consistency of observation data from different systems.

[0075] Data from microwave radiometers and wind profiler radars located at the same site as the radiosonde were used. The data was normalized to ensure it was on the same scale, facilitating model training. The data was divided into training, validation, and test sets (e.g., 70% training, 15% validation, 15% test). Temperature and humidity profile corrections used regression models (e.g., neural networks, random forests, support vector regression), while wind speed and direction corrections used regression or classification models (e.g., neural networks, gradient boosting trees). The validation set was used to evaluate model performance, calculating metrics such as root mean square error (RMSE), mean bias, and correlation coefficient (R). If model performance was insufficient, the model structure or hyperparameters were adjusted. In real-time data processing, the trained model was used to correct the microwave radiometer and wind profiler radar data in real time.

[0076] 3D Analysis Module

[0077] This invention aims to develop a spatial super-resolution downscaling scheme for conventional meteorological elements by utilizing deep learning algorithms and combining them with the characteristics of basic meteorological elements. It involves modeling and training relevant basic elements such as temperature, humidity, and wind, analyzing and optimizing the results to form an optimal model, thus preparing for subsequent system integration. It includes:

[0078] 1) GAN-based super-resolution downscaling model

[0079] Generative Adversarial Networks (GANs) have unique advantages in super-resolution downscaling tasks. A GAN consists of two main parts: a generator and a discriminator.

[0080] When performing super-resolution downscaling on data such as CRA5 and Guizhou Province gridded data products, the generator's role is to convert low-resolution meteorological data into high-resolution data. It achieves this by learning the mapping relationship between low-resolution data and the corresponding high-resolution data.

[0081] The discriminator is responsible for distinguishing the high-resolution data generated by the generator from real high-resolution data. It determines the data's authenticity by evaluating various features, such as statistical distribution and spatial texture. During training, the generator and discriminator compete and learn from each other. The generator continuously optimizes itself to generate more realistic high-resolution data to fool the discriminator, while the discriminator continuously improves its discrimination ability. As training progresses, the quality of the high-resolution data generated by the generator continuously improves, gradually approaching that of real high-resolution data.

[0082] 2) Transformer-based super-resolution model

[0083] The Transformer model, with its powerful feature extraction and global information processing capabilities, has also demonstrated great potential in the field of super-resolution downscaling. For meteorological data, the Transformer can capture long-distance dependencies between different meteorological elements. In its model structure, the Transformer's multi-head attention mechanism can simultaneously focus on different feature dimensions of the data. When performing super-resolution processing on gridded data, it can comprehensively analyze the features of meteorological data at different altitude levels. Furthermore, the Transformer's positional encoding can effectively handle spatial location information in meteorological data. Whether on a planar latitude and longitude coordinate system or in the vertical direction, positional encoding allows the model to accurately understand the spatial relationships of meteorological elements, thereby generating meteorological element data with correct spatial distribution characteristics more precisely during the super-resolution downscaling process.

[0084] 3) MLP-based fusion correction model

[0085] Building upon the previous super-resolution downscaling, observational data are further integrated to generate more accurate high-resolution feature data;

[0086] First, data from different data sources, processed by a super-resolution downscaling model, along with other relevant feature data, are used as input to the MLP. The MLP performs deep feature extraction and information fusion on these input data through multiple hidden layers. In the hidden layers, the connection weights between neurons are continuously adjusted during training to learn the optimal fusion strategy between different data sets. Specifically, based on experimental data, the MLP can fuse it with high-resolution data to form the final product.

[0087] Product creation module

[0088] Following the NetCDF format's CF conventions and grib2 format encoding rules, the ground analysis results were re-extracted and organized to obtain product files in NetCDF and grib2 formats conforming to the CF conventions. Simultaneously, diagnostic data for the fusion analysis was generated, including OB, OA, data acquisition details for a single fusion, and convergence results in the core minimization algorithm. This information provides auxiliary data support for better utilizing the analysis field and further improvements.

[0089] The device system of this invention uses an MTV architecture. Models, the model layer, is responsible for database modeling; Views, the view layer, is used to handle the content displayed to the user, such as HTML; Controller, the controller, handles the part of the content that interacts with the user and provides responses.

[0090] Because the system uses and generates a large amount of data during operation, this data includes: observation data from vertical equipment such as microwave radiometers, cloud radar, lidar, and wind profiler radar; other external business data; real-time field data on temperature, humidity, wind, and water condensate fusion generated by the system; and secondary product data generated by algorithms. Overall, the data types are numerous and diverse, with inconsistent formats and levels of standardization. Therefore, a robust data management solution is needed to effectively manage this data. Consequently, the system adopts a hybrid management approach using Xugu and MongoDB to centrally manage the data involved.

[0091] The system design adopts a technical approach of B / S + MTV + Xugu + Gbase, which has significant advantages in its architecture and integration process. Each of these technologies is currently the most advanced, and their integration can maximize their advantages.

[0092] Example 3

[0093] The present invention also includes a computer device storing program instructions that, when executed, implement a method for generating a three-dimensional atmospheric field fusion product based on machine learning. It also includes a computer-readable storage medium storing the three-dimensional atmospheric field fusion product, which includes temperature, humidity, wind speed, wind direction, and radar reflectivity factor data in a gridded manner.

[0094] This invention achieves breakthrough visualization of multiple meteorological elements, including temperature, humidity, wind field, and radar reflectivity factor, using a three-dimensional spatial grid. It has been fully deployed and deeply integrated with existing operational systems, significantly enhancing meteorological data visualization capabilities. The invention also focuses on expanding into emerging application scenarios such as disaster prevention and mitigation, and the low-altitude economy, deeply exploring the value potential of meteorological data in areas such as industrial empowerment and disaster early warning, and continuously injecting innovative momentum into the high-quality development of the meteorological industry to empower new productive forces.

[0095] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for generating 3D atmospheric field fusion products based on machine learning, characterized in that, Includes the following steps: Step S1, Background Field Downscaling: Obtain the grib2 format grid background field and high-precision terrain data, and match the coarse-resolution model forecast field with the terrain data through bilinear or cubic spline interpolation to generate a high spatiotemporal resolution initial guess field. Step S2, observation data preprocessing: Perform extreme value checks, time-varying checks, and consistency checks on ground, radiosonde, wind profiler, radar single-station, satellite, and microwave radiometer data, and construct a unified data structure; Step S3, Observational data correction: Based on the radiosonde data, a machine learning model is used to normalize and correct the microwave radiometer and wind profiler radar data. Step S4, 3D Real-Time Analysis: Model and train relevant basic elements such as temperature, humidity, and wind, analyze and optimize the results to form the optimal model; Step S5, Product Generation: Encode the analysis results according to NetCDF's CF convention and grib2 format, and output three-dimensional temperature, humidity, wind field and radar reflectivity factor products, and generate diagnostic data simultaneously.

2. The method for generating three-dimensional atmospheric field fusion products based on machine learning according to claim 1, characterized in that, The terrain matching in step S1 specifically includes: parsing the terrain data with the highest precision, converting it into a static format of the fusion system, and aligning the background field with the vertical structure of the terrain through interpolation.

3. The method for generating three-dimensional atmospheric field fusion products based on machine learning according to claim 1, characterized in that, The correction in step S3 includes: temperature and humidity profile correction using a neural network / random forest / support vector regression model; wind speed and direction correction using a gradient boosting tree classification model; specifically: S31. Divide microwave radiometer and wind profiler radar data with co-located sounding data into a 70% training set, a 15% validation set, and a 15% test set. S32. Optimize model hyperparameters based on root mean square error (RMSE), mean bias (Bias), and correlation coefficient (R) of the validation set.

4. The method for generating three-dimensional atmospheric field fusion products based on machine learning according to claim 1, characterized in that, The model in step S4 includes: The super-resolution downscaling model based on GAN consists of two main parts: a generator and a discriminator. Based on the Transformer super-resolution model, for meteorological data, the Transformer can capture the long-distance dependencies between different meteorological elements; The MLP-based fusion correction model further fuses observation data on the basis of previous super-resolution downscaling to generate more accurate high-resolution feature data.

5. The method for generating three-dimensional atmospheric field fusion products based on machine learning according to claim 4, characterized in that, In step S4, the Transformer model processes meteorological data at different altitudes through a multi-head attention mechanism and uses location coding to accurately resolve spatial relationships in latitude, longitude, and vertical direction.

6. The method for generating three-dimensional atmospheric field fusion products based on machine learning according to claim 1, characterized in that, The products in step S5 include: A 3D live field with 100-meter resolution and updates every 10 minutes; Products with hourly temperature / humidity / wind field and minute-level radar reflectivity factor.

7. A machine learning-based three-dimensional atmospheric field fusion product generation device, characterized in that... ,include: Background field downscaling module, used for terrain matching and interpolation processing; The observation preprocessing module enables quality control and structured storage of multi-source data; The data correction module performs vertical observation correction based on machine learning. The 3D analysis module deploys GAN, Transformer, and MLP models. The product creation module generates standardized grid products and diagnostic data.

8. The machine learning-based three-dimensional atmospheric field fusion product generation device according to claim 1, characterized in that, It employs a hybrid storage approach combining the Xugu database and Gbase to store structured / semi-structured data, and manages data flow through an MTV architecture, which includes: The model layer handles database modeling; The view layer manages the user interface display; The controller layer coordinates user interaction responses.

9. A machine learning-based 3D atmospheric field fusion product generation device, including computer equipment, characterized in that, It stores program instructions that, when executed, implement the method of any one of claims 1-6.

10. The machine learning-based three-dimensional atmospheric field fusion product generation device according to claim 9, characterized in that, It also includes a computer-readable storage medium that stores a three-dimensional atmospheric field fusion product, containing temperature, humidity, wind speed, wind direction, and radar reflectivity factor data in a gridded manner.

Images