Method and system for generating a new high resolution energy history reanalysis dataset

Abstract

To provide a method and a system for generating a new energy history re-analysis data set of high resolution.SOLUTION: A method for providing more comprehensive auxiliary information for training a large-scale AI weather model, comprising a step of collecting observation data of wind and solar resources in a target area, high-level atmospheric state variables, surface variables and forced variables of a Mesoscale re-analysis weather data set over a whole period, and static data of a lower surface required for operating a Earth system combined model through multi-source data fusion, a data preprocessing step, a data assimilation step, a step of generating a high-resolution wind and solar re-analysis data set in a predetermined period, a step of comparing the high-resolution wind and solar re-analysis data set with a wind and solar down-scaling data set, and analyzing a difference characteristic therebetween through a deep learning algorithm; Training a data assimilation and optimization module, and generating a new high-resolution energy history re-analysis dataset consisting of a plurality of members.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the field of data processing technology, and more particularly to a method and system for generating new high-resolution energy history reanalysis datasets. [Background technology]

[0002] Accurate prediction of wind speed and solar radiation incident on the ground is the primary challenge in accurately forecasting wind and solar power generation, which are new energy sources. As an emerging cutting-edge technology, AI large-scale meteorological models play a crucial role in accurately forecasting wind and solar power generation. Their importance can be seen in several key areas: 1. Improved forecast accuracy. While traditional power forecasting methods are limited by the complexity of numerical models and data processing capabilities, AI large-scale meteorological models utilize advanced technologies such as deep learning and machine learning, combining large amounts of historical data with real-time weather information to precisely simulate weather patterns and significantly improve the accuracy of wind and solar power generation forecasts. 2. Refinement management. AI models can adopt differentiated forecasting strategies for different topography, climatic conditions, and power generation characteristics. In the case of wind power, this means using different model parameters for light and strong wind weather to improve forecast accuracy. This helps new energy generation companies procure resources more efficiently, thereby reducing wind waste (unused wind energy) and solar waste (unused solar energy). 3. Optimizing operating costs. Accurate power forecasting helps with advance planning and scheduling of the power system, thereby reducing the need for backup capacity and reducing economic losses due to inaccurate forecasts. This means that power system operators can better balance supply and demand, reducing power system stress and costs caused by variability in new energy generation. 4. Strengthening power system stability. High-precision power forecasting helps ensure the stable operation of the power system. In particular, in power systems with a high proportion of new energy, accurate predictions of renewable energy generation are crucial for avoiding problems such as power system overload and frequency fluctuations. 5. Facilitating energy trading and market mechanisms. Accurate power forecast data is the basis for new energy to participate in power market bidding, helping new energy generation companies to participate more effectively in market transactions and achieve higher economic benefits. 6. Responding to climate change.In response to the increase in extreme weather caused by climate change, large-scale AI weather models can improve prediction capabilities for extreme weather, enable new energy power generation systems to better adapt to climate change, and increase the resilience of the entire energy system. In summary, large-scale AI weather models are not only a key technology for improving the power generation efficiency of new energy, but also an important foundational technology for promoting energy transition and realizing green, low-carbon development.

[0003] The number of parameters in large-scale AI meteorological models can reach hundreds of millions. They utilize operators for solving partial differential equations or network structures commonly used in computer vision. They are trained on atmospheric reanalysis data to obtain models with short-term and even next-season forecasting capabilities. Existing technologies still have common shortcomings in industrial application research of AI meteorological models. The lack of domestically produced, controllable, high-resolution new energy historical reanalysis datasets is a key issue that urgently needs to be addressed. Currently, training information for large-scale AI meteorological models relies primarily on overseas datasets, leaving many inadequate and room for optimization in terms of accuracy, resolution, controllability, and effectiveness for domestic industrial applications. Existing wind and solar energy datasets face several technical challenges in terms of data resolution, precision, and real-time performance. First, current datasets often rely on traditional meteorological station and satellite remote sensing technologies, which often lack the data update frequency and spatial resolution to meet the needs of sophisticated energy management and real-time decision-making. For example, meteorological stations are distributed unevenly and data collection intervals are long. Satellite data, although having a wide coverage area, is affected by sensor resolution and cloud obscuration, making it difficult to provide continuous and highly accurate data. Furthermore, conventional methods often suffer from low computational efficiency when processing large-scale data, which is a particularly serious challenge during real-time data processing and emergency response. Furthermore, compatibility issues when fusing data from different sources often result in inaccurate analysis results, such as the scale mismatch between ground-based observation data and remote sensing data. Therefore, it is necessary to design a new high-resolution energy history reanalysis dataset generation method and system to solve the above problems. Summary of the Invention [Problem to be solved by the invention]

[0004] The technical problem to be solved by this invention is to provide a method and system for generating a new high-resolution energy history reanalysis dataset. By combining AI algorithms, data assimilation techniques, ensemble forecasting methods, and coupled Earth system models and using dynamic downscaling techniques, a new high-resolution energy history reanalysis dataset consisting of multiple members can be generated, providing more comprehensive supporting material for training AI large-scale meteorological models. [Means for solving the problem]

[0005] In order to achieve the above technical effects, the technical solutions adopted in the present invention are as follows:

[0006] 1. A method for generating a new high-resolution energy history reanalysis dataset, comprising: collecting data; collecting observation data of wind and solar resources within a target area by multi-source data fusion; collecting upper atmospheric state variables, surface variables, and forcing variables over a full period of a mesoscale reanalysis meteorological dataset; collecting subsurface static data necessary to operate the coupled Earth system model; Preprocessing the data; Preprocessing the upper atmosphere state variables, land surface variables, and forcing variables in the collected mesoscale reanalysis meteorological dataset to obtain the initial and boundary fields required to run the coupled Earth system model, preprocessing the lower surface static data in the collected mesoscale reanalysis meteorological dataset to obtain the regional topography files required to run the coupled Earth system model, and substituting them into the coupled Earth system model to generate wind and solar downscaling datasets; performing data assimilation; generating high-resolution wind and solar reanalysis datasets within a predetermined time period using data assimilation; training a data assimilation and optimization module; comparing and contrasting the high-resolution wind and solar reanalysis dataset with the wind and solar downscaling dataset, and using a deep learning algorithm to analyze the difference characteristics between the high-resolution wind and solar reanalysis dataset and the wind and solar downscaling dataset, thereby obtaining a data assimilation and optimization module by training; and generating high spatiotemporal resolution wind and solar reanalysis datasets.

[0007] Preferably, the multi-source data includes satellite remote sensing data, ground weather station data, drone monitoring data, radar data and historical weather records; High-resolution satellite images are used as satellite remote sensing data, and solar radiation intensity and cloud cover rate data are extracted. The following surface reflectance and atmospheric correction parameters are calculated using satellite data processing formulas: JPEG2026022604000002.jpg53134In formula, L * is the radiance of a single pixel received by the sensor, DN is the gray level of the original image pixel, gain and bias are the gain and bias values ​​corresponding to the sensor, ρ is the ground reflectance of the pixel, and ρ * is the top-of-atmosphere apparent reflectance of a ground object, d is the astronomical unit distance from the ground, E0 is the solar irradiance, θ is the solar zenith angle, S is the spherical albedo of the atmosphere, and L a * is the radiance of atmospheric path radiation entering the sensor, and F d is the downward radiation from the sun to the ground, T is the upward radiation from the ground to the sensor, Surface weather station data includes meteorological parameters consisting of wind speed, temperature and humidity; Drone monitoring data includes high-resolution meteorological element images and vertical profile information of meteorological parameters collected by drones; The radar data includes high-resolution rainfall intensity, cloud layer movement data, and wind speed and direction data collected by the radar system. The precipitation information obtained and processed from the radar data is R=a·Z b Including, where R denotes the rainfall rate, Z is the radar echo strength, and a and b are coefficients related to the particular radar equipment and configuration.

[0008] Preferably, the collection of subsurface static data necessary to operate the coupled Earth system model includes the collection of land and ocean topography, land cover, and land use data.

[0009] Preferably, the data pre-processing includes: preprocessing the upper atmospheric state variables, land surface variables, and forcing variables in a mesoscale reanalysis meteorological dataset to generate the initial and boundary fields required to run a coupled Earth system model; preprocessing the subsurface static data to generate regional topographic files necessary to run the coupled Earth system model; The method includes substituting the generated initial field, boundary field, and regional topography files into a coupled Earth system model, setting the grid point size of the model to 1 kilometer, setting the time resolution to 1 hour, and running the coupled Earth system model to perform dynamic downscaling simulations to obtain high-spatiotemporal resolution wind and solar downscaling datasets with a spatial resolution of 1 kilometer and a temporal resolution of 1 hour.

[0010] Preferably, during data assimilation, the step of utilizing data assimilation to generate high resolution wind and solar reanalysis datasets within a predetermined time period comprises: selecting a data assimilation method, and integrating the collected observation data of wind and solar resources in the target area into the simulation work of the coupled Earth system model through mathematical calculations and statistical processes, and updating the model state to generate high-resolution wind and solar reanalysis data sets within a predetermined period, wherein the data assimilation method includes one or more of a Kalman filter, an ensemble Kalman filter, a three-dimensional variational method, and a four-dimensional variational method; The mathematical model of data assimilation is x a =xb +K(y-Hx b ) and where x a is the analysis state, which is the result after the model simulation and update, and x b is the background state, which is the initial simulation result based on the model, including the wind and solar downscaling data sets obtained by simulation during data preprocessing; y is the actual observation data; H is the mapping function from the initial simulation state to the observation variables; and K is the Kalman gain, which indicates the weight of the observation data on the model simulation result.

[0011] Preferably, the step of comparing and contrasting the high-resolution wind and solar reanalysis dataset with the wind and solar downscaled dataset, analyzing the difference characteristics between the two by deep learning, and training the data assimilation and optimization module includes: Selecting graph neural networks (GNNs) to perform deep learning and training a data assimilation and optimization module, specifically: a. Preprocessing wind and solar downscaling datasets obtained by simulation of a coupled Earth system model into training input data and preprocessing high-resolution wind and solar reanalysis datasets generated by data assimilation into training target output data; The pre-processing includes normalization and feature encoding, and converts the data structure into a graph structure, where each node represents a specific geographic location, and the features of the node include wind speed, wind direction, solar radiation incident on the ground surface, and terrain features; b. defining edges between nodes based on geographic proximity, connecting neighboring weather stations or points within the same climate region, and dynamically adjusting the weight of the edge according to the distance or topographical barriers between the two points; c. Using graph convolutional layers (GCNs) to process graph structure data defined as follows: JPEG2026022604000003.jpg31162d. There are steps to set up multi-layer graph convolutional layers (GCNs) and concatenate a fully connected layer after the last graph convolutional layer to obtain the required data assimilation and optimization module.

[0012] Preferably, the step of generating high spatiotemporal resolution wind and solar reanalysis datasets comprises: The step includes inputting the high-spatiotemporal resolution wind and solar downscaling datasets with 1-kilometer spatial resolution and 1-hour temporal resolution over a fixed period within the target area obtained in the data preprocessing step into a trained data assimilation and optimization module, and outputting high-spatiotemporal resolution wind and solar reanalysis datasets over a fixed period within the target area.

[0013] Preferably, the initial field obtained by data preprocessing is subjected to initial perturbation processing using the growth mode propagation method and the singular vector method to obtain an initial field set having initial value perturbation information, and the number of members of the initial field set is 10 or more.

[0014] Preferably, multiple combinations of parameterization schemes, including a microphysics scheme, a boundary layer and near-surface layer scheme, and a longwave and shortwave radiation scheme, are set for the coupled Earth system model, and a comprehensive comparison and selection is performed by quantitatively calculating the anomaly correlation coefficient, root mean square error, absolute error, and predictive score between the simulation results of the coupled model and the actual observation values. At least three combinations of parameterization schemes are selected, and the processes of data assimilation, training of the data assimilation and optimization module, and generation of high-spatial and temporal resolution wind and solar reanalysis datasets are repeatedly performed to generate multiple sets of 30-year wind and solar reanalysis datasets with 1-kilometer spatial resolution and 1-hour temporal resolution within the target area, thereby forming a database.

[0015] Preferably, the system includes a data collection module, a data pre-processing module, a data assimilation module, a deep learning training module, a data assimilation and optimization module, and a dataset generation module; The data collection module includes a multi-source data fusion module for performing a multi-source data fusion method to collect observation data of wind and solar resources within the target area; The data collection module is used to collect upper atmospheric state variables, surface variables, and forcing variables over the full cycle of mesoscale reanalysis meteorological datasets, and to collect the lower surface static data necessary to run coupled Earth system models. The data preprocessing module is used to preprocess the upper atmosphere state variables, surface variables and forcing variables in the collected mesoscale reanalysis meteorological dataset to obtain the initial fields and boundary fields required for running the coupled earth system model, preprocess the static data of the lower surface in the collected mesoscale reanalysis meteorological dataset to obtain the regional terrain file required for running the coupled earth system model, and substitute it into the coupled earth system model to generate wind and solar downscaling datasets, perform normalization processing and feature encoding on the data to be processed, convert the data structure into a graph structure, implement the growing mode propagation method and the singular vector method, and perform initial perturbation processing on the generated initial fields to obtain an initial field set with initial value perturbation information; The data assimilation module is used to implement a data assimilation method, integrate the collected observation data of wind and solar resources in the target area into the simulation work of the earth system coupled model through mathematical calculations and statistical processes, update the model state, and generate high-resolution wind and solar reanalysis datasets within a predetermined period; The deep learning training module includes graph neural networks (GNNs), multi-layer graph convolutional layers (GCNs), and fully connected layers, and is used to compare and contrast high-resolution wind and solar reanalysis datasets with wind and solar downscaling datasets, analyze the differences between them using deep learning, and train the data assimilation and optimization module. The data assimilation and optimization module uses the high-spatiotemporal resolution wind and solar downscaled datasets with 1-kilometer spatial resolution and 1-hour temporal resolution over a fixed period within the target region obtained in the data preprocessing step as input, performs data assimilation, and outputs high-spatiotemporal resolution wind and solar reanalysis datasets over a fixed period within the target region. [Effects of the Invention]

[0016] The beneficial effects of the present invention are as follows: 1. Using the initial value perturbation method to obtain multiple initial fields, different parameterization schemes are used to drive the coupled Earth system model for simulation, and observation data is combined with the data assimilation and optimization module to generate more sets of possible outcomes for wind and solar resources. This quantitatively takes into account the chaotic characteristics of the Earth system and the uncertainties of the coupled Earth system model itself, providing more comprehensive auxiliary decision-making for the assessment scheme of wind and solar resources.

[0017] 2. The data assimilation and optimization module obtained by training the deep learning algorithm can learn deep relationships and models from multidimensional data. These relationships exceed the capture capability of traditional statistical models. At the same time, it significantly reduces the computational resources and time required for traditional data assimilation methods combined with coupled Earth system models, improving the efficiency of generating high-spatiotemporal resolution wind and solar reanalysis datasets for the target area.

[0018] 3. The creation of high-spatiotemporal resolution wind and solar reanalysis datasets is crucial for training large-scale AI models for new energy power forecasting, significantly improving their understanding of the complex climatic conditions and geographical features of the target region. Accurate reanalysis datasets enable large-scale AI models to identify and learn the spatiotemporal change characteristics of key energy meteorological parameters such as wind speed and solar radiation, thereby enabling effective forecasting of new energy power in complex environments. Therefore, fine-grained wind and solar reanalysis datasets with high spatiotemporal resolution are the foundation for building and training high-performance large-scale AI models for wind and solar power forecasting, and are an important foundation for meeting the development of new productivity in the energy industry.

[0019] 4. By adding an implementation and update mechanism and combining it with AI models, it can achieve rapid response and processing of real-time data, and continuously update and optimize data sets, thereby supporting the creation of implementation decisions. Furthermore, compared with traditional data assimilation methods, the application of AI models reduces the need for human involvement in data processing and analysis, improving the efficiency and automation level of data processing flows.

[0020] 5. The structured data assimilation and optimization module is not limited to processing a single type of data, but can simultaneously process data from different sources, such as satellite images, surface meteorological observations, drone data, and radar data, thereby realizing data fusion and assimilation and providing a comprehensive solution to complex environmental problems. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a flow chart of the method of the present invention. [Figure 2] FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0022] Example 1 As shown in FIG. 1, a method for generating a new high-resolution energy history reanalysis dataset includes: collecting data; collecting observation data of wind and solar resources within a target area by multi-source data fusion; collecting upper atmospheric state variables, surface variables, and forcing variables over a full period of a mesoscale reanalysis meteorological dataset; collecting subsurface static data necessary to operate the coupled Earth system model; Preprocessing the data; Preprocessing the upper atmosphere state variables, land surface variables, and forcing variables in the collected mesoscale reanalysis meteorological dataset to obtain the initial and boundary fields required to run the coupled Earth system model, preprocessing the lower surface static data in the collected mesoscale reanalysis meteorological dataset to obtain the regional topography files required to run the coupled Earth system model, and substituting them into the coupled Earth system model to generate wind and solar downscaling datasets; performing data assimilation; generating high-resolution wind and solar reanalysis datasets within a predetermined time period using data assimilation; training a data assimilation and optimization module; comparing and contrasting the high-resolution wind and solar reanalysis dataset with the wind and solar downscaling dataset, and using a deep learning algorithm to analyze the difference characteristics between the high-resolution wind and solar reanalysis dataset and the wind and solar downscaling dataset, thereby obtaining a data assimilation and optimization module by training; and generating high spatiotemporal resolution wind and solar reanalysis datasets.

[0023] Preferably, the multi-source data includes satellite remote sensing data, ground weather station data, drone monitoring data, radar data and historical weather records; High-resolution satellite images are used as satellite remote sensing data, and solar radiation intensity and cloud cover rate data are extracted. The following surface reflectance and atmospheric correction parameters are calculated using satellite data processing formulas: JPEG2026022604000004.jpg55137In formula, L * is the radiance of a single pixel received by the sensor, DN is the gray level of the original image pixel, gain and bias are the gain and bias values ​​corresponding to the sensor, ρ is the ground reflectance of the pixel, and ρ * is the top-of-atmosphere apparent reflectance of a ground object, d is the astronomical unit distance from the ground, E0 is the solar irradiance, θ is the solar zenith angle, S is the spherical albedo of the atmosphere, and L a * is the radiance of atmospheric path radiation entering the sensor, and F d is the downward radiation from the sun to the ground, T is the upward radiation from the ground to the sensor, Surface weather station data includes meteorological parameters consisting of wind speed, temperature and humidity; Drone monitoring data includes high-resolution meteorological element images and vertical profile information of meteorological parameters collected by drones; The radar data includes high-resolution rainfall intensity, cloud layer movement data, and wind speed and direction data collected by the radar system. The precipitation information obtained and processed from the radar data is R=a·Z b Including, where R denotes the rainfall rate, Z is the radar echo strength, and a and b are coefficients related to the particular radar equipment and configuration.

[0024] Preferably, the collection of subsurface static data necessary to operate the coupled Earth system model includes the collection of land and ocean topography, land cover, and land use data.

[0025] Preferably, the data pre-processing includes: preprocessing the upper atmospheric state variables, land surface variables, and forcing variables in a mesoscale reanalysis meteorological dataset to generate the initial and boundary fields required to run a coupled Earth system model; preprocessing the subsurface static data to generate regional topographic files necessary to run the coupled Earth system model; The method includes substituting the generated initial field, boundary field, and regional topography files into a coupled Earth system model, setting the grid point size of the model to 1 kilometer, setting the time resolution to 1 hour, and running the coupled Earth system model to perform dynamic downscaling simulations to obtain high-spatiotemporal resolution wind and solar downscaling datasets with a spatial resolution of 1 kilometer and a temporal resolution of 1 hour.

[0026] Preferably, during data assimilation, the step of utilizing data assimilation to generate high resolution wind and solar reanalysis datasets within a predetermined time period comprises: selecting a data assimilation method, and integrating the collected observation data of wind and solar resources in the target area into the simulation work of the coupled Earth system model through mathematical calculations and statistical processes, and updating the model state to generate high-resolution wind and solar reanalysis data sets within a predetermined period, wherein the data assimilation method includes one or more of a Kalman filter, an ensemble Kalman filter, a three-dimensional variational method, and a four-dimensional variational method; The mathematical model of data assimilation is x a =x b +K(y-Hx b ) and where x a is the analysis state, which is the result after the model simulation and update, and x b is the background state, which is the initial simulation result based on the model, including the wind and solar downscaling data sets obtained by simulation during data preprocessing; y is the actual observation data; H is the mapping function from the initial simulation state to the observation variables; and K is the Kalman gain, which indicates the weight of the observation data on the model simulation result.

[0027] Preferably, the step of comparing and contrasting the high-resolution wind and solar reanalysis dataset with the wind and solar downscaled dataset, analyzing the difference characteristics between the two by deep learning, and training the data assimilation and optimization module includes: This involves selecting graph neural networks (GNNs) to perform deep learning and training a data assimilation and optimization module, specifically: a. Preprocessing wind and solar downscaling datasets obtained by simulation of a coupled Earth system model into training input data and preprocessing high-resolution wind and solar reanalysis datasets generated by data assimilation into training target output data; The pre-processing includes normalization and feature encoding, and converts the data structure into a graph structure, where each node represents a specific geographic location, and the features of the node include wind speed, wind direction, solar radiation incident on the ground surface, and terrain features; b. defining edges between nodes based on geographic proximity, connecting neighboring weather stations or points within the same climate region, and dynamically adjusting the weight of the edge according to the distance or topographical barriers between the two points; c. Using graph convolutional layers (GCNs) to process graph structure data defined as follows: JPEG2026022604000005.jpg31164d. There are steps to set up multi-layer graph convolutional layers (GCNs) and concatenate a fully connected layer after the last graph convolutional layer to obtain the required data assimilation and optimization module.

[0028] Preferably, the step of generating high spatiotemporal resolution wind and solar reanalysis datasets comprises: The step involves inputting the high-spatiotemporal resolution wind and solar downscaling datasets with 1-kilometer spatial resolution and 1-hour temporal resolution over a fixed period within the target region obtained in the data preprocessing step into a trained data assimilation and optimization module, which outputs a 30-year high-spatiotemporal resolution wind and solar reanalysis dataset within the target region.

[0029] Preferably, the initial field obtained by data preprocessing is subjected to initial perturbation processing using the growth mode propagation method and the singular vector method to obtain an initial field set having initial value perturbation information, and the number of members of the initial field set is 10 or more.

[0030] Preferably, multiple combinations of parameterization schemes, including a microphysics scheme, a boundary layer and near-surface layer scheme, and a longwave and shortwave radiation scheme, are set for the coupled Earth system model, and a comprehensive comparison and selection is performed by quantitatively calculating the anomaly correlation coefficient, root mean square error, absolute error, and predictive score between the simulation results of the coupled model and the actual observation values. At least three combinations of parameterization schemes are selected, and the processes of data assimilation, training of the data assimilation and optimization module, and generation of high-spatial and temporal resolution wind and solar reanalysis datasets are repeatedly performed to generate multiple sets of 30-year wind and solar reanalysis datasets with 1-kilometer spatial resolution and 1-hour temporal resolution within the target area, thereby forming a database. Example 2

[0031] As shown in FIG. 2, this embodiment provides a new high-spatial-temporal resolution energy history reanalysis dataset generation system based on an AI large-scale model, which includes: a data collection module, a data preprocessing module, a data assimilation module, a deep learning training module, a data assimilation and optimization module, and a dataset generation module; The data collection module includes a multi-source data fusion module for performing a multi-source data fusion method to collect observation data of wind and solar resources within the target area; The data collection module is used to collect upper atmospheric state variables, surface variables, and forcing variables over the full cycle of mesoscale reanalysis meteorological datasets, and to collect the lower surface static data necessary to run coupled Earth system models. The data preprocessing module is used to preprocess the upper atmosphere state variables, surface variables and forcing variables in the collected mesoscale reanalysis meteorological dataset to obtain the initial fields and boundary fields required for running the coupled earth system model, preprocess the static data of the lower surface in the collected mesoscale reanalysis meteorological dataset to obtain the regional terrain file required for running the coupled earth system model, and substitute it into the coupled earth system model to generate wind and solar downscaling datasets, perform normalization processing and feature encoding on the data to be processed, convert the data structure into a graph structure, implement the growing mode propagation method and the singular vector method, and perform initial perturbation processing on the generated initial fields to obtain an initial field set with initial value perturbation information; The data assimilation module is used to implement a data assimilation method, integrate the collected observation data of wind and solar resources in the target area into the simulation work of the earth system coupled model through mathematical calculations and statistical processes, update the model state, and generate high-resolution wind and solar reanalysis datasets within a predetermined period; The deep learning training module includes graph neural networks (GNNs), multi-layer graph convolutional layers (GCNs), and fully connected layers, and is used to compare and contrast high-resolution wind and solar reanalysis datasets with wind and solar downscaling datasets, analyze the differences between them using deep learning, and train the data assimilation and optimization module. The data assimilation and optimization module uses the high-spatiotemporal resolution wind and solar downscaled datasets with 1-kilometer spatial resolution and 1-hour temporal resolution over a fixed period within the target region obtained in the data preprocessing step as input, performs data assimilation, and outputs high-spatiotemporal resolution wind and solar reanalysis datasets over a fixed period within the target region.

[0032] In some alternative embodiments, the data collection module includes a satellite remote sensing data collection unit for collecting high-resolution satellite imagery, extracting solar radiation intensity and cloud cover data, and calculating surface reflectance and atmospheric correction parameters using satellite data processing equations, such as: JPEG2026022604000006.jpg55170, L * is the radiance of a single pixel received by the sensor, DN is the gray level of the original image pixel, gain and bias are the gain and bias values ​​corresponding to the sensor, ρ is the ground reflectance of the pixel, and ρ * is the top-of-atmosphere apparent reflectance of a ground object, d is the astronomical unit distance from the ground, E0 is the solar irradiance, θ is the solar zenith angle, S is the spherical albedo of the atmosphere, and L a * is the radiance of atmospheric path radiation entering the sensor, and F d is the downward radiation from the sun to the ground, and T is the upward radiation from the ground to the sensor.

[0033] In some alternative embodiments, the data collection module includes a radar data acquisition and processing unit for acquiring and processing precipitation information, the precipitation information being: R=a·Z b Including, where R denotes the rainfall rate, Z is the radar echo strength, and a and b are coefficients related to the particular radar equipment and configuration.

[0034] In some alternative embodiments, the data assimilation module includes a data assimilation framework unit for performing data assimilation as follows: x a =x b +K(y-Hx b ), where x a is the analysis state, which is the result after the model simulation and update, and x bis the background state, which is the initial simulation result based on the model, including the wind and solar downscaling data sets obtained by simulation during data preprocessing; y is the actual observation data; H is the mapping function from the initial simulation state to the observation variables; and K is the Kalman gain, which indicates the weight of the observation data on the model simulation result.

[0035] In some alternative embodiments, the graph convolution layers GCNs of the deep learning training module are used to process graph structure data defined as follows: JPEG2026022604000007.jpg32163 (Example 3)

[0036] According to the method for generating a new high-resolution energy history reanalysis dataset provided by the present invention, the method steps of a specific embodiment include data collection, AI model construction, dataset generation, and a real-time update mechanism.

[0037] Step 1: Data collection. The present invention uses multi-source data fusion technology to collect observation data of wind and solar resources within a target area, and the data sources mainly include satellite remote sensing data, ground meteorological station data, drone monitoring data, radar data, and historical meteorological records. Here, high-resolution satellite images are used as satellite remote sensing data, and solar radiation intensity and cloud cover data are extracted. Satellite data processing formulas are used to calculate the surface reflectance and atmospheric correction parameters as follows: JPEG2026022604000008.jpg55162, L * is the radiance of a single pixel received by the sensor, DN is the gray level of the original image pixel, gain and bias are the gain and bias values ​​corresponding to the sensor, ρ is the ground reflectance of the pixel, and ρ * is the top-of-atmosphere apparent reflectance of a ground object, d is the astronomical unit distance from the ground, E0 is the solar irradiance, θ is the solar zenith angle, S is the spherical albedo of the atmosphere, and La * is the radiance of atmospheric path radiation entering the sensor, and F d is the downward radiation from the sun to the ground, and T is the upward radiation from the ground to the sensor.

[0038] Ground-based weather station data includes meteorological parameters such as wind speed, temperature and humidity, and these data points are distributed at strategic locations within the target area to validate and calibrate the satellite data.

[0039] Regarding drone monitoring data, drones can provide high-resolution meteorological element images and vertical profile information of necessary meteorological parameters, especially in areas where data acquisition is difficult using conventional methods.

[0040] Regarding radar data, the radar system can provide high-resolution rainfall intensity, cloud layer movement data, wind speed, wind direction, and other data information, which is extremely important for analyzing the impact of solar radiation and wind energy resources in the target area. For example, radar data can be used to obtain and process precipitation information in the following way: R=a·Z b where R denotes the amount of rainfall, Z is the radar echo intensity, and a and b are coefficients related to the particular radar instrument and configuration. Such data are important to understand how rainfall affects solar radiation conditions, for example, the reduction of solar radiation due to cloud obscuration, and help optimize light energy estimation and management.

[0041] In step 2, the upper atmosphere state variables, land surface variables, and forcing variables for the entire cycle of mesoscale reanalysis meteorological datasets are collected. The entire cycle, for example, the entire cycle of ERA5, covers the period from 1940 to the present, but this is by way of example only and is not limiting. In addition to ERA5, commonly used reanalysis datasets include MERRA2, JRA-55, and NCEP / NCAR reanalysis datasets, etc., whose spatial resolution is generally about 25 kilometers. Static data on the lower surface required for running the Earth system coupled model, such as land and ocean topography data, land cover data, land use data, etc., are collected and selected according to the requirements of the project implementation. For example, GEOG topography data can be used as land data. There are no restrictions on the source and type of topography data.

[0042] In step 3, the upper atmosphere state variables, surface variables, and forcing variables in the mesoscale reanalysis meteorological dataset obtained in step 2 are preprocessed to obtain the initial and boundary fields required for running the coupled model. The static data of the lower surface is preprocessed to obtain the regional topography file required for running the Earth system coupled model, which is created according to the spatial resolution required for project implementation and has no restrictions.

[0043] The above results are substituted into a coupled Earth system model, the model's grid point size is set to 1 kilometer, and the time resolution is set to 1 hour. The coupled Earth system model is then used to perform a dynamical downscaling simulation. The coupled Earth system model can be selected based on the target location. For example, an atmospheric model, ocean model, and ocean wave model can be combined using the MCT coupler. Commonly used atmospheric models include WRF and MM5, ocean models include ROMS, FVCOM, and HYCOM, and ocean wave models include SWAN and WAVEWATCHIII. These are merely examples and are not intended to be limiting. A high-spatiotemporal resolution wind and solar downscaling dataset with a spatial resolution of 1 kilometer and a temporal resolution of 1 hour is obtained.

[0044] In step 4, conventional data assimilation techniques are used to generate high-resolution wind and solar reanalysis datasets for the past 5 to 10 years. This period is preferably selected according to the deployment time of on-site observation equipment, and is based on the time when multiple types of observation data, such as satellite remote sensing data, surface meteorological station data, drone monitoring data, and radar data, are first obtained. A data assimilation method suitable for the target region, such as Kalman filter (KF), ensemble Kalman filter (EnKF), three-dimensional variational method (3D-Var), or four-dimensional variational method (4D-Var), is selected. The observation data collected in step 1 is integrated into the simulation work of the coupled Earth system model through mathematical calculation and statistical processes to improve the accuracy of the model and update the model state, so that the model simulation can reflect the state most similar to the actual observation. In this embodiment, the following framework is usually adopted for data assimilation: x a =x b +K(y-Hx b ), where x a is the analysis state, which is the result after the model simulation and update, and x b is the background state, which is the initial simulation result based on the model, including the wind and solar downscaling data sets obtained by simulation during data preprocessing; y is the actual observation data; H is the mapping function from the initial simulation state to the observation variables; and K is the Kalman gain, which indicates the weight of the observation data on the model simulation result.

[0045] In step 5, the high-resolution wind and solar reanalysis datasets generated in step 4 using conventional data assimilation techniques are compared with the wind and solar downscaled datasets obtained in step 3 directly from the coupled Earth system model simulation. The differences between the two datasets are analyzed using a deep learning algorithm to train a data assimilation and optimization module. Deep learning algorithms that can be used to generate the data assimilation and optimization module include graph neural networks (GNNs), which can effectively handle non-Euclidean structures in data, such as spatial relationships in geographic data. In meteorological data generation, GNNs can capture complex interactions between different geographic locations, such as spatial correlations in wind and climate models, thereby improving the model's ability to process complex terrain and multi-source data. A graph neural network is a neural network that processes graph-structured data. In the context of meteorological data generation, each geographic location, such as a weather station or a remote sensing satellite pixel point, can be viewed as a node in the graph, and the spatial relationships and interactions between nodes can be represented as edges in the graph. GNNs learn representations of nodes by aggregating their own features and the features of their neighbors to capture complex spatial dependencies.

[0046] a. First, the wind and solar downscaling dataset obtained in step 3, which is directly obtained by simulating the Earth system coupled model, is preprocessed to serve as training input data; the high-resolution wind and solar reanalysis dataset obtained in step 4, which is generated in combination with traditional data assimilation techniques, is preprocessed to serve as training target output data; the preprocessing includes normalization and feature encoding, and the data structure is converted into a graph structure, where each node represents a specific geographical location, and the node features include wind speed, wind direction, surface incident solar radiation, and terrain features.

[0047] b. The edges of the graph can then be defined, where edges between nodes can be defined based on geographic proximity, connecting adjacent weather stations or points within the same climate region, and the weight of the edge can be dynamically adjusted depending on the distance between the two points or other environmental factors such as topographical barriers.

[0048] c. Graph convolutional networks (GCNs) are used to process graph structure data. The GCNs can maintain the geographical and meteorological characteristics of each node while aggregating information from neighboring nodes, enabling a deeper understanding of the local geographical environment. The GCNs can be defined as follows: JPEG2026022604000009.jpg31165

[0049] d. Multi-layer graph convolutional layers (GCNs) can realize deep feature aggregation and improve the model's ability to capture a wide range of meteorological spatial models. By connecting a fully connected layer after the last graph convolutional layer, the required data assimilation and optimization module can be obtained.

[0050] In Step 6, the data assimilation and optimization module obtained in Step 5 is applied to the high-spatiotemporal resolution wind and solar downscaling dataset with 1-kilometer spatial resolution and 1-hour temporal resolution for the past 30 years within the target region obtained in Step 3 to obtain a high-spatiotemporal resolution wind and solar reanalysis dataset for the past 30 years within the target region.

[0051] In step 7, the initial field obtained in step 3 is subjected to initial perturbation processing using the growing mode propagation method and the singular vector method to obtain an initial field set containing initial value perturbation information. An appropriate initial value perturbation method can be selected as needed, but is not limited to this. It is recommended that the number of members of the initial field set be 10 or more.

[0052] In addition, multiple combinations of parameterization schemes are selected for the coupled Earth system model, including microphysics schemes such as the Kessler scheme, the Lin scheme, and the WSM5 scheme; boundary layer and near-surface layer schemes such as the YSU-revised MM5, the Monin-Obukhov scheme, and the QNSE scheme; and longwave and shortwave radiation schemes such as the CAM-RRTMG scheme, the CAM-RRTMG fast scheme, and the New Goddard scheme. The combinations of parameterization schemes are selected based on a comprehensive and objective comparative evaluation method, and at least three combinations of parameterization schemes are selected through a comprehensive comparison and selection method using quantitative calculations of the anomalous correlation coefficient, root mean square error, absolute error, and predictive score between the simulation results of the coupled model and the actual observation values.

[0053] Repeat steps 4 to 6 to generate a database of multiple sets (e.g., 10 × 3 = 30 sets) of 30-year, 1-kilometer spatial resolution, 1-hour temporal resolution wind and solar reanalysis datasets within the target area.

[0054] In step 8, an implementation and update mechanism is added, and a real-time data update system is designed. A streaming data processing framework and an event-driven mechanism are used to ensure significant improvements in the accuracy, spatiotemporal resolution, and real-time updates of the data of the present invention. Here, platforms such as Apache Kafka or Apache Flink are used for streaming data processing, and new incoming data is processed in real time. Event-driven updates trigger the data processing and update flow when new data is input, ensuring that the dataset continuously reflects the latest weather and energy conditions.

Claims

1. collecting data; collecting observation data of wind and solar resources within a target area by multi-source data fusion; collecting upper atmospheric state variables, surface variables, and forcing variables over a full period of a mesoscale reanalysis meteorological dataset; collecting subsurface static data necessary to operate the coupled Earth system model; Preprocessing the data; Preprocessing the upper atmosphere state variables, land surface variables, and forcing variables in the collected mesoscale reanalysis meteorological dataset to obtain the initial and boundary fields required to run the coupled Earth system model, preprocessing the lower surface static data in the collected mesoscale reanalysis meteorological dataset to obtain the regional topography files required to run the coupled Earth system model, and substituting them into the coupled Earth system model to generate wind and solar downscaling datasets; performing data assimilation; generating high-resolution wind and solar reanalysis datasets within a predetermined time period using data assimilation; training a data assimilation and optimization module; comparing and contrasting the high-resolution wind and solar reanalysis dataset with the wind and solar downscaling dataset, and using a deep learning algorithm to analyze the difference characteristics between the high-resolution wind and solar reanalysis dataset and the wind and solar downscaling dataset, thereby obtaining a data assimilation and optimization module by training; generating high spatiotemporal resolution wind and solar reanalysis datasets; A method for generating a new high-resolution energy history reanalysis dataset.

2. Multi-source data includes satellite remote sensing data, surface weather station data, drone monitoring data, radar data and historical weather records. High-resolution satellite images are used as satellite remote sensing data, and solar radiation intensity and cloud cover rate data are extracted. The following surface reflectance and atmospheric correction parameters are calculated using satellite data processing formulas: In the ceremony, L ＊ is the radiance of a single pixel received by the sensor, DN is the gray level of the original image pixel, gain and bias are the gain and bias values ​​corresponding to the sensor, ρ is the ground reflectance of the pixel, and ρ ＊ is the apparent top-of-atmosphere reflectance of a ground object, d is the astronomical unit distance from the ground, and E ０ is the solar irradiance, θ is the solar zenith angle, S is the spherical albedo of the atmosphere, and L ａ ＊ is the radiance of the atmospheric path radiation entering the sensor, and F ｄ is the downward radiation from the sun to the ground, T is the upward radiation from the ground to the sensor, Surface weather station data includes meteorological parameters consisting of wind speed, temperature and humidity; Drone monitoring data includes high-resolution meteorological element images and vertical profile information of meteorological parameters collected by drones; Radar data includes high-resolution rainfall intensity, cloud layer movement data, and wind speed and direction data information collected by radar systems.

2. A method for generating a new high-resolution energy history reanalysis dataset according to claim 1.

3. Collection of static subsurface data necessary to operate coupled Earth system models includes collection of land and ocean topography, land cover, and land use data; 2. A method for generating a new high-resolution energy history reanalysis dataset according to claim 1.

4. For data preprocessing, preprocessing the upper atmospheric state variables, land surface variables, and forcing variables in a mesoscale reanalysis meteorological dataset to generate the initial and boundary fields required to run a coupled Earth system model; preprocessing the subsurface static data to generate regional topographic files necessary to run the coupled Earth system model; and substituting the generated initial field, boundary field and regional topography files into a coupled Earth system model, setting the grid point size of the model to 1 kilometer and the time resolution to 1 hour, and running the coupled Earth system model to perform dynamic downscaling simulations to obtain high spatiotemporal resolution wind and solar downscaling datasets with a spatial resolution of 1 kilometer and a temporal resolution of 1 hour.

2. A method for generating a new high-resolution energy history reanalysis dataset according to claim 1.

5. During data assimilation, utilizing data assimilation to generate high-resolution wind and solar reanalysis datasets within a predetermined time period includes: selecting a data assimilation method, and integrating the collected observation data of wind and solar resources in the target area into the simulation work of the coupled Earth system model through mathematical calculations and statistical processes, and updating the model state to generate high-resolution wind and solar reanalysis data sets within a predetermined period, wherein the data assimilation method includes one or more of a Kalman filter, an ensemble Kalman filter, a three-dimensional variational method, and a four-dimensional variational method; The mathematical model of data assimilation is x ａ = x ｂ +K(y-Hx ｂ ) and Here, x ａ refers to the analysis state, which is the result after the model has been simulated and updated, and x ｂ is the background state, which is the initial simulation result based on the model, including the wind and solar downscaling datasets obtained by simulation during data preprocessing; y is the actual observation data; H is a mapping function from the initial simulation state to the observation variables; K is the Kalman gain, which indicates the weight of the observation data on the model simulation result; 5. A method for generating a new high-resolution energy history reanalysis dataset according to claim 4.

6. The step of comparing and contrasting the high-resolution wind and solar reanalysis dataset with the wind and solar downscaled dataset, analyzing the difference characteristics between the two using deep learning, and training a data assimilation and optimization module includes: Selecting graph neural networks (GNNs) to perform deep learning and training a data assimilation and optimization module, specifically: a) preprocessing wind and solar downscaling datasets obtained by simulation of a coupled Earth system model into training input data and preprocessing high-resolution wind and solar reanalysis datasets generated by data assimilation techniques into training target output data; The pre-processing includes normalization and feature encoding to convert the data structure into a graph structure, where each node represents a specific geographic location, and the features of the node include wind speed, wind direction, solar radiation incident on the ground surface, and terrain features; b. defining edges between nodes based on geographic proximity, connecting neighboring weather stations or points within the same climate region, and dynamically adjusting the weight of the edge according to the distance or topographical barriers between the two points; c. Using a graph convolution layer GCNs to process graph structure data defined as follows: d. Setting up multi-layer graph convolution layers GCNs and concatenating a fully connected layer after the last graph convolution layer to obtain the required data assimilation and optimization module; 6. A method for generating a new high-resolution energy history reanalysis dataset, as claimed in claim 5.

7. generating high spatiotemporal resolution wind and solar reanalysis datasets; a step of inputting the high spatiotemporal resolution wind and solar downscaling dataset with 1 kilometer spatial resolution and 1 hourly temporal resolution within the target area and over a fixed period of time obtained in the data pre-processing step into a trained data assimilation and optimization module, and outputting a high spatiotemporal resolution wind and solar reanalysis dataset within the target area and over a fixed period of time; 7. A method for generating a new high-resolution energy history reanalysis dataset according to claim 6.

8. Using the growth mode propagation method and the singular vector method, an initial perturbation process is performed on the initial field obtained by the data preprocessing to obtain an initial field set having initial value perturbation information, and the number of members of the initial field set is 10 or more.

8. A method for generating a new high-resolution energy history reanalysis dataset according to claim 7.

9. A plurality of parameterization scheme combinations, including a microphysics scheme, a boundary layer and near-surface layer scheme, and a longwave and shortwave radiation scheme, are set for the coupled Earth system model, and a comprehensive comparison and selection is performed by quantitatively calculating the anomaly correlation coefficient, root mean square error, absolute error, and predictive score between the simulation results of the coupled model and the actual observation values, to select at least three parameterization scheme combinations. The processes of data assimilation, training of the data assimilation and optimization module, and generation of high-spatial and temporal resolution wind and solar reanalysis datasets are repeatedly performed, and a plurality of 30-year wind and solar reanalysis datasets with 1-kilometer spatial resolution and 1-hour temporal resolution within the target area are generated to form a database.

9. A method for generating a new high-resolution energy history reanalysis dataset according to claim 8.

10. A system used in the method for generating a high-resolution new energy history reanalysis dataset according to any one of claims 1 to 9, comprising: The system includes a data collection module, a data pre-processing module, a data assimilation module, a deep learning training module, a data assimilation and optimization module, and a dataset generation module; The data collection module includes a multi-source data fusion module for performing a multi-source data fusion method to collect observation data of wind and solar resources within the target area; The data collection module is used to collect upper atmospheric state variables, surface variables, and forcing variables over the full cycle of mesoscale reanalysis meteorological datasets, and to collect the lower surface static data necessary to run coupled Earth system models. The data preprocessing module is used to preprocess the upper atmosphere state variables, surface variables and forcing variables in the collected mesoscale reanalysis meteorological dataset to obtain the initial fields and boundary fields required for running the coupled earth system model, preprocess the static data of the lower surface in the collected mesoscale reanalysis meteorological dataset to obtain the regional terrain file required for running the coupled earth system model, and substitute it into the coupled earth system model to generate wind and solar downscaling datasets, perform normalization processing and feature encoding on the data to be processed, convert the data structure into a graph structure, implement the growing mode propagation method and the singular vector method, and perform initial perturbation processing on the generated initial fields to obtain an initial field set with initial value perturbation information; The data assimilation module is used to implement a data assimilation method, integrate the collected observation data of wind and solar resources in the target area into the simulation work of the earth system coupled model through mathematical calculations and statistical processes, update the model state, and generate high-resolution wind and solar reanalysis datasets within a predetermined period; The deep learning training module includes graph neural networks (GNNs), multi-layer graph convolutional layers (GCNs) and fully connected layers, and is used to compare and contrast the high-resolution wind and solar reanalysis dataset with the wind and solar downscaling dataset, analyze the difference characteristics between them using deep learning, and obtain the data assimilation and optimization module by training; The data assimilation and optimization module is used to perform data assimilation using high-spatiotemporal resolution wind and solar downscaling datasets with 1-kilometer spatial resolution and 1-hour temporal resolution over a fixed period within the target area obtained in the data preprocessing step as input, and to output high-spatiotemporal resolution wind and solar reanalysis datasets over a fixed period within the target area. A system for use in a method for generating a new high-resolution energy history reanalysis dataset.

Images