Sand-dust weather assimilation data generation method and device based on fusion of AI algorithm re-optimization

Abstract

The application provides a sand-dust weather assimilation data generation method and device fused with AI algorithm reoptimization, and relates to the technical field of computer science and atmospheric science fusion. The method comprises the following steps: acquiring multi-channel observation data provided by a satellite; introducing a Bayesian optimization process to make background error and observation error independent of each other, and obtaining analyzed field data after assimilation; and constructing a joint optimization framework, which responds to the input analyzed field data, generates preliminary post-optimization data field as the CNN global feature by using CNN, obtains SVR local correction features by SVR, and realizes the balance between the CNN global feature and the SVR local correction features through dynamic weighted fusion. Through the combination and comparison of the collaborative processing among the multi-satellite channel data and the actual sand-dust weather process, the reconstruction accuracy of the large-scale sand-dust data is significantly improved, and more accurate support is provided for the construction of the current sand-dust assimilation data.

Description

Technical Field

[0001] This application relates to the field of computer science and atmospheric science integration technology, particularly to the intersection of data assimilation, deep learning and statistical learning, and especially to a method and apparatus for generating assimilation data for dust storm weather. Background Technology

[0002] Data assimilation techniques have been widely used in atmospheric science. By combining observational data with numerical models, data assimilation can effectively improve data accuracy. However, for in-depth analysis of the environmental and climate effects of dust aerosols, existing studies are mostly based on a combination of multi-source observations and traditional numerical simulations. Therefore, data fluctuations between the initial background field and the boundary field further accumulate errors between simulated and actual data.

[0003] In the field of data assimilation, the most widely used methods currently include ensemble Kalman filtering (EnKF) and four-dimensional variational assimilation (4DVar). The former constructs a dynamic structure of error covariance through multiple perturbation simulations and adjusts the predicted values ​​based on the current data, making it suitable for handling the uncertainty propagation of nonlinear system data. However, its efficiency is low for high-dimensional atmospheric models with multiple time windows, and it may face numerical instability. The latter relies on optimization theory to optimize the minimum objective cost function of background field observation and analysis field simulation within a certain time window, making it suitable for applications requiring high resolution, multiple time nodes, and strong constraints.

[0004] Against this backdrop, an improved method combining four-dimensional variational and ensemble Kalman filtering (4DEnVar) is proposed. By introducing the EnKF ensemble assimilation method to estimate the statistical characteristics of the system, and combining it with variational methods to optimize initial conditions, computational efficiency is improved while maintaining high accuracy. Furthermore, to address the issue of unstable data error fluctuations before and after assimilation, artificial intelligence algorithms, especially deep learning methods, have gradually become an important supplement to improving the representation and observation assimilation performance of complex systems. They have also demonstrated strong nonlinear fitting and feature extraction capabilities in fields such as remote sensing inversion, pollution source identification, and meteorological variable reconstruction, greatly expanding the application scope and effectiveness of assimilation techniques. Currently, the main techniques for error re-optimization of data assimilation based on artificial intelligence algorithms include classical neural networks, supervised learning, and generative adversarial networks. These techniques each have their own advantages and disadvantages when used individually, and each has its own requirements for application scenarios and computational resources. Therefore, the dust weather assimilation data construction method designed in this application based on data assimilation and artificial intelligence algorithm error re-optimization will focus more on improving computational efficiency, accuracy, and the ability to handle nonlinear and high-dimensional systems, especially in the fields of extreme dust prediction and dust multi-time window change prediction, which have important application potential. Summary of the Invention

[0005] This application provides a method and apparatus for generating assimilated data for dust storm weather, aiming to create a 4DEnVar_DeepSVR data assimilation scheme based on a simpler artificial intelligence algorithm built with Python for error re-optimization and collaborative analysis. Specifically, it combines deep learning neural networks and statistical learning algorithms to perform post-optimization on errors generated by variable subset assimilation, and the final model algorithm can be applied to the reconstruction of dust storm weather process data. By combining and comparing multi-satellite channel data with actual dust storm weather processes, the reconstruction accuracy of large-scale dust storm data is significantly improved, providing more accurate support for the overall construction of current dust storm data.

[0006] Specifically, this application utilizes artificial intelligence algorithms to re-optimize data errors, combining deep learning, statistical learning, and variational set conformance algorithms to overcome systematic biases in current data assimilation processes, particularly in initial field settings and dust aerosol data simulation. By combining the spatial feature extraction capabilities of CNNs with the advantages of SVR in small-sample nonlinear modeling, and introducing a cyclic iterative mechanism to dynamically correct the background error covariance structure, the application enhances the modeling and assimilation capabilities for high-dimensional nonlinear perturbation processes. This will contribute to providing a more advanced, comprehensive, accurate, and convenient method for dust data assimilation.

[0007] Firstly, this application provides a method for generating assimilation data for dust storm weather, including:

[0008] Acquire multi-channel observation data provided by satellite; wherein, the multi-channel observation data includes background field data and observation field data;

[0009] The 4DEnVar method, which combines four-dimensional variational and ensemble Kalman filtering, is used as the basic assimilation model. A Bayesian optimization process is introduced to make the background error and observation error independent of each other. The multi-channel observation data is input into the basic assimilation model for assimilation calculation, and the assimilated analysis field data is output.

[0010] A joint optimization framework is constructed, which integrates Convolutional Neural Network (CNN) and Support Vector Regression (SVR). Responding to the input analysis field data, the framework utilizes the multi-layer convolutional structure of CNN to capture the spatial correlation and multi-scale features of data across multiple time windows, generating a preliminary post-optimized data field as the global feature of CNN. The high-frequency residuals are then modeled a second time using the ε-insensitive loss function of SVR to obtain the SVR local correction features. These high-frequency residuals represent the difference between the preliminary post-optimized data field and the actual observed data. A dynamic weighted fusion is used to balance the global features of CNN and the local correction features of SVR, with the fused data serving as the final assimilation data for the dust storm.

[0011] In one possible design, a multi-layer convolutional CNN structure is used to capture the spatial correlation and multi-scale features of data across multiple time windows, generating a preliminary post-optimized data field as global features for the CNN, including:

[0012] Based on CNN, a spatially adaptive dynamic weight function is introduced. This dynamic weight function enhances the contribution of assimilated data in extreme value mutation regions through gradient weighting, while suppressing oscillation noise introduced in the assimilation process of stable regions based on the background field similarity of multiple time windows.

[0013] An architecture incorporating residual connections is constructed within the joint optimization framework, and the loss function of the architecture is determined.

[0014] Through a loop iteration mechanism, combined with the dynamic weight function and loss function, optimized data that simultaneously satisfies extreme value fidelity and stability in the stationary region is generated as global features of the CNN.

[0015] In one possible design, the dynamic weighting function is expressed as:

[0016]

[0017] In the formula, For dynamic weighting functions, For background field uncertainty parameters, To analyze the gradient of the field data, The background field data is denoted by max, which represents the maximum value. exp is an exponential function with the natural constant as the base. ISDI is the dust intensity index, used to quantify the intensity of dust during dusty weather.

[0018] In one possible design, the loss function is expressed as:

[0019]

[0020] In the formula, L For loss function, The analysis field data is further optimized for the CNN algorithm. For observation field data, Analyzing the field data, TV represents the total variation that measures the degree of spatial fluctuation. For discrete data X, its calculation formula is: , where X i,j This represents the value of matrix X at position (i,j).

[0021] In one possible design, optimized data that simultaneously satisfies extreme value fidelity and stability in stationary regions is generated as global features for the CNN using the following formula:

[0022]

[0023] In the formula, R ( x ) represents the regularization constraint term based on the CNN framework. The loss function for analyzing background and extreme region data. The loss function for analyzing data in the background field and flat regions. X opt To simultaneously satisfy the requirements of extreme value fidelity and stable data in the steady-state region, X b y represents the background field data, y represents the observation field data, X represents the extreme region analysis data, and arg min is an abbreviation for "argument of the minimum", which is the variable value that minimizes the objective function.

[0024] In one possible design, the dynamic weighted fusion constructs an adaptive weight function based on gradient sensitivity to achieve a dynamic balance between CNN global features and SVR local modified features. The expression for the adaptive weight function is:

[0025]

[0026] In the formula, α , β These represent the contribution weights for controlling the gradient-sensitive region and the stable region, respectively. This refers to the gradient of the global features of the CNN, i.e., the gradient of the observed data after the CNN algorithm has been further optimized. These are the SVR local correction features, i.e., the assimilated data obtained after correction by the SVR algorithm. These are global features of the CNN, i.e., the assimilated data obtained after correction by the CNN algorithm. W DeepSVR ( x ) represents the adaptive weights, exp is the exponential function with the natural constant as the base, and max is the maximum value. The feature scale parameter is used to correct the difference between local features of SVR and global features of CNN.

[0027] In one possible design, the satellite includes the FY4B satellite and / or the Himawari-9 satellite.

[0028] Secondly, this application provides a dust weather assimilation data generation device, the device comprising:

[0029] The data acquisition module is configured to acquire multi-channel observation data provided by the satellite; wherein the multi-channel observation data includes background field data and observation field data;

[0030] The data processing module is configured to use the 4DEnVar method, which combines four-dimensional variational and ensemble Kalman filtering, as the basic assimilation model. A Bayesian optimization process is introduced to make the background error and observation error independent of each other. The multi-channel observation data is input into the basic assimilation model for assimilation calculation, and the assimilated analysis field data is output.

[0031] The joint optimization module is configured to construct a joint optimization framework that integrates a convolutional neural network (CNN) and a support vector regression (SVR). Responding to the input analysis field data, the framework utilizes the multi-layer convolutional structure of the CNN to capture the spatial correlation and multi-scale features of data across multiple time windows, generating a preliminary post-optimized data field as the global feature of the CNN. The high-frequency residuals are then modeled a second time using the ε-insensitive loss function of the SVR to obtain the local correction features of the SVR. These high-frequency residuals represent the difference between the preliminary post-optimized data field and the actual observed data. Dynamic weighted fusion is used to balance the global features of the CNN and the local correction features of the SVR, with the fused data serving as the final assimilation data for the dust storm.

[0032] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the dust weather assimilation data generation method as described in the first aspect and various possible designs of the first aspect.

[0033] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the dust weather assimilation data generation method described in the first aspect and various possible designs of the first aspect.

[0034] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the dust weather assimilation data generation method described in the first aspect and various possible designs of the first aspect.

[0035] The method and apparatus for generating dust weather assimilation data provided in this application have at least the following beneficial effects:

[0036] This application innovatively solves the problem of high computational resource requirements and unsatisfactory final results in current data assimilation methods. By introducing Bayesian posterior perturbation matrix information into 4DEnVar, this invention can better compare and select data after preliminary assimilation, thereby significantly improving the accuracy of the analysis field in subsequent models and enhancing the training effect of the models.

[0037] Specifically, 4DEnVar introduces a Bayesian optimization process to make background error and observation error independent of each other. On the one hand, this effectively avoids neglecting small fluctuations in extreme and stable data due to the model's pursuit of minimizing the loss function during computation. On the other hand, the deep learning and statistical learning parts enhanced by artificial intelligence algorithms employ a perturbation observation method, ensuring that the analysis field for each time window uses different observation errors, thus adhering to the statistical nature of observation errors. Compared to the high computational power requirements of traditional WRF-DA methods, this invention significantly reduces the time and material consumption problems associated with excessive computation in the assimilation model process while improving data assimilation accuracy. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] Figure 1 A schematic diagram illustrating the implementation principle of a method for generating assimilation data for sandstorm weather provided in this application embodiment;

[0040] Figure 2 A flowchart illustrating a method for generating assimilation data for sandstorm weather, provided in an embodiment of this application;

[0041] Figure 3 The image shows a comparison of the dust data provided in the embodiments of this application before and after model assimilation.

[0042] Figure 4 Error comparison diagram between the original data and the 4DEnVar_DeepSVR model provided in the embodiments of this application;

[0043] Figure 5 This is a structural diagram of the dust weather assimilation data generation device provided in an embodiment of this application.

[0044] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0046] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0047] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0048] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0049] This application provides a method for generating assimilated data for dust storms. It uses a four-dimensional variational and ensemble Kalman filtering (4DEnVar) method as the basic data assimilation method, and further employs a joint artificial intelligence algorithm to optimize the assimilated data for errors. This invention simultaneously utilizes multi-channel satellite data from both FY4B and Himawari-9, calculating and retrieving corrected channel data to convert it into dust storm data.

[0050] like Figure 1 The diagram illustrates the implementation principle of a dust storm assimilation data generation method provided in this application. This method expands the 4DEnVar assimilation part based on multi-source channel data from satellites, integrating only the assimilated channel data into dust storm analysis field data. A neural network algorithm is introduced to reconstruct the error covariance matrix of the current analysis field data. A support vector regression algorithm is also introduced to re-optimize the error in different gradient-sensitive regions of the analysis field data, jointly constructing a 4DEnVar_DeepSVR model using the neural network and support vector regression algorithm. Specifically, firstly, a spatiotemporal feature vector is constructed using a CNN to correct the background error covariance. Then, combined with SVR, multi-scale decomposition and nonlinear correction are performed on the residuals of the assimilated comparison data. A dynamic recurrent mechanism enhances the adaptability of the joint framework to the time-varying background field. While maintaining computational efficiency, this method optimizes the subgrid process capture and outlier handling processes by using the Swish activation function and RBF kernel (C=1.5, γ=0.01, ε=0.05) respectively, reducing the peak bias of smoothed data in the dusty region (25<IDSI<30) by 15%, which is significantly better than the 4DEnVar assimilation method alone.

[0051] The following section will elaborate on the implementation process, mechanism, and principle of this method by detailing its specific implementation steps. Figure 2 As shown, the method for generating assimilation data for dust storm weather includes the following steps S10 to S30.

[0052] S10: Acquire multi-channel observation data provided by the satellite; the multi-channel observation data includes background field data and observation field data.

[0053] S20: Based on the 4DEnVar method combining four-dimensional variational and ensemble Kalman filtering, a Bayesian optimization process is introduced to make background error and observation error independent of each other. Multi-channel observation data are input into the basic assimilation model for assimilation calculation, and the assimilated analysis field data are output.

[0054] For example, multi-channel observation data from the FY4B and Himawari-9 satellites are used as the observation field and background field inputs to the 4DEnVar model. In the 4DEnVar model, the optimization capabilities of four-dimensional variational analysis (4DVAR) and the ensemble Kalman filtering (EnKF) ensemble assimilation properties are combined, while a Bayesian optimization process is introduced to make background errors and observation errors independent, performing assimilation calculations on the input multi-channel observation data. After calculation, the assimilated analysis field data is output. The four-dimensional variational method optimizes the initial field by minimizing the error between observed and simulated values ​​within a time window, while the ensemble Kalman filtering uses ensemble samples to estimate the error covariance. 4DEnVar combines both, possessing both the optimization accuracy of the variational method and the statistical properties of ensemble assimilation, enabling more accurate analysis of atmospheric conditions. In this embodiment, atmospheric conditions refer to the state of the assimilated data of dust weather within a continuous time window. In the assimilation process, Bayesian optimization treats background error and observation error as random variables and constructs their probability distributions using Bayes' theorem, making them independent of each other. This allows for a more reasonable fusion of background field information and observation information, thereby improving the accuracy of the assimilated analysis field data.

[0055] S30: Construct a joint optimization framework that integrates Convolutional Neural Network (CNN) and Support Vector Regression (SVR). The joint optimization framework responds to the input analysis field data and utilizes the multi-layer convolutional structure of CNN to capture the spatial correlation and multi-scale features of data from multiple time windows, generating a preliminary post-optimized data field as the global feature of CNN. The high-frequency residuals are then modeled a second time using the ε-insensitive loss function of SVR to obtain the local correction features of SVR. The high-frequency residuals are the difference between the preliminary post-optimized data field and the actual observed data. The balance between the global features of CNN and the local correction features of SVR is achieved through dynamic weighted fusion, and the fused data is used as the final assimilation data for dust weather.

[0056] Step S30 is the core step of this application. Due to cloud cover and sensor data noise limitations, the selected FY4B and Himawari-9 satellite observation data have a weak ability to capture data features in extreme value regions before assimilation. While assimilation can partially compensate for this deficiency, it introduces background field bias in stable regions. To address this contradiction, this embodiment introduces a spatially adaptive dynamic weight function based on the CNN algorithm within the DeepSVR algorithm optimization framework (joint optimization framework). :

[0057]

[0058] In the formula, For background field uncertainty parameters, To analyze the gradient of the field data, The background field data is denoted by max, which represents the maximum value. exp is an exponential function with the natural constant as the base. ISDI is the dust intensity index, used to quantify the intensity of dust during dusty weather.

[0059] This dynamic weighting function enhances the contribution of assimilated data to abrupt changes in extreme value regions through gradient weighting. In stable data regions, it suppresses oscillating noise introduced by the assimilation process based on the similarity of the background field over multiple time windows. Based on this, an architecture incorporating residual connections is constructed, with its loss function L defined as:

[0060]

[0061] In the formula, The analysis field data is further optimized for the CNN algorithm. For observation field data, In analyzing field data, TV represents the total variation that measures the degree of spatial fluctuation. For discrete data... X The calculation formula is as follows: ,in X i,j Representation matrix X The value at position (i,j).

[0062] In the aforementioned joint optimization based on dynamic weights and convolutional neural networks (CNNs), the core objective is to balance the background field ( X b ), observation field ( y obs ) and assimilation data analysis field ( X a The contribution of ) generates optimized data that simultaneously satisfies extreme value fidelity and stability in the stationary region. X opt Its mathematical expression is:

[0063]

[0064] In the formula, R ( x ) represents the regularization constraint term based on the CNN framework. The loss function for analyzing background and extreme region data. Let y be the loss function for the background field and the analysis data of the flat region, y be the observation field data, X be the analysis data of the extreme region, and arg min be the abbreviation for "argument of the minimum", which is the variable value that minimizes the objective function.

[0065] Meanwhile, in the joint optimization framework built based on Support Vector Regression (SVR) and CNN, CNN extracts global features and utilizes its multi-layer convolutional structure to capture the spatial correlation and multi-scale features between data from multiple time windows, generating a preliminary post-optimized data field. Subsequently, in nonlinear residual modeling, SVR is used with an ε-insensitive loss function to perform secondary modeling on the high-frequency residuals output by CNN, suppressing overfitting. Simultaneously, dynamic weighted fusion constructs an adaptive weight function based on gradient sensitivity to achieve a dynamic balance between global features from CNN and local corrected features from SVR. Its mathematical expression is:

[0066]

[0067] In the formula, α=0.6 and β=0.4 control the contribution weights of the gradient-sensitive region and the stable region, respectively. This refers to the gradient of the global features of the CNN, i.e., the gradient of the observed data after the CNN algorithm has been further optimized. These are the SVR local correction features, i.e., the assimilated data obtained after correction by the SVR algorithm. These are global features of the CNN, i.e., the assimilated data obtained after correction by the CNN algorithm. W DeepSVR ( x ) represents adaptive weights. The feature scale parameter is used to correct the difference between local features of SVR and global features of CNN.

[0068] For example, after obtaining the adaptive weights, the fusion of CNN global features and SVR local modified features can be achieved using the following formula to obtain the fused data as the final assimilation data for sandstorm weather:

[0069] y ( x )= W DeepSVR ( x )· y CNN ( x)+(1- W DeepSVR ( x ))· y SVR ( x )

[0070] In the formula, y ( x () represents the merged data.

[0071] In some embodiments, to achieve the optimal combination of model outputs, a two-layer Stacking ensemble architecture is adopted, combining the above-mentioned aspects. The combined use of CNN and SVR jointly corrects the errors in the data before and after assimilation. The assimilated data, compared with actual sandstorm weather events, shows that the efficient feature extraction capability of CNN and the strong generalization ability of SVR effectively complement each other, alleviating the overfitting problem of a single model. Through dynamic weight fusion and Stacking ensemble, the contradiction between the local degradation of CNN and the global sensitivity of SVR is resolved. Furthermore, the SVR ε-insensitive loss function is used to quantify the tolerance threshold for residual correction, enhancing the overall robustness of the model. Finally, the optimized data after model correction exhibits a more continuous spatial distribution characteristic within the transition range between smooth and extreme value regions.

[0072] Therefore, in addition to using posterior uncertainty information as a reference for assimilation, the 4DEnVar_DeepSVR model algorithm of this invention improves the accuracy of the data by re-optimizing the error during the data iteration process. Furthermore, it provides data assimilation operators with more ways to compare the merits of different artificial intelligence algorithm choices. Since the traditional 4DEnVar algorithm assumes that the error of the state variable follows a Gaussian distribution and uses a linear model to describe the error distribution, but the fluctuation error in actual data may have a complex nonlinear structure, this embodiment uses a deep learning neural network to optimize the nonlinear distribution characteristics of the state variable error. These networks can automatically capture the complex patterns of error through training samples (the difference between the assimilation model simulated data and the original baseline data); simultaneously, statistical learning algorithms are used to generate an error field consistent with the observed data, providing more accurate error estimation. The trained deep learning model is embedded into the model assimilation process, replacing the traditional error modeling and optimization steps. During the backward iterative propagation process of assimilation analysis field with the original background field and observation field, the neural network can adaptively adjust the learning rate and weights to accelerate the optimization of the target loss function of the 4DEnVar_DeepSVR algorithm, thereby optimizing the entire model assimilation process.

[0073] To further illustrate the feasibility and advancement of the method proposed in this application, this embodiment uses multi-channel data from FY4B and Himawari-9 satellites as observation data and background field data for experiments. The comparison results of the dust data in the target area before and after model assimilation are as follows: Figure 3 As shown, the error comparison between the original data and the 4DEnVar_DeepSVR model is as follows: Figure 4 As shown.

[0074] from Figure 3 As can be seen from the dust index distribution, the spatial continuity and concentration gradient of dust distribution in the first row (Initial data) time series are poor. For example, the red high-value area on April 7, 2023 is fragmented and scattered, which cannot accurately reflect the actual sheet-like distribution of dust. Although the second row (4DEnVar assimilation) has improved the original data to some extent, there is still a sense of ambiguity in the extreme dust areas (such as the boundary of high-concentration dust belts), and the gradient transition is not natural enough. The third row (4DEnVar_DeepSVR, the method of this application) has achieved a significant improvement. The spatial distribution of dust is more continuous, and the concentration gradient at the provincial boundary is more refined. The large-scale dust area on April 9, 2023, not only maintains the accuracy of overall coverage, but also shows clear gradation characteristics in local small-scale areas (such as the dust front at the inter-provincial boundary), which fully reflects the synergistic effect of the method of this application in the preservation of extreme values ​​and the stability of stable areas.

[0075] Combined Figure 4 The error and correlation analysis shown in the left figure shows that the relative error distribution of this method has a significantly higher log frequency in the low interval (<0.05) than the comparison method, indicating that the overall data bias is greatly suppressed. In the right figure, the row correlation coefficient distribution shows that the Pearson correlation coefficient of this method is concentrated in the interval of 0.7~0.9, and the peak value is higher and the distribution is more concentrated, proving that it has a stronger linear correlation with the actual observation.

[0076] Combination Figure 3 and Figure 4 The experimental results fully validate the feasibility of the proposed 4DEnVar_DeepSVR method from two dimensions: the spatial consistency of dust distribution and the statistical characteristics of data errors. It can effectively integrate multi-source satellite observation data to achieve high-precision assimilation of dust data. At the same time, it shows significant improvements. Compared with the original data and the traditional 4DEnVar method, it has made breakthrough improvements in fidelity in extreme value areas, stability in stable areas, and overall error control, providing reliable technical support for the accurate monitoring and analysis of dust weather.

[0077] In summary, the method proposed in this application can efficiently assimilate extreme dust storm data as well as dust storm data from other stable regions. This model effectively combines Bayesian optimization, the 4DEnVar data assimilation model, and deep learning and statistical learning in the error re-optimization artificial intelligence algorithm to generate better assimilated analysis field data. This addresses the current problems of inaccurate fluctuation ranges after assimilation of dust storm data from extreme and stable regions, and the excessive computational demands of the assimilation process. This innovation lies in ensuring the accuracy and usability of the data through multiple backward iterations of the AI ​​algorithm, which plays a significant role in the prediction process of inserting various types of physical mechanism meteorological elements in subsequent large-scale AI models.

[0078] Furthermore, this method allows users to replace traditional numerical models by training statistical learning algorithms with minimum loss functions and dynamic error weights. The novel data assimilation model described above enhances the ability to perform variational set conformation (4DEnVar) using non-explicit transformation functions (part of the neural network). Simultaneously, this invention addresses the difficulty of current data assimilation algorithms in handling such non-explicit functions, and by changing parameters such as the number of observations, time window length, data ensemble size, observation error, numerical model, number of iterations, and location length, operators can construct data assimilation scenarios as needed.

[0079] This application also provides a dust weather assimilation data generation device, such as... Figure 5 As shown, the dust weather assimilation data generation device includes:

[0080] The data acquisition module 501 is configured to acquire multi-channel observation data provided by the satellite; wherein the multi-channel observation data includes background field data and observation field data;

[0081] The data processing module 502 is configured to use the 4DEnVar method, which combines four-dimensional variational and ensemble Kalman filtering, as the basic assimilation model. It introduces a Bayesian optimization process to make the background error and observation error independent of each other. The multi-channel observation data is input into the basic assimilation model for assimilation calculation, and the assimilated analysis field data is output.

[0082] The joint optimization module 503 is configured to construct a joint optimization framework that integrates a convolutional neural network (CNN) and a support vector regression (SVR). Responding to the input analysis field data, the framework utilizes the multi-layer convolutional structure of the CNN to capture the spatial correlation and multi-scale features of data across multiple time windows, generating a preliminary post-optimized data field as the global feature of the CNN. It then uses the ε-insensitive loss function of the SVR to perform secondary modeling of the high-frequency residuals, obtaining the SVR local correction features. The high-frequency residuals represent the difference between the preliminary post-optimized data field and the actual observed field data. Dynamic weighted fusion is used to achieve a balance between the global features of the CNN and the local correction features of the SVR, with the fused data serving as the final assimilation data for the dust storm.

[0083] This application provides an electronic device. The electronic device may include a processor and a memory, wherein the processor and the memory can communicate; exemplarily, the processor and the memory communicate via a communication bus.

[0084] The processor executes computer execution instructions stored in memory, causing the processor to perform the scheme in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0086] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0087] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer performs the technical solution of the dust weather assimilation data generation method described in the above embodiments.

[0088] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the sandstorm weather assimilation data generation method in the above embodiments.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0090] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0091] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0092] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0093] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0094] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0095] Buses can be Industry Standard Architecture (ISA) buses, Peripheral Component Interconnect (PCI) buses, or Extended Industry Standard Architecture (EISA) buses, etc. Buses can be categorized into address buses, data buses, control buses, etc.

[0096] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0097] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0098] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for generating assimilation data for sandstorm weather, characterized in that, The method includes: Acquire multi-channel observation data provided by satellite; wherein, the multi-channel observation data includes background field data and observation field data; The 4DEnVar method, which combines four-dimensional variational and ensemble Kalman filtering, is used as the basic assimilation model. A Bayesian optimization process is introduced to make the background error and observation error independent of each other. The multi-channel observation data is input into the basic assimilation model for assimilation calculation, and the assimilated analysis field data is output. A joint optimization framework is constructed, which integrates Convolutional Neural Network (CNN) and Support Vector Regression (SVR). Responding to the input analysis field data, the framework utilizes the multi-layer convolutional structure of CNN to capture the spatial correlation and multi-scale features of data across multiple time windows, generating a preliminary post-optimized data field as the global feature of CNN. The high-frequency residuals are then modeled a second time using the ε-insensitive loss function of SVR to obtain the SVR local correction features. These high-frequency residuals represent the difference between the preliminary post-optimized data field and the actual observed data. A dynamic weighted fusion is used to balance the global features of CNN and the local correction features of SVR, with the fused data serving as the final assimilation data for the dust storm. This paper utilizes a multi-layer convolutional CNN structure to capture the spatial correlation and multi-scale features of data from multiple time windows, generating a preliminary post-optimized data field as the global features of the CNN, including: Based on CNN, a spatially adaptive dynamic weight function is introduced. This dynamic weight function enhances the contribution of assimilated data in extreme value mutation regions through gradient weighting, while suppressing oscillation noise introduced in the assimilation process of stable regions based on the background field similarity of multiple time windows. An architecture incorporating residual connections is constructed within the joint optimization framework, and the loss function of the architecture is determined. Through a loop iteration mechanism, combined with the dynamic weight function and loss function, optimized data that simultaneously satisfies extreme value fidelity and stability in the steady region is generated as global features of the CNN. The dynamic weighting function is expressed as follows: In the formula, For dynamic weighting functions, For background field uncertainty parameters, To analyze the gradient of the field data, The background field data is denoted by max, which is the maximum value, exp is the exponential function with the natural constant as the base, and ISDI is the dust intensity index, which is used to quantify the dust intensity in dusty weather. The loss function is expressed as: In the formula, L For loss function, The analysis field data is further optimized for the CNN algorithm. For observation field data, Analyze field data, TV To measure the total variation in the degree of spatial fluctuation; The optimized data that simultaneously satisfies extreme value fidelity and stability in the stationary region is generated using the following formula as the analysis field data after further optimization by the CNN algorithm: In the formula, R ( x ) represents the regularization constraint term based on the CNN framework. The loss function for analyzing background and extreme region data. The loss function for analyzing data in the background field and flat regions. X b For background field data, y For observation field data, X For extreme region analysis data, arg min is the variable value that minimizes the subsequent objective function; The dynamic weighted fusion is based on gradient sensitivity to construct an adaptive weight function, achieving a dynamic balance between CNN global features and SVR local modified features. The expression of the adaptive weight function is as follows: In the formula, α and β are the contribution weights for controlling the gradient-sensitive region and the stable region, respectively. This refers to the gradient of the global features of the CNN, that is, the gradient of the observed data after the CNN algorithm has been further optimized. These are SVR local correction features, i.e., the assimilated data obtained after correction by the SVR algorithm. These are global features of the CNN, i.e., the assimilated data obtained after correction by the CNN algorithm. W DeepSVR ( x ) represents the adaptive weights, exp is the exponential function with the natural constant as the base, and max is the maximum value. The feature scale parameter is used to correct the local features of SVR and the global features of CNN.

2. The method for generating assimilated data for sandstorm weather according to claim 1, characterized in that, The satellites include the FY4B satellite and / or the Himawari-9 satellite.

3. A dust weather assimilation data generation device, used to implement the dust weather assimilation data generation method as described in any one of claims 1-2, characterized in that, The device includes: The data acquisition module is configured to acquire multi-channel observation data provided by the satellite; wherein the multi-channel observation data includes background field data and observation field data; The data processing module is configured to use the 4DEnVar method, which combines four-dimensional variational and ensemble Kalman filtering, as the basic assimilation model. A Bayesian optimization process is introduced to make the background error and observation error independent of each other. The multi-channel observation data is input into the basic assimilation model for assimilation calculation, and the assimilated analysis field data is output. The joint optimization module is configured to construct a joint optimization framework that integrates a convolutional neural network (CNN) and a support vector regression (SVR). Responding to the input analysis field data, the framework utilizes the multi-layer convolutional structure of the CNN to capture the spatial correlation and multi-scale features of data across multiple time windows, generating a preliminary post-optimized data field as the global feature of the CNN. The high-frequency residuals are then modeled a second time using the ε-insensitive loss function of the SVR to obtain the local correction features of the SVR. These high-frequency residuals represent the difference between the preliminary post-optimized data field and the actual observed data. Dynamic weighted fusion is used to balance the global features of the CNN and the local correction features of the SVR, with the fused data serving as the final assimilation data for the dust storm.

4. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the dust weather assimilation data generation method as described in any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the dust weather assimilation data generation method as described in any one of claims 1-2.

Images