A method for rendering extreme rainstorm weather based on generative artificial intelligence

Abstract

The present invention discloses a method for rendering extreme rainstorm weather based on generative artificial intelligence, which belongs to the technical field of image rendering. In the training stage, the present application produces and processes a data set corresponding to rainstorm weather, uses image blocks with features such as wind force and precipitation to establish a training set, and then uses a generative artificial intelligence network to learn these data to obtain a high-simulation rendering model. For new predicted precipitation data, the present application can automatically process and output a highly realistic rainstorm weather visual effect. The present application can efficiently generate realistic rainstorm weather effects and achieve a high level of visual accuracy. The use of the present application can more efficiently process and display the upcoming rainstorm weather conditions, provide the public and decision makers with more intuitive and easy-to-understand weather information, and effectively support emergency management and disaster prevention work.

Description

Technical Field

[0001] The present invention belongs to the technical field of image rendering, and in particular relates to an extreme rainstorm weather rendering method based on generative artificial intelligence. Background Art

[0002] Extreme weather events, such as heavy rain, heavy snow, and heat waves, pose a huge threat to both human society and the natural environment, so there is a need to simulate and predict extreme weather events.

[0003] In the field of heavy rain weather rendering, the main means at present still relies on the professional knowledge and experience of meteorologists. Meteorologists analyze the shape, wind direction, temperature and other data of clouds in two-dimensional or three-dimensional meteorological images to identify areas that may cause heavy rain weather. Then meteorologists manually draw on the two-dimensional images to predict future weather conditions and issue corresponding warnings.

[0004] Some weather stations have begun to introduce some auxiliary forecasting equipment to predict possible extreme weather based on big data analysis or pattern recognition, which will be further confirmed and refined by meteorologists.

[0005] However, the existing technical means have the following problems:

[0006] 1. Insufficient manual forecasting: As precipitation data continues to increase, the resources and manpower available for analysis and forecasting are insufficient. This not only makes the workload difficult to manage, but also may lead to unstable forecast quality due to differences in professional levels among meteorologists. In addition, manual analysis methods are limited in efficiency and accuracy, especially when faced with large-scale or global precipitation data analysis, the risk of false alarms and missed reports increases significantly.

[0007] 2. Insufficient preliminary automated screening: Although preliminary screening through computer programs can reduce some unnecessary workload, this technical solution does not completely overcome the limitations of manual analysis. In addition, the accuracy and reliability of the preliminary automated system still depends on the algorithm settings and the quality of the input data.

[0008] 3. Insufficient machine learning algorithms: There are multiple challenges in practical applications. The formation of rainstorm weather involves many complex factors, such as the nonlinear characteristics of precipitation systems and the influence of different geographical and climatic conditions, which makes it extremely complicated to design a widely applicable model. The performance of machine learning models depends largely on the quality and diversity of training data. If the training data does not fully represent the actual meteorological conditions, the prediction accuracy of the model may be affected. For the rendering of rainstorm images, if certain specific types of rainstorm events are lacking in the training data, the model may not be able to accurately identify these events. The type of rainstorm weather in each region is likely to require specific algorithm adjustments and optimizations. At the same time, the acquisition and processing of high-quality training data is also a difficult problem, especially when the types are diverse and the data is noisy.

[0009] 4. Numerical data is not intuitive enough: Most of the existing meteorological precipitation forecasts are in numerical form. Although they are highly accurate and standardized in the professional field, it is often difficult for the general public to understand and absorb these data. Numerical forecasts usually include parameters such as temperature, humidity, wind speed, and air pressure. These data need to be interpreted professionally to be transformed into information that has practical guiding significance for people's daily lives. Summary of the invention

[0010] In order to solve the problems raised in the above background technology, the present invention provides an extreme rainstorm weather rendering method based on generative artificial intelligence to solve the problems of upper limit of manual prediction, insufficient preliminary automated screening, limitations of machine learning algorithms and lack of intuitiveness of numerical data.

[0011] To achieve the above object, the present invention provides the following technical solutions:

[0012] A method for rendering extreme rainstorm weather based on generative artificial intelligence includes the following steps:

[0013] S1: Obtain rainstorm data and rainstorm images from the meteorological data source, form a rainstorm data set including paired data of rainstorm data and rainstorm images, and divide the rainstorm data set into a training set, a validation set, and a test set;

[0014] S2: Calibrate and correct the data in the rainstorm dataset;

[0015] S3: Set up the ControlNet network structure, introduce a zero convolution layer into the ControlNet network structure and add a new permutation self-attention module. The output of the permutation self-attention module is residually connected with the output of the feature encoder of the ControlNet network structure.

[0016] S4: Add a weather physics model control module to the ControlNet network structure. The weather physics model control module is used to make the generated image conform to the actual physical laws. Set a dynamic control module. The dynamic control module is used to dynamically adjust the precipitation intensity and distribution during the image generation process according to the input precipitation parameters. Insert the permutation self-attention module into the middle layer of the U-Net encoder and decoder of the ControlNet network structure to obtain the ControlNet model.

[0017] S5: Combine the training set, validation set, and test set and use the ControlNet model training process to train the ControlNet model;

[0018] S6: Use the trained ControlNet model to render each image uploaded by the user;

[0019] S7: Display the rendering results on the interface.

[0020] Preferably, S1 is specifically:

[0021] S1.1: Obtain the required rainstorm data and rainstorm images from the meteorological data source. The rainstorm data includes horizontal wind, vertical wind and total precipitation;

[0022] S1.2: Use linear interpolation to fill missing values ​​in the rainstorm data, identify and process outliers in the rainstorm data through the interquartile range method, and apply Gaussian filtering to remove noise in the rainstorm images to form a rainstorm dataset including paired data of rainstorm data and rainstorm images.

[0023] Preferably, S2 is specifically:

[0024] The rainstorm dataset is scored, and the rainstorm weather features in the rainstorm images are corrected. For extreme weather events such as rainstorms, the total precipitation in the rainstorm images is corrected pixel by pixel based on historical precipitation data and rainstorm image analysis results. The corrected rainstorm dataset is divided into training set, validation set and test set.

[0025] Preferably, in S3, the introduced new permutation self-attention module includes multiple arranged self-attention groups, each self-attention group consists of multiple PABs and a 3×3 convolutional layer.

[0026] Preferably, in S4, the replacement self-attention module is inserted into the multi-scale feature fusion position of the middle layer of the U-Net encoder and decoder of the ControlNet network structure.

[0027] Preferably, in S5, the ControlNet network structure first extracts features of the rainstorm image through a convolutional neural network (CNN), then uses a codec structure in the ControlNet network structure to encode the extracted features into high-dimensional feature representations and integrate them to generate a high-resolution rainstorm image, and then decodes the extracted high-dimensional features into control signals for generating the image;

[0028] The zero convolution layer optimizes the resolution and details of the high-resolution rainstorm image by adjusting the convolution kernel size to 3x3 and using the "SAME" padding strategy. Then, by controlling the convolution kernel size and padding method, the influence of the characteristics of the rainstorm image is changed and the control weight setting is optimized.

[0029] The permutation self-attention module is used to permute the rainstorm feature map. The permuted feature map is input into the self-attention module of the ControlNet network structure. The feature relationship is captured by calculating the attention weights between the features. Finally, the output of the permutation self-attention module is residually connected with the output of the feature encoder to generate a high-resolution meteorological image.

[0030] The ControlNet network structure locks and clones all parameters of Stable Diffusion to a trainable copy, trains the copy using an external condition vector, and stacks multiple ControlNet blocks for training. The ControlNet model is trained based on the parameters in the meteorological data to achieve scene rendering with controllable precipitation.

[0031] Preferably, in S5, the ControlNet model training process is used, combined with the permuted self-attention module, and a learning rate of 0.0001 is selected for model fine-tuning.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] In the visualization rendering of rainstorm weather events, this application has developed a generative artificial intelligence-based extreme rainstorm weather rendering method to address the problems of small precipitation data sets, unrealistic or inconsistent effects, etc., which can efficiently generate realistic rainstorm weather effects and achieve a high level of visual accuracy.

[0034] During the training phase, a data set corresponding to heavy rain weather is created and processed. A training set is established using image blocks with features such as wind speed and precipitation. Subsequently, a generative artificial intelligence network is used to learn these data to obtain a highly realistic rendering model. For new predicted precipitation data, the rendering model can be used to automatically process it and output a highly realistic visual effect of heavy rain weather.

[0035] By using this application, weather forecasting agencies can more efficiently process and display upcoming rainstorm weather conditions, provide the public and decision makers with more intuitive and easy-to-understand weather information, and effectively support emergency management and disaster prevention work. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flowchart of the application;

[0037] Figure 2 It is a schematic diagram of the structure of the encoder and decoder;

[0038] Figure 3 This is a schematic diagram of the ControlNet structure;

[0039] Figure 4 Schematic diagram of the structure of CNN. DETAILED DESCRIPTION

[0040] In order to facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] Embodiment 1:

[0042] A method for rendering extreme rainstorm weather based on generative artificial intelligence, such as Figure 1 , Figure 2 , Figure 3 and Figure 4 As shown, the following steps are included:

[0043] Step 1: Obtain the required rainstorm data and rainstorm images from the meteorological data source. The rainstorm data includes horizontal wind, vertical wind and total precipitation. Then use linear interpolation to fill missing values, use the IQR (interquartile range) method to identify and process outliers, apply Gaussian filtering to remove noise, and then form paired data according to the date and time period of the data. The paired data are divided into training set, validation set and test set, and the Z-score standardization method is used to ensure that the model uses consistent standardized parameters at each stage.

[0044] Step 2: Based on step 1, calibrate the rainstorm dataset of real meteorological information. Through meteorological data experts, the paired data are scored and the rainstorm weather characteristics in the rainstorm images are carefully corrected to ensure the authenticity and accuracy of the training data. For extreme weather events such as rainstorms, experts will perform pixel-by-pixel correction on the total precipitation in the rainstorm images based on historical precipitation data and rainstorm image analysis results to ensure that the distribution of precipitation is consistent with the actual observation data, so as to improve the model's ability to predict and generate rainstorm weather.

[0045] Step 3: Use the ControlNet network structure, such as Figure 3 As shown in the figure, the ControlNet network structure first extracts rainstorm image features from the prepared scene data through a convolutional neural network CNN. The image features include edges, textures, etc. Then, the encoder-decoder structure is used to encode the input data into a high-dimensional feature representation and integrate these features to generate a high-resolution rainstorm image. The extracted high-dimensional features are then decoded into control signals for the generated image. In order to optimize the model performance, zero convolutional layers are introduced into the network to enhance the expressiveness of the feature map. The resolution and details of the feature map are optimized by adjusting the convolution kernel size to 3x3 and using the 'SAME' padding strategy. By controlling the convolution kernel size and padding method, the influence of the rainstorm image features is changed, and the control weight setting is optimized to balance the importance of the ControlNet network structure and the prompt generation prompt. In order to enhance the model’s ability to understand and render extreme weather features, a new permutation self-attention module is added to the ControlNet network structure. It contains multiple permutation self-attention groups, each of which consists of several PABs and a 3×3 convolutional layer to permute the rainstorm feature map. The permuted feature map is input into the self-attention module, and a wider range of feature relationships is captured by calculating the attention weights between features. Finally, the output of the permutation self-attention module is residually connected to the output of the feature encoder, which enhances the interaction between high-level features while retaining the original information, thereby generating high-resolution meteorological images.

[0046] The ControlNet network structure locks and clones all parameters of Stable Diffusion to a trainable copy, trains the copy using an external condition vector, and stacks multiple ControlNet blocks for training. The ControlNet model is trained based on the parameters in the meteorological data to achieve scene rendering with controllable precipitation.

[0047] This enables it to learn how to generate results based on new control conditions. The model is trained based on parameters such as precipitation in meteorological data, and ultimately scene rendering with controllable precipitation is achieved.

[0048] Step 4: Add a weather physics model control module to the ControlNet architecture to make the generated rainstorm weather image conform to the actual physical laws. Then, by fusing multimodal data (satellite images) as the control input of the ControlNet architecture, the generation accuracy of the rainstorm weather scene is improved. Design a dynamic control module that can dynamically adjust the precipitation intensity and distribution during the image generation process according to the input precipitation parameters, and generate rainstorms of different intensities and forms by adjusting the parameters. The introduced permutation self-attention is an attention mechanism that can reorganize and rearrange the input features. Insert the permutation self-attention module into the middle layer of the U-Net encoder and decoder, especially at the location of multi-scale feature fusion, so as to better capture cross-scale information. Combined with the dynamic adjustment ability of the permutation self-attention, the model can flexibly adjust the feature permutation strategy when processing different control input maps, thereby optimizing the quality of the generated rainstorm weather map.

[0049] Step 5: Use the ControlNet model training process, combined with the unique features of permuted self-attention, select a learning rate of 0.0001 for model fine-tuning to avoid overfitting, and combine early stopping to prevent over-optimization during training. Consider adding diversity loss or permutation consistency loss to ensure that the permutation operation does not cause the quality of the generated image to degrade. For each image uploaded by the user, the ControlNet model predicts whether each image block meets the requirements of the rainstorm area. Use this information to generate a graph that represents the distribution of rainstorm features on the image.

[0050] Step 6: To optimize the rendering effect, an intuitive UI interface is designed to allow users to adjust parameters such as precipitation and wind speed by dragging sliders, and preview the generated rainstorm weather effect map in real time. The interface provides two modes: text description and image preview to enhance the user experience and obtain more diverse and accurate extreme weather areas.

[0051] The present invention aims to solve the problem of high-resolution, high-realism rendering of rainstorm weather events, especially in the case of limited data sets. In the visual rendering of rainstorm weather events, the problems of small precipitation data sets, unrealistic or inconsistent effects, etc., this application can efficiently generate realistic rainstorm weather effects and achieve a high level of visual accuracy.

[0052] During the training phase, a data set corresponding to heavy rain weather is created and processed, and a training set is established using image blocks with features such as wind speed and precipitation. Subsequently, a generative artificial intelligence network is used to learn from this data to obtain a highly realistic rendering model.

[0053] For new predicted precipitation data, this application can automatically process and output highly realistic rainstorm weather visual effects. Using this application, weather forecasting agencies can more efficiently process and display upcoming rainstorm weather conditions, provide the public and decision makers with more intuitive and easy-to-understand weather information, and effectively support emergency management and disaster prevention work.

Claims

1. A method for rendering extreme rainstorm weather based on generative artificial intelligence, characterized in that: The following steps are involved: S1: Obtain rainstorm data and rainstorm images from the meteorological data source, form a rainstorm data set including paired data of rainstorm data and rainstorm images, and divide the rainstorm data set into a training set, a validation set, and a test set; S2: Calibrate and correct the data in the rainstorm dataset; S3: Set up the ControlNet network structure, introduce a zero convolution layer into the ControlNet network structure and add a new permutation self-attention module. The output of the permutation self-attention module is residually connected with the output of the feature encoder of the ControlNet network structure. S4: Add a weather physics model control module to the ControlNet network structure. The weather physics model control module is used to make the generated image conform to the actual physical laws. Set a dynamic control module. The dynamic control module is used to dynamically adjust the precipitation intensity and distribution during the image generation process according to the input precipitation parameters. Insert the permutation self-attention module into the middle layer of the U-Net encoder and decoder of the ControlNet network structure to obtain the ControlNet model. S5: Combine the training set, validation set, and test set and use the ControlNet model training process to train the ControlNet model; S6: Use the trained ControlNet model to render each image uploaded by the user; S7: Display the rendering results on the interface.

2. The method for rendering extreme rainstorm weather based on generative artificial intelligence according to claim 1, characterized in that: S1 is specifically: S1.1: Obtain the required rainstorm data and rainstorm images from the meteorological data source. The rainstorm data includes horizontal wind, vertical wind and total precipitation; S1.2: Use linear interpolation to fill missing values ​​in the rainstorm data, identify and process outliers in the rainstorm data through the interquartile range method, and apply Gaussian filtering to remove noise in the rainstorm images to form a rainstorm dataset including paired data of rainstorm data and rainstorm images.

3. The method for rendering extreme rainstorm weather based on generative artificial intelligence according to claim 1, characterized in that: S2 is specifically: The rainstorm dataset is scored, and the rainstorm weather features in the rainstorm images are corrected. For extreme weather events such as rainstorms, the total precipitation in the rainstorm images is corrected pixel by pixel based on historical precipitation data and rainstorm image analysis results. The corrected rainstorm dataset is divided into training set, validation set and test set.

4. The method for rendering extreme rainstorm weather based on generative artificial intelligence according to claim 1, characterized in that: In S3, the new permutation self-attention module introduced includes multiple arranged self-attention groups, each of which consists of multiple PABs and a 3×3 convolutional layer.

5. The method for rendering extreme rainstorm weather based on generative artificial intelligence according to claim 4, characterized in that: In S4, the permuted self-attention module is inserted into the multi-scale feature fusion position of the middle layer of the U-Net encoder and decoder of the ControlNet network structure.

6. The method for rendering extreme rainstorm weather based on generative artificial intelligence according to claim 5, characterized in that: In S5, the ControlNet network structure first extracts features from the rainstorm image through a convolutional neural network (CNN), then uses the encoder-decoder structure in the ControlNet network structure to encode the extracted features into high-dimensional feature representations and integrate them to generate a high-resolution rainstorm image, and then decodes the extracted high-dimensional features into control signals for the generated image; The zero convolution layer optimizes the resolution and details of the high-resolution rainstorm image by adjusting the convolution kernel size to 3x3 and using the "SAME" padding strategy. Then, by controlling the convolution kernel size and padding method, the influence of the characteristics of the rainstorm image is changed and the control weight setting is optimized. The permutation self-attention module is used to permute the rainstorm feature map. The permuted feature map is input into the self-attention module of the ControlNet network structure. The feature relationship is captured by calculating the attention weights between the features. Finally, the output of the permutation self-attention module is residually connected with the output of the feature encoder to generate a high-resolution meteorological image. The ControlNet network structure locks and clones all parameters of stable diffusion to a trainable copy, trains the copy using external condition vectors, and trains the ControlNet model based on the parameters in meteorological data by stacking multiple ControlNet blocks and performing training to achieve scene rendering of controllable precipitation.

7. The method for rendering extreme rainstorm weather based on generative artificial intelligence according to claim 6, characterized in that: In S5, the ControlNet model training process is used, combined with the permuted self-attention module, and a learning rate of 0.0001 is selected for model fine-tuning.

Images