Radar Echo Prediction Method and Device Based on Latent Diffusion Model

Abstract

The present invention provides a radar echo prediction method and device based on a latent diffusion model, relating to the technical field of radar echo prediction, including: inputting environmental grid data and the live echo sequence in the pixel space of a target area into the latent diffusion model; through a multi-layer convolutional neural network in the conditional encoder, successively extracting features from each environmental grid data in the order of decreasing scale to generate the environmental conditional features of the target area; converting the live echo sequence in the pixel space into a low-dimensional dense vector of the live echo in the latent space through the echo encoder, and performing noise addition processing; under the action of the low-dimensional dense vector of the live echo in the latent space and the environmental conditional features, successively processing the low-dimensional dense vector of the live echo after noise addition through multiple cascaded noise reduction structures in the denoising model, and then converting it into a predicted echo sequence in the pixel space through the echo decoder; so as to alleviate the technical problem of low accuracy of radar echo prediction results existing in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of radar echo prediction, and in particular, to a radar echo prediction method and device based on a latent diffusion model. Background Art

[0002] Currently, radar echo prediction results are commonly used to estimate precipitation, and radar echo extrapolation methods include traditional advection methods, deterministic deep learning methods, and methods using image generation models.

[0003] The inventor has found through research that in practical applications, the traditional advection method predicts the future echo field by translating the current radar echo field in a certain direction. Since it does not consider the evolution process of the precipitation system, the precipitation result estimated in this way has too high an intensity. When the deterministic deep learning method is applied to radar echo prediction, it can better capture the precipitation area, but due to the lack of sensitivity to the change of precipitation intensity, the prediction result obtained in this way is relatively fuzzy. The method using the image generation model has limited accuracy of the prediction result of image extrapolation because it does not consider the actual factors that have an important impact on the evolution of radar echoes. Summary of the Invention

[0004] The purpose of the present invention is to provide a radar echo prediction method and device based on a latent diffusion model to alleviate the technical problem of low accuracy of radar echo prediction results existing in the prior art.

[0005] In a first aspect, the present invention provides a radar echo prediction method based on a latent diffusion model, including:

[0006] Preprocess the radar data and environmental data of the target area to obtain the actual echo sequence in the pixel space and multiple environmental grid data, and input the environmental grid data and the actual echo sequence in the pixel space into the latent diffusion model; wherein, the latent diffusion model is composed of an echo encoder, a noise addition model, a noise reduction model, a conditional encoder, and an echo decoder;

[0007] Through the multi-layer convolutional neural network in the conditional encoder, extract features from each environmental grid data in turn in the order of decreasing scale to generate the environmental condition features of the target area;

[0008] The actual echo sequence in the pixel space is converted into a low-dimensional dense vector of the actual echo in the latent space by the echo encoder;

[0009] Input the low-dimensional dense vector of the actual echo into the noise addition model, and output the low-dimensional dense vector of the actual echo after noise addition through the sequential processing of multiple cascaded noise addition structures in the noise addition model;

[0010] Under the action of the live echo low-dimensional dense vector in the latent space and the environmental condition features, the denoised live echo low-dimensional dense vector is sequentially processed by a plurality of cascaded denoising structures in the denoising model, and a predicted echo low-dimensional dense vector is output;

[0011] The predicted echo low-dimensional dense vector is converted into a predicted echo sequence in the pixel space by the echo decoder.

[0012] In an alternative embodiment, the environmental grid data includes terrain information, satellite observation data, model forecast data, spatial information, and temporal information; the multi-layer convolutional neural network includes a plurality of feature extraction branches; each feature extraction branch is composed of a convolutional layer, an activation function, and a max pooling layer;

[0013] The step of generating the environmental condition features of the target area by sequentially extracting features from each of the environmental grid data in descending order of scale through the multi-layer convolutional neural network in the conditional encoder includes:

[0014] Input the terrain information at the first preset scale into the first feature extraction branch for processing, and output a first feature extraction result;

[0015] Overlay the first feature extraction result and the satellite observation data at the second preset scale, and input the result into the second feature extraction branch for processing, and output a second feature extraction result;

[0016] Overlay the second feature extraction result and the model forecast data at the third preset scale, and input the result into the third feature extraction branch for processing, and output a third feature extraction result;

[0017] Overlay the third feature extraction result and the temporal information and spatial information at the fourth preset scale to obtain the environmental condition features of the target area.

[0018] In an alternative embodiment, the step of sequentially processing the denoised live echo low-dimensional dense vector by a plurality of cascaded denoising structures in the denoising model under the action of the live echo low-dimensional dense vector in the latent space and the environmental condition features to output a predicted echo low-dimensional dense vector includes:

[0019] Input the live echo low-dimensional dense vector in the latent space, the environmental condition features, and the denoised live echo low-dimensional dense vector into the first denoising structure of the denoising model;

[0020] After being affected by the live echo low-dimensional dense vector in the latent space and the environmental condition features by the first denoising structure, the denoised live echo low-dimensional dense vector is denoised, and a first denoised live echo low-dimensional dense vector is output;

[0021] Repeat the following process until each noise reduction structure in the noise reduction model has been traversed and the first noise reduction actual echo low-dimensional dense vector output is used as the predicted echo low-dimensional dense vector, including:

[0022] Input the actual echo low-dimensional dense vector of the latent space, the environmental condition features, and the first noise reduction actual echo low-dimensional dense vector into the second noise reduction structure;

[0023] Under the influence of the actual echo low-dimensional dense vector of the latent space and the environmental condition features by the second noise reduction structure, perform noise reduction processing on the first noise reduction actual echo low-dimensional dense vector, and output the second noise reduction actual echo low-dimensional dense vector;

[0024] Use the second noise reduction actual echo low-dimensional dense vector as the new first noise reduction actual echo low-dimensional dense vector.

[0025] In an alternative embodiment, the method further includes:

[0026] Perform weighted summation on the differences between each current frame and the previous frame within the echo range of the target area of the predicted echo low-dimensional dense vector to determine the loss function of the potential diffusion model.

[0027] In an alternative embodiment, the loss function can be calculated by the following formula:

[0028]

[0029] Where, is the tensor of the previous frame in the predicted echo low-dimensional dense vector, is the tensor of the current frame in the predicted echo low-dimensional dense vector, is the number of image frames in the predicted echo low-dimensional dense vector, is the weight of the i-th interval of the current frame within the echo range of the target area, is the weight of the current frame when the j-th frame image is the current frame.

[0030] In an alternative embodiment, the method further includes:

[0031] Estimate the precipitation in the target area using the predicted echo sequence in the pixel space.

[0032] In an alternative embodiment, the steps of preprocessing the radar data and environmental data of the target area to obtain the actual echo sequence in the pixel space and multiple environmental grid data include:

[0033] Filter out outliers and interference noise in the radar data of the target area, and then slice it according to a preset scale to generate a live echo sequence in the pixel space of the first preset scale;

[0034] Slice the environmental data of the target area according to the corresponding preset resolution to generate multiple environmental grid data of preset scales.

[0035] In a second aspect, the present invention provides a radar echo prediction device based on a latent diffusion model, including:

[0036] A preprocessing module preprocesses the radar data and environmental data of the target area to obtain a live echo sequence in the pixel space and multiple environmental grid data, and inputs the environmental grid data and the live echo sequence in the pixel space into the latent diffusion model; wherein, the latent diffusion model is composed of an echo encoder, a noise addition model, a noise reduction model, a conditional encoder, and an echo decoder;

[0037] A first feature extraction module sequentially extracts features from each of the environmental grid data in descending order of scale through a multi-layer convolutional neural network in the conditional encoder to generate environmental conditional features of the target area;

[0038] A second feature extraction module converts the live echo sequence in the pixel space into a low-dimensional dense vector of the live echo in the latent space through the echo encoder;

[0039] A noise addition module inputs the low-dimensional dense vector of the live echo into the noise addition model, and outputs a low-dimensional dense vector of the live echo after noise addition through sequential processing of multiple cascaded noise addition structures in the noise addition model;

[0040] A noise reduction module sequentially processes the low-dimensional dense vector of the live echo after noise addition through multiple cascaded noise reduction structures in the noise reduction model under the action of the low-dimensional dense vector of the live echo in the latent space and the environmental conditional features, and outputs a low-dimensional dense vector of the predicted echo;

[0041] A decoding module converts the low-dimensional dense vector of the predicted echo into a predicted echo sequence in the pixel space through the echo decoder.

[0042] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a program stored on the memory and capable of running on the processor. When the processor executes the program, the method described in any one of the foregoing embodiments is implemented.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the method described in any one of the foregoing embodiments is implemented.

[0044] The embodiments of the present invention provide a method and device for predicting radar echoes based on a latent diffusion model. First, environmental factors that affect radar echoes are encoded to generate conditional features. Then, through the echo encoder and echo decoder in the latent diffusion model, the pixel space is transformed into a low-dimensional dense latent space for diffusion to achieve richer feature learning. Using these environmental features and low-dimensional dense vector features as conditions, the low-dimensional dense vector of the noisy actual echo after being processed by the latent diffusion model for noise addition is denoised to obtain a relatively accurate low-dimensional dense vector of the predicted echo, which is then transformed into a predicted echo sequence in the pixel space.

[0045] Other features and advantages of the present invention will be described in the following specification. And, partly, they will become obvious from the specification, or be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0046] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0048] Figure 1 It is a flowchart of a method for predicting radar echoes based on a latent diffusion model provided by an embodiment of the present invention;

[0049] Figure 2 It is a schematic diagram of the architecture of a latent diffusion model provided by an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of the architecture of a conditional encoder provided by an embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram for comparing the true value and predicted value of a deterministic model provided by an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram for comparing the true value and predicted value of an image extrapolation model provided by an embodiment of the present invention;

[0053] Figure 6 It is a schematic diagram for comparing the true value and predicted value of a latent diffusion model provided by an embodiment of the present invention;

[0054] Figure 7 Schematic diagram of functional modules of a radar echo prediction device based on a latent diffusion model provided by an embodiment of the present invention;

[0055] Figure 8 Schematic diagram of the hardware architecture of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0057] Currently, the inventor has found through research that the radar echo prediction method based on image extrapolation lacks an accurate description of the movement and generation / elimination characteristics of precipitation cloud clusters. The movement of precipitation cloud clusters is not only strongly correlated with conditions such as wind speed, wind direction, and terrain, but also its generation / elimination process is closely related to phenomena such as humidity, temperature, and whether strong convection occurs. These environmental factors have an important impact on the evolution of radar echoes, while traditional image extrapolation methods often ignore these factors, resulting in limited accuracy and reliability of prediction results. In the radar echo prediction with a long time series, due to the large time span, external environmental factors may change significantly. For example, during the movement of the echo, the wind force may change, resulting in a change in the echo movement speed, etc. These changes have an important impact on the movement and generation / elimination characteristics of precipitation cloud clusters. The method based on image extrapolation often fails to effectively integrate these external environmental characteristics, resulting in a decrease in prediction accuracy.

[0058] Based on this, a radar echo prediction method and device based on a latent diffusion model provided by an embodiment of the present invention can achieve relatively accurate radar echo prediction by considering the environmental factors that affect radar echoes and the richer learning expression of the latent diffusion model in the latent space.

[0059] To facilitate the understanding of this embodiment, first, a radar echo prediction method based on a latent diffusion model disclosed in an embodiment of the present invention will be introduced in detail. This method can be applied to intelligent control devices such as a host computer, a server, and a controller.

[0060] Figure 1 Flowchart of a radar echo prediction method based on a latent diffusion model provided by an embodiment of the present invention.

[0061] Refer to Figure 1 As shown, the method may include the following steps:

[0062] Step S102: Preprocess the radar data and environmental data of the target area to obtain the live echo sequence in the pixel space and multiple environmental grid data, and input the environmental grid data and the live echo sequence in the pixel space into the latent diffusion model;

[0063] Among them, the target area can be understood as the actual geographical area range for predicting and estimating the radar echo result; the latent diffusion model provided by the embodiment of the present invention is a pre-trained model; the latent diffusion model is composed of an echo encoder, a noise addition model, a noise reduction model, a conditional encoder, and an echo decoder, as Figure 2 shown; each component in the latent diffusion model executes steps S104 - S112 as shown in Figure 2 shown.

[0064] Step S104: Through the multi-layer convolutional neural network in the conditional encoder, sequentially extract features from each environmental grid data in the order of decreasing scale to generate the environmental condition features of the target area.

[0065] Here, the conditional encoder uses a multi-layer convolutional neural network to encode environmental grid data at different scales, so as to realize feature extraction in multi-source environmental grid data.

[0066] Step S106: The live echo sequence in the pixel space is converted into a low-dimensional dense vector of the live echo in the latent space through the echo encoder.

[0067] The structure of the echo encoder is not limited, and models such as VAE or VQGAN can be used, as Figure 2 shown. In the embodiment of the present invention, the VAE Encoder is taken as an example for illustration; the echo encoder is a single-channel single-frame radar data encoder, and the encoder extracts features from the input high-dimensional vector and converts it into a sequence of feature maps of low-dimensional dense vectors, that is, the low-dimensional dense vector of the live echo; for example, it is restricted that the size of each image frame in the input live echo sequence in the pixel space is 512x512 and the number of channels is 1; the size of the image frame in the output low-dimensional dense vector of the live echo is 64x64 and the number of channels is 4.

[0068] Step S108: Input the low-dimensional dense vector of the live echo into the noise addition model, and output the low-dimensional dense vector of the live echo after noise addition through the sequential processing of multiple cascaded noise addition structures in the noise addition model.

[0069] For example, the low-dimensional dense vector of the live echo is input into the second noise addition structure for further noise addition processing after being processed by the first noise addition structure in the noise addition model, and the above processing process is repeated until each noise addition structure in the noise addition model has executed the noise addition operation and then terminated, and the low-dimensional dense vector of the live echo after noise addition is output.

[0070] Step S110, under the action of the live echo low-dimensional dense vector in the latent space and the environmental condition features, the denoised live echo low-dimensional dense vector is sequentially processed through multiple cascaded denoising structures in the denoising model, and the predicted echo low-dimensional dense vector is output.

[0071] The radar echo extrapolation prediction that fuses external environmental features is realized through the denoising model; specifically, based on the powerful generation ability of the latent diffusion model and the correct encoding of conditional information by the conditional encoder, the conditional injection based on the DiT Denoise denoising structure in the diffusion model is realized, and environmental factors such as time, space, terrain, satellite observation, and model prediction are fully fused as conditions in the denoising stage of the latent diffusion model to realize accurate and efficient radar echo prediction.

[0072] Step S112, the predicted echo low-dimensional dense vector is converted into a predicted echo sequence in the pixel space by the echo decoder.

[0073] The echo decoder VAE Decoder is a single-channel single-frame radar data decoder, which is used to restore the low-dimensional latent space vector to a continuous frame of radar echo in the high-dimensional pixel space.

[0074] In a preferred embodiment of practical application, first, the environmental factors that affect each pair of radar echoes are encoded to generate conditional features, and then the pixel space is transferred to a low-dimensional dense latent space through the echo encoder and echo decoder in the latent diffusion model for diffusion to achieve richer feature learning; using this environmental feature and low-dimensional dense vector feature as conditions, the denoised live echo low-dimensional dense vector after being processed by the latent diffusion model with noise addition is denoised to obtain a relatively accurate predicted echo low-dimensional dense vector, and then it is converted into a predicted echo sequence in the pixel space; this method has a relatively high accuracy in obtaining the radar echo prediction result.

[0075] In some embodiments, the radar data is quality controlled, screened, and normalized through data preprocessing, and the environmental condition data is standardized and constructed; exemplarily, step S102 includes:

[0076] Step 1.1), filtering out the outliers and interference noises in the radar data of the target area, and then slicing according to a preset scale to generate a live echo sequence in the pixel space of the first preset scale.

[0077] Slice the radar data at a scale of 512x512 to meet the input requirements of the potential diffusion model and ensure the consistency and processability of the radar data. Then, perform quality control on the clutter in the radar echo to reduce the impact of noise on subsequent analysis. In practical applications, filters or other signal processing components can be used to identify and remove outliers. At the same time, small echoes in the radar data are also filtered to reduce interference and noise in the data and improve the data quality. Finally, data normalization is performed to make the data have a unified scale, which helps the model to learn and generalize better.

[0078] Specifically, for the sliced data of the composite reflectivity mosaic of the accessed radar echo, the radar data is a continuous radar observation sequence with an image frame scale of 512*512. Small echoes below 5 dBz are filtered out and normalized.

[0079] Step 1.2), slice the environmental data of the target area according to the corresponding preset resolution to generate environmental grid data of multiple preset scales.

[0080] Here, the terrain information, satellite observations, and model forecast data in the environmental data are respectively formed into standardized grid data with resolutions of 0.01 degrees, 0.02 degrees, and 0.4 degrees through interpolation and other means, sliced according to the same area as the radar, and the time and space position information is extended and mapped onto the grid.

[0081] Specifically, the terrain information is interpolated onto a 4-km grid and converted into slices with a scale of 512*512, the satellite observation data is interpolated onto an 8-km grid and converted into slices with a scale of 256*256, the model forecast data is interpolated onto a 16-km grid and converted into slices with a scale of 128*128, and the time and space are mapped into slices of 64*64 to ensure the alignment of the slices of the radar data and the external environmental data.

[0082] In practical applications, the environmental grid data includes terrain information, satellite observation data, model forecast data, spatial information, and time information; as Figure 3 shown, the multi-layer convolutional neural network includes multiple feature extraction branches; each feature extraction branch consists of a convolutional layer, an activation function, and a max-pooling layer; the conditional encoder can achieve feature extraction and learning of environmental factors such as time, space, terrain, satellite observations, and model forecasts at different scales;

[0083] Exemplarily, this step S104 can be implemented through the following steps, specifically including:

[0084] Step 2.1), input the terrain information of the first preset scale of 512x512 into the first feature extraction branch for processing, and output the first feature extraction result.

[0085] Among them, the convolutional layer Conv, activation function Relu, and max pooling layer MaxPooling of the first feature extraction branch are processed layer by layer, and the first feature extraction result is output.

[0086] In step 2.2), the first feature extraction result and satellite observation data of the second preset scale of 256x256 are superimposed and then input into the second feature extraction branch for processing, and the second feature extraction result is output.

[0087] Here, the processing process of the second feature extraction branch is similar to that of the first feature extraction branch, and will not be elaborated here.

[0088] In step 2.3), the second feature extraction result and model forecast data of the third preset scale of 128x128 are superimposed and then input into the third feature extraction branch for processing, and the third feature extraction result is output.

[0089] Similarly, the processing process of the third feature extraction branch is similar to that of the first feature extraction branch, and will not be elaborated here.

[0090] In step 2.4), the third feature extraction result is superimposed with the time information and space information of the fourth preset scale of 64×64 to obtain the environmental condition features of the target area.

[0091] It can be understood that first, the terrain information is subjected to feature extraction through a convolutional layer (Conv), then the ReLU activation function is applied to increase the non-linear expression ability of the network, and then the max pooling layer (MaxPooling) is used to reduce the feature dimension and extract important features. And so on, satellite observation data, model forecast data, and spatio-temporal feature information are fused in turn to form the final conditional features (Condition Feature), providing rich information reference for subsequent prediction tasks. This multi-convolution conditional encoder fuses various environmental condition data at different scales to form an environmental feature extraction network, and the output feature size of this network is 64×64.

[0092] In some embodiments, the noise reduction step S110 of the embodiment of the present invention may specifically include the following steps:

[0093] In step 3.1), the low-dimensional dense vector of the actual echo in the latent space, the environmental condition features, and the low-dimensional dense vector of the actual echo after adding noise are input into the first noise reduction structure of the noise reduction model.

[0094] In step 3.2), under the influence of the low-dimensional dense vector of the actual echo in the latent space and the environmental condition features by the first noise reduction structure, the low-dimensional dense vector of the actual echo after adding noise is subjected to noise reduction processing, and the first noise-reduced low-dimensional dense vector of the actual echo is output.

[0095] Step 3.3), repeat the processing of the following steps 3.3.1) - 3.3.3) until each noise reduction structure in the noise reduction model is traversed and the first noise reduction live echo low-dimensional dense vector output is used as the predicted echo low-dimensional dense vector, including:

[0096] Step 3.3.1), input the live echo low-dimensional dense vector of the latent space, environmental condition features, and the first noise reduction live echo low-dimensional dense vector into the second noise reduction structure;

[0097] Step 3.3.2), under the influence of the live echo low-dimensional dense vector and environmental condition features in the latent space by the second noise reduction structure, perform noise reduction processing on the first noise reduction live echo low-dimensional dense vector, and output the second noise reduction live echo low-dimensional dense vector;

[0098] Step 3.3.3), use the second noise reduction live echo low-dimensional dense vector as the new first noise reduction live echo low-dimensional dense vector.

[0099] It should be noted that the construction of the noise reduction model in the embodiment of the present invention is to accurately predict and denoise radar echo data. By using the internal Transformer structure of the multi-layer DIT (Diffusion Transformer) module, the radar echo data can be gradually denoised. In each inference process, the latent space vector representation of the live radar echo sequence is used as the conditional input, and the conditional features generated by the Condition Encoder are fused. These features are derived from terrain information, satellite observation data, model forecast data, and time and space information, providing environmental and background information for the model as the basis for radar echo extrapolation, enabling the model to accurately predict the generation, disappearance, and movement of radar echoes. The model output result is the vector representation of the predicted radar echo sequence in the latent space, that is, the predicted echo low-dimensional dense vector.

[0100] Fuse the latent space vector of the live radar echo sequence and the environmental feature vector before the Patch Embedding of the DIT model. The number of layers of the DIT module is 6 to 12 layers, which can balance the generation effect and inference speed. The input is an echo sequence for a period of time (such as 1 hour), and the output is the subsequent echo sequence. Considering the timeliness of short-term nowcasting and the generation effect, the autoregressive iteration steps of the model are optimally set between 50 and 100. The training uses MSELoss, and the echo encoder and echo decoder are frozen during training.

[0101] Based on the foregoing embodiments, the prediction ability for large values and extreme echo values can be achieved through the optimization training steps weighted by loss. In the training of the deep learning model for radar echo prediction, due to the uneven distribution of the strength of radar echoes, common problems include that the echoes of 10 - 20 dbz account for a relatively large proportion, while the echoes of 60 - 70 dbz rarely appear. This uneven distribution may cause the prediction effect of the model to be biased towards the average. As the forecast time increases, the size of the echo becomes average, and the echo area spreads out, resulting in the lack of prediction of extreme values. The embodiments of the present invention ensure that the prediction results of the potential diffusion model can be more accurate and there will be no lack of extreme data through a new loss function setting; the method further includes:

[0102] Weighted sum of the differences between the predicted echo low-dimensional dense vectors within the echo range of the target area for each current frame and the previous frame is performed to determine the loss function of the potential diffusion model.

[0103] The loss function can be calculated by the following formula:

[0104]

[0105] Wherein, is the tensor of the previous frame in the predicted echo low-dimensional dense vector, is the tensor of the current frame in the predicted echo low-dimensional dense vector, is the number of image frames in the predicted echo low-dimensional dense vector, is the weight of the i-th interval within the echo range of the target area for the current frame, is the weight of the current frame when the j-th frame image is the current frame.

[0106] During the application process, the weighting of this interval is defined in reverse according to the probability of the echo appearing in different ranges, and this weighting relationship is "amplified" after the time series becomes longer, so that the model will pay more attention to those extreme echo values that rarely appear but are crucial for the forecasting skill during the application process, thereby improving the overall prediction performance. This method helps to improve the model's forecasting ability for extreme weather events, especially in the radar echo prediction of severe convective weather.

[0107] For the potential diffusion model involved in the above application process, the echo encoder, echo decoder, and conditional feature encoder need to be frozen during training, while focusing on training the rest of the network. The weighted weight values in the above loss function are not static, but are dynamically adjusted according to the morphology and effect of the predicted echo during the training process. This dynamic adjustment strategy can more finely control the training process to optimize the model performance. The initial weight values should be set considering the possibility of echoes of different sizes and weighted inversely to ensure that the model can pay sufficient attention to those less frequent but equally important echo types at the beginning of training. Through this method, the potential diffusion model can learn various echo features more evenly, thus providing more accurate and reliable prediction results in practical applications.

[0108] In practical applications, the radar echo prediction potential diffusion model that fuses environmental conditions realizes the integration of environmental factors in echo prediction and makes more accurate predictions; using the predicted results, that is, the predicted echo sequence in the pixel space, can more accurately estimate the precipitation in the target area.

[0109] Figure 4 It is a schematic diagram of the comparison between the echo prediction of a deterministic model and the echo ground truth; the top sequence is the echo ground truth for the next hour, and the bottom sequence is the echo prediction value for the next hour; Figure 5 It is a schematic diagram of the comparison between the echo prediction output by an image extrapolation model without fusing external conditions and the echo ground truth; the top sequence is the echo ground truth for the next hour, and the bottom sequence is the echo prediction value for the next hour; it can be seen that compared with the echo ground truth, the predicted echoes output by the above two models do not conform to the actual generation and disappearance of echoes.

[0110] The embodiment of the present invention has prediction skills for extreme values and echo details compared with traditional deterministic models. Based on the above two advantages, the embodiment of the present invention can output prediction results with both extreme values and details, and a radar echo sequence with a small deviation in the large value position, such as Figure 6 As shown, they are the radar echo prediction sequences at 08:00 on May 1, 2024 and 19:00 on August 20, Beijing time; the actual echo in the previous hour is the input of the potential diffusion model, and the echo prediction for the next hour is output; the top sequence is the echo ground truth for the next hour, and the bottom sequence is the echo prediction value for the next hour; it can be seen that the potential diffusion model provided by the embodiment of the present invention can accurately predict extreme values and echo details and has good prediction skills.

[0111] In some embodiments, as Figure 7 shown, the embodiment of the present invention provides a radar echo prediction device based on a potential diffusion model, including:

[0112] The preprocessing module preprocesses the radar data and environmental data of the target area to obtain the real-time echo sequence in the pixel space and multiple environmental grid data, and inputs the environmental grid data and the real-time echo sequence in the pixel space into the latent diffusion model; wherein, the latent diffusion model consists of an echo encoder, a noise addition model, a noise reduction model, a conditional encoder and an echo decoder;

[0113] The first feature extraction module sequentially extracts features from each of the environmental grid data in the order of decreasing scale through the multi-layer convolutional neural network in the conditional encoder to generate the environmental condition features of the target area;

[0114] The second feature extraction module converts the real-time echo sequence in the pixel space into a low-dimensional dense vector of the real-time echo in the latent space through the echo encoder;

[0115] The noise addition module inputs the low-dimensional dense vector of the real-time echo into the noise addition model, and outputs the low-dimensional dense vector of the real-time echo after noise addition through the sequential processing of multiple cascaded noise addition structures in the noise addition model;

[0116] The noise reduction module sequentially processes the low-dimensional dense vector of the real-time echo after noise addition through multiple cascaded noise reduction structures in the noise reduction model under the action of the low-dimensional dense vector of the real-time echo in the latent space and the environmental condition features, and outputs the low-dimensional dense vector of the predicted echo;

[0117] The decoding module converts the low-dimensional dense vector of the predicted echo into a predicted echo sequence in the pixel space through the echo decoder.

[0118] Furthermore, the environmental grid data includes terrain information, satellite observation data, model forecast data, spatial information and time information; the multi-layer convolutional neural network includes multiple feature extraction branches; each feature extraction branch consists of a convolutional layer, an activation function and a max pooling layer; the first feature extraction module is specifically configured to input the terrain information at the first preset scale into the first feature extraction branch for processing, and output the first feature extraction result; superimpose the first feature extraction result and the satellite observation data at the second preset scale, and input the result into the second feature extraction branch for processing, and output the second feature extraction result; superimpose the second feature extraction result and the model forecast data at the third preset scale, and input the result into the third feature extraction branch for processing, and output the third feature extraction result; superimpose the third feature extraction result and the time information and spatial information at the fourth preset scale to obtain the environmental condition features of the target area.

[0119] Further, the noise reduction module is specifically configured to input the live echo low-dimensional dense vector in the latent space, the environmental condition features, and the low-dimensional dense vector of the live echo after noise addition into the first noise reduction structure of the noise reduction model; under the influence of the live echo low-dimensional dense vector in the latent space and the environmental condition features by the first noise reduction structure, perform noise reduction processing on the low-dimensional dense vector of the live echo after noise addition, and output the first noise-reduced live echo low-dimensional dense vector; repeat the following process until each noise reduction structure in the noise reduction model is traversed and the output first noise-reduced live echo low-dimensional dense vector is used as the predicted echo low-dimensional dense vector, including: inputting the live echo low-dimensional dense vector in the latent space, the environmental condition features, and the first noise-reduced live echo low-dimensional dense vector into the second noise reduction structure; under the influence of the live echo low-dimensional dense vector in the latent space and the environmental condition features by the second noise reduction structure, perform noise reduction processing on the first noise-reduced live echo low-dimensional dense vector, and output the second noise-reduced live echo low-dimensional dense vector; using the second noise-reduced live echo low-dimensional dense vector as the new first noise-reduced live echo low-dimensional dense vector.

[0120] Further, the device is further configured to perform weighted summation on the differences between each current frame and the previous frame within the echo range of the target area of the predicted echo low-dimensional dense vector to determine the loss function of the potential diffusion model.

[0121] Further, the loss function can be calculated by the following formula:

[0122]

[0123] where is the tensor of the previous frame in the predicted echo low-dimensional dense vector, is the tensor of the current frame in the predicted echo low-dimensional dense vector, is the number of image frames in the predicted echo low-dimensional dense vector, is the weight of the i-th interval of the current frame within the echo range of the target area, is the weight of the current frame when the j-th frame image is the current frame.

[0124] Further, the device is further configured to estimate the precipitation in the target area by using the predicted echo sequence in the pixel space.

[0125] Further, the preprocessing module is specifically configured to filter out outliers and interference noises in the radar data of the target area, and then slice it according to a preset scale to generate a live echo sequence in the pixel space of the first preset scale; slice the environmental data of the target area according to the corresponding preset resolution to generate multiple environmental grid data of the preset scale.

[0126] An electronic device provided by an embodiment of the present invention. In this embodiment, the electronic device may be, but is not limited to, a computer device with analysis and processing capabilities such as a personal computer (PC), a laptop computer, a monitoring device, a server, etc.

[0127] As an exemplary embodiment, refer to Figure 8 , the electronic device 110 includes a communication interface 111, a processor 112, a memory 113, and a bus 114. The processor 112, the communication interface 111, and the memory 113 are connected through the bus 114; the above-mentioned memory 113 is used to store a computer program that supports the processor 112 to execute the above method, and the above-mentioned processor 112 is configured to execute the program stored in the memory 113.

[0128] The machine-readable storage medium mentioned in this article can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof.

[0129] The non-volatile medium can be non-volatile memory, flash memory, storage drives (such as hard disk drives), any type of storage disk (such as optical discs, DVDs, etc.), or similar non-volatile storage media, or a combination thereof.

[0130] It can be understood that the specific operation methods of the functional modules in this embodiment can refer to the detailed descriptions of the corresponding steps in the above method embodiment, and will not be repeated here.

[0131] The computer-readable storage medium provided by the embodiment of the present invention stores a computer program in the readable storage medium, and when the computer program code is executed, it can implement the method described in any of the above embodiments. For the specific implementation, refer to the method embodiment and will not be repeated here.

[0132] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0133] In addition, in the description of the embodiments of the present invention, unless otherwise clearly defined and limited, the terms "installed", "connected", and "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0134] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0135] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments or easily conceive of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention.

Claims

1. A radar echo prediction method based on a potential diffusion model, characterized in that: include: Preprocessing radar data and environmental data of the target area to obtain a live echo sequence in pixel space and a plurality of environmental grid data, and inputting the environmental grid data and the live echo sequence in pixel space into a potential diffusion model; wherein the potential diffusion model is composed of an echo encoder, a noise addition model, a noise reduction model, a conditional encoder and an echo decoder; By using a multi-layer convolutional neural network in the conditional encoder, feature extraction is performed on each of the environmental grid data in order from large to small scales to generate environmental condition features of the target area; The echo encoder converts the live echo sequence in the pixel space into a low-dimensional dense vector of the live echo in the latent space; Inputting the live echo low-dimensional dense vector into the noise adding model, and outputting the noisy live echo low-dimensional dense vector through sequential processing of a plurality of serially connected noise adding structures in the noise adding model; Under the action of the low-dimensional dense vector of the live echo in the latent space and the characteristics of the environmental conditions, the low-dimensional dense vector of the live echo after adding noise is processed in sequence by a plurality of serially connected noise reduction structures in the noise reduction model to output a predicted low-dimensional dense vector of the echo; The predicted echo low-dimensional dense vector is converted into a predicted echo sequence in pixel space by the echo decoder; Under the action of the low-dimensional dense vector of the live echo in the latent space and the characteristics of the environmental conditions, the step of sequentially processing the low-dimensional dense vector of the live echo after adding noise through a plurality of serially connected noise reduction structures in the noise reduction model to output a predicted low-dimensional dense vector of the echo comprises: Inputting the low-dimensional dense vector of the actual echo in the latent space, the environmental condition feature and the low-dimensional dense vector of the actual echo after adding noise into the first denoising structure of the denoising model; The first denoising structure performs denoising on the noisy live echo low-dimensional dense vector under the influence of the live echo low-dimensional dense vector in the latent space and the environmental condition characteristics, and outputs a first denoised live echo low-dimensional dense vector.

2. The method according to claim 1, characterized in that The environmental grid data includes terrain information, satellite observation data, model forecast data, spatial information and time information; the multi-layer convolutional neural network includes multiple feature extraction branches; Each feature extraction branch consists of a convolutional layer, an activation function, and a maximum pooling layer; The step of extracting features from each of the environmental grid data in order from large to small scales through a multi-layer convolutional neural network in the conditional encoder to generate environmental condition features of the target area includes: Inputting terrain information of a first preset scale into a first feature extraction branch for processing, and outputting a first feature extraction result; The first feature extraction result and the satellite observation data of the second preset scale are superimposed and input into the second feature extraction branch for processing, and the second feature extraction result is output; The second feature extraction result and the model forecast data of the third preset scale are superimposed and input into the third feature extraction branch for processing, and the third feature extraction result is output; The third feature extraction result and the time information and space information of the fourth preset scale are superimposed to obtain the environmental condition characteristics of the target area.

3. The method according to claim 1, characterized in that The step of repeatedly performing the following processing until each noise reduction structure in the noise reduction model is traversed and the first noise reduction actual echo low-dimensional dense vector outputted is used as the predicted echo low-dimensional dense vector comprises: Inputting the low-dimensional dense vector of the actual echo in the latent space, the environmental condition feature and the first denoised actual echo low-dimensional dense vector into the second denoising structure; The second denoising structure performs denoising processing on the first denoised live echo low-dimensional dense vector under the influence of the live echo low-dimensional dense vector in the latent space and the environmental condition characteristics, and outputs a second denoised live echo low-dimensional dense vector; The second denoised live echo low-dimensional dense vector is used as a new first denoised live echo low-dimensional dense vector.

4. The method according to claim 1, characterized in that: The method further comprises: The difference between each current frame and the previous frame of the predicted echo low-dimensional dense vector within the echo range of the target area is weighted summed to determine the loss function of the potential diffusion model.

5. The method according to claim 4, characterized in that The loss function can be calculated by the following formula: in, is the tensor of the previous frame in the predicted echo low-dimensional dense vector, is the tensor of the current frame in the predicted echo low-dimensional dense vector, is the number of image frames in the predicted echo low-dimensional dense vector, is the weight of the i-th interval of the current frame within the echo range of the target area, is the weight of the current frame when the j-th image frame is the current frame.

6. The method according to claim 1, characterized in that The method further comprises: The precipitation in the target area is estimated using the predicted echo sequence in the pixel space.

7. The method according to claim 1, characterized in that The step of preprocessing radar data and environmental data of the target area to obtain a live echo sequence in pixel space and multiple environmental grid data includes: Filtering outliers and interference noise in radar data of the target area, and then slicing according to a preset scale to generate a live echo sequence in a pixel space of a first preset scale; The environmental data of the target area is sliced ​​according to the corresponding preset resolution to generate environmental grid data of multiple preset scales.

8. A radar echo prediction device based on a potential diffusion model, characterized in that: include: A preprocessing module preprocesses the radar data and environmental data of the target area to obtain a live echo sequence in pixel space and a plurality of environmental grid data, and inputs the environmental grid data and the live echo sequence in pixel space into a potential diffusion model; wherein the potential diffusion model is composed of an echo encoder, a noise addition model, a noise reduction model, a conditional encoder and an echo decoder; A first feature extraction module, which extracts features from each of the environmental grid data in descending order of scale through a multi-layer convolutional neural network in the conditional encoder to generate environmental condition features of the target area; A second feature extraction module converts the live echo sequence in the pixel space into a live echo low-dimensional dense vector in the latent space via the echo encoder; A noise adding module, inputting the low-dimensional dense vector of the live echo into the noise adding model, and outputting the low-dimensional dense vector of the live echo after the noise is added through sequential processing of a plurality of serially connected noise adding structures in the noise adding model; A denoising module, under the action of the low-dimensional dense vector of the live echo in the latent space and the characteristics of the environmental conditions, sequentially processes the low-dimensional dense vector of the live echo after adding noise through a plurality of serially connected denoising structures in the denoising model, and outputs a predicted low-dimensional dense vector of the echo; A decoding module, which converts the predicted echo low-dimensional dense vector into a predicted echo sequence in pixel space via the echo decoder; The denoising module is also used to input the low-dimensional dense vector of the actual echo in the latent space, the environmental condition characteristics and the low-dimensional dense vector of the actual echo after noise addition into the first denoising structure of the denoising model; the first denoising structure performs denoising on the low-dimensional dense vector of the actual echo after noise addition under the influence of the low-dimensional dense vector of the actual echo in the latent space and the environmental condition characteristics, and outputs a first denoised low-dimensional dense vector of the actual echo.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a program stored in the memory and capable of being run on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images