Deep learning rainfall prediction enhancement method fusing graph structure and frequency domain features and integration framework

Abstract

The invention relates to the technical field of deep learning and meteorological science crossing, in particular to a deep learning rainfall prediction enhancement method fusing a graph structure and frequency domain characteristics and an integration framework, and the method comprises the steps: obtaining rainfall data of a historical time-space sequence as input data; inputting the input data into a spatial coding module to extract initial spatial features; inputting the initial spatial features into a graph frequency fusion enhancement module to generate enhanced features; wherein the graph-frequency fusion enhancement module is used for enhancing the expression capability of a local space structure and multi-scale frequency domain features of a precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of a graph neural network; and inputting the enhanced features into a backbone network module of a rainfall prediction model, carrying out spatio-temporal evolution modeling, and outputting a rainfall prediction result of a future spatio-temporal sequence. According to the method, the accuracy and reliability of heavy rainfall prediction can be improved, and more powerful technical support is provided for weather forecast and disaster early warning.

Description

Technical Field

[0001] The embodiments of this application relate to the interdisciplinary fields of deep learning and meteorological science, and in particular to a deep learning precipitation prediction enhancement method and integration framework that integrates graph structure and frequency domain features. Background Technology

[0002] Deep learning has made significant progress in precipitation prediction, with related methods broadly categorized into deterministic models and probabilistic generative models. In recent years, with the rapid development of deep learning technology, it has achieved remarkable progress in meteorological forecasting, particularly in precipitation prediction. Currently, mainstream deep learning methods can be mainly divided into two categories: deterministic models and probabilistic generative models.

[0003] Deterministic models are represented by recursive architectures such as Convolutional Long Short-Term Memory (ConvLSTM), Predictive Recurrent Neural Networks (PredRNN) series, Eidetic 3D Long Short-Term Memory (E3D-LSTM), and Motion-Aware Units (MAUs), as well as non-recursive architectures such as Simple VideoPrediction (SimVP), Earthformer, Temporal Attention Units (TAUs), and AlphaPre. These models can effectively simulate large-scale spatiotemporal evolution processes and perform well in terms of prediction speed and computational efficiency. However, because these models primarily optimize point prediction errors, they often produce overly smoothed precipitation field predictions, making it difficult to preserve the crucial frontal structure and local convection details in heavy precipitation forecasts.

[0004] To mitigate the limitations of deterministic models, researchers have turned to probabilistic generative models, particularly diffusion models. Representative methods, such as Masked Conditional Video Diffusion (MCVD) and Dynamic Generative Diffusion Model (DGDM), characterize the uncertainty and chaotic nature of future weather by randomly sampling latent variables, generating precipitation fields with richer local structures than deterministic models. However, despite their advantages in representing stochasticity, diffusion models still struggle to fully capture the crucial spatial organization, multi-scale variations, and frequency domain patterns in precipitation systems. These models typically emphasize temporal evolution and under-model the local topological relationships and spectral components that determine precipitation formation, limiting their performance in complex climate transition zones. Summary of the Invention

[0005] In view of this, embodiments of this application propose a deep learning precipitation prediction enhancement method and integrated framework that integrates graph structure and frequency domain features. The aim is to leverage the complementary advantages of graph neural networks and frequency domain feature extraction to improve the accuracy and reliability of heavy precipitation prediction, thereby providing stronger technical support for weather forecasting and disaster early warning.

[0006] To achieve the above objectives, embodiments of this application propose a deep learning-based precipitation prediction enhancement method that integrates graph structure and frequency domain features. The method includes the following steps: Historical spatiotemporal precipitation data is used as input data; the historical spatiotemporal precipitation data includes observations at multiple time points and covers a predetermined geographical area; The input data is fed into the encoding module to extract initial spatial features; The initial spatial features are input into the image-frequency fusion enhancement module to generate enhanced features. The image-frequency fusion enhancement module enhances the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks. The enhanced features are input into the backbone network module of the precipitation prediction model to perform spatiotemporal evolution modeling and output the precipitation prediction results for future spatiotemporal sequences.

[0007] To achieve the above objectives, embodiments of this application also propose an integrated framework for deep learning-based precipitation prediction enhancement based on image-frequency fusion (i.e., a system for deep learning-based precipitation prediction enhancement based on image-frequency fusion), the integrated framework comprising: The acquisition module is used to acquire historical spatiotemporal precipitation data as input data; wherein, the historical spatiotemporal precipitation data includes observations at multiple time points and covers a predetermined geographical area; The encoding module is used to input data into the encoding module to extract initial spatial features; The image-frequency fusion enhancement module is used to input the initial spatial features into the image-frequency fusion enhancement module to generate enhanced features; wherein, the image-frequency fusion enhancement module enhances the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks; The backbone network module is used to input enhanced features into the backbone network module of the precipitation prediction model, perform spatiotemporal evolution modeling, and output precipitation prediction results for future spatiotemporal sequences.

[0008] To achieve the above objectives, embodiments of this application also propose an electronic device, including a processor and a memory, wherein the memory stores instructions executable by the processor, and the processor is configured to execute the instructions such that the electronic device can implement the deep learning precipitation prediction enhancement method based on image-frequency fusion as described above.

[0009] To achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of a deep learning-based precipitation prediction enhancement method based on image-frequency fusion as described above.

[0010] This application proposes a deep learning-based precipitation prediction enhancement method that integrates graph structure and frequency domain features. First, historical spatiotemporal precipitation data is acquired as input data. Then, the input data is fed into an encoding module to extract initial spatial features. Next, the initial spatial features are input into a graph-frequency fusion enhancement module to generate enhanced features. Finally, the enhanced features are input into the backbone network module of the precipitation prediction model for spatiotemporal evolution modeling, outputting the precipitation prediction results for future spatiotemporal sequences. Since the historical spatiotemporal precipitation data includes observations from multiple time points and covers a predetermined geographical area, and the graph-frequency fusion enhancement module enhances the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks, this scheme can leverage the complementary advantages of graph neural networks and frequency domain feature extraction to improve the accuracy and reliability of heavy precipitation prediction, providing stronger technical support for weather forecasting and disaster early warning.

[0011] Optionally, the initial spatial features are input into the graph-frequency fusion enhancement module to generate enhanced features. The specific execution process of the graph-frequency fusion enhancement module includes: extracting Fourier frequency domain features from the initial spatial features to obtain frequency domain features, and enhancing the frequency domain features using a learnable frequency domain filter to characterize the multi-scale spectral components of precipitation evolution; dynamically constructing a spatial graph structure based on the initial spatial features; wherein nodes in the graph spatial structure represent grid points in the precipitation field, and edges represent the spatial adjacency relationships between grid points; using a graph neural network to perform message passing on the spatial graph structure to capture spatial dependencies and obtain graph structure features; and fusing the enhanced frequency domain features and graph structure features through a cross-domain attention mechanism to obtain enhanced features.

[0012] Optionally, the precipitation prediction model uses an autoregressive approach for prediction; the step of inputting enhanced features into the backbone network module of the precipitation prediction model to perform spatiotemporal evolution modeling and outputting precipitation prediction results for future spatiotemporal sequences includes: using the prediction output at the current moment as the input for the prediction at the next moment, iteratively generating precipitation prediction sequences for multiple future moments, and the image-frequency fusion enhancement module uses the initial spatial features output by the encoder as intermediate features for enhancement in each prediction step.

[0013] Optionally, the image-frequency fusion enhancement module is based on a general plug-in representation, and the method further includes: obtaining the type of precipitation prediction model; and determining the position of the image-frequency fusion enhancement module in the prediction model based on the type of precipitation prediction model.

[0014] Optionally, the position of the image-video fusion enhancement module in the prediction model is determined based on the type of precipitation prediction model, including: if the precipitation prediction model is a deterministic model, the image-video fusion enhancement module is placed between the spatial coding module and the cyclic unit or conversion module of the backbone network module; if the precipitation prediction model is a probabilistic generation model, the image-video fusion enhancement module is placed after the initial encoder and before the U-Net denoising network; wherein, the probabilistic generation model is a diffusion model.

[0015] Optionally, after inputting the enhanced features into the backbone network module of the precipitation prediction model, performing spatiotemporal evolution modeling, and outputting the precipitation prediction results of the future spatiotemporal sequence, the method provided in the embodiments of this application further includes: during the training phase, receiving historical precipitation sequences through the precipitation prediction model, obtaining the future spatiotemporal sequence through forward propagation, and calculating the scalar loss value between the prediction result and the true label based on the regression loss function; wherein, the scalar loss value is used to reflect the overall deviation of the current prediction of the precipitation prediction model; calculating the gradient of the scalar loss value with respect to all learnable parameters of the model through the backpropagation algorithm, and using the optimizer to update the parameters in the opposite direction of the gradient to minimize the loss, thereby improving the prediction accuracy of the precipitation prediction model; wherein, the learnable parameters include the weights in the backbone network module and the image-frequency fusion enhancement module, and the image-frequency fusion enhancement module participates in the entire gradient descent optimization process as a differentiable component.

[0016] Optionally, during the training phase, the precipitation prediction model receives historical precipitation sequences, obtains future spatiotemporal sequences through forward propagation, and calculates a scalar loss value between the prediction results and the true labels based on a loss function. This includes: obtaining the precipitation prediction results of the future precipitation field obtained through forward propagation and the corresponding true labels for each training sample; comparing the precipitation prediction results with the true labels based on a regression loss function, calculating the mean squared error between the precipitation prediction results and the true labels, and using this as the scalar loss value; wherein, the regression loss function... This can be expressed by the following formula: ; in, This represents the total number of pixels in the future precipitation field. For the first The actual label value of each pixel. This represents the corresponding pixel value predicted by the model. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies of this application will be briefly introduced below. Obviously, the following drawings are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings described herein are only used to explain this application and are not intended to limit this application.

[0018] Figure 1 This is a flowchart of a deep learning precipitation prediction enhancement method that fuses graph structure and frequency domain features, provided in one embodiment of this application; Figure 2This is a schematic diagram of the structure of a video-frequency fusion enhancement module provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a precipitation prediction model provided in one embodiment of this application; Figure 4 This is a schematic diagram of the structure of an integrated framework for enhancing deep learning precipitation prediction by fusing graph structures and frequency domain features, provided in another embodiment of this application. Figure 5 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. Those skilled in the art will understand that many technical details have been presented in the embodiments of this application to facilitate better understanding. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of this application. The following embodiments can be combined with and referenced by each other without contradiction.

[0020] Deep learning has made significant progress in precipitation prediction, with related methods broadly categorized into deterministic models and probabilistic generative models. In recent years, with the rapid development of deep learning technology, it has achieved remarkable progress in meteorological forecasting, particularly in precipitation prediction. Currently, mainstream deep learning methods can be mainly divided into two categories: deterministic models and probabilistic generative models.

[0021] Deterministic models are represented by recursive architectures such as ConvLSTM, PredRNN series, E3D-LSTM, and MAU, as well as non-recursive architectures such as SimVP, Earthformer, TAU, and AlphaPre. These models can simulate large-scale spatiotemporal evolution processes well and perform well in terms of prediction speed and computational efficiency. However, because these models mainly optimize point prediction errors, they often produce overly smoothed precipitation field predictions, making it difficult to retain the frontal structure and local convection details that are crucial in heavy precipitation forecasting.

[0022] To mitigate the limitations of deterministic models, researchers have turned to probabilistic generative models, particularly diffusion models. Representative methods such as MCVD and DGDM, by randomly sampling latent variables, can characterize the uncertainty and chaotic nature of future weather, generating precipitation fields with richer local structures than deterministic models. However, despite their advantages in representing stochasticity, diffusion models still struggle to fully capture the crucial spatial organization, multi-scale variations, and frequency domain patterns of precipitation systems. These models typically emphasize temporal evolution and under-model the local topological relationships and spectral components that determine precipitation formation, thus limiting their performance in complex climate transition zones.

[0023] While existing deep learning-based precipitation prediction methods outperform traditional numerical weather prediction in terms of speed and generalization ability, they still have significant shortcomings. Models often fail to adequately model the local spatial structure and multi-scale frequency domain characteristics of precipitation fields, leading to overly smoothed results and difficulty in accurately representing typical convective organization, scale coupling, and sudden changes during heavy precipitation events. Furthermore, existing methods often extract features only in a single spatial or temporal domain, failing to simultaneously capture the synergistic relationship between "local topology and global spectral structure" in the precipitation system. This results in considerable room for improvement in the prediction accuracy and stability for complex weather patterns.

[0024] Overall, whether using deterministic or probabilistic generative frameworks, existing deep learning methods generally lack a unified mechanism capable of simultaneously modeling local spatial topology and global frequency features. This deficiency is particularly pronounced in regions with significant climatic heterogeneity, such as the semi-arid-monsoon transition zone, where local convection structures exhibit strong spatial inhomogeneity and multi-scale coupling, making them difficult for traditional models to effectively characterize.

[0025] In view of this, embodiments of this application propose a deep learning precipitation prediction enhancement method and integration framework that integrates graph structure and frequency domain features. Specifically, it consists of a Graph Frequency Fusion Enhancer (GFFE) module and a framework that can effectively integrate multiple baseline models with the GFFE module. The aim is to leverage the complementary advantages of graph neural networks and frequency domain feature extraction to improve the accuracy and reliability of heavy precipitation prediction, thereby providing stronger technical support for weather forecasting and disaster early warning.

[0026] One embodiment of this application proposes a deep learning precipitation prediction enhancement method that fuses graph structure and frequency domain features, applied to an electronic device. The electronic device can be a terminal or a server; this embodiment and subsequent embodiments will use a server as an example. The implementation details of the deep learning precipitation prediction enhancement method that fuses graph structure and frequency domain features proposed in this embodiment are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution.

[0027] The specific process of the deep learning precipitation prediction enhancement method that integrates graph structure and frequency domain features proposed in this embodiment can be described as follows: Figure 1 As shown, it includes: Step 101: Obtain historical spatiotemporal precipitation data as input data.

[0028] The historical spatiotemporal precipitation data includes observations at multiple time points and covers a predetermined geographical area.

[0029] For example, historical spatiotemporal precipitation data can originate from meteorological observation data, which can be specifically defined as ground station observation data (e.g., datasets provided by meteorological stations) or publicly available datasets such as radar inversion. This data can be a spatiotemporal series covering a predetermined geographical area.

[0030] Observations at multiple time points can represent observations with a defined temporal resolution and historical duration. For example, in a dataset application, the input data consists of historical data from six consecutive moments, with an interval of one hour between adjacent moments, i.e., a 6-hour historical sequence. For a radar dataset, the data could be five historical moments, with an interval of six minutes. A radar dataset can be a radar dataset from a specific region or area; this application does not impose specific restrictions on the region or specific area.

[0031] A predefined geographic region can represent an area with a defined spatial extent and resolution. For example, a dataset of a mountain range might cover an area from 31°N to 40°N and from 103°E to 113°E, with a spatial resolution of 0.125 degrees, forming a regular grid of 72 (latitude) x 81 (longitude) points. Radar datasets, on the other hand, are typically resampled uniformly to a 64x64 grid resolution to fit the model input.

[0032] Input data is represented as a multidimensional tensor in a computer. Its standard shape can be described as follows: ,in: Represents the number of historical time steps (e.g., 6 or 5). The number of channels represents the precipitation variable in this embodiment, so it is usually... . and These represent the height and width of the spatial grid (e.g., 72x81 or 64x64).

[0033] For example, the input data could be data fed into a precipitation prediction model.

[0034] For example, a precipitation prediction model uses historical precipitation data from 6 specific times to predict precipitation values ​​for the next 24 times, with a 1-hour time interval between any two adjacent times. The data covers a spatial range of 31°N to 40°N and 103°E to 113°E, with a spatial resolution of 0.125 degrees, resulting in a 72×81 grid. The shape of the input data is [6, 1, 72, 81] (6 historical times, 1 input variable, 72×81 grid points), and the shape of the prediction results and true labels is [24, 1, 72, 81] (24 future times, 1 variable, 72×81 grid points).

[0035] In addition, the framework supports publicly available radar datasets such as radar datasets (6-minute time resolution) and SEVIR datasets (5-minute time resolution). For both radar and SEVIR datasets, the precipitation prediction model takes 5 historical time points as input and predicts 20 future time points. To unify the spatial scale, both the Shanghai radar and SEVIR data are resampled to a spatial resolution of 64×64. The shape of the input data is [5, 1, 64, 64] (5 historical time points, 1 input variable, 64×64 grid points), and the shape of the prediction results and ground truth labels is [20, 1, 64, 64] (20 future time points, 1 variable, 64×64 grid points).

[0036] For example, the precipitation prediction model can be any of the following models: Predformer, Earthformer, ConvLSTM, E3D-LSTM, MAU, SimVPv1, PredRNN, PredRNN++, PredRNNv2, SimVPv2, TAU, MCVD, DGDM, AlphaPre, etc., and this application does not impose any specific limitations on it.

[0037] As a possible implementation, to improve model training efficiency and stability, after step 101, the following steps may also be included: data preprocessing, such as performing a logarithmic transformation on precipitation values ​​to alleviate their strong skewed distribution, and performing normalization processing to make the data distribution closer to a Gaussian type.

[0038] Step 102: Input the input data into the encoding module to extract the initial spatial features.

[0039] Understandably, the encoding module can transform raw, pixel-level precipitation observation data into higher-dimensional, more abstract feature representations for subsequent processing.

[0040] For example, the encoding module can be implemented using a convolutional neural network consisting of multiple stacked convolutional and pooling layers (or strided convolutions). The convolutional layers perform a sliding window operation on the input data using their kernels, effectively capturing local spatial patterns in the precipitation field, such as the edges of rainbands, the initial morphology of convective clouds, and the outlines of large-scale precipitation systems. The pooling layers progressively reduce the spatial resolution and increase the receptive field, thereby extracting more robust spatial features while preserving key information.

[0041] After processing by the encoding module, the original input tensor It is transformed into a higher-dimensional initial spatial feature tensor, the shape of which can be represented as .in It is the dimension of the feature channels (much greater than 1). and These are the reduced spatial height and width after downsampling. These initial spatial features contain key spatial information extracted from the raw data, but have not yet undergone enhancement processing tailored to the specific structure of the precipitation system.

[0042] Step 103: Input the initial spatial features into the image-frequency fusion enhancement module to generate enhanced features.

[0043] Among them, the image-frequency fusion enhancement module combines Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks to enhance the ability to express the local spatial structure and multi-scale frequency domain features of precipitation fields.

[0044] In one possible embodiment, step 103, the specific execution flow of the image-frequency fusion enhancement module includes: Fourier frequency domain feature extraction is performed on the initial spatial features to obtain frequency domain features, and learnable frequency domain filters are used to enhance the frequency domain features to characterize the multi-scale spectral components of precipitation evolution. Based on the initial spatial features, a spatial graph structure is dynamically constructed. A graph neural network is used to pass messages through the spatial graph structure to capture spatial dependencies and obtain graph structure features. Through a cross-domain attention mechanism, the enhanced frequency domain features and graph structure features are fused to obtain enhanced features.

[0045] In the graph space structure, nodes represent grid points in the precipitation field, and edges represent the spatial adjacency relationships between grid points.

[0046] For example, the image-frequency fusion enhancement module may include a frequency domain feature extraction path, a spatial topology modeling path, and a feature fusion unit.

[0047] For example, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of an image-frequency fusion enhancement module provided in an embodiment of this application. The image-frequency fusion enhancement module may include a frequency domain feature extraction path, a graph neural network module, and a cross-domain multi-scale attention mechanism module.

[0048] In the frequency domain feature extraction pathway, a two-dimensional real-valued Fast Fourier Transform (FFT) is performed on the initial spatial features, mapping them from the spatial domain to the frequency domain. This transform decomposes the image information into spectral components of different frequencies, where low frequencies correspond to large-scale smooth variations, such as reflecting large-scale precipitation trends; high frequencies correspond to details, edges, and sudden structures, such as edge structures and sudden convective details (like rainband fronts) in heavy precipitation. In the frequency domain, a learnable frequency-domain filter is applied to enhance high-frequency components closely related to heavy precipitation (e.g., signals characterizing sharp boundaries of convective systems and the core of sudden rainfall). The enhanced frequency-domain signal is then restored back to the spatial domain via an inverse Fourier transform to obtain the enhanced frequency-domain features.

[0049] In the graph neural network module, each location in the spatial grid is treated as a node in the graph. Edges between nodes are not based on fixed geographical adjacency, but are dynamically constructed based on a composite metric that integrates geographical distance and feature similarity. This allows for effective connections even between geographically non-adjacent areas with similar meteorological behaviors (such as two distant convection zones influenced by the same topography). Graph neural networks are used for message passing on the constructed dynamic graph. By aggregating information from neighboring nodes to update the representation of each node, the graph neural network effectively captures data-driven, non-uniform local spatial dependencies.

[0050] In the cross-domain multi-scale attention mechanism module, each location in the spatial domain can adaptively focus on the frequency domain components most relevant to it. Finally, a gated fusion network is used to residually connect the features processed by the two pathways and the original initial spatial features, adaptively combining them to ensure that while preserving the original information, enhanced frequency domain features and graph structure features obtained from the graph-frequency fusion enhancement module are injected, ultimately generating enhanced features. The enhanced features possess the local convection organization structure and global multi-scale spectral characteristics of the precipitation field, thus significantly improving the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field.

[0051] Step 104: Input the enhanced features into the backbone network module of the precipitation prediction model to perform spatiotemporal evolution modeling and output the precipitation prediction results for the future spatiotemporal sequence.

[0052] For example, the precipitation prediction model also includes a backbone network module. The backbone network module is used to simulate the dynamic evolution of the precipitation system over time; it receives the enhanced features output by the image-frequency fusion enhancement module and learns its evolution from the current time to the future time.

[0053] In one possible embodiment, the precipitation prediction model uses an autoregressive approach for prediction; step 104 includes: using the prediction output at the current moment as the input for the prediction at the next moment, iteratively generating a precipitation prediction sequence for multiple future moments, and the image-frequency fusion enhancement module uses the initial spatial features output by the encoder as intermediate features for enhancement in each prediction step.

[0054] For example, an autoregressive approach uses its past predictions to extrapolate the future, thereby enabling the generation of temporally coherent sequences of arbitrary length.

[0055] It should be noted that the image-frequency fusion enhancement module is integrated into the loop of the precipitation prediction model. In each autoregressive iteration, the intermediate features of the current step are enhanced in real time, thereby ensuring that the modeling ability of precipitation field structure (especially heavy precipitation structure) can be continuously injected and maintained during the 24-step prediction process, effectively preventing the prediction results from becoming structurally blurred or overly smoothed as the prediction time increases.

[0056] For example, let's take a specific model instance and prediction scenario as an example: using PredRNN++ as the backbone network, utilizing the past 6 hours ( Historical precipitation data for the next 24 hours ( ) The precipitation field has a spatial resolution of 72x81.

[0057] Precipitation prediction models receive a 6-hour historical data sequence. As input data, the historical sequence is first converted into an initial spatial feature representation by the model's spatial encoder. These initial spatial features are then fed into the graph-frequency fusion enhancement module. The module performs frequency domain filtering and dynamic graph structure modeling to generate an enhanced feature rich in detail and spatial topology. The enhanced feature is then input into the PredRNN++'s spatiotemporal memory unit (Translator). This unit updates its internal state based on the historical state (which can be empty or zero in the initial state) and the current enhanced feature, and outputs the first future time step (…). Prediction results That is, the prediction result of the first frame. .

[0058] Starting from the second frame, the model performs autoregressive iterative processing, that is... to The enhancement effect of the image-frequency fusion enhancement module occurs within each iteration, acting on the intermediate features of the current iteration; specifically, it transforms the prediction output from the previous time step... Input data for the current time step In prediction At that time, the input data is In prediction At that time, the input is And so on. During the enhancement process of the feature extraction and image-frequency fusion enhancement module, the predicted values ​​constitute... The model's spatial encoder again maps the data to intermediate spatial features at the current time step. These newly generated intermediate spatial features are immediately fed into the GFFE module for the same enhancement process as in the first step. The enhanced features from the current step are then fed into the spatiotemporal memory unit of PredRNN++. At this point, the memory unit has stored the data from the historical sequence to the current time step (…). All evolutionary information of the unit. The unit combines its internal historical state with new enhancement features to update its state and generate the current time step. Prediction results .Will As the next moment ( The above regression iteration process is repeated with the input of ) until all 24 prediction frames are generated.

[0059] In one possible embodiment, the image-frequency fusion enhancement module is based on a general plug-in representation, and the method further includes: obtaining the type of precipitation prediction model; and determining the position of the image-frequency fusion enhancement module in the prediction model based on the type of precipitation prediction model.

[0060] Understandably, by using the image-frequency fusion enhancement module based on a general plugin representation, it can be seamlessly embedded into various existing prediction architectures.

[0061] For example, the image and video fusion enhancement module can be embodied into two main architectures, such as deterministic models or probabilistic generative models, depending on the selected baseline model.

[0062] In one possible embodiment, the position of the image-video fusion enhancement module in the prediction model is determined based on the type of the precipitation prediction model, including: if the precipitation prediction model is a deterministic model, the image-video fusion enhancement module is placed between the spatial coding module and the loop unit or conversion module of the backbone network module; if the precipitation prediction model is a probabilistic generation model, the image-video fusion enhancement module is placed after the initial encoder and before the U-Net denoising network; wherein the probabilistic generation model is a diffusion model.

[0063] like Figure 3 As shown, Figure 3This is a schematic diagram of the structure of a precipitation prediction model provided in an embodiment of this application. For example, if the precipitation prediction model is a deterministic model, the image-frequency fusion enhancement module is placed after the spatial emcoder module; if the precipitation prediction model is a probabilistic generation model, the image-frequency fusion enhancement module is placed before the U-Net denoising network to enhance the encoded historical condition features, and the enhanced features are used as the initial conditions for the denoising process input into the U-Net denoising network.

[0064] For example, deterministic models could be PredRNN, E3D-LSTM, etc. Image-video fusion enhancement modules are embedded after the encoder and before the recurrent unit (such as LSTM) or the Transformer backbone. For example, in PredRNN++, GFFE enhances the encoded features before inputting them into the spatiotemporal LSTM unit, thereby maintaining structural fidelity during temporal evolution.

[0065] For example, the probabilistic generation model can be a model such as MCVD or DGDM. The image and video fusion enhancement module is embedded after the conditional encoder and before the U-Net denoising network to provide high-quality initial conditions for the diffusion process, thereby guiding random generation to more accurately reproduce the fine-grained organization of heavy precipitation.

[0066] In one possible embodiment, after step 104, the method provided by the embodiments of this application further includes: during the training phase, by receiving historical precipitation sequences through a precipitation prediction model, obtaining future spatiotemporal sequences through forward propagation, calculating a scalar loss value between the prediction result and the true label based on a regression loss function; calculating the gradient of the scalar loss value with respect to all learnable parameters of the model through a backpropagation algorithm, and using an optimizer to update the parameters in the opposite direction of the gradient to minimize the loss, thereby improving the prediction accuracy of the precipitation prediction model.

[0067] The scalar loss value reflects the overall bias of the precipitation prediction model's current forecasts. Learnable parameters include the weights in the backbone network module and the image-frequency fusion enhancement module, which participates in the entire gradient descent optimization process as a differentiable component.

[0068] In one possible embodiment, during the training phase, a precipitation prediction model receives historical precipitation sequences, obtains future spatiotemporal sequences via forward propagation, and calculates a scalar loss value between the prediction results and the true labels based on a loss function. This includes: in each training sample, obtaining the precipitation prediction results of the future precipitation field obtained through forward propagation, and the corresponding true labels for each time step; comparing the precipitation prediction results with the true labels based on a regression loss function, calculating the mean squared error between the precipitation prediction results and the true labels, and using this as the scalar loss value; wherein, the regression loss function... This can be expressed by the following formula: ; in, This represents the total number of pixels in the future precipitation field. For the first The actual label value of each pixel. This represents the corresponding pixel value predicted by the model.

[0069] For example, historical precipitation data (i.e., historical precipitation sequences) is input into the precipitation prediction model and integrated with an image and video fusion enhancement module. The model uses the difference between the predicted results and the true labels as a supervision signal, and uses the mean squared error as the main regression loss function to guide the optimization of model parameters. Specifically, in each training sample, the model receives a historical precipitation sequence as input, and after internal spatiotemporal modeling (such as a recurrent unit, Transformer, or diffusion process), outputs a precipitation prediction field for the next few hours; simultaneously, the dataset provides real precipitation observations for the corresponding time steps. The system will then follow the regression loss function... The predicted and actual fields are compared pixel-by-pixel at the same spatial location and time step. The squared difference between the two is calculated and averaged to obtain a scalar loss value. This loss value reflects the overall bias of the model's current prediction: the larger the value, the further the prediction deviates from the reality. Subsequently, the system uses the backpropagation algorithm to calculate the gradient of this loss with respect to all learnable parameters of the model (including convolutional kernels, graph neural network weights, frequency domain filters, attention parameters, etc. in the backbone network and GFFE module), and uses an optimizer (such as AdamW) to update the parameters in the reverse direction of the gradient, so that the next prediction is closer to the true value. This process is iterated repeatedly on a large number of training samples, and the model gradually learns to capture the spatiotemporal patterns of precipitation evolution, especially the structural features and dynamic trends of heavy precipitation events. Since the image and video fusion enhancement module is designed as a differentiable plug-in module, its internal frequency domain filtering, dynamic graph construction, and cross-domain fusion mechanisms can all be effectively trained through gradient descent, thereby co-optimizing with the backbone model and ultimately improving the overall prediction accuracy and physical consistency. The entire training process is completely end-to-end, requiring no additional pre-training or manual parameter tuning, ensuring the model's generalization ability and practicality in real-world weather scenarios.

[0070] This application proposes a deep learning-based precipitation prediction enhancement method that integrates graph structure and frequency domain features. First, historical spatiotemporal precipitation data is acquired as input data. Then, the input data is fed into an encoding module to extract initial spatial features. Next, the initial spatial features are input into a graph-frequency fusion enhancement module to generate enhanced features. Finally, the enhanced features are input into the backbone network module of the precipitation prediction model for spatiotemporal evolution modeling, outputting the precipitation prediction results for future spatiotemporal sequences. Since the historical spatiotemporal precipitation data includes observations from multiple time points and covers a predetermined geographical area, and the graph-frequency fusion enhancement module enhances the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks, this scheme can leverage the complementary advantages of graph neural networks and frequency domain feature extraction to improve the accuracy and reliability of heavy precipitation prediction, providing stronger technical support for weather forecasting and disaster early warning.

[0071] In summary, the image-frequency fusion enhancement module combines the spatial topology modeling capabilities of graph neural networks with Fourier frequency domain feature extraction, enabling the model to simultaneously capture local convective structures and global spectral characteristics, thereby more accurately representing the multi-scale organizational patterns inherent in heavy precipitation. As a lightweight and pluggable module, the image-frequency fusion enhancement module can be seamlessly integrated into existing deterministic or generative prediction frameworks, significantly improving the model's ability to structurally represent complex precipitation systems and its forecasting performance. Compared to traditional numerical weather prediction methods, deep learning-based prediction methods outperform traditional methods in terms of prediction effectiveness, inference speed, and data utilization. Therefore, the deep learning-based precipitation prediction enhancement method based on image-frequency fusion proposed in this application can leverage the complementary advantages of graph neural networks and frequency domain feature extraction to improve the accuracy and reliability of heavy precipitation prediction, providing stronger technical support for weather forecasting and disaster early warning.

[0072] The steps described above are for clarity only. In implementation, they can be combined into one step, or some steps can be broken down into multiple steps, as long as they involve the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0073] Another embodiment of this application proposes an integrated framework for deep learning-based precipitation prediction enhancement based on image-frequency fusion (i.e., a system for deep learning-based precipitation prediction enhancement based on image-frequency fusion). The details of this system for deep learning-based precipitation prediction enhancement based on image-frequency fusion are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this example. Figure 4This is a schematic diagram of the structure of a deep learning-based precipitation prediction enhancement system based on image-frequency fusion proposed in this embodiment, including: The acquisition module 210 is used to acquire historical spatiotemporal precipitation data as input data; wherein, the historical spatiotemporal precipitation data includes observations at multiple time points and covers a predetermined geographical area; The encoding module 220 is used to input input data to the encoding module in order to extract initial spatial features; The image-frequency fusion enhancement module 230 is used to input the initial spatial features into the image-frequency fusion enhancement module to generate enhanced features; wherein, the image-frequency fusion enhancement module enhances the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks; The backbone network module 240 is used to input the enhanced features into the backbone network module of the precipitation prediction model, perform spatiotemporal evolution modeling, and output the precipitation prediction results of the future spatiotemporal sequence.

[0074] It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiments, and this embodiment can be implemented in conjunction with the above method embodiments. The relevant technical details and technical effects mentioned in the above method embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above method embodiments.

[0075] It is worth mentioning that all modules and units involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units do not exist in this embodiment.

[0076] Another embodiment of this application provides an electronic device, such as Figure 5 As shown, it includes a processor 31 and a memory 32. The memory 32 stores instructions that the processor 31 can execute. When the processor 31 is configured to execute the instructions, the electronic device can implement a deep learning precipitation prediction enhancement method based on image-frequency fusion as described in the above method embodiment.

[0077] The memory and processor are connected via a bus, which includes any number of interconnecting buses and bridges, connecting various circuits of one or more processors and the memory. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0078] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0079] Another embodiment of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, can implement a deep learning precipitation prediction enhancement method based on image-frequency fusion as described in the above method embodiments.

[0080] That is, those skilled in the art will understand that all or part of the steps in the above method embodiments can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to cause a device (such as a microcontroller, chip, etc.) or processor to execute all or part of the steps of the method described in the method embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0081] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application. For those skilled in the art, several improvements and modifications can be made without departing from the principles of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A deep learning-based precipitation prediction enhancement method based on image-frequency fusion, characterized in that, The method includes: Historical spatiotemporal precipitation data is used as input data; the historical spatiotemporal precipitation data includes observations at multiple time points and covers a predetermined geographical area; The input data is fed into the encoding module to extract initial spatial features; The initial spatial features are input into the image-frequency fusion enhancement module to generate enhanced features. The image-frequency fusion enhancement module enhances the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks. The enhanced features are input into the backbone network module of the precipitation prediction model to perform spatiotemporal evolution modeling and output the precipitation prediction results for future spatiotemporal sequences.

2. The method according to claim 1, characterized in that, The initial spatial features are input into the image-frequency fusion enhancement module to generate enhanced features. The specific execution flow of the image-frequency fusion enhancement module includes: Fourier frequency domain features are extracted from the initial spatial features to obtain frequency domain features, and learnable frequency domain filters are used to enhance the frequency domain features to characterize the multi-scale spectral components of precipitation evolution. Based on the initial spatial features, a spatial graph structure is dynamically constructed; where nodes in the graph spatial structure represent grid points in the precipitation field, and edges represent the spatial adjacency relationships between grid points. Graph neural networks are used to perform message passing on spatial graph structures in order to capture spatial dependencies and obtain graph structure features; Enhanced features are obtained by fusing enhanced frequency domain features and graph structure features through a cross-domain attention mechanism.

3. The method according to claim 1, characterized in that, The precipitation prediction model uses an autoregressive approach for prediction; the enhanced features are input into the backbone network module of the precipitation prediction model to perform spatiotemporal evolution modeling, and output the precipitation prediction results for the future spatiotemporal sequence, including: The current prediction output is used as the input for the next prediction, and the precipitation prediction sequence for multiple future times is generated iteratively. In each prediction step, the image-frequency fusion enhancement module uses the initial spatial features output by the encoder as intermediate features for enhancement.

4. The method according to claim 1, characterized in that, The image-frequency fusion enhancement module is based on a general plug-in representation, and the method further includes: Identify the type of precipitation prediction model; Based on the type of precipitation prediction model, the position of the image-frequency fusion enhancement module in the prediction model is determined.

5. The method according to claim 4, characterized in that, The determination of the location of the image-frequency fusion enhancement module in the prediction model based on the type of precipitation prediction model includes: If the precipitation prediction model is a deterministic model, then the image-frequency fusion enhancement module is placed between the spatial coding module and the backbone network module in a loop unit or conversion module. If the precipitation prediction model is a probabilistic generation model, then the image-frequency fusion enhancement module is placed after the initial encoder and before the U-Net denoising network; where the probabilistic generation model is a diffusion model.

6. The method according to claim 1, characterized in that, After inputting the enhanced features into the backbone network module of the precipitation prediction model, performing spatiotemporal evolution modeling, and outputting the precipitation prediction results for the future spatiotemporal sequence, the method further includes: During the training phase, the precipitation prediction model receives historical precipitation sequences, obtains future spatiotemporal sequences through forward propagation, and calculates the scalar loss value between the prediction results and the true labels based on the regression loss function. The scalar loss value is used to reflect the overall degree of deviation of the current prediction of the precipitation prediction model. The gradient of the scalar loss value with respect to all learnable parameters of the model is calculated using the backpropagation algorithm. The optimizer is then used to update the parameters in the opposite direction of the gradient to minimize the loss, thereby improving the prediction accuracy of the precipitation prediction model. The learnable parameters include the weights in the backbone network module and the image-frequency fusion enhancement module. The image-frequency fusion enhancement module participates in the entire gradient descent optimization process as a differentiable component.

7. The method according to claim 1, characterized in that, During the training phase, the precipitation prediction model receives historical precipitation sequences, obtains future spatiotemporal sequences through forward propagation, and calculates a scalar loss value between the prediction results and the true labels based on a loss function, including: In each training sample, the precipitation prediction result of the future precipitation field obtained through forward propagation, and the real label of the corresponding time step are obtained; Based on the regression loss function, the precipitation forecast results are compared with the true labels, the mean squared error between the precipitation forecast results and the true labels is calculated, and this error is used as the scalar loss value; whereby, the regression loss function... This can be expressed by the following formula: ； in, This represents the total number of pixels in the future precipitation field. For the first The actual label value of each pixel. This represents the corresponding pixel value predicted by the model.

8. An integrated framework for enhancing deep learning-based precipitation prediction based on image-frequency fusion, characterized in that, The integration framework includes: The acquisition module is used to acquire historical spatiotemporal precipitation data as input data; wherein, the historical spatiotemporal precipitation data includes observations at multiple time points and covers a predetermined geographical area; The encoding module is used to input the input data to extract the initial spatial features; The image-frequency fusion enhancement module is used to input the initial spatial features into the image-frequency fusion enhancement module to generate enhanced features; wherein, the image-frequency fusion enhancement module enhances the ability to express the local spatial structure and multi-scale frequency domain features of the precipitation field by combining Fourier frequency domain feature extraction and spatial topology modeling of graph neural networks; The backbone network module is used to input enhanced features into the backbone network module of the precipitation prediction model, perform spatiotemporal evolution modeling, and output precipitation prediction results for future spatiotemporal sequences.

9. An electronic device, characterized in that, include: A processor and a memory, wherein the memory stores instructions executable by the processor, and the processor is configured to, when executing the instructions, enable the electronic device to implement a deep learning precipitation prediction enhancement method based on image-frequency fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can implement a deep learning precipitation prediction enhancement method based on image-frequency fusion as described in any one of claims 1 to 7.

Images