Radar image extrapolation model based on space-time frequency domain fusion

Abstract

The invention provides a radar image extrapolation model based on time-space frequency domain fusion, and belongs to the technical field of meteorological prediction. The radar image extrapolation model comprises the following steps: encoding an input radar image sequence frame into a low-dimensional feature vector through an encoder; respectively inputting the encoded features into a time sequence feature processing module and a spatial feature extraction module; the time sequence features and the space features are fused to generate comprehensive space-time features, the fused features are input into a decoder, and future radar image frames are decoded; the space, time and frequency domain information of the radar image is comprehensively captured by combining the global space feature, the local space feature, the time feature and the frequency domain feature, meanwhile, an efficient calculation method is adopted, the extrapolation precision and the calculation efficiency are remarkably improved, and a brand new solution is provided for the radar image extrapolation technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of meteorological prediction, and specifically, relates to a radar image extrapolation model based on spatio-temporal-frequency domain fusion. Background Art

[0002] Radar image extrapolation technology is an important research direction in the fields of meteorological prediction, disaster warning, and military reconnaissance. Radar images can capture information such as precipitation and cloud movement in the atmosphere in real time, providing key data support for short-term weather forecasting and extreme weather event warnings. However, there are many technical challenges in existing radar image extrapolation, mainly reflected in the following aspects:

[0003] (1) Spatial complexity. Radar images have a high degree of spatial complexity, especially in high echo regions (such as thunderstorms and heavy rains), which show non-uniform distribution and clump-like characteristics. Traditional methods (such as optical flow method, convolutional neural network, etc.) are difficult to capture both the global spatial distribution and local detail features simultaneously, resulting in insufficient extrapolation accuracy. For example, the optical flow method is prone to failure when dealing with complex motion patterns, while the convolutional neural network has limited performance in capturing global spatial features.

[0004] (2) Temporal dependence. The radar image sequence has strong temporal dependence, and its dynamic change process is affected by various factors (such as wind speed, air pressure, etc.). Traditional time series models (such as ARIMA, RNN, etc.) have limited performance in dealing with long-term dependence and non-linear dynamic changes, and it is difficult to accurately predict future radar images. For example, the ARIMA model performs poorly in dealing with non-linear dynamic changes, while the RNN is prone to the problem of gradient disappearance when dealing with long-term dependence.

[0005] (3) Insufficient utilization of frequency domain features. Radar images contain rich periodic, trend, and long-range dependence features in the frequency domain, but most existing methods only extract features from the spatial domain or time domain, and fail to fully utilize the frequency domain information, which limits the performance of the extrapolation model. For example, existing methods cannot handle the periodic information presented in radar frames, such as the cyclic appearance of precipitation systems, so the periodic features in the frequency domain are wasted in this way.

[0006] (4) Computational efficiency problem. Radar image data usually has the characteristics of high resolution and multi-channels. Traditional methods have a high computational complexity when dealing with large-scale data, and it is difficult to meet the real-time requirement. In addition, although existing deep learning models (such as Transformer) have excellent performance, their computational complexity is \(O(N^2)\), which limits their application in high-resolution radar images. Summary of the Invention

[0007] Based on the above background, the present invention proposes a radar image extrapolation model based on spatio-temporal-frequency domain fusion. By combining global spatial features, local spatial features, temporal features, and frequency domain features, it comprehensively captures the spatial, temporal, and frequency domain information of radar images. At the same time, by adopting an efficient calculation method, it significantly improves the extrapolation accuracy and calculation efficiency, providing a new solution for radar image extrapolation technology.

[0008] The present invention is realized through the following technical solutions: a radar image extrapolation model based on spatio-temporal-frequency domain fusion,

[0009] The radar image extrapolation model includes an encoder, a temporal feature processing module, a spatial feature processing module, and a decoder;

[0010] The encoder is a feature extraction module that encodes the input radar image sequence into a feature representation;

[0011] The temporal feature processing module includes an inter-frame comprehensive feature module and an inter-frame extreme feature processing module, which extract the temporal features in the radar image sequence. Through the weighted fusion of the inter-frame comprehensive feature module and the inter-frame extreme feature module, the future temporal features of the radar image are obtained;

[0012] The spatial feature processing module includes a global spatial feature processing module and a local spatial feature processing module. The encoded features are respectively processed by the global spatial feature processing module and the local spatial feature processing module and then preliminarily connected to obtain the future spatial features of the radar image;

[0013] The decoder is a feature reconstruction module that reconstructs the spatio-temporal features obtained after passing through the temporal feature processing module and the spatial feature processing module into a future radar image sequence.

[0014] Further, the encoder is composed of multiple ConvD modules, and each ConvD module consists of a convolutional layer + an activation function + a normalization operation;

[0015] The encoder maps the input radar image sequence to the feature space through multiple convolutional operations, and extracts representative spatio-temporal features;

[0016] The representative spatio-temporal features include the intensity, shape, and motion trend of the radar echo.

[0017] Further, in the temporal feature processing module,

[0018] The inter-frame comprehensive feature module captures the comprehensive temporal changes between frames in the radar image sequence, models the global temporal dependencies between frames through dilated convolution and depth convolution, and extracts the overall temporal changes;

[0019] The inter-frame extreme feature processing module captures the extreme changes between frames in the radar image sequence, extracts the maximum value in the inter-frame features through max pooling operation, and captures the change of radar value from small to large; through the fully connected layer, the attention weights of the inter-frame extreme features are generated to adjust the relative importance of the inter-frame features.

[0020] Further, in the spatial feature processing module,

[0021] The global spatial feature processing module transforms the radar image from the spatial domain to the frequency domain through Fourier transform, and utilizes the global information in the frequency domain to enhance the perception ability of the model, so as to capture the global spatial features.

[0022] Further, the global spatial feature processing module first performs two-dimensional Fourier transform on the input radar image to map the spatial distribution of the image into the spectral information in the frequency domain;

[0023] Then, frequency domain features are extracted through linear transformation in the frequency domain to ensure that the output dimension is consistent with the input, model the overall spatial distribution of the radar image, and capture the global information in the frequency domain;

[0024] Finally, the extracted frequency domain features are transformed back to the spatial domain through inverse Fourier transform, and then the global spatial features of the radar image are obtained.

[0025] Further, the local spatial feature processing module captures the local spatial features of the radar image by stacking multiple depthwise separable convolutions, and increases the fitting ability of the model to the local spatial features;

[0026] The depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution;

[0027] Among them, the depthwise convolution performs convolution operations on each input channel respectively to extract the local features of each channel; while the pointwise convolution fuses the channels of the output of the depthwise convolution through 1x1 convolution to obtain deeper image spatial features.

[0028] Further, the decoder consists of multiple ConvDT modules, and the ConvDT module consists of a transposed convolution layer + activation function + normalization operation;

[0029] The decoder gradually reconstructs the spatio-temporal features into a high-dimensional radar image sequence through multi-layer transposed convolution operations, while restoring the spatial resolution of the radar image, reconstructing the dynamic evolution information of the radar echo, and reconstructing the detailed information in the radar image.

[0030] A running method of a radar image extrapolation model based on spatio-temporal frequency domain fusion: The method specifically includes the following steps:

[0031] Step 1: Encode the input radar image sequence frames into low-dimensional feature vectors through an encoder;

[0032] Step 2: Input the encoded features into the temporal feature processing module and the spatial feature extraction module respectively; among them, the temporal feature processing module is responsible for capturing the dynamic temporal relationship between frames. The comprehensive feature module and the extreme feature module in it can extract the overall change and extreme change in time respectively. After the two modules are fused, the future temporal features of the radar map can be generated;

[0033] The spatial feature extraction module mines the global spatial features through the global spatial feature processing module; captures the local spatial features through the local spatial feature processing module; after the encoded features pass through these two sub-modules respectively and then are preliminarily connected, the comprehensive future spatial features can be obtained;

[0034] Step 3: Fuse the temporal features and spatial features to generate comprehensive spatio-temporal features.

[0035] Step 4: Input the fused features into the decoder to decode the future radar image frames.

[0036] An electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0037] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of the above method are implemented.

[0038] Advantages of the present invention

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] This model has multiple advantages in the radar image extrapolation task. Through the collaborative effect of the temporal feature processing module and the spatial feature processing module, the model can accurately capture the dynamic evolution of high echo regions; in addition, the global spatial feature processing module mines frequency domain features through Fourier transform, which also enhances the model's learning ability for complex spatial patterns; and through the depthwise separable convolution of the local spatial feature processing module, the model can efficiently capture the local features of clump-like high echo regions such as thunderstorm gales, which can also improve the accuracy of the second-stage thunderstorm gale recognition task. Description of the drawings

[0041] Figure 1 It is the overall architecture of the model of the present invention.

[0042] Figure 2 It is the encoder structure of the present invention.

[0043] Figure 3This is the decoder structure of the present invention.

[0044] Figure 4 It is the timing feature processing module of the present invention.

[0045] Figure 5 It is the spatial feature processing module of the present invention. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The experimental methods used in the following examples are conventional methods unless otherwise specified. The materials, reagents, methods and instruments used are conventional materials, reagents, methods and instruments in the art unless otherwise specified, and can be obtained through commercial channels by those skilled in the art.

[0048] The present invention proposes a radar image extrapolation model based on time-space frequency domain fusion, the structure of which is as follows: Figure 1 As shown:

[0049] The structure of the model includes an encoder, a temporal feature processing module, a spatial feature processing module, and a decoder.

[0050] Among them, the spatial feature processing module is divided into a global spatial feature processing module and a local spatial feature processing module. The global spatial feature processing module realizes the mapping of spatial features to the frequency domain through the Fourier transform method, so as to mine more spatial nonlinear features and periodic structures; the local spatial feature processing module captures local features through deep separable convolution, especially high-echo clumping areas such as thunderstorms and gales.

[0051] The encoder and decoder included in this model are as follows:

[0052] Encoder is a feature extraction module in the model. Its main function is to encode the input radar image sequence into feature representation. Its structure is as follows: Figure 2 As shown in the figure, the Encoder consists of multiple ConvD modules, which consist of convolution layer + activation function + normalization operation. The Encoder maps the input radar image sequence to the feature space through multi-layer convolution operation and extracts representative spatiotemporal features. These features include key information such as the intensity, shape, and motion trend of the radar echo.

[0053] The Encoder formula is expressed as:

[0054] latent = ConvD n (ConvD n-1 (...ConvD1(X)))

[0055] The Decoder is the feature reconstruction module in the model. Its main function is to reconstruct the spatio-temporal features obtained after passing through the temporal feature processing module and the spatial feature processing module into a future radar image sequence. The structure is as Figure 3 shown. The structure of the Decoder consists of multiple ConvDT modules, and each ConvDT module is composed of a transposed convolution layer + activation function + normalization operation. The Decoder gradually reconstructs the spatio-temporal features into a high-dimensional radar image sequence through multiple layers of transposed convolution operations. This process not only restores the spatial resolution of the radar image but also reconstructs the dynamic evolution information of the radar echo, and can effectively reconstruct the detailed information in the radar image. In addition, the Decoder also combines the intermediate features of the Encoder with the decoding process through skip connections, retaining more detailed information, thereby improving the prediction accuracy.

[0056] The Decoder formula is expressed as:

[0057] enc skip = skip_adjust(enc)

[0058] hid0 = latent

[0059] hid i = ConvDT i (hid i-1 ) for i = 1,..., n - 1

[0060] Y = ConvD n (concate(hid n-1 , enc skip ))

[0061] output = readout(Y)

[0062] The temporal feature processing module of this model is specifically:

[0063] The time feature processing module, which is specifically used to capture the time evolution features in the radar image sequence. The structure is as Figure 4As shown. In this module, through two sub-modules, namely the inter-frame comprehensive feature module and the inter-frame extreme feature processing module, the temporal changes of the radar image sequence are modeled from different perspectives. From the comprehensive perspective, it can effectively handle the dependency relationship between consecutive radar frames, thereby improving the accuracy and efficiency of radar image extrapolation; from the extreme perspective, it can focus on the dynamic evolution of high echo regions. For example, during a thunderstorm with strong winds, the radar echo value changes from small to large. Therefore, it also helps to improve the prediction accuracy of high echo regions.

[0064] The inter-frame comprehensive feature module in the temporal feature processing module is used to capture the comprehensive temporal changes between frames in the radar image sequence. Its core is to model the global temporal dependency between frames through dilated convolution and depth convolution. Among them, dilated convolution can expand the receptive field without increasing the number of parameters, thereby capturing the long-range temporal dependency between frames; depth convolution can perform convolution operations on each input channel separately, reducing the computational amount while retaining the independence between channels, which helps to retain the temporal features between consecutive frames. Finally, 1x1 convolution is used to adjust the number of channels to further fuse the inter-frame comprehensive features.

[0065] The inter-frame extreme feature processing module in the temporal feature processing module is used to capture the extreme changes between frames in the radar image sequence. It relies on max pooling and fully connected layers. Among them, the max pooling operation can extract the maximum value in the inter-frame features, which helps to retain the significant changes in the high echo regions of the radar map and can capture the change of radar values from small to large. The fully connected layer is used to generate the attention weights of the inter-frame extreme features to adjust the relative importance of the inter-frame features.

[0066] The final output feature of the temporal feature processing module is the weighted fusion of the inter-frame comprehensive feature module and the inter-frame extreme feature module. The specific formula is as follows:

[0067] F global =Conv 1×1 (Dilated Conv(DW Conv(Input))),

[0068] F extreme =FC(MaxPool(Input)),

[0069]

[0070] The temporal feature processing module of the present invention has multiple advantages.

[0071] First, the parallel application of the two modules enables the model to extract rich temporal variation features from the radar image sequence. The inter-frame comprehensive feature module can help the model capture the overall temporal evolution trend in the radar image sequence, which is crucial for modeling the long-term temporal dependencies in the radar image sequence. And through the inter-frame extreme features, the model can capture the extreme temporal variations in the radar image sequence, especially the significant changes in the high echo regions, which helps to model the occurrence process of extreme weather events such as severe thunderstorm winds.

[0072] Second, this module replaces the traditional serial temporal processing units (such as LSTM, GRU, etc.) with a parallel computing mechanism, significantly improving the computing efficiency and enabling the module to quickly process prediction tasks under large-scale radar image sequences.

[0073] The spatial feature processing module of this model is specifically as follows:

[0074] In this model, a spatial feature processing module is designed specifically for capturing the spatial features of radar images, and its structure is as Figure 5 shown. This module includes two sub-modules, namely the global spatial feature module and the local spatial feature module, which can extract the spatial features of radar images from the global and local scales respectively. The key is to capture the spatial feature distribution in the high echo regions.

[0075] The core idea of the global spatial feature module is to transform the radar image from the spatial domain to the frequency domain through Fourier transform, and use the global information in the frequency domain to enhance the model's perception ability, so as to capture the global spatial features. It first performs a two-dimensional Fourier transform on the input radar image to map the spatial distribution of the image to the spectral information in the frequency domain. Then, frequency domain features are extracted through a linear transform in the frequency domain, and this step can model the overall spatial distribution of the radar image. Then, the extracted frequency domain features are transformed back to the spatial domain through the inverse Fourier transform, and thus the final global spatial features are obtained.

[0076] The specific steps are as follows:

[0077] 1. Fourier transform.

[0078] In this step, the input vector is transformed from the spatial domain to the frequency domain using the Fourier transform. The formula is as follows, where k represents the frequency and x represents the position in the spectral sequence. Through the FFT, the input vector is transformed into its frequency domain representation:

[0079]

[0080] 2. Linear transform.

[0081] In the frequency domain, a linear transformation is performed on the frequency domain representation through a multi-layer perceptron. The purpose of this linear transformation is to map the frequency domain representation to a linear space, ensuring that the output dimension is consistent with the input. In this way, the global information in the frequency domain can be effectively captured while avoiding the high computational complexity of the traditional self-attention mechanism. The formula for this step is as follows:

[0082]

[0083] 3. Inverse Fourier transform.

[0084] After the linear transformation, the frequency domain representation is converted back to the spatial domain through the inverse Fourier transform, thereby retaining the global spatial feature representation of the radar image. The formula is as follows, where, is the output after the inverse Fourier transform, and F + (k) is the result of the linear transformation:

[0085]

[0086] The core idea of the local spatial feature module is to capture the local spatial features of the radar image by stacking multiple depthwise separable convolutions, especially the clump-like features in the high echo region, so as to increase the model's fitting ability for local spatial features. The depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution. Among them, the depthwise convolution performs convolution operations on each input channel separately to extract the local features of each channel. The pointwise convolution then fuses the outputs of the depthwise convolution through 1x1 convolution to obtain deeper image spatial features.

[0087] For the high echo regions in the radar map, their distributions are often relatively concentrated and have high intensities, presenting a local blocky shape. Through the depthwise separable convolution, these information can be effectively fused.

[0088] In the local spatial feature module, multiple depthwise separable convolution layers are stacked, and they use different sizes of convolution kernels such as 3x3, 5x5, and 7x7 respectively to capture local spatial features of different scales. For example, a smaller convolution kernel like 3x3 is good at discovering the detailed features in the input feature map, such as the edges and texture information of the high echo region; a medium-sized convolution kernel of 5x5 can grasp the local features in the feature map, such as the clump-like features of the high echo region; while using a larger convolution kernel like 7x7 can discover slightly larger local weather system features, such as the overall distribution of the high echo region.

[0089] The final output feature of the spatial feature processing module is the weighted fusion of the global spatial feature and the local spatial feature. Through the global spatial feature module, the global spatial distribution of the radar image can be captured to model the overall spatial feature, which helps to understand the overall structure of the radar image. While through the local spatial feature module, the local spatial features of the radar image can be captured, especially the clump-like features in the high echo area, which is crucial for modeling the spatial distribution of extreme weather events such as severe thunderstorm winds.

[0090] The specific formula in the feature fusion stage is as follows:

[0091] F global = Inverse FFT(Linear Transform(FFT(Input)))

[0092] F local = Multi-DWConv(Input),

[0093] output = α·F global + β·F local ,

[0094] In summary, the spatial feature processing module in this model has the advantages of effectively combining global and local features, extracting multi-scale features, high computational efficiency, and strong flexibility. Among them, the global feature module based on Fourier transform captures global periodicity, trend, and long-range dependence features through frequency domain analysis, while the local feature module based on depthwise separable convolution extracts local high-frequency details and clump-like features through multi-scale convolutional kernels. The collaborative work of the two realizes the comprehensive modeling of complex spatial data. At the same time, the complexity of Fourier transform is only (O(N log N)), and the number of parameters of depthwise separable convolution is also less than that of traditional ordinary convolution, so it can significantly improve the computational efficiency and enable the module to efficiently process high-resolution data. In addition, this module is flexibly designed, can adapt to different tasks and datasets, and can be extended by adjusting the stacking level or changing the convolutional kernel size, providing a powerful feature extraction ability for spatio-temporal data modeling.

[0095] Based on the above, a running method of a radar image extrapolation model based on spatio-temporal frequency domain fusion: In the running process of the model,

[0096] First, the input radar image sequence frames are encoded into low-dimensional feature vectors through an encoder.

[0097] Second, the encoded features are respectively input into the temporal feature processing module and the spatial feature extraction module.

[0098] Among them, the temporal feature processing module is responsible for capturing the dynamic temporal relationship between frames. The comprehensive feature module and the extreme feature module therein can extract the overall change and extreme change in time respectively. After the two modules are fused, the future temporal features of the radar chart can be generated.

[0099] The spatial feature extraction module mines global spatial features through the global spatial feature processing module (based on Fourier transform); captures local spatial features through the local spatial feature processing module (based on depthwise separable convolution). After the encoded features pass through these two sub-modules respectively and then are preliminarily connected, the comprehensive future spatial features can be obtained.

[0100] Next, the temporal features and spatial features are fused to generate comprehensive spatio-temporal features.

[0101] Finally, the fused features are input into the decoder to decode the future radar image frames.

[0102] An electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0103] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of the above method are implemented.

[0104] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory of the method described in the present invention is intended to include but not limited to these and any other suitable types of memory.

[0105] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means such as coaxial cable, optical fiber, digital subscriber line (DSL), or wireless means such as infrared, wireless, microwave, etc. The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium such as a floppy disk, hard disk, magnetic tape, an optical medium such as a high-density digital video disc (DVD), or a semiconductor medium such as a solid-state disc (SSD), etc.

[0106] In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware processor, or executed and completed by the combination of the hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0107] It should be noted that the processor in the embodiments of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0108] The above has introduced in detail a radar image extrapolation model based on spatio-temporal-frequency domain fusion proposed by the present invention, and expounded the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A radar image extrapolation model based on spatio-temporal-frequency domain fusion, characterized in that: The radar image extrapolation model includes an encoder, a temporal feature processing module, a spatial feature processing module, and a decoder; The encoder is a feature extraction module that encodes the input radar image sequence into a feature representation; The temporal feature processing module includes an inter-frame comprehensive feature module and an inter-frame extreme feature processing module, which extracts the temporal features in the radar image sequence. Through the weighted fusion of the inter-frame comprehensive feature module and the inter-frame extreme feature module, the future temporal features of the radar image are obtained; The spatial feature processing module includes a global spatial feature processing module and a local spatial feature processing module. The encoded features are respectively passed through the global spatial feature processing module and the local spatial feature processing module and then preliminarily connected to obtain the future spatial features of the radar image; The decoder is a feature reconstruction module that reconstructs the spatio-temporal features obtained after passing through the temporal feature processing module and the spatial feature processing module into a future radar image sequence.

2. The radar image extrapolation model according to claim 1, characterized in that: The encoder is composed of multiple ConvD modules, and each ConvD module consists of a convolutional layer + an activation function + a normalization operation; The encoder maps the input radar image sequence to the feature space through multi-layer convolutional operations, and extracts representative spatio-temporal features; The representative spatio-temporal features include the intensity, shape, and motion trend of the radar echo.

3. The radar image extrapolation model according to claim 2, wherein: In the temporal feature processing module, The inter-frame comprehensive feature module captures the comprehensive temporal changes between frames in the radar image sequence, and models the global temporal dependencies between frames through dilated convolution and depth convolution to extract the overall temporal changes; The inter-frame extreme feature processing module captures the extreme changes between frames in the radar image sequence. It extracts the maximum value in the inter-frame features through a max pooling operation to capture the change of the radar value from small to large; through a fully connected layer, the attention weights of the inter-frame extreme features are generated to adjust the relative importance of the inter-frame features.

4. The radar image extrapolation model according to claim 3, wherein: In the spatial feature processing module, The global spatial feature processing module transforms the radar image from the spatial domain to the frequency domain through Fourier transform, and utilizes the global information in the frequency domain to enhance the perception ability of the model, thereby capturing global spatial features.

5. The radar image extrapolation model according to claim 4, characterized in that: The global spatial feature processing module first performs a two-dimensional Fourier transform on the input radar image to map the spatial distribution of the image into spectral information in the frequency domain; Then, frequency domain features are extracted through a linear transformation in the frequency domain to ensure that the output dimension is consistent with the input, model the overall spatial distribution of the radar image, and capture the global information in the frequency domain; Then, the extracted frequency domain features are transformed back to the spatial domain through an inverse Fourier transform to obtain the global spatial features of the radar image.

6. The radar image extrapolation model according to claim 5, characterized in that: The local spatial feature processing module captures the local spatial features of the radar image by stacking multiple depthwise separable convolutions, increasing the fitting ability of the model to local spatial features; The depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution; wherein, the depthwise convolution performs convolution operations on each input channel respectively to extract local features of each channel; while the pointwise convolution fuses the channels of the output of the depthwise convolution through 1x1 convolution to obtain deeper spatial features of the image.

7. The radar image extrapolation model according to claim 6, characterized in that: The decoder is composed of multiple ConvDT modules, and each ConvDT module is composed of a transposed convolution layer + an activation function + a normalization operation; The decoder gradually reconstructs the spatio-temporal features into a high-dimensional radar image sequence through multiple layers of transposed convolution operations, while restoring the spatial resolution of the radar image, reconstructing the dynamic evolution information of the radar echo, and reconstructing the detailed information in the radar image.

8. A method for running a radar image extrapolation model based on spatio-temporal frequency domain fusion according to any one of claims 1 to 7, characterized in that: The method specifically includes the following steps: Step 1, encoding the input radar image sequence frames into low-dimensional feature vectors through an encoder; Step 2, respectively inputting the encoded features into a temporal feature processing module and a spatial feature extraction module; wherein, the temporal feature processing module is responsible for capturing the dynamic temporal relationship between frames, and the comprehensive feature module and the extreme feature module therein can respectively extract the overall change and extreme change in time. After the two modules are fused, the future temporal features of the radar image can be generated; The spatial feature extraction module mines global spatial features through a global spatial feature processing module; captures local spatial features through a local spatial feature processing module; after the encoded features pass through these two sub-modules respectively and then are preliminarily connected, comprehensive future spatial features can be obtained; Step 3, fusing the temporal features and the spatial features to generate comprehensive spatio-temporal features. Step 4, inputting the fused features into the decoder to decode future radar image frames.

9. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

Images