A radar echo extrapolation method based on time series motion decomposition and dominant fusion

Abstract

This application discloses a radar echo extrapolation method based on temporal motion decomposition and dominant fusion, relating to the field of spatiotemporal sequence prediction technology. The method proposes an innovative network fusion paradigm, specifically including: acquiring implicit long-term / short-term temporal features and explicit global / transient motion features during processing using stacked hierarchical recurrent networks; then, innovatively constructing query vectors using temporal features and key-value vectors using motion features, and fusing them using a heterogeneous cross-attention mechanism to form a fusion architecture "dominated by an implicit state modeling network and referenced by an explicit motion modeling network." Based on this, through a series of collaborative designs such as directional decoupling, reference confidence estimation, and long-term feature constraints on short-term features, noise in long-term / short-term motion features is effectively suppressed, enhancing system stability. This method can improve the problems of blurred texture details and attenuation in high-value echo regions in predicted images.

Description

Technical Field

[0001] This application relates to the field of spatiotemporal sequence prediction technology, particularly radar echo extrapolation technology in radar meteorology, specifically a radar echo extrapolation method based on temporal motion decomposition and dominant fusion. Background Technology

[0002] Radar echo extrapolation is a core technology in short-term weather forecasting, essentially a complex spatiotemporal sequence prediction problem. Models based on recurrent neural networks (RNNs), such as predictive recurrent neural networks (PredRNNs) and their variants, have achieved success in this field due to their powerful temporal modeling capabilities. However, such models typically learn the system state implicitly, which has inherent limitations when modeling explicit and complex motion patterns in the physical scene (such as the movement, deformation, and generation / disappearance of echo cells). This leads to motion ambiguity in the prediction results and a decline in high-value echo intensity over time.

[0003] To introduce explicit motion modeling, existing technologies have proposed modules such as Motion RNNs, which decompose and track motion trends by designing dedicated memory units (such as momentum memory and periodic memory). However, these explicit motion modules estimate motion vectors directly from data, often containing significant noise and exhibiting poor long-term stability. Simply concatenating or paralleling them with implicit state networks such as PredRNNs can lead to the noise from the explicit modules interfering with the stable learning of the implicit state networks, potentially resulting in performance degradation. Summary of the Invention

[0004] The embodiments of this application aim to provide a radar echo extrapolation method based on temporal motion decomposition and dominant fusion, so as to solve or alleviate the technical problems in the prior art that are caused by the insufficient motion modeling capability of implicit state network, resulting in prediction ambiguity and attenuation, and the impact on system stability caused by the introduction of noise due to the simple fusion of explicit motion modules.

[0005] The core concept of this application lies in proposing a "dominant-reference" fusion paradigm. Specifically, an implicit state network (such as PredRNN) capable of stably modeling the long-term evolution of the system is placed in a dominant position, while a network that can provide instantaneous, explicit motion cues but is accompanied by noise (such as Motion GRU) is placed in a reference position. Through a carefully designed attention fusion mechanism, the dominant network "actively inquires" about the reference network and has the right to decide the degree and method of adopting reference information. This allows the network to absorb beneficial motion information while filtering out noise to the maximum extent, achieving a synergistic gain of "1+1>2".

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions: A radar echo extrapolation method based on temporal motion decomposition and dominant fusion includes: Obtain radar echo sequence images of a specified area; Radar echo sequence maps are processed using stacked hierarchical recurrent networks; During the processing, for a specified time and a specified layer, the long-term and short-term temporal features of the next layer at the specified time and the specified layer are obtained based on PredRNN. The overall motion features and transient motion features of the echo between the specified time and the next layer at the specified time are obtained based on the Motion Gated Recurrent Unit (GRU). Here, the specified time is any time in the radar echo sequence diagram, and the specified layer is any layer of the hierarchical recurrent network except for the last layer. A first query vector is constructed based on long-term time features, a first key vector and a first value vector are constructed based on the overall motion features of the echo, long-term reference features are determined based on the first query vector, the first key vector and the first value vector, and long-term enhancement features are determined based on the long-term time features and the long-term reference vector. A second query vector is constructed based on short-term time features, a second key vector and a second value vector are constructed based on echo transient motion features, short-term reference features are determined based on the second query vector, the second key vector and the second value vector, and short-term enhancement features are determined based on the short-term time features and the short-term reference features. Extract short-term filtering features that meet the constraints of long-term enhancement features from short-term enhancement features; The hidden state features of the next layer at a specified time are determined based on long-term enhancement features and short-term filtering features. After the hierarchical recurrent network performs calculations for each time step and each layer in the manner described above, the predicted image for each time step is determined based on the hidden state features in the last layer of the hierarchical recurrent network.

[0007] Optionally, a first key vector and a first value vector are constructed based on the overall motion characteristics of the echo, including: ; ; in, The first key vector, The learnable convolution operator corresponding to the first key vector. The overall motion characteristics of the echo, This indicates a decoupling process between target assignment and direction, used to retain the direction but reduce the assignment information. , Describing the L2 norm, This represents the convolution operation. For the first value vector, For the learnable convolution operator corresponding to the first value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0008] Optionally, the second key vector and the second value vector are constructed using the transient motion characteristics of the echo, including: ; ; in, This is the second key vector. The learnable convolution operator corresponding to the second key vector. The transient motion characteristics of the echo This indicates a decoupling process between the target's assignment and direction, used to retain the direction but reduce the assignment information. , Describing the L2 norm, This represents the convolution operation. For the second value vector, For the learnable convolution operator corresponding to the second value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0009] Optionally, determining long-term reference features based on the first query vector, the first key vector, and the first value vector includes: constructing a first mask based on long-term time features; calculating the first Hadamard product of the first query vector and the first mask; determining a first similarity based on the product of the first Hadamard product and the first key vector; and determining long-term reference features based on the first value vector and the first similarity.

[0010] Optionally, determining short-term reference features based on the second query vector, the second key vector, and the second value vector includes: constructing a second mask based on short-term temporal features; calculating the second Hadamard product of the second query vector and the second mask; determining a second similarity based on the product of the second Hadamard product and the second key vector; and determining short-term reference features based on the second value vector and the second similarity.

[0011] Optionally, a first mask is constructed based on long-term temporal characteristics, including: ; in, As the first mask, For the learnable convolution operator corresponding to the first mask, For long-term time characteristics, It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0012] Optionally, the first similarity is determined based on the product of the first Hadamard product and the first key vector, including: ; in, The first similarity score, This is the first query vector. As the first mask, The process of calculating the Hadamard product, For the first Hadamah, The first key vector, for transpose, Indicates the channel dimension. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0013] Optionally, a second mask is constructed based on short-term temporal characteristics, including: ; in, For the second mask, For the learnable convolution operator corresponding to the second mask, It is a short-term time characteristic. It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0014] Optionally, the second similarity is determined based on the product of the second Hadamard product and the second key vector, including: ; in, For the second similarity, This is the second query vector. For the second mask, The process of calculating the Hadamard product, For the second Hadamah, This is the second key vector. for transpose, Indicates the channel dimension. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0015] Optionally, long-term reference features are determined based on the first value vector and the first similarity, including: ; in, For long-term reference characteristics, For reference confidence level, , Learnable convolution operators used to compute reference confidence levels. The overall motion characteristics of the echo, It is the sigmoid activation function. This represents the convolution operation. The first similarity score, For normalized exponential functions, For the first value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0016] Optionally, short-term reference features are determined based on the second value vector and the second similarity, including: ; in, For short-term reference characteristics, For reference confidence level, , Learnable convolution operators used to compute reference confidence levels. The transient motion characteristics of the echo It is the sigmoid activation function. This represents the convolution operation. For the second similarity, For normalized exponential functions, For the second value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0017] Optionally, long-term enhancement features are determined based on long-term time features and long-term reference features, including: ; in, For long-term enhancement features, For long-term time characteristics, For long-term reference characteristics, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0018] Optionally, short-term enhancement features are determined based on short-term time features and short-term reference features, including: ; in, This is a short-term enhancement feature. It is a short-term time characteristic. For short-term reference characteristics, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0019] Optionally, short-term filtering features that meet the constraints of long-term enhancement features are extracted from the short-term enhancement features, including: The constraint matrix for long-term enhanced features is constructed as follows: ; in, For the constraint matrix, For learnable convolution operators used to compute constraint matrices, For long-term enhancement features, It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; Short-term filtering features are obtained in the following way: ; in, For short-term filtering features, For the constraint matrix, The process of calculating the Hadamard product, For a learnable convolution operator used to compute short-term filtered features, This is a short-term enhancement feature. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0020] Optionally, the hidden state features of the next layer at a specified time are determined based on long-term enhancement features and short-term filtering features, including: ; in, For a specified time, the hidden state feature of the next layer. For long-term enhancement features, This is a short-term filtering feature; This represents a function for computing hidden state features based on PredRNN.

[0021] Optionally, determining the predicted image at each time step based on the hidden state features in the last layer of the hierarchical recurrent network includes: decoding the hidden state features corresponding to each time step in the last layer of the hierarchical recurrent network to obtain the predicted image at each time step.

[0022] Optionally, radar echo sequence maps are processed using stacked hierarchical recurrent networks, including: Retrieve the last short-term temporal feature output by the last layer of the vertical memory unit at the specified time. The last short-term time feature is determined as the input of the first layer at a specified time, so that the first layer at the specified time can calculate the short-term time feature corresponding to this layer.

[0023] The key design principle behind the above technical solution for achieving "dominant-reference" fusion and system stability lies in: (i) Establishing Fusion Dominance through Attention Vector Construction. When fusing the temporal and motion features via a cross-attention mechanism, the core design of this scheme is to consistently construct the query vector using long-term / short-term temporal features from PredRNN, while constructing the key and value vectors using motion features from Motion GRU. This design ensures that the implicit state modeling network maintains control over the fusion process, enabling it to actively filter and integrate explicit motion information, rather than being passively influenced by it.

[0024] (II) Implementation of noise suppression-oriented fusion process control. To overcome the inherent noise problem of explicit motion features, this scheme introduces multiple control mechanisms in the fusion process. First, the direction and assignment of motion features are decoupled to reduce the interference of amplitude noise on attention weight allocation. Second, by introducing the reference confidence level calculated from the motion features, the reliability of motion reference information is dynamically evaluated and its contribution is adjusted accordingly, thereby achieving adaptive noise suppression at the system level.

[0025] (III) Constructing Hierarchical Stability Constraints. To ensure the long-term stability of the system during multi-timestep and multi-network-layer propagation, this scheme establishes a constraint relationship between long-term states and short-term states. Specifically, by utilizing the enhanced long-term features, the enhanced short-term features are guided and filtered, so that short-term dynamic changes conform to the long-term evolution trend, effectively preventing error accumulation and divergence.

[0026] Through the above-mentioned collaborative design centered on "dominance control" and supplemented by "noise suppression" and "stability constraints", this scheme constructs an organic fusion architecture, thereby effectively utilizing explicit motion information to improve prediction accuracy while ensuring the overall robustness of the system.

[0027] The technical solution provided in this application, through an innovative fusion architecture "dominated by implicit state networks and referenced by explicit motion networks," and a series of collaborative noise suppression and stability designs, achieves the following beneficial effects: 1. Significantly improves motion prediction accuracy: The dominant network can stably absorb explicit motion cues provided by the reference network, effectively correcting the bias of traditional PredRNN in motion trajectory prediction and suppressing spurious attenuation of high-value echoes.

[0028] 2. Enhanced detail preservation: By enhancing and filtering short-term features, the texture information of the prediction results is enriched, alleviating the blurring and fading of the predicted image.

[0029] 3. Ensuring overall system stability: The combined effects of dominant design, noise suppression mechanisms, and long-term constraints prevent the instability of explicit motion modules from spreading to the system, ensuring the reliability of long-sequence prediction.

[0030] 4. Provides a scalable fusion paradigm: The "dominant-reference" fusion concept and specific implementation methods can be applied to other spatiotemporal prediction tasks that require the fusion of stable state models and noisy observation models.

[0031] In summary, the predicted image output by the technical solution of this application has clearer texture details and a better effect on improving high-value echo attenuation.

[0032] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0033] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrative descriptions and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are considered similar elements, and wherein: Figure 1 This is a flowchart illustrating a radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in an embodiment of this application. Figure 2 This is a schematic diagram of a stacked hierarchical circular network provided in an embodiment of this application; Figure 3 This is a schematic diagram of the input and output of the interlayer Motion GRU module provided in an embodiment of this application; Figure 4 This is a schematic diagram of the internal algorithm flow of a Motion GRU module provided in an embodiment of this application; Figure 5 This is a schematic diagram of the internal algorithm flow of a STA-LSTM module provided in an embodiment of this application; Figure 6 This is a schematic diagram of a prediction result provided in an embodiment of this application; Figure 7 This is a schematic diagram of another prediction result provided in an embodiment of this application; Figure 8 This is a schematic diagram of a radar echo extrapolation device based on temporal motion decomposition and dominant fusion provided in an embodiment of this application. Detailed Implementation

[0034] To provide a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this application. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0035] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0036] Unless otherwise stated, the term "multiple" means two or more.

[0037] In this embodiment, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0038] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0039] Figure 1 This is a flowchart illustrating a radar echo extrapolation method based on temporal motion decomposition and dominant fusion, provided in an embodiment of this application. This radar echo extrapolation method based on temporal motion decomposition and dominant fusion can be executed on a local computer or a cloud server.

[0040] Combination Figure 1 As shown, the radar echo extrapolation method based on temporal motion decomposition and dominant fusion includes: S101. Obtain the radar echo sequence map of the specified area.

[0041] The specified area mentioned above refers to the specific area that needs to be analyzed.

[0042] After identifying the area, real-time radar echo images of that area are acquired. The acquisition process must ensure the accuracy of the radar system and the real-time nature of the data to obtain the most accurate meteorological data. Accurate radar image acquisition and processing are crucial for meteorological data analysis and processing.

[0043] The acquired radar images may contain some incomplete or inaccurate images. These images may be affected by external factors that cause the radar signal to be reflected or refracted, thus affecting the accuracy of the images. Therefore, they need to be screened.

[0044] First, identify and filter out radar images that are affected by electromagnetic interference.

[0045] Secondly, radar echo images containing zero information are filtered out; these images may not have recorded any useful meteorological information due to radar system malfunctions or other technical problems.

[0046] After the above processing, the following processing is also performed: First, based on the model's input requirements, the original size of the radar image needs to be adjusted to conform to the model's input specifications; this involves image scaling to ensure that the image's resolution and size are suitable for model analysis.

[0047] Next, normalization is performed. Normalization adjusts the data values ​​of the radar image to a specific range. In this application, it is normalized to between 0 and 1 so that the model can better process these data. Normalization not only helps improve the prediction accuracy of the model, but also speeds up the training of the model.

[0048] The radar echo sequence map obtained through the above processing can be effectively used by the model.

[0049] The aforementioned model refers to a model capable of running the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in the embodiments of this application.

[0050] This model could be a weather model.

[0051] S102. Process radar echo sequence diagrams using stacked hierarchical cyclic networks.

[0052] S103. During the processing, for a specified time and a specified layer, the long-term and short-term time features of the next layer at the specified time and the specified layer are obtained based on PredRNN, and the overall motion features and transient motion features of the echo between the specified time and the next layer at the specified time are obtained based on Motion GRU.

[0053] The specified time is any time in the radar echo sequence diagram, and the specified layer is any layer in the hierarchical cyclic network except for the last layer.

[0054] S104. Construct a first query vector based on long-term time features, construct a first key vector and a first value vector based on the overall motion features of the echo, determine long-term reference features based on the first query vector, the first key vector and the first value vector, and determine long-term enhancement features based on the long-term time features and the long-term reference vector.

[0055] Optionally, a first key vector and a first value vector are constructed based on the overall motion characteristics of the echo, including: ; ; in, The first key vector, The learnable convolution operator corresponding to the first key vector. The overall motion characteristics of the echo, This represents the convolution operation. For the first value vector, For the learnable convolution operator corresponding to the first value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0056] Furthermore, the first key vector and the first value vector are constructed based on the overall motion characteristics of the echo, including: ; ; in, The first key vector, The learnable convolution operator corresponding to the first key vector. The overall motion characteristics of the echo, This indicates a decoupling process between target assignment and direction, used to retain the direction but reduce the assignment information. , Describing the L2 norm, This represents the convolution operation. For the first value vector, For the learnable convolution operator corresponding to the first value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0057] by Decoupling the assignment and direction of the overall motion characteristics of the echo while retaining the direction information reduces the noise in the overall motion characteristics of the echo reduced by Motion GRU, thereby making the whole system more stable.

[0058] Optionally, determining long-term reference features based on the first query vector, the first key vector, and the first value vector includes: constructing a first mask based on long-term time features; calculating the first Hadamard product of the first query vector and the first mask; determining a first similarity based on the product of the first Hadamard product and the first key vector; and determining long-term reference features based on the first value vector and the first similarity.

[0059] Specifically, the first mask is constructed based on long-term time characteristics, including: ; in, As the first mask, For the learnable convolution operator corresponding to the first mask, For long-term time characteristics, It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0060] Specifically, the first similarity is determined based on the product of the first Hadamard product and the first key vector, including: ; in, The first similarity score, This is the first query vector. As the first mask, The process of calculating the Hadamard product, For the first Hadamah, The first key vector, for transpose, Indicates the channel dimension. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0061] The first mask mentioned above can indicate which locations in the long-term time characteristics are allowed to be corrected by the overall motion state of the echo. Since the first mask is determined based on the long-term time characteristics, this strengthens the dominant role of RredRNN and the reference role of MotionGRU.

[0062] Furthermore, long-term reference features are determined based on the first value vector and the first similarity, including: ; in, For long-term reference characteristics, For reference confidence level, , Learnable convolution operators used to compute reference confidence levels. The overall motion characteristics of the echo, It is the sigmoid activation function. This represents the convolution operation. The first similarity score, For normalized exponential functions, For the first value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0063] conventional The output is candidate options. In this embodiment, the first similarity is determined based on the first mask. Therefore, using... Processing this first similarity yields the results of determining which positions require state correction and to what extent.

[0064] Furthermore, conventional cross-attention mechanisms assume that both key and value vectors are entirely reliable. This is because the computational results are needed to replace the original features of both vectors used for cross-attention. In this embodiment, since PredRNN is the primary driver and Motion GRU is used as a reference, the reference level needs to be estimated; that is, the reference confidence level is used to represent the reference level. For example, if the output noise of Motion GRU is too high, the reference level is reduced to avoid introducing too much noise into the system and to maintain system stability and accuracy.

[0065] Furthermore, long-term enhancement features are determined based on long-term time characteristics and long-term reference characteristics, including: ; in, For long-term enhancement features, For long-term time characteristics, For long-term reference characteristics, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0066] The process of determining long-term enhancement features indicates that the original features of the two entities involved in the cross-attention process are not replaced by the cross-attention calculation results, but rather the cross-attention calculation results are used to provide a reference for the PredRNN output. Specifically, in this embodiment, long-term time features... This refers to the output of PredRNN, a long-term reference feature. This is a correction to the output of PredRNN.

[0067] S105. Construct a second query vector based on short-term time features, construct a second key vector and a second value vector based on echo transient motion features, determine short-term reference features based on the second query vector, the second key vector and the second value vector, and determine short-term enhancement features based on the short-term time features and the short-term reference features.

[0068] Optionally, the second key vector and the second value vector are constructed using the transient motion characteristics of the echo, including: ; ; in, This is the second key vector. The learnable convolution operator corresponding to the second key vector. The transient motion characteristics of the echo This represents the convolution operation. For the second value vector, For the learnable convolution operator corresponding to the second value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0069] Furthermore, the second key vector and the second value vector are constructed based on the transient motion characteristics of the echo, including: ; ; in, This is the second key vector. The learnable convolution operator corresponding to the second key vector. The transient motion characteristics of the echo This indicates a decoupling process between the target's assignment and direction, used to retain the direction but reduce the assignment information. , Describing the L2 norm, This represents the convolution operation. For the second value vector, For the learnable convolution operator corresponding to the second value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0070] by By decoupling the assignment and direction of the transient motion characteristics of the echo, while retaining the direction, the assignment information is weakened. This reduces the noise in the overall motion characteristics of the echo reduced by Motion GRU, thereby making the whole system more stable.

[0071] Optionally, determining short-term reference features based on the second query vector, the second key vector, and the second value vector includes: constructing a second mask based on short-term temporal features; calculating the second Hadamard product of the second query vector and the second mask; determining a second similarity based on the product of the second Hadamard product and the second key vector; and determining short-term reference features based on the second value vector and the second similarity.

[0072] Specifically, the second mask is constructed based on short-term time characteristics, including: ; in, For the second mask, For the learnable convolution operator corresponding to the second mask, It is a short-term time characteristic. It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0073] Specifically, the second similarity is determined based on the product of the second Hadamard product and the second key vector, including: ; in, For the second similarity, This is the second query vector. For the second mask, The process of calculating the Hadamard product, For the second Hadamah, This is the second key vector. for transpose, Indicates the channel dimension. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0074] The second mask mentioned above can indicate which locations in the short-term time features are allowed to be corrected by the echo transient motion state. Since the second mask is determined based on the long-term time features, this strengthens the dominant role of RredRNN and the reference role of MotionGRU.

[0075] Furthermore, short-term reference features are determined based on the second value vector and the second similarity, including: ; in, For short-term reference characteristics, For reference confidence level, , Learnable convolution operators used to compute reference confidence levels. The transient motion characteristics of the echo It is the sigmoid activation function. This represents the convolution operation. For the second similarity, For normalized exponential functions, For the second value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0076] conventional The output is candidate options. In this embodiment, the second similarity is determined based on the second mask. Therefore, using... Processing this second similarity yields the results of determining which positions require state correction and to what extent.

[0077] Furthermore, conventional cross-attention mechanisms assume that both key and value vectors are entirely reliable. This is because the computational results are needed to replace the original features of both vectors used for cross-attention. In this embodiment, since PredRNN is the primary driver and Motion GRU is used as a reference, the reference level needs to be estimated; that is, the reference confidence level is used to represent the reference level. For example, if the output noise of Motion GRU is too high, the reference level is reduced to avoid introducing too much noise into the system and to maintain system stability and accuracy.

[0078] Furthermore, short-term enhancement features are determined based on short-term time characteristics and short-term reference characteristics, including: ; in, This is a short-term enhancement feature. It is a short-term time characteristic. For short-term reference characteristics, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0079] The process of determining short-term enhancement features indicates that the original features of the two entities involved in the cross-attention process are not replaced by the cross-attention calculation results, but rather the cross-attention calculation results are used to provide a reference for the PredRNN output. Specifically, in this embodiment, short-term temporal features... This refers to the output of PredRNN, specifically the short-term reference features. This is a correction to the output of PredRNN.

[0080] S106. Extract short-term filtering features that meet the constraints of long-term enhancement features from short-term enhancement features.

[0081] Optionally, short-term filtering features that meet the constraints of long-term enhancement features are extracted from the short-term enhancement features, including: The constraint matrix for long-term enhanced features is constructed as follows: ; in, For the constraint matrix, For learnable convolution operators used to compute constraint matrices, For long-term enhancement features, It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; Short-term filtering features are obtained in the following way: ; in, For short-term filtering features, For the constraint matrix, The process of calculating the Hadamard product, For a learnable convolution operator used to compute short-term filtered features, This is a short-term enhancement feature. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

[0082] In this embodiment, long-term enhancement features It can be regarded as a base, constraint matrix This indicates a mask that allows for corrections, thus enhancing short-term features. Disturbances can only be applied within the allowable range of the base. In this way, short-term noise is suppressed by long-term conditions, which helps the system maintain stability.

[0083] S107. Determine the hidden state features of the next layer at a specified time based on the long-term enhancement features and the short-term filtering features.

[0084] Optionally, the hidden state features of the next layer at a specified time are determined based on long-term enhancement features and short-term filtering features, including: ; in, For a specified time, the hidden state feature of the next layer. For long-term enhancement features, This is a short-term filtering feature; This represents a function for computing hidden state features based on PredRNN.

[0085] The above processing procedure is illustrated by taking the processing procedure between a specified layer and the next layer and the processing procedure of the next layer after the specified layer as examples. Since the specified layer is any layer of the hierarchical circular network except the last layer, the above processing procedure does not include the specific processing procedure of the first layer.

[0086] The processes described in S104 to S107 can be encapsulated in a module. Specifically, this module can be named STA-LSTM (Self Attention LSTM). Here, self-attention refers to the overall self-attention calculation process performed on the results obtained from PredRNN and Motion GRU within the module. Of course, after distinguishing between PredRNN and Motion GRU, the overall process can be considered as self-attention, while the specific implementation locally is a cross-attention calculation process.

[0087] Optionally, the first layer of the hierarchical recurrent network also uses the STA-LSTM module, that is, the first layer also executes the above S104 to S107 processes, but with slight differences. The differences between the first layer and other layers include the following two points: The first echo input feature corresponding to the first layer is the radar echo image in the radar echo sequence diagram; The process of obtaining the short-term time features corresponding to the first layer includes: obtaining the last short-term time feature output by the last vertical memory unit of the last layer at the specified time; determining the last short-term time feature as the input of the first layer at the specified time, so that the first layer at the specified time can calculate the short-term time feature corresponding to this layer.

[0088] S108. After the hierarchical recurrent network is calculated for each time step and each layer in the manner described above, the predicted image for each time step is determined based on the hidden state features in the last layer of the hierarchical recurrent network.

[0089] Compared to non-last layers, the STA-LSTM module corresponding to the last layer has an extra step: in the last layer of the hierarchical recurrent network, the hidden state features corresponding to each time step are decoded to obtain the predicted image at each time step.

[0090] The radar echo extrapolation prediction method based on the attention mechanism provided in this application can achieve the following technical effects: In the process of processing radar echo sequence maps using stacked hierarchical recurrent networks, long-term and short-term time features of the next layer at a specified time are obtained based on PredRNN, and overall and transient motion features of the echo between the specified layer at a specified time and the next layer at a specified time are obtained based on Motion GRU.

[0091] Next, a first query vector is constructed using long-term temporal features, and a first key vector and a first value vector are constructed using the overall motion features of the echo. Long-term reference features are then determined based on these three vectors. Conversely, a second query vector is constructed using short-term temporal features, and a second key vector and a second value vector are constructed using the transient motion features of the echo. Short-term reference features are then determined based on these three vectors. This process is based on cross-attention. Typically, the semantics of two objects processed using cross-attention are aligned, and the final output is the result of the cross-attention mechanism.

[0092] In this technical solution, the long-term time and short-term time features obtained based on PredRNN are semantically incompatible with the overall motion features and transient motion features of the echo obtained based on Motion GRU. For example, the results obtained by PredRNN focus on the implicit system state and have long-term stability, while the results obtained based on Motion GRU focus on the explicit motion state and are less likely to have long-term stability.

[0093] In this scenario, constructing the first or second query vector using long-term or short-term time features obtained from PredRNN means that PredRNN is "asking questions," essentially handing over the prediction leadership to PredRNN. Furthermore, after obtaining long-term and short-term reference features, the final fusion result is not based on either feature. Instead, long-term enhancement features are determined based on the long-term and reference features, and short-term enhancement features are determined based on the short-term and reference features. This means that long-term features again participate in determining long-term enhancement features, and short-term features again participate in determining short-term enhancement features. Thus, fusing the results obtained from PredRNN and Motion GRU is actually led by PredRNN, with Motion GRU serving as a reference, supplement, or correction. Furthermore, since the results obtained based on Motion GRU focus on displaying motion state, they are not easy to maintain long-term stability. This also indicates that the results obtained based on Motion GRU have a large amount of noise. Therefore, instead of using Motion GRU as the main driver and PredRNN as the reference, we use PredRNN as the main driver and Motion GRU as the reference for fusion. This can reduce the impact of noise in Motion GRU on the overall system and maintain the stability of the radar extrapolation process.

[0094] After fusing the results obtained by using PredRNN as the primary source and Motion GRU as a reference, PredRNN can ensure long-term stability while also absorbing the explicit motion features exhibited by Motion GRU. These explicit motion features can shift the long-term temporal features obtained based on PredRNN towards a direction that is more consistent with actual motion, that is, it has a suppressive effect on the attenuation of high-value echo regions over time. At the same time, these explicit motion features can also enrich the short-term temporal features obtained based on PredRNN.

[0095] Next, short-term filtering features that meet the constraints of long-term enhancement features are extracted from the short-term enhancement features to filter out the noise absorbed by PredRNN from Motion GRU and retain the details that meet the constraints of long-term enhancement features. These details can suppress blurring, fading and other issues in the final predicted image.

[0096] In addition, the more details are preserved at each moment, the less likely information is to be lost during long-term evolution, and it also has a suppressive effect on the attenuation phenomenon in the high-value echo region.

[0097] Figure 2 This is a schematic diagram of a stacked hierarchical recurrent network provided in an embodiment of this application, used to illustrate the overall processing procedure of the hierarchical recurrent network.

[0098] refer to Figure 2 , All are STA-LSTM modules, their subscripts Indicates the number of floors; Representing radar echo sequence diagrams A radar echo image at a given moment; express The predicted image at any given time is the actual image after extrapolation of the radar echo; express Time of the first The short-term temporal characteristics of the output of the vertical memory unit of the layer; express Time of the first Long-term temporal characteristics of the output of the lateral memory units of the layer; express Time of the first Hidden state characteristics of the layer.

[0099] Figure 3 This is a schematic diagram of the input and output of the interlayer Motion GRU module provided in the embodiments of this application.

[0100] refer to Figure 3 , express time Figure 3 upper side (No. Echo input characteristics of the STA-LSTM module (layer); express time Figure 3 Top and bottom sides (the first) Layer and first Echo transient motion characteristics between STA-LSTM modules (layers); express time Figure 3 Top and bottom sides (the first) Layer and first Overall motion characteristics of echoes between STA-LSTM modules (layers); express time Figure 3 Top and bottom sides (the first) Layer and first Echo transient motion characteristics between STA-LSTM modules (layers); express time Figure 3 Top and bottom sides (the first) Layer and first Overall motion characteristics of echoes between STA-LSTM modules (layers); express time Figure 3 lower side (the first) Hidden state features of the STA-LSTM module (layer); express time Figure 3 lower side (the first) Short-term time characteristics of the STA-LSTM module (layer); The function of Motion GRU can be summarized as follows: ; Depend on Figure 3 lower side (the first) (Layer) STA-LSTM module points to Figure 3 upper side (No. The dashed line in the STA-LSTM module represents the Motion High Way, indicating a direct transmission operation that does not pass through the Motion GRU module.

[0101] Figure 4 This is a schematic diagram of the internal algorithm flow of a Motion GRU module provided in an embodiment of this application.

[0102] Combination Figure 4 As shown, an example is provided for S103 to obtain the overall motion characteristics and transient motion characteristics of the echo between a specified layer and the next layer at a specified time based on Motion GRU.

[0103] Optionally, based on Motion GRU, the overall motion characteristics and transient motion characteristics of the echo between a specified layer at a specified time and the next layer at a specified time are obtained, including: Based on the first hidden state characteristics of a specified layer at a specified time, the first echo transient motion characteristics of the specified layer at the previous time between the specified layer and the next layer, and the first echo overall motion characteristics, determine the first echo input characteristics of the next layer at a specified time, and the second echo transient motion characteristics and the second echo overall motion characteristics of the next time between the specified layer and the next layer.

[0104] Furthermore, the specific steps for obtaining the transient motion characteristics of the second echo in the above process include: The first hidden state features are encoded to obtain the encoded echo output features; The first splicing feature is obtained by splicing and encoding the output echo features and the first echo transient motion features. The first concatenated features are processed according to the updated convolution kernel to obtain the updated gate features; The first concatenated features are processed according to the reset convolution kernel to obtain the reset gate features; The second splicing feature is obtained by concatenating the encoded output echo, the reset gate feature, and the Hadamard product of the transient motion feature of the first echo; The second concatenated features are processed based on the reset convolution kernel to obtain the reset features; The updated residual features are obtained by subtracting the update gate features from 1. The transient change characteristics of the echo are determined based on the Hadamard product of the update gate characteristics and the reset characteristics, and the Hadamard product of the update residual characteristics and the first hidden state characteristics. The transient motion characteristics of the second echo are determined by summing the transient change characteristics of the second echo with the overall motion characteristics of the second echo.

[0105] Specifically, the calculation formula for this process is as follows: ; ; ; ; ; in, To update the gate features, It is the sigmoid activation function. To update the convolution kernel, the scale is 1. 1, For convolution operations, For splicing operations, For encoding operations, Features of the first hidden state The transient motion characteristics of the first echo To reset the door features, To reset the convolution kernel, the scale is 1. 1, For reset characteristics, To reset the convolution kernel, the scale is 1. 1, The process of calculating the Hadamard product, i.e., dot product of corresponding positions, It is the hyperbolic tangent function. Echo transient characteristics, echo transient characteristics The spatiotemporal coherence that can reflect transient changes is reflected in the transient characteristics of echoes. During the calculation process, the transient motion characteristics of the first echo Equivalent to a motion filter from the previous moment, it can capture transient changes and echo transient characteristics. This represents the position transition of each pixel between adjacent states. Representing pixel-level displacement, it belongs to the "displacement space," which is the memory unit of the spatiotemporal state transition space. , different, The transient motion characteristics of the second echo, The second echo represents the overall motion characteristics, indicating both trend momentum and transient motion characteristics. Echo transient change characteristics Combined with trend momentum.

[0106] To encode the echo output characteristics, This is the first splicing feature. This is the second splicing feature. To update the residual features.

[0107] At the specified time This indicates that the specified layer is... In the case described, the transient motion characteristics of the echo between the specified layer at the specified time and the next layer at the specified time in S103 are the aforementioned second transient motion characteristics of the echo. .

[0108] Optionally, the specific process of obtaining the overall motion characteristics of the second echo includes: Obtain the difference between the transient motion characteristics of the first echo and the overall motion characteristics of the first echo; The overall motion characteristics of the second echo are determined based on the difference characteristics and the overall motion characteristics of the first echo.

[0109] Specifically, the calculation formula for this process is as follows: ; in, The overall motion characteristics of the second echo, The overall motion characteristics of the first echo, The default value for the update step size is 0.5, which can be modified based on experience. This step, which represents the transient motion characteristics of the first echo, is equivalent to momentum renewal.

[0110] In the above formula, the transient motion characteristics of the first echo of the motion filter from the previous moment are used. As a basis for estimating the trend of motion at a specified moment.

[0111] At the specified time This indicates that the specified layer is... In the case described, the overall motion characteristics of the echo between the specified layer at the specified time and the next layer at the specified time in S103 are the aforementioned second overall motion characteristics of the echo. .

[0112] Combined again Figure 3 As shown, the function of Motion GRU is: The foregoing embodiments respectively illustrate the acquisition of the transient motion characteristics of the second echo. Overall motion characteristics of the second echo The calculation process is as follows, and the first echo input features are discussed below. The calculation process is illustrated by example.

[0113] Combined again Figure 4 First echo input characteristics The calculation process includes: The first hidden state features are encoded to obtain the encoded echo output features; The encoded echo output features are broadcast based on the broadcast weight matrix to obtain a mask for the overall motion features of the second echo. Warp operation is performed on the coded echo output features and the overall motion features of the second echo, and the Hadamard product of the Warp operation and the mask is determined as the echo propagation features of the first hidden state features. Perform Dec operation processing on the echo propagation characteristics; The Dec operation processing result and the first hidden state feature are concatenated to obtain the third concatenated feature; Construct a third modulation gate based on the third splicing feature; The first hidden state feature and the Dec operation processing result are modulated according to the third modulation gate, and the modulation result is determined as the first echo input feature.

[0114] Specifically, the calculation formula for this process is as follows: ; ; ; ; in, Mask of the overall motion characteristics of the second echo. Features of the first hidden state For encoding operations, For broadcast weight matrix, It is the sigmoid activation function. Broadcast operation is a function that processes data of different dimensions and can preserve the mask. Features of the first hidden state The tensors must have the same dimension. When the tensor dimensions are inconsistent, the broadcast mechanism will automatically adjust the tensor dimensions to ensure that the computation can proceed smoothly. It is a transformation operation that maps a pixel from its previous state to its position in the next state. Echo propagation characteristics This is the third modulation gate, equivalent to the output of the third modulation gate. for convolution kernel, For convolution operations, For splicing operations, It is a clustering operation that utilizes deep neural networks. For the first echo input features, The process of calculating the Hadamard product is, in other words, the dot product of corresponding positions.

[0115] To encode the echo output characteristics, This is the result of the Dec operation. This is the third splicing feature.

[0116] The final output of Motion GRU It is input The gated result of the decoder output, where the decoder output is passed through a motion-based filter. The Wrap operation provides explicit transitions. Motion GRU can effectively simulate spatiotemporally changing motion by capturing transient changes and motion trends separately and fusing them into a unified unit.

[0117] Figure 5 This is a schematic diagram of the internal algorithm flow of a STA-LSTM module provided in an embodiment of this application.

[0118] The content within the two dashed boxes represents the algorithm flow for obtaining the long-term and short-term time features of the next layer at a specified time using PredRNN.

[0119] Optionally, based on PredRNN, long-term temporal features of the next layer at a specified time are obtained, including: The first echo input feature is processed according to the first input weight matrix in the first input gate, the second hidden state feature is processed according to the first output weight matrix in the first input gate, and the output of the first input gate is determined according to the two processing results. The first echo input feature is processed according to the second input weight matrix in the first modulation gate, the second hidden state feature is processed according to the second output weight matrix in the first modulation gate, and the output of the first modulation gate is determined according to the two processing results. The first echo input feature is processed according to the third input weight matrix in the first forget gate, the second hidden state feature is processed according to the third output weight matrix in the first forget gate, and the output of the first forget gate is determined based on the two processing results. Based on the output of the first input gate, the output of the first modulation gate, the output of the first forget gate, and the first long-term time characteristics of the horizontal memory unit of the next layer of the specified layer at the specified time, determine the second long-term time characteristics of the output of the horizontal memory unit of the next layer of the specified layer at the specified time.

[0120] Specifically, the above process can be achieved using the following formula: ; ; ; ; The output of the first input gate, It is the sigmoid activation function. This is the first input weight matrix. For convolution operations, For the first echo input features, This is the first output weight matrix. This is a feature of the second hidden state. The output of the first modulation gate, It is the hyperbolic tangent function. The second input weight matrix This is the second output weight matrix. This is the third input weight matrix. This is the third output weight matrix. This is the second long-term time feature. This is the first long-term characteristic. The process of calculating the Hadamard product is, in other words, the dot product of corresponding positions.

[0121] At the specified time This indicates that the specified layer is... In the case of S103, the long-term time feature of the layer below the specified time is the second long-term time feature mentioned above. .

[0122] Optionally, short-term temporal features of the next layer at a specified time can be obtained based on PredRNN, including: The first echo input feature is processed according to the fourth input weight matrix of the second input gate, the first short-term time feature is processed according to the first memory weight matrix of the second input gate, and the output of the second input gate is determined based on the two processing results. The first echo input feature is processed according to the fifth input weight matrix of the second modulation gate, the first short-term time feature is processed according to the second memory weight matrix of the second modulation gate, and the output of the second modulation gate is determined based on the two processing results. The first echo input feature is processed according to the sixth input weight matrix of the second forget gate, the first short-term time feature is processed according to the third memory weight matrix of the second forget gate, and the output of the second forget gate is determined based on the two processing results. Based on the output of the second input gate, the output of the second modulation gate, the output of the second forget gate, and the first short-term time characteristic, determine the second short-term time characteristic of the output of the vertical memory unit of the next layer of the specified layer at a specified time.

[0123] Specifically, the above process can be achieved using the following formula: ; ; ; ; in, The output of the second input gate, It is the sigmoid activation function. This is the fourth input weight matrix. For convolution operations, For the first echo input features, This is the first memory weight matrix. This is the first short-term time characteristic. This is the output of the second modulation gate. It is the hyperbolic tangent function. This is the fifth input weight matrix. This is the second memory weight matrix. This is the output of the second forget gate. This is the sixth input weight matrix. This is the third memory weight matrix. This is the second short-term time characteristic. The process of calculating the Hadamard product is, in other words, the dot product of corresponding positions.

[0124] At the specified time This indicates that the specified layer is... In the case of S103, the short-term time feature of the next layer below the specified time is the aforementioned second short-term time feature. .

[0125] To continue combining Figure 5 The following is an example of S107, “Determine the hidden state features of the next layer at a specified time based on long-term enhancement features and short-term filtering features”.

[0126] Optionally, the hidden state features of the next layer at a specified time are determined based on long-term enhancement features and short-term filtering features, including: ; ; in, As a feature of fusion, It is the sigmoid activation function. This is the seventh input weight matrix. For convolution operations, For the first echo input features, This is the fourth input weight matrix. This is a feature of the second hidden state. For memorizing the weight matrix, For long-term enhancement features, For short-term filtering features, For a specified time, the hidden state feature of the next layer. The process of calculating the Hadamard product, i.e., dot product of corresponding positions, It is the hyperbolic tangent function. for The convolution kernel.

[0127] Refer again Figure 3 As shown, Figure 3 The lower STA-LSTM is directed towards the Motion High Way. Figure 3 The STA-LSTM on the upper side passes two parameters: the first hidden state feature of the specified layer at a specified time. and the first short-term time feature of a specified layer at a specified time. .

[0128] Among them, the first short-term time characteristic The foregoing embodiments can be used to calculate the second short-term time feature. and the hidden state features of the next layer below the specified layer at a specified time. .

[0129] Obtain the hidden state features of the next layer below a specified layer at a specified time. Then, based on the features of the first hidden state Hidden state features of the next layer below a specified layer at a specified time. The update process is as follows: ; Among them, the left side of the equation For the updated hidden state features, the right side of the equation is... The hidden state features before the update. The calculation method for the fusion feature is as follows: , As a feature of fusion, It is the sigmoid activation function. This is the seventh input weight matrix. For convolution operations, For the first echo input features, This is the fourth input weight matrix. This is a feature of the second hidden state. For memorizing the weight matrix, For long-term enhancement features, For short-term filtering features, Used to indicate how much of the characteristics are retained that do not change over time. This is the first hidden state feature.

[0130] At this point, the first hidden state feature is updated based on the first hidden state feature, and the updated hidden state feature is used as the echo output feature of the next layer at the specified time and the specified layer.

[0131] This can reduce the attenuation of motion information between multiple layers.

[0132] The radar echo extrapolation method based on temporal motion decomposition and dominant fusion is implemented through a trained neural network model.

[0133] The training process of the neural network model uses reverse scheduled sampling (RSS), specifically: The model framework outputs a predicted image for the next time step after each input image. Therefore, during model training, except for the first time step, each time step has both the current real-world image and the predicted image from the previous time step. To force the model to learn more about long-term dynamics, the input phase initially uses the predicted output from the previous time step as the input for the current time step. As training progresses over time steps, the probability of using the predicted image is gradually reduced, and the current real-world image is used instead.

[0134] Figure 6 This is a schematic diagram of a prediction result provided in an embodiment of this application. Figure 7 This is a schematic diagram of another prediction result provided in an embodiment of this application. PredRNN_V2 is a specific PredRNN model.

[0135] In the field of meteorology, a region of high-value echoes can be regarded as an echo cell. Over time, the echo cell in reality moves and deforms continuously.

[0136] Observation and comparison Figure 6 and Figure 7 The prediction results of the two models shown indicate that the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in this embodiment of the application performs better in terms of both location and intensity prediction for high-value echoes from 72 minutes to 2 hours. A magnified view of the prediction results at 2 hours clearly shows that the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in this embodiment of the application is significantly superior in predicting high-value echoes.

[0137] The visualization results above show that the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in this application embodiment can better address the problem of high-value echo attenuation.

[0138] In some embodiments, the radar echo extrapolation method apparatus based on temporal motion decomposition and dominant fusion includes a processor and a memory storing program instructions. The processor is configured to execute the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in the foregoing embodiments when executing the program instructions.

[0139] Figure 8 This is a schematic diagram of the radar echo extrapolation method device based on temporal motion decomposition and dominant fusion provided in the embodiments of this application.

[0140] Combination Figure 8 As shown, the radar echo extrapolation method based on temporal motion decomposition and dominant fusion includes the following apparatus: The processor 81 and memory 82 may also include a communication interface 83 and a bus 84. The processor 81, communication interface 83, and memory 82 can communicate with each other via the bus 84. The communication interface 83 can be used for information transmission. The processor 81 can call logical instructions in the memory 82 to execute the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in the foregoing embodiments.

[0141] Furthermore, the logic instructions in the aforementioned memory 82 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0142] The memory 82, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 81 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 82, thereby implementing the methods in the above-described method embodiments.

[0143] The memory 82 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 82 may include high-speed random access memory and may also include non-volatile memory.

[0144] This application provides a computer-readable storage medium storing computer-executable instructions configured to execute the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in the foregoing embodiments.

[0145] This application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to execute the radar echo extrapolation method based on temporal motion decomposition and dominant fusion provided in the foregoing embodiments.

[0146] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0147] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0148] The foregoing description and accompanying drawings fully illustrate embodiments of this application to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Additionally, when used in this application, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Unless otherwise specified, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes that element. In this document, each embodiment may focus on describing the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, then the relevant parts can be referred to the description of the method section.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0150] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A radar echo extrapolation method based on temporal motion decomposition and dominant fusion, characterized in that, include: Obtain radar echo sequence images of a specified area; The radar echo sequence map is processed using a stacked hierarchical recurrent network; During the processing, for a specified time and a specified layer, the long-term and short-term time features of the next layer at the specified time and the specified layer are obtained based on PredRNN, and the overall motion features and transient motion features of the echo between the specified time and the specified layer and the next layer at the specified time are obtained based on Motion GRU; wherein, the specified time is any time in the radar echo sequence diagram, and the specified layer is any layer of the hierarchical recurrent network except the last layer. A first query vector is constructed using the long-term time features, a first key vector and a first value vector are constructed using the overall motion features of the echo, a long-term reference feature is determined based on the first query vector, the first key vector and the first value vector, and a long-term enhancement feature is determined based on the long-term time features and the long-term reference feature. A second query vector is constructed using the short-term time features, a second key vector and a second value vector are constructed using the echo transient motion features, a short-term reference feature is determined based on the second query vector, the second key vector and the second value vector, and a short-term enhancement feature is determined based on the short-term time features and the short-term reference vector. Extract short-term filtering features that meet the constraints of the long-term enhancement features from the short-term enhancement features; The hidden state features of the next layer at a specified time are determined based on the long-term enhancement features and the short-term filtering features. After the hierarchical recurrent network performs calculations for each time step and each layer in the manner described above, the predicted image for each time step is determined based on the hidden state features in the last layer of the hierarchical recurrent network.

2. The radar echo extrapolation prediction method according to claim 1, characterized in that, Constructing a first key vector and a first value vector based on the overall motion characteristics of the echo includes: ； ； in, For the first key vector, The learnable convolution operator corresponding to the first key vector. The overall motion characteristics of the echo, This indicates a decoupling process between target assignment and direction, used to retain the direction but reduce the assignment information. , Represents the L2 norm. This represents the convolution operation. For the first value vector, The learnable convolution operator corresponding to the first value vector. Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; Constructing a second key vector and a second value vector based on the transient motion characteristics of the echo includes: ； ； in, This is the second key vector. The learnable convolution operator corresponding to the second key vector. The transient motion characteristics of the echo, This indicates a decoupling process between the target's assignment and direction, used to retain the direction but reduce the assignment information. , Represents the L2 norm. This represents the convolution operation. For the second value vector, The learnable convolution operator corresponding to the second value vector. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

3. The radar echo extrapolation prediction method according to claim 1, characterized in that, Determining long-term reference features based on the first query vector, the first key vector, and the first value vector includes: constructing a first mask based on the long-term time features; calculating a first Hadamard product between the first query vector and the first mask; determining a first similarity based on the product of the first Hadamard product and the first key vector; and determining the long-term reference features based on the first value vector and the first similarity. Determining short-term reference features based on the second query vector, the second key vector, and the second value vector includes: constructing a second mask based on the short-term time features; calculating a second Hadamard product between the second query vector and the second mask; determining a second similarity based on the product of the second Hadamard product and the second key vector; and determining the short-term reference features based on the second value vector and the second similarity.

4. The radar echo extrapolation prediction method according to claim 3, characterized in that, Constructing a first mask based on the long-term time characteristics includes: ； in, For the first mask, The learnable convolution operator corresponding to the first mask. This refers to the long-term time characteristic. It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; The first similarity is determined based on the product of the first Hadamard product and the first key vector, including: ； in, For the first similarity, This is the first query vector. For the first mask, The process of calculating the Hadamard product, This is the first Hadama product. For the first key vector, for transpose, Indicates the channel dimension. Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; Constructing a second mask based on the aforementioned short-term temporal characteristics includes: ； in, For the second mask, The learnable convolution operator corresponding to the second mask. For the aforementioned short-term time characteristics, It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; The second similarity is determined based on the product of the second Hadamard product and the second key vector, including: ； in, For the second similarity, This is the second query vector. For the second mask, The process of calculating the Hadamard product, This is the second Hadamarda product. This is the second key vector. for transpose, Indicates the channel dimension. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

5. The radar echo extrapolation prediction method according to claim 3, characterized in that, Determining the long-term reference feature based on the first value vector and the first similarity includes: ； in, For the aforementioned long-term reference feature, For reference confidence level, , The learnable convolution operator used to calculate the reference confidence level The overall motion characteristics of the echo, It is the sigmoid activation function. This represents the convolution operation. For the first similarity, For normalized exponential functions, For the first value vector, Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; Determining the short-term reference feature based on the second value vector and the second similarity includes: ； in, For the aforementioned short-term reference feature, For reference confidence level, , The learnable convolution operator used to calculate the reference confidence level The transient motion characteristics of the echo It is the sigmoid activation function. This represents the convolution operation. For the second similarity, For normalized exponential functions, For the second value vector, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

6. The radar echo extrapolation prediction method according to claim 1, characterized in that, Determining long-term enhancement features based on the long-term time features and the long-term reference features includes: ； in, For the long-term enhancement feature, This refers to the long-term time characteristic. For the aforementioned long-term reference feature, Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; The short-term enhancement features are determined based on the short-term time features and the short-term reference features, including: ； in, For the aforementioned short-term enhancement feature, For the aforementioned short-term time characteristics, For the aforementioned short-term reference feature, Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

7. The radar echo extrapolation prediction method according to claim 1, characterized in that, Extracting short-term filtering features that meet the constraints of the long-term enhancement features from the short-term enhancement features includes: The constraint matrix of the long-term enhancement feature is constructed as follows: ； in, Let be the constraint matrix. For a learnable convolution operator used to compute the constraint matrix, For the long-term enhancement feature, It is the sigmoid activation function. This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the next level below the specified level; The short-term filtering features are obtained in the following manner: ； in, For the short-term filtering feature, Let be the constraint matrix. The process of calculating the Hadamard product, For the learnable convolution operator used to compute the short-term filtered features, For the aforementioned short-term enhancement feature, This represents the convolution operation. Indicates a specified time. Indicates the specified layer. Indicates the layer below the specified layer.

8. The radar echo extrapolation prediction method according to claim 1, characterized in that, Determining the hidden state features of the next layer at a specified time based on the long-term enhancement features and the short-term filtering features includes: ； in, For a specified time, the hidden state feature of the next layer. For long-term enhancement features, This is a short-term filtering feature; This represents a function for computing hidden state features based on PredRNN.

9. The radar echo extrapolation prediction method according to any one of claims 1 to 8, characterized in that, The predicted image at each time step is determined based on the hidden state features in the last layer of the hierarchical recurrent network, including: In the last layer of the hierarchical recurrent network, the hidden state features corresponding to each time step are decoded to obtain the predicted image at each time step.

10. The radar echo extrapolation prediction method according to any one of claims 1 to 8, characterized in that, The radar echo sequence map is processed using a stacked hierarchical recurrent network, including: Retrieve the last short-term temporal feature output by the last layer of the vertical memory unit at the specified time. The last short-term time feature is determined as the input of the first layer at a specified time, so that the first layer at the specified time can calculate the short-term time feature corresponding to this layer.

Images