Precipitation nowcast image generation method, system, electronic device, and storage medium

Abstract

The application discloses a precipitation near-forecast image generation method and system, an electronic device and a storage medium, and relates to the technical field of weather forecasting. The method comprises the following steps: acquiring m frames of to-be-predicted images that need to be subjected to precipitation near-forecast; performing downsampling on all the to-be-predicted images to obtain image features, and performing time position coding on all the to-be-predicted images to obtain first coding information; coding all the image features and the first coding information to obtain second coding information; coding n time points of the m frames of to-be-predicted images at time T=[m+1, m+2,..., m+n] to obtain third coding information; decoding the second coding information and the third coding information to obtain n decoded images, the number of the decoded images being the same as the number of the time points; and performing upsampling on the n decoded images to obtain n frames of precipitation near-forecast images.

Description

Technical Field

[0001] This invention relates to the field of weather forecasting technology, and in particular to a method, system, electronic device, and storage medium for generating near-term precipitation forecast images. Background Technology

[0002] With the development of deep learning technology, the use of deep learning for weather forecasting has gradually been accepted by meteorologists. In precipitation nowcasting, a typical model is single-frame input and single-frame output, recursively generating the required radar image sequence, such as PredRNN (Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs) and Memory In Memory (A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics). This recursive generation method uses predicted radar images to predict the next radar image, so the error from each prediction accumulates and increases over time. This unavoidable accumulation of errors directly leads to a sharp decline in the prediction performance of the single-frame input, single-frame output model over time.

[0003] To address the drawbacks of single-frame input / single-frame output models, researchers have explored multi-frame input / multi-frame output prediction models for precipitation near-term prediction. However, existing multi-frame input / multi-frame output prediction models have not performed well. The reason lies in the fact that these models do not fully leverage the advantages of MIMO (Multi-In-Multi-Out). For example, the transformer model used in Vit (An image is worth 16x16 words: Transformers for image recognition at scale) only employs linear operations in its multi-head attention module. This operation loses spatial correlation in the radar image sequence, and its feedforward layer is also implemented using fully connected layers, directly leading to the loss of spatiotemporal correlation between frames in the sequence. Therefore, this method of prediction struggles to meet the required accuracy. Another model based on convolutional neural networks (SimVP: Simpler Yet Better Video Prediction) aims to perform predictions solely through convolutional neural networks. Since convolution operations only extract features from a local area of ​​the radar image, they cannot effectively capture the global feature information of the radar image, thus inevitably leading to poorer prediction results.

[0004] Therefore, existing precipitation nowcasting methods have poor prediction performance, and there is an urgent need for a precipitation nowcasting method with better prediction performance. Summary of the Invention

[0005] The main objective of this invention is to provide a method, system, electronic device, and storage medium for generating near-term precipitation forecast images, aiming to solve the technical problem of poor forecasting performance in existing near-term precipitation forecasting methods.

[0006] To achieve the above objectives, the first aspect of the present invention provides a method for generating near-term precipitation prediction images, comprising: acquiring m frames of images to be predicted for near-term precipitation prediction; downsampling all images to be predicted to obtain image features, and performing time and position encoding on all images to be predicted to obtain first encoding information; encoding all image features and the first encoding information to obtain second encoding information; encoding n time points T = [m+1, m+2, ..., m+n] of the m frames of images to be predicted to obtain third encoding information; decoding the second encoding information and the third encoding information to obtain n decoded images, wherein the number of decoded images is the same as the number of time points; and upsampling the n decoded images to obtain n frames of near-term precipitation prediction images.

[0007] Further, encoding all image features and the first encoding information includes: using a pre-built two-dimensional multi-head attention mechanism to extract attention from the image features and the temporal location encoding to obtain first extracted information; normalizing the first extracted information, the image features, and the temporal location encoding to obtain first normalized data; associating the first normalized data with spatiotemporal correlation in a pre-built local spatiotemporal correlation network to obtain temporal correlation information and spatial correlation information between different images to be predicted; and normalizing the temporal correlation information and the spatial correlation information with the first normalized data to obtain second encoding information.

[0008] Further, the decoding of the second encoded information and the third encoded information to obtain the decoded n images includes: using a pre-built two-dimensional multi-head attention mechanism to extract attention from the third encoded information to obtain second extracted information; normalizing the second extracted information and the third encoded information to obtain second normalized data; using a pre-built two-dimensional multi-head attention mechanism to extract attention from the second normalized data and the first encoded information to obtain third extracted information; normalizing the third extracted information and the second normalized data to obtain third normalized data; associating the third normalized data with spatiotemporal correlation in a pre-built local spatiotemporal correlation network to obtain temporal correlation information and spatial correlation information between predicted images; and normalizing the temporal correlation information and spatial correlation information between predicted images with the third normalized data to obtain the decoded n images.

[0009] Furthermore, the method for constructing the two-dimensional multi-head attention mechanism includes: pre-constructing a multi-head attention mechanism for a transformer model; replacing the generation method of the query, key-value, and value items in the multi-head attention mechanism with generating the query, key-value, and value items using convolution; and replacing the linear operation of attention weighting the query, key-value, and value items in the multi-head attention mechanism with a convolution operation to obtain the two-dimensional multi-head attention mechanism.

[0010] Furthermore, the method for constructing the two-dimensional multi-head attention mechanism also includes: removing the prediction occlusion mechanism from the multi-head attention mechanism.

[0011] Furthermore, the method for constructing the local spatiotemporal correlation network includes: replacing the preceding network of a pre-constructed transformer model with a three-dimensional convolutional neural network to obtain a local spatiotemporal correlation network, which is used to perform convolution operations in the temporal and spatial domains to preserve the spatial and temporal correlations between images.

[0012] Furthermore, the method further includes: encoding and integrating all image features and time position codes into an encoder, and replacing the encoder in the pre-built transformer model; decoding the second encoded information and the third encoded information to obtain n decoded images, wherein the number of decoded images is the same as the number of time points, and integrating them into a decoder, and replacing the decoder in the pre-built transformer model.

[0013] A second aspect of the present invention provides a precipitation nowcasting image generation system, comprising: an image acquisition module for acquiring m frames of images to be predicted for precipitation nowcasting; an image processing module for downsampling all images to be predicted to obtain image features, and performing time-location encoding on all images to be predicted to obtain first encoding information; an encoder module for encoding all image features and the first encoding information to obtain second encoding information; a time point encoding module for encoding n time points T = [m+1, m+2, ..., m+n] of the m frames of images to be predicted to obtain third encoding information; a decoder module for decoding the second encoding information and the third encoding information to obtain n decoded images, wherein the number of decoded images is the same as the number of time points; and an upsampling module for upsampling the n decoded images to obtain n frames of precipitation nowcasting images.

[0014] A third aspect of the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any one of the above-described methods for generating near-term precipitation prediction images.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements any one of the above-described methods for generating precipitation near-term prediction images.

[0016] This invention provides a method, system, electronic device, and storage medium for generating near-term precipitation prediction images. Its advantages lie in the following: by using a multi-frame input and multi-frame output approach (inputting m frames and outputting n frames), it overcomes the shortcomings of single-frame input and single-frame output, preventing a sharp decline in prediction accuracy over time. Furthermore, because the downsampled image features possess spatial characteristics, and this invention combines the temporal features of n time points from the m frames to be predicted, it can better utilize the global feature information of the image to be predicted. This overcomes the shortcomings of existing multi-frame input and multi-frame output methods, which can only extract features locally from radar images and cannot effectively grasp the global feature information of radar images. Therefore, the precipitation near-term prediction image generation method, system, electronic device, and storage medium provided by this invention have better prediction results for near-term precipitation prediction. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the precipitation now-prediction image generation method according to an embodiment of the present invention;

[0019] Figure 2 This is a flowchart illustrating the functional implementation of the precipitation now-prediction image generation method according to an embodiment of the present invention.

[0020] Figure 3 This is a framework diagram of the precipitation now-predictive image generation system according to an embodiment of the present invention;

[0021] Figure 4 This is a schematic block diagram of the electronic device according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] Please see Figure 1 This is a method for generating near-term precipitation prediction images, comprising:

[0024] S101. Obtain m frames of images to be predicted for precipitation near-term forecasting;

[0025] S102. Downsample all images to be predicted to obtain image features, and perform time-position encoding on all images to be predicted to obtain the first encoding information;

[0026] S103. Encode all image features and the first coding information to obtain the second coding information;

[0027] S104. Encode the n time points T = [m+1,m+2,…m+n] of the m-frame image to be predicted to obtain the third encoded information;

[0028] S105. Decode the second and third encoded information to obtain n decoded images. The number of decoded images is the same as the number of time points.

[0029] S106. Upsample the decoded n images to obtain n frames of precipitation near-term prediction images.

[0030] The precipitation nowcasting image generation method provided in this embodiment overcomes the shortcomings of single-frame input and single-frame output by using a multi-frame input and multi-frame output approach, which inputs m frames of images and outputs n frames of images. This prevents the prediction effect from decreasing sharply over time. In addition, because the image features obtained by downsampling have spatial features, and this invention combines the temporal features of n time points of the m frames of images to be predicted, it can better utilize the global feature information of the images to be predicted. This overcomes the shortcomings of existing multi-frame input and multi-frame output methods, which can only extract features locally in radar images and cannot effectively grasp the global feature information of radar images. Therefore, the precipitation nowcasting image generation method, system, electronic device, and storage medium provided by this invention have better precipitation nowcasting prediction effects.

[0031] In one embodiment, the execution of steps S101-S106 can also use the structural framework of the transformer model. By optimizing the transformer model, the output of precipitation near-term prediction images can be achieved. In this embodiment, a transformer model is first constructed, and then all image features and time location codes are encoded and integrated into an encoder, which replaces the encoder in the pre-constructed transformer model. The second and third encoded information are decoded to obtain n decoded images. The number of decoded images is the same as the number of time points, which are integrated into a decoder and replace the decoder in the pre-constructed transformer model.

[0032] In one embodiment, encoding all image features and the first encoding information includes:

[0033] The first extracted information is obtained by using a pre-built two-dimensional multi-head attention mechanism to extract attention from image features and temporal location codes.

[0034] The first extracted information, image features, and time location encoding are normalized to obtain the first normalized data;

[0035] The first normalized data is spatiotemporally correlated in a pre-constructed local spatiotemporal correlation network to obtain the temporal and spatial correlation information between different images to be predicted.

[0036] The temporal and spatial correlation information is normalized with the first normalized data to obtain the second encoded information.

[0037] In this embodiment, a two-dimensional multi-head attention mechanism, a local spatiotemporal correlation network, and two normalization modules can form an encoder. This encoder is an optimized version of the transformer model encoder, which aims to leverage the advantages of the transformer model to achieve multiple inputs.

[0038] In one embodiment, decoding the second and third encoded information to obtain n decoded images includes:

[0039] The third encoded information is extracted using a pre-built two-dimensional multi-head attention mechanism to obtain the second extracted information;

[0040] The second extracted information and the third encoded information are normalized to obtain the second normalized data;

[0041] A pre-built two-dimensional multi-head attention mechanism is used to extract attention from the second normalized data and the first encoded information to obtain the third extracted information;

[0042] The third extracted information and the second normalized data are normalized to obtain the third normalized data;

[0043] The third normalized data is spatiotemporally correlated in a pre-constructed local spatiotemporal correlation network to obtain the temporal and spatial correlation information between the predicted images.

[0044] The temporal and spatial correlation information between the predicted images and the third normalized data are normalized to obtain the n decoded images.

[0045] In this embodiment, two two-dimensional multi-head attention mechanisms, a local spatiotemporal correlation network, and three normalization modules can form a decoder. This decoder is a novel multi-output decoder designed to replace the decoder of the transformer model. The purpose is to utilize the transformer structure to achieve multiple outputs.

[0046] Because the transformer module possesses unique permutation invariance, this embodiment encodes n time points T = [m+1, m+2, ..., m+n] (where m is the length of the input image sequence and n is the length of the output image sequence). The resulting encoding is then input into the decoder, allowing for the generation of predicted images for all n time points from m+1 to m+n in a single pass. This method is simple, fast, and integrates well with the two improved designs mentioned above, resulting in a model with excellent prediction performance.

[0047] In one embodiment, the method for constructing a two-dimensional multi-head attention mechanism includes:

[0048] The multi-head attention mechanism of the transformer model is pre-built, and the query, key-value, and value item generation method of the multi-head attention mechanism is replaced by generating query, key-value, and value items using convolution;

[0049] By replacing the linear operations of attention weighting for queries, key-value pairs, and value items in the multi-head attention mechanism with convolutional operations, a two-dimensional multi-head attention mechanism is obtained.

[0050] In this embodiment, the traditional multi-head attention (MHA) mechanism in the transformer is modified. Specifically, the generation of Q (Query), K (Key), and V (Value) is changed by using convolution to obtain Q, K, and V. Furthermore, the linear operation of attention weighting for these three values ​​is replaced with convolution operations. Experiments show that this approach significantly improves prediction performance.

[0051] In one embodiment, the method for constructing a two-dimensional multi-head attention mechanism further includes: removing the prediction occlusion mechanism from the multi-head attention mechanism.

[0052] In this embodiment, the occlusion mechanism (mask) in the traditional multi-head attention mechanism of the transformer is used to predict occlusion. However, since this embodiment adds time points and predicts from the embedding of time points, since the occlusion prediction mechanism is not needed, removing the mask will result in better prediction performance.

[0053] In one embodiment, the method for constructing a local spatiotemporal correlation network includes:

[0054] The preceding network of the pre-built transformer model is replaced with a 3D convolutional neural network to obtain a local spatiotemporal correlation network, which is used to perform convolution operations in the temporal and spatial domains to preserve the spatial and temporal correlations between images.

[0055] In this embodiment, because the traditional transformer model's FFN (FeedForward Network) only has fully connected layers, resulting in the loss of many correlations, a three-dimensional convolutional neural network is used for model prediction. By performing convolution operations simultaneously in the temporal and spatial domains, the model can preserve both the spatial correlation between image sequences and the temporal correlation between different images. Experiments show that this stronger spatiotemporal correlation leads to better prediction results.

[0056] The execution of each step can form an improved transformer model, and its functional implementation process is as follows: Figure 2 As shown, after inputting m frames of images to be detected, the images are downsampled to obtain image features, and the images are then encoded temporally and positionally. The image features and temporally and positionally encoded images are input into the encoder for two-dimensional multi-head attention extraction and normalization. Subsequently, they are input into the local spatiotemporal correlation module for further normalization, and M image encoding information are output to the decoder. Then, after encoding n time points T = [m+1, m+2, ..., m+n], the time point encodings are input into the decoder. The decoder performs two-dimensional multi-head attention extraction on the time point encodings and normalizes them. The normalized data and the M image encoding information are then subjected to two-dimensional multi-head attention extraction and normalization. Subsequently, the normalized data is input into the local spatiotemporal correlation module for further normalization. The decoder outputs N decoded data, which are then upsampled to output n frames of precipitation near-term prediction images.

[0057] Please see Figure 3This application also provides a precipitation nowcasting image generation system, including: an image acquisition module 1, an image processing module 2, an encoder module 3, a time point encoding module 4, a decoder module 5, and an upsampling module 6; the image acquisition module 1 is used to acquire m frames of images to be predicted for precipitation nowcasting; the image processing module 2 is used to downsample all images to be predicted to obtain image features, and to perform time and position encoding on all images to be predicted to obtain first encoding information; the encoder module 3 is used to encode all image features and the first encoding information to obtain second encoding information; the time point encoding module 4 is used to encode n time points T = [m+1, m+2, ..., m+n] of the m frames of images to be predicted to obtain third encoding information; the decoder module 5 is used to decode the second encoding information and the third encoding information to obtain n decoded images, the number of decoded images being the same as the number of time points; the upsampling module 6 is used to upsample the decoded n images to obtain n frames of precipitation nowcasting images.

[0058] The precipitation nowcasting image generation system provided in this embodiment overcomes the shortcomings of single-frame input and single-frame output by using a multi-frame input and multi-frame output method, which inputs m frames of images and outputs n frames of images. This prevents the prediction effect from decreasing sharply over time. In addition, because the image features obtained by downsampling have spatial features, and this invention combines the temporal features of n time points of the m frames of images to be predicted, it can better utilize the global feature information of the images to be predicted. This overcomes the shortcomings of existing multi-frame input and multi-frame output methods, which can only extract features locally in radar images and cannot effectively grasp the global feature information of radar images. Therefore, the precipitation nowcasting image generation method, system, electronic device, and storage medium provided by this invention have better precipitation nowcasting prediction effects.

[0059] In one embodiment, the encoder module 3 includes: a first two-dimensional multi-head attention unit, a first normalization unit, a first local spatiotemporal correlation unit, and a second normalization unit; the first two-dimensional multi-head attention unit is used to perform attention extraction on image features and temporal location codes using a pre-built two-dimensional multi-head attention mechanism to obtain first extracted information; the first normalization unit is used to normalize the first extracted information, image features, and temporal location codes to obtain first normalized data; the first local spatiotemporal correlation unit is used to perform spatiotemporal correlation association on the first normalized data in a pre-built local spatiotemporal correlation network to obtain temporal correlation information and spatial correlation information between different images to be predicted; the second normalization unit is used to normalize the temporal correlation information and spatial correlation information with the first normalized data to obtain second encoded information.

[0060] In one embodiment, the decoder module 5 includes: a second two-dimensional multi-head attention unit, a third normalization unit, a fourth normalization unit, a second local spatiotemporal correlation unit, and a fifth normalization unit; the second two-dimensional multi-head attention unit is used to extract attention from the third encoded information using a pre-built two-dimensional multi-head attention mechanism to obtain second extracted information; the third normalization unit is used to normalize the second extracted information and the third encoded information to obtain second normalized data; the third two-dimensional multi-head attention unit is used to extract attention from the second normalized data and the first encoded information using a pre-built two-dimensional multi-head attention mechanism to obtain third extracted information; the fourth normalization unit is used to normalize the third extracted information and the second normalized data to obtain third normalized data; the second local spatiotemporal correlation unit is used to perform spatiotemporal correlation association of the third normalized data in a pre-built local spatiotemporal correlation network to obtain temporal correlation information and spatial correlation information between predicted images; the fifth normalization unit is used to normalize the temporal correlation information and spatial correlation information between predicted images and the third normalized data to obtain n decoded images.

[0061] In one embodiment, the precipitation near-term prediction image generation system further includes a construction module for constructing the two-dimensional multi-head attention unit in the above embodiment. The construction module includes: a multi-head attention mechanism construction unit for pre-constructing the multi-head attention mechanism of the transformer model; and a replacement unit for replacing the generation method of the query, key-value, and value items of the multi-head attention mechanism with the generation of query, key-value, and value items using convolution, and replacing the linear operation of attention weighting of the query, key-value, and value items of the multi-head attention mechanism with convolution operation to obtain the two-dimensional multi-head attention mechanism unit.

[0062] In one embodiment, the building module further includes a deletion unit for deleting the predictive occlusion mechanism in the multi-head attention mechanism.

[0063] In one embodiment, the building module further includes a local spatiotemporal correlation network building unit, which is used to pre-build the preceding network of the transformer model, replace the preceding network with a three-dimensional convolutional neural network to obtain a local spatiotemporal correlation network, which is used to perform convolution operations in the temporal and spatial domains to preserve the spatial and temporal correlations between images.

[0064] In one embodiment, the precipitation nowcasting image generation system further includes a replacement module for replacing the encoder module with the encoder in a pre-built transformer model and replacing the decoder module in the pre-built transformer model.

[0065] This application provides an electronic device; please refer to [link / reference]. Figure 4 The electronic device includes a memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor 602. When the processor 602 executes the computer program, it implements the precipitation now-prediction image generation method described above.

[0066] Furthermore, the electronic device also includes at least one input device 603 and at least one output device 604.

[0067] The aforementioned memory 601, processor 602, input device 603, and output device 604 are connected via bus 605.

[0068] The input device 603 can specifically be a camera, touch panel, physical buttons, or mouse, etc. The output device 604 can specifically be a display screen.

[0069] The memory 601 can be a high-speed random access memory (RAM) or a non-volatile memory, such as a disk storage device. The memory 601 is used to store a set of executable program code, and the processor 602 is coupled to the memory 601.

[0070] Furthermore, this application embodiment also provides a computer-readable storage medium, which may be disposed in the electronic device in the above embodiments, and the computer-readable storage medium may be the aforementioned memory 601. The computer-readable storage medium stores a computer program, which, when executed by the processor 602, implements the precipitation nowcasting image generation method described in the foregoing embodiments.

[0071] Furthermore, the storage medium of the computer can also be a USB flash drive, a portable hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk, or any other medium that can store program code.

[0072] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0073] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0074] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0075] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0076] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0077] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0078] The above is a description of a method, system, electronic device, and storage medium for generating near-term precipitation prediction images provided by the present invention. For those skilled in the art, based on the ideas of the embodiments of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for generating near-term precipitation prediction images, characterized in that, include: To obtain m frames of images of precipitation near-term forecasting; All images to be predicted are downsampled to obtain image features, and all images to be predicted are time-position encoded to obtain the first encoding information; Encode all image features and the first encoding information to obtain the second encoding information; The time points T=[m+1,m+2,…m+n] of the m-frame image to be predicted are encoded to obtain the third encoded information; The second encoded information and the third encoded information are decoded to obtain n decoded images, wherein the number of decoded images is the same as the number of time points; Upsample the decoded n images to obtain n frames of near-term precipitation prediction images; The step of encoding all image features and the first encoding information includes: The image features and the temporal location encoding are subjected to attention extraction using a pre-built two-dimensional multi-head attention mechanism to obtain the first extracted information; The first extracted information and the image features and time location encoding are normalized to obtain the first normalized data; The first normalized data is spatiotemporally correlated in a pre-constructed local spatiotemporal correlation network to obtain the temporal correlation information and spatial correlation information between different images to be predicted. The time correlation information and the spatial correlation information are normalized with the first normalized data to obtain the second coded information; The step of decoding the second encoded information and the third encoded information to obtain n decoded images includes: The third encoded information is extracted using a pre-built two-dimensional multi-head attention mechanism to obtain the second extracted information; The second extracted information and the third encoded information are normalized to obtain the second normalized data; The second normalized data and the first encoded information are subjected to attention extraction using a pre-built two-dimensional multi-head attention mechanism to obtain the third extracted information; The third extracted information and the second normalized data are normalized to obtain the third normalized data; The third normalized data is spatiotemporally correlated in a pre-constructed local spatiotemporal correlation network to obtain temporal and spatial correlation information between predicted images. The temporal and spatial correlation information between the predicted images and the third normalized data are normalized to obtain the n decoded images.

2. The precipitation nowcasting image generation method according to claim 1, characterized in that, The method for constructing the two-dimensional multi-head attention mechanism includes: A multi-head attention mechanism for the transformer model is pre-built, and the generation method of the query, key-value, and value items in the multi-head attention mechanism is replaced by generating the query, key-value, and value items using convolution; The linear operation of attention weighting for query, key, and value items in the multi-head attention mechanism is replaced with a convolution operation to obtain a two-dimensional multi-head attention mechanism.

3. The precipitation nowcasting image generation method according to claim 2, characterized in that, The method for constructing the two-dimensional multi-head attention mechanism also includes: Remove the predictive occlusion mechanism from the multi-head attention mechanism.

4. The method for generating precipitation nowcasting images according to claim 1, characterized in that, The method for constructing the local spatiotemporal correlation network includes: A preceding network of a transformer model is pre-built, and then replaced with a three-dimensional convolutional neural network to obtain a local spatiotemporal correlation network, which is used to perform convolution operations in the temporal and spatial domains to preserve the spatial and temporal correlations between images.

5. The method for generating precipitation nowcasting images according to claim 1, characterized in that, The method further includes: The encoder integrates all image features and temporal location encodings into a single encoder, replacing the encoder in the pre-built transformer model. The second and third encoded information are decoded to obtain n decoded images. The number of decoded images is the same as the number of time points, and they are integrated into a decoder to replace the decoder in the pre-built transformer model.

6. A precipitation nowcasting image generation system, characterized in that, include: The image acquisition module is used to acquire m frames of images to be predicted for the near-term precipitation forecast. The image processing module is used to downsample all images to be predicted to obtain image features, and to perform temporal and positional encoding on all images to be predicted to obtain first encoded information. An encoder module is used to encode all image features and first encoded information to obtain second encoded information. The encoding of all image features and first encoded information includes: using a pre-built two-dimensional multi-head attention mechanism to extract attention from the image features and the temporal location encoding to obtain first extracted information; normalizing the first extracted information, the image features, and the temporal location encoding to obtain first normalized data; associating the first normalized data with spatiotemporal correlation in a pre-built local spatiotemporal correlation network to obtain temporal correlation information and spatial correlation information between different images to be predicted; and normalizing the temporal correlation information and the spatial correlation information with the first normalized data to obtain second encoded information. The temporal encoding module is used to encode n time points T=[m+1,m+2,…m+n] of the m-frame image to be predicted, obtaining third encoded information. Decoding the second and third encoded information to obtain n decoded images includes: using a pre-built two-dimensional multi-head attention mechanism to extract attention from the third encoded information, obtaining second extracted information; normalizing the second extracted information and the third encoded information to obtain second normalized data; using the pre-built two-dimensional multi-head attention mechanism to extract attention from the second normalized data and the first encoded information, obtaining third extracted information; normalizing the third extracted information and the second normalized data to obtain third normalized data; associating the third normalized data with spatiotemporal correlation in a pre-built local spatiotemporal correlation network to obtain temporal and spatial correlation information between the predicted images; and normalizing the temporal and spatial correlation information between the predicted images and the third normalized data to obtain the n decoded images. The decoder module is used to decode the second encoded information and the third encoded information to obtain n decoded images, wherein the number of decoded images is the same as the number of time points; The upsampling module is used to upsample the decoded n images to obtain n frames of precipitation near-term prediction images.

7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 5.

Images