A weather radar puzzle missing echo repair method and system based on deep learning

Abstract

The application belongs to the technical field of weather radars, and discloses a weather radar puzzle missing echo repairing method and system based on deep learning, which comprises the following steps: acquiring weather radar puzzle data to construct a radar echo data set, selecting time-continuous data after preprocessing to construct a complete data set, randomly setting a missing area to construct a loss data set, and dividing training / validation / testing sets; combining Bi-ConvLSTM and U-Net to construct a sequence repairing model and training and optimizing; combining model output and observation results to generate a mask Mask based on missing discrimination and superimposed fusion to form a final missing echo repairing product. The application effectively repairs missing radar echoes and further improves the completeness and continuity of weather radar puzzle data.

Description

Technical Field

[0001] This invention belongs to the field of weather radar technology, and particularly relates to weather radar data quality control, specifically to a method and system for repairing missing echoes in weather radar mosaics based on deep learning. Background Technology

[0002] Weather radar is an advanced meteorological detection device that plays a vital role in many key areas, including severe convective weather early warning and forecasting, disaster prevention and mitigation, aviation safety, and weather modification operations, thanks to its superior technical performance. However, in actual radar operation, data loss in the scanned area is a common problem, severely limiting the quantitative application of its data. Factors contributing to data loss include beam obstruction (terrain, buildings, trees, or other objects blocking the radar beam, creating wedge-shaped blind spots behind objects), wind farm echo interference, and radar hardware malfunctions.

[0003] Furthermore, the detection range of a single radar is limited. Both the physical limitations of equipment performance and the objective constraints imposed by the Earth's curvature and beam attenuation dictate that its effective radius is typically only a few hundred kilometers. When faced with frontal precipitation or mesoscale convective systems stretching over thousands of miles, a single radar can often only capture local fragments, making it difficult to present a complete weather process. Radar mosaic data, on the other hand, stitches together observation data from multiple weather radars within a region of interest at the same time point, forming radar echo data covering the entire area. It is one of the most frequently used and crucial types of data in weather forecasting operations. With the help of radar mosaic data, meteorological personnel can track weather changes and disaster situations in real time, comprehensively analyze and judge weather trends, clarify weather changes or potential disaster situations, accurately analyze potential severe weather, and issue early warnings, thereby effectively reducing the probability of disasters and mitigating the severity of disaster losses. Therefore, radar mosaic data has become a crucial basis for improving the accuracy of severe weather defense and early warning dissemination.

[0004] However, the data gaps inherent in individual radars act like "genetic defects," directly inherited by the radar mosaic products. In radar junctions or overlapping areas, these defects amplify into continuous data voids, creating false blind spots or echo break zones, severely disrupting the continuity of radar observations. Furthermore, if a radar suddenly malfunctions, its coverage area instantly forms large "windows," exacerbating the data gaps. This is undoubtedly a fatal weakness for disasters like torrential rains and hailstorms that require seamless monitoring.

[0005] To address this issue, researchers have actively explored various methods. Early weather radar data restoration primarily relied on traditional methods, such as interpolation, terrain correction, and vertical reflectivity profile (VPR) to complete missing data. While these methods are simple in principle and computationally efficient, they generally lack the ability to capture the high-dimensional semantic features of radar data and are prone to oversmoothing in complex missing scenarios. In recent years, the field of artificial intelligence (AI) has developed rapidly. AI technologies, represented by deep learning, have become increasingly active in various scenarios for radar data quality control due to their outstanding ability to solve complex problems and nonlinear modeling. Convolutional neural networks (CNNs), generative adversarial networks (GANs), and diffusion models have shown significant advantages in restoration accuracy and efficiency. However, deep learning methods are prone to oversmoothing in data restoration tasks, making restoration traces easily identifiable visually, and their completion capabilities and accuracy are limited by the scenario. Therefore, further research and improvement of radar echo data missing restoration methods still have significant scientific and application value.

[0006] Currently, a number of studies on the repair of missing weather radar echo data have been conducted both domestically and internationally. However, these studies mostly focus on typical missing scenarios such as beam obstruction, or on repair based on multi-elevation radar data, achieving some success in specific application scenarios. However, the causes of missing radar echoes are highly diverse, and the generalization ability of existing methods still needs systematic evaluation. Meanwhile, research on the problem of missing echoes during radar mosaicking is relatively scarce. Moreover, while deep learning methods have strong modeling capabilities, most rely on known missing masks, making it difficult to adapt to the complex and diverse missing patterns in actual operational applications. Therefore, achieving robust repair of missing radar echoes without relying on explicit missing priors remains an urgent problem to be solved. To address this, for cases of large-area echo missingness in radar mosaicking data caused by radar hardware failures, data delays or loss, or mosaicking program errors, this paper utilizes the powerful feature extraction capabilities of deep learning to construct an intelligent model that can effectively repair missing echo data in weather radar mosaicking. This model does not use missing masks during training and inference stages, but learns the changing patterns of radar echo sequences to complete the data repair of missing areas. This provides an effective technical means for the high-quality application of weather radar mosaic data. Summary of the Invention

[0007] To address the frequent issue of missing echo data in existing weather radar mosaics, this invention provides a deep learning-based method and system for repairing missing echoes in weather radar mosaics. This method does not use a missing mask during training and inference phases, but instead repairs missing areas by reconstructing the radar echo sequence based on its spatiotemporal variation patterns, thereby further improving the completeness and continuity of weather radar mosaic data.

[0008] To achieve the above objectives, the present invention provides the following solution: A deep learning-based method for repairing missing echoes in weather radar mosaics, the method comprising: S1: Obtain weather radar mosaic data, extract radar composite reflectivity factor data of the area of ​​interest from the weather radar mosaic data, and form a radar echo dataset; S2: Preprocess the radar echo dataset and select multiple radar echo data that are consecutive in time as a group of valid radar data to construct a complete dataset; S3: Based on the complete dataset, construct the corresponding loss dataset by randomly setting the missing radar echo regions; and divide the complete dataset and the loss dataset into training dataset, validation dataset and test dataset as needed. S4: Combining the bidirectional convolutional long short-term memory network Bi-ConvLSTM and the U-shaped convolutional neural network U-Net, a radar missing echo repair model Bi-ConvLSTM-UNet is constructed with continuous time series weather radar mosaic data as input and output; S5: Based on the training and validation datasets, train and evaluate the radar missing echo repair model, and obtain the best repair model through hyperparameter optimization; S6: Combine the output of the optimal repair model with the radar echo observation results to generate a radar echo mask based on the mask construction rule based on missing discrimination; S7: Overlay the radar echo mask onto the output of the optimal repair model and the radar echo observation results to obtain the final weather radar mosaic missing echo repair product.

[0009] Preferred methods for preprocessing radar echo datasets include: The radar echo data is normalized, and radar echo data with a precipitation echo ratio lower than a preset threshold are removed. Radar echo values ​​with a precipitation echo area smaller than the threshold are set to 0.

[0010] Preferably, methods for constructing the corresponding loss dataset by randomly setting missing radar echo regions include: For each group of valid radar data in the complete dataset, 1 to M radar echo data are randomly selected for loss processing, that is: the location and size of the loss area are randomly determined, and the radar echo value within the loss area is set to 0; wherein, the location and size of each loss area in each group of valid radar data are kept consistent, and a loss dataset is constructed.

[0011] The preferred radar missing echo repair model Bi-ConvLSTM-UNet, which uses continuous time series weather radar mosaic data as input and output, uses UNet as the overall framework and includes three parts: the coding layer, the bottleneck layer, and the decoding layer. The coding layer consists of three cascaded IR-Block modules, which are used to extract the spatial structure features of radar echoes at different scales step by step. The bottleneck layer consists of IR-Block and Bi-ConvLSTM. IR-Block is used to further regularize and enhance the high-level features from the last level of the coding layer, while Bi-ConvLSTM is used to model the bidirectional temporal dependency of the feature sequence, thereby characterizing the dynamic evolution of radar echoes in the time dimension. The decoding layer consists of alternating PS-Block and IR-Block. PS-Block is used as the upsampling unit, and PixelShuffle is used to restore spatial resolution. Depthwise separable convolution is combined to reshape the upsampled features at the spatial and channel levels. Subsequently, the feature representations of the corresponding encoding and decoding layers are fused through skip connections, and the high-level semantic information and shallow spatial details are further integrated through IR-Block. Finally, the P_C2D module completes the channel mapping and non-linear activation to output the repair result.

[0012] Preferred methods for training and evaluating radar missing echo repair models based on training and validation datasets include: The radar missing echo repair model was trained using a partitioned training dataset. The trained radar missing echo repair model was evaluated using a partitioned validation dataset. The evaluation metrics were peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). The performance of the model was evaluated by calculating the PSNR and SSIM values ​​of the model repair results and the actual observation results.

[0013] Preferably, the method for generating a radar echo mask includes: A mask construction rule based on missing data discrimination is adopted, namely: the radar echo repair result output by the obtained best repair model and the input radar echo observation result are subjected to loss discrimination. If the radar echo observation result corresponding to a certain grid point is 0 but the radar echo repair result is not 0, the grid point is marked as 0; otherwise, it is marked as 1, thus obtaining the radar echo mask.

[0014] Preferably, the method for overlaying a radar echo mask onto the output of the optimal restoration model and the radar echo observation results to obtain the final weather radar mosaic missing echo restoration product includes: The generated radar echo mask is superimposed on the radar echo repair result and radar echo observation result output by the best repair model. A mask grid point marked as 1 indicates a valid observation, and the radar echo value is based on the radar echo observation result; a mask grid point marked as 0 indicates a missing observation, and the radar echo value is based on the radar echo repair result; finally, the missing echo repair product of the weather radar mosaic is obtained.

[0015] The present invention also provides a weather radar mosaic missing echo repair system based on deep learning. The system is used to implement the aforementioned method and includes: an acquisition module, a preprocessing module, a partitioning module, a construction module, a training module, a discrimination module, and an overlay module. The acquisition module is used to acquire weather radar mosaic data, extract radar composite reflectivity factor data of the area of ​​interest from the weather radar mosaic data, and form a radar echo dataset. The preprocessing module is used to preprocess the radar echo dataset and select multiple radar echo data that are consecutive in time as a group of valid radar data to construct a complete dataset. The partitioning module is used to construct a corresponding loss dataset based on the complete dataset by randomly setting radar echo missing regions; and to jointly partition the complete dataset and the loss dataset into training dataset, validation dataset and test dataset as needed. The construction module is used to combine the bidirectional convolutional long short-term memory network Bi-ConvLSTM and the U-shaped convolutional neural network U-Net to construct a radar missing echo repair model Bi-ConvLSTM-UNet with continuous time series weather radar mosaic data as input and output. The training module is used to train and evaluate the radar missing echo repair model based on the training dataset and the validation dataset, and obtain the best repair model through hyperparameter optimization. The discrimination module is used to combine the output of the best repair model with the radar echo observation results, and generate a radar echo mask based on the mask construction rules based on the missing discrimination. The overlay module is used to overlay the radar echo mask onto the output of the optimal repair model and the radar echo observation results to obtain the final weather radar mosaic missing echo repair product.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention focuses on the issue of missing echo portions in weather radar mosaic data caused by radar hardware failures, data arrival delays, and mosaic program errors. It employs deep learning methods to learn the changing patterns of radar echo sequences through a model, thereby repairing the missing data areas. This invention can successfully repair large-scale echoes within missing areas, and the echo intensity and variation patterns closely match the actual situation. Simultaneously, it ensures that non-missing areas are not significantly affected during the reconstruction process, providing an effective technical means for the high-quality application of weather radar mosaic data. Attached Figure Description

[0017] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for repairing missing echoes in weather radar mosaics based on deep learning, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a weather radar mosaic missing echo repair model based on deep learning according to an embodiment of the present invention; wherein, (A) is a schematic diagram of the overall structure of the Bi-ConvLSTM-UNet model; (B) is a schematic diagram of the structure of the IR-Block, PS-Block and P_C2D modules; and (C) is a schematic diagram of the Bi-ConvLSTM structure. Figure 3 The images show a comparison between weather radar mosaic data with partial echo loss during a precipitation event according to an embodiment of the present invention, radar echoes repaired by the method of the present invention, and actual observations; wherein, (a) is a schematic diagram of radar echo observation during a precipitation event; (b) is a schematic diagram of radar echoes with partial echo loss; and (c) is a schematic diagram of radar echoes repaired by the method of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] Example 1 like Figure 1 As shown, this invention provides a method for repairing missing echoes in weather radar mosaics based on deep learning, including the following steps: S1: Obtain weather radar network mosaic product data, extract radar composite reflectivity factor (CR) products for the area of ​​interest, and obtain radar echo data; S2: Preprocess the radar echo data obtained in step S1, normalize the radar echo data, remove radar echo data with a low proportion of precipitation echoes, set the radar echo value of precipitation echo area less than the threshold to 0, and select N (N≥10) radar echo data with continuous time as a group of valid radar data, thereby constructing a complete dataset. S3: Based on the complete dataset obtained in step S2, construct the corresponding loss dataset by randomly setting the missing radar echo regions; and divide the complete dataset and the loss dataset into training dataset, validation dataset and test dataset as needed. S4: Combining Bi-ConvLSTM and U-Net, a bi-directional temporal modeling mechanism is introduced at the bottleneck layer under the encoder-decoder framework to construct a radar missing echo repair model Bi-ConvLSTM-UNet that takes continuous time series weather radar mosaic data as input and outputs the corresponding time series radar echo reconstruction results. S5: Based on the training and validation datasets obtained in step S3, train and evaluate the radar missing echo repair model, and obtain the best repair model through hyperparameter optimization. S6: Combine the output of the best repair model obtained in step S5 with the radar echo observation results to generate a radar echo mask using a mask construction rule based on missing discrimination. S7: Overlay the radar echo mask generated in step S6 onto the output of the optimal repair model and the radar echo observation results to obtain the final weather radar mosaic missing echo repair product.

[0022] Furthermore, in the aforementioned deep learning-based weather radar mosaic missing echo repair method for improving the completeness and continuity of weather radar mosaic data, step S1, the method for obtaining radar echo data is as follows: S11: Obtain weather radar network mosaic product data, extract radar combined reflectivity factor product data within the area of ​​interest (specified latitude and longitude range) to form radar echo data.

[0023] Furthermore, in the aforementioned deep learning-based weather radar mosaic missing echo repair method for improving the completeness and continuity of weather radar mosaic data, step S2 involves the following preprocessing method for the radar echo data: S21: Acquire radar echo data, focusing only on radar echoes that produce significant precipitation (value range 10~75dBZ); set echo data below 10dBZ to 0dBZ, set echo data above 75dBZ to 75dBZ, and normalize the radar echo data by maximum and minimum values ​​to form radar echo data with a value range of [0, 1]. S22: For the radar echo data obtained in S21, count the number of grid points of precipitation echo (radar echo value greater than 0), and remove radar echo data whose proportion (number of precipitation echo grid points / total number of radar echo data grid points) is less than the threshold. S23: For the radar echo data obtained in S22, identify precipitation echo connectivity areas and set the radar echo values ​​of areas with precipitation echo area (number of grid points in precipitation echo connectivity areas) less than a threshold to 0. S24: Set the number of valid sequences N, and select N radar echo data that are consecutive in time as a group of valid radar data, thereby constructing a complete dataset. This invention adopts a data construction method based on a fixed-length time window. Specifically, a time window that can accommodate 15 frames of radar echoes is set, and continuous radar echo sequences with a time interval of 6 minutes are sequentially input into this window. When the window accumulates 15 consecutive frames of data, the 15-frame sequence is extracted as a complete sample and added to the dataset. Then, the window is cleared and a new 15-frame continuous sequence is started. During the data reading process, if it is detected that the time interval between adjacent frames does not meet the 6-minute continuity condition, and the number of frames in the current window is less than 15, the sequence is considered an invalid sample, the window is directly cleared, and the currently detected frame causing the time discontinuity is used as the new starting frame, and the above sample construction process is repeated. If the last group has less than 15 frames after data processing, it is removed.

[0024] Furthermore, in the aforementioned deep learning-based weather radar mosaic missing echo repair method for improving the completeness and continuity of weather radar mosaic data, step S3, the method for obtaining the training dataset, validation dataset, and test dataset is as follows: S31: For each group of valid radar data in the complete dataset obtained in step S2, randomly select 1 to M radar echo data for loss processing, that is: randomly determine the location and size of the loss area, and set the radar echo value within the loss area to 0; wherein, the location and size of each loss area in each group of valid radar data are consistent, thereby constructing a loss dataset; S32: Match the complete dataset obtained in step S2 with the loss dataset obtained in step S31, and divide them into training dataset, validation dataset and test dataset according to a certain ratio.

[0025] Furthermore, in the aforementioned deep learning-based weather radar mosaic missing echo repair method for improving the completeness and continuity of weather radar mosaic data, step S4, the method for constructing the deep learning-based weather radar mosaic missing echo repair model, is as follows: S41: A weather radar mosaic missing echo repair model, Bi-ConvLSTM-UNet, is constructed by combining the bidirectional convolutional long short-term memory network Bi-ConvLSTM and the U-shaped convolutional neural network U-Net. This model takes continuous temporal radar echo data as input and outputs the corresponding temporal radar missing echo repair results. The model uses the encoder-decoder structure of U-Net to extract and reconstruct the spatial features of radar echoes hierarchically. Bi-ConvLSTM is introduced between the encoder and decoder layers to perform bidirectional temporal modeling of the high-level spatial feature sequence, jointly characterizing the correlation of radar echoes in both the temporal and spatial dimensions. Through this approach, end-to-end holistic modeling and repair of incomplete radar echo sequences is achieved without relying on prior information about the location of missing regions, providing a model foundation for obtaining spatiotemporally continuous radar echo repair results. Specifically, the radar missing echo repair model Bi-ConvLSTM-UNet, which uses continuous time-series weather radar mosaic data as input and output, uses U-Net as its overall framework and includes three parts: an encoder layer, a bottleneck layer, and a decoder layer. The coding layer consists of three cascaded IR-Block modules, which are used to extract the spatial structure features of radar echoes at different scales step by step. The bottleneck layer receives high-level feature representations from the final IR-Block of the coding layer and is composed of the IR-Block and the bidirectional convolutional long short-term memory network Bi-ConvLSTM. The IR-Block is used to further regularize and enhance the high-level features of the final IR-Block of the coding layer, and the Bi-ConvLSTM is used to perform bidirectional temporal modeling of the feature sequence. By fusing forward and backward temporal information, the dynamic evolution characteristics of radar echo in the time dimension are characterized. The decoding layer consists of alternating PS-Block and IR-Block modules, with a P_C2D module at the end of the decoding process. The PS-Block acts as an upsampling unit, restoring spatial resolution through PixelShuffle and reshaping the upsampled features at both the spatial and channel levels using depthwise separable convolution. Subsequently, feature maps of the same scale from the encoder and decoder are fused and concatenated along the channel dimension to obtain the fused feature representation. The IR-Block further integrates high-level semantic information and shallow spatial details. Finally, the P_C2D module completes the channel mapping and nonlinear activation of the decoded features, outputting the radar echo restoration result.

[0026] Furthermore, in the aforementioned deep learning-based weather radar mosaic missing echo repair method for improving the completeness and continuity of weather radar mosaic data, step S5 involves the following method for training, evaluating, and optimizing the radar echo repair model: S51: Train the radar missing echo repair model using the training dataset divided in step S3; S52: The radar missing echo restoration model trained in step S51 is evaluated using the validation dataset partitioned in step S3. The main evaluation metrics are Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). PSNR measures the "sharpness" of the overall numerical restoration; a higher value indicates a smaller pixel-level error between the restored result and the true echo, and higher numerical accuracy. SSIM is used to evaluate the structure, texture, and edge restoration effect of the radar echo (such as the shape and gradient of strong echo areas). The closer the SSIM is to 1, the more consistent the visual / physical structure of the restored result is with the true echo. The model's performance is evaluated by calculating the PSNR and SSIM values ​​of the model's restoration results and the actual observation results. S53: Adjust hyperparameters such as learning rate, batch size, and loss weight, and repeat the model training and evaluation steps S51 and S52. Finally, select the model with the highest evaluation metrics (PSNR and SSIM) as the best-fit model to ensure the accuracy and reliability of the model in practical applications.

[0027] Furthermore, in the aforementioned deep learning-based weather radar mosaic missing echo repair method for improving the completeness and continuity of weather radar mosaic data, step S6, the method for generating the radar echo mask, is as follows: S61: Adopt a mask construction rule based on missing data discrimination, that is: perform loss discrimination on the radar echo repair result output by the best repair model obtained in step S5 and the input radar echo observation result. If the radar echo observation result corresponding to a certain grid point is 0 but the radar echo repair result is not 0, mark the grid point as 0; otherwise, mark it as 1, thus obtaining the radar echo mask.

[0028] Furthermore, in the aforementioned deep learning-based weather radar mosaic missing echo repair method for improving the completeness and continuity of weather radar mosaic data, step S7, the method for obtaining the weather radar mosaic missing echo repair product is as follows: S71: The radar echo mask generated in step S6 is superimposed on the radar echo repair result and radar echo observation result output by the best repair model obtained in step S5. A grid point marked as 1 in the mask indicates a valid observation, and the radar echo value is based on the radar echo observation result; a grid point marked as 0 in the mask indicates a missing observation, and the radar echo value is based on the radar echo repair result; finally, the weather radar mosaic missing echo repair product is obtained.

[0029] Example 2 Based on the same concept as Embodiment 1, this invention provides an embodiment of a deep learning-based method for repairing missing echoes in weather radar mosaic data to improve the completeness and continuity of weather radar mosaic data: Step S1: Obtain the weather radar mosaic product data of the SWAN (Short-Time Nowcasting) system for severe weather, which is operated by a meteorological observatory in a certain province from March to September 2022-2023. Extract the radar composite reflectivity factor (CR) product for a certain city area to obtain radar echo data. Step S2: Preprocess the radar echo data obtained in Step S1 by normalizing the radar echo data, removing radar echo data with a low proportion of precipitation echo grid points, setting the radar echo value of precipitation echo area less than the threshold to 0, and selecting multiple radar echo data with continuous time as a group of valid radar data to construct a complete dataset. Step S3: Based on the complete dataset obtained in Step S2, construct the corresponding loss dataset by randomly setting the missing radar echo regions; and divide the complete dataset and the loss dataset into training dataset, validation dataset and test dataset as needed. Step S4: Combining Bi-ConvLSTM and U-Net, a bi-directional temporal modeling mechanism is introduced at the bottleneck layer under the encoder-decoder framework, and the upsampling strategy in the feature reconstruction process is optimized to construct a radar missing echo repair model Bi-ConvLSTM-UNet that takes continuous time series weather radar mosaic data as input and outputs the corresponding time series radar echo reconstruction results. Step S5: Based on the training dataset and validation dataset obtained in step S3, train and evaluate the radar missing echo repair model, and obtain the best repair model through hyperparameter optimization. Step S6: Combine the output of the optimal repair model obtained in step S5 with the radar echo observation results, and generate a radar echo mask using a mask construction rule based on missing discrimination. Step S7: Overlay the radar echo mask generated in step S6 onto the output of the optimal repair model and the radar echo observation results to obtain the final weather radar mosaic missing echo repair product.

[0030] Furthermore, in step S1, the specific steps for obtaining the grid data of the numerical model forecast product are as follows: S11: Read the weather radar mosaic product data of the SWAN (Short-Time Nowcasting System) for severe weather, which is operated by a meteorological observatory in a certain province from March to September 2022 to 2023. Extract the combined reflectivity factor product (CR) for a certain city area (28.72~32.72°N, 112.42~116.42°E) to form radar echo data with a resolution of 1km and 400×400 grid points.

[0031] Furthermore, in step S2, the specific steps for preprocessing the radar echo data are as follows: S21: Normalize the radar echo data according to the maximum and minimum values ​​within the range of 10~75dBZ to obtain radar echo data with a value range of [0, 1]. Before normalization, radar echo values ​​less than 10dBZ are set to 0dBZ, and radar echo values ​​greater than 75dBZ are set to 75dBZ. S22: For the radar echo data obtained in S21, calculate the proportion of the number of grid points of precipitation echo (radar echo value greater than 0) in the total number of radar echo grid points, and remove radar echo data with a proportion less than the threshold (take 5%). S23: For the radar echo data processed by S22, the precipitation echo connectivity area is identified using the "8" neighborhood method, and the radar echo value of the area (number of grid points in the precipitation echo connectivity area) is less than the threshold (20) is set to 0. S24: For the radar echo data processed in S23, select N (N=15) radar echo data points that are consecutive in time as a group of valid radar data, thereby constructing a complete dataset.

[0032] Furthermore, in step S3, the specific steps for obtaining the training dataset, validation dataset, and test dataset are as follows: S31: Based on each group of valid radar data in the complete dataset obtained in step S2, randomly select 1 to 3 radar echo data for loss processing, that is: randomly determine the location (top left, bottom left, top right, bottom right, and center) and size (100×100, 150×150, 200×200) of the loss area, and set the radar echo value within the loss area to 0; wherein, the location and size of each loss area in each group of valid radar data are consistent, thereby constructing a loss dataset; S32: Match the complete dataset obtained in S2 with the loss dataset obtained in S31, and jointly divide them into training dataset, validation dataset, and test dataset. To maintain the temporal continuity of the data, 866 valid radar data points from March to May 2022 are designated as the validation dataset, 1034 valid radar data points from June to September 2022 are designated as the test dataset, and 2141 valid radar data points from March to September 2023 are designated as the training dataset.

[0033] Furthermore, in step S4, the specific steps for constructing the deep learning-based radar missing echo repair model are as follows: S41: A Bi-ConvLSTM-UNet model for repairing missing echoes in weather radar mosaics is constructed by combining a bidirectional convolutional long short-term memory network (Bi-ConvLSTM) with a U-shaped convolutional neural network (U-Net). The model takes 15 past radar echo data as input and 15 repaired radar echo data as output (spatial resolution of 1km, temporal resolution of 6min).

[0034] Specifically, Bi-ConvLSTM-UNet uses U-Net as its overall framework and consists of three parts: the coding layer, the bottleneck layer, and the decoding layer (e.g., ...). Figure 2(As shown). The encoding layer consists of cascaded depthwise separable convolutional modules (IR-Blocks). The first layer does not perform downsampling to avoid unnecessary impact on the local spatial structure representation due to resolution changes in the early feature extraction stage, thus learning the local spatial features of the radar echo at the original spatial resolution. Subsequent IR-Blocks perform stepwise downsampling with a depthwise convolution stride of 2, reducing spatial resolution while expanding the effective receptive field and increasing the channel dimension, gradually constructing a hierarchical spatial feature representation, providing a stable spatial feature foundation for the overall sequence reconstruction. The bottleneck layer receives high-level feature representations from the last-level IR-Block of the encoding layer and further regularizes and enhances them through the IR-Block. On this basis, a bidirectional convolutional long short-term memory network Bi-ConvLSTM is introduced to perform bidirectional temporal modeling of the feature sequence. By concatenating the forward and backward outputs in the feature dimension, contextual information at different time scales is combined to characterize the dynamic evolution features of the radar echo across the entire sequence, enhancing the model's ability to model temporal continuity and evolution trends. The forward and backward feature update processes are respectively represented as follows: , ,in, for t Input feature map at any time, for t Hide your status at all times. , This is a temporal feature update function based on a convolutional gated recursive mechanism. In this implementation, bidirectional features can be fused through concatenation: In one optional implementation, the aforementioned temporal feature update function can be implemented by a convolutional long short-term memory network, but is not limited to this. The decoding layer uses a depthwise separable convolutional upsampling module (PS-Block) based on pixel rearrangement as the upsampling unit. First, it uses pixel rearrangement (PixelShuffle) to restore spatial resolution, and then combines depthwise separable convolution to reorganize the upsampled features at the spatial and channel levels. Subsequently, it fuses the feature representations of the corresponding encoding layer through skip connections, and further integrates high-level semantic information and shallow spatial details through IR-Block. The above process is repeated step by step to restore spatial resolution. Finally, the output normalization module (P_C2D) performs pointwise convolutional mapping on the decoded features, and restricts the output to [0, 1] through the ReLU1 activation function f(x)=min(max(0,x),1) to ensure the rationality of the reconstructed radar echo sequence in terms of numerical range and physical meaning. Without relying on missing location masks or explicitly missing prior information, this model directly uses incomplete radar echo time series as input. By jointly modeling the correlation of radar echoes in the time and space dimensions, it achieves end-to-end overall reconstruction of missing regions, providing effective modeling support for obtaining radar echo repair results that are coherent in spatiotemporal structure and reasonable in physical characteristics.

[0035] Furthermore, in step S5, the specific steps for training, evaluating, and optimizing the hyperparameters of the radar missing echo repair model to obtain the optimal repair model are as follows: S51: Use the training dataset divided in step S3 to train the radar echo repair model. The goal of model training is to minimize the error between the radar echo prediction results and the actual observations (including echo intensity, morphology, etc.). S52: The radar echo repair model trained in step S51 is evaluated using the validation dataset divided in step S3. The main evaluation metrics are peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

[0036] PSNR measures the "sharpness" of the overall numerical restoration. A higher value indicates a smaller pixel-level error between the restored result and the true echo, signifying higher numerical accuracy. The formula for calculating PSNR is: Where MAX is the maximum value of the radar echo (75dBZ in this case, which becomes 1 after normalization), and MSE is the mean square error.

[0037] SSIM is used to evaluate the structure, texture, and edge restoration of radar echoes (such as the shape and gradient of strong echo areas). The closer the SSIM is to 1, the more consistent the visual / physical structure of the restored result is with the real echo. The formula for calculating SSIM is: Where μ is the mean, σ 2 It is variance. It is the covariance, and C1 and C2 are constants to avoid the denominator being 0.

[0038] The performance of the model can be evaluated by calculating the PSNR and SSIM values ​​of the model repair results and the actual observation results. S53: Adjust hyperparameters such as learning rate, batch size, and loss weight, and repeat the model training and evaluation steps S51 and S52. Finally, select the model with the highest evaluation metrics (PSNR and SSIM) as the best-fit model to ensure the accuracy and reliability of the model in practical applications.

[0039] In this embodiment, the PSNR and SSIM values ​​of the optimal repair model both reached a high level, indicating that the model can effectively repair the missing radar echoes in the weather radar mosaic and has high practicality and reliability.

[0040] Furthermore, in step S6, the specific steps for obtaining the radar echo mask are as follows: S61: Adopt a mask construction rule based on missing data discrimination: Perform loss discrimination on the output of the best repair model obtained in step S5 and the input observation results. If the observation result corresponding to a certain grid point is 0 but the repair result is not 0, mark the grid point as 0; otherwise, mark it as 1, thus obtaining the radar echo mask.

[0041] Furthermore, in step S7, the specific steps for obtaining the weather radar mosaic missing echo repair product are as follows: S71: The radar echo mask generated in step S6 is superimposed on the output of the best repair model obtained in step S5 and the radar echo observation results. A mask grid point marked as 1 indicates a valid observation, and the radar echo value is based on the radar echo observation results; a mask grid point marked as 0 indicates a missing observation, and the radar echo value is based on the radar echo repair results; finally, the weather radar mosaic missing echo repair product is obtained.

[0042] In this embodiment, the deep learning model selected is Bi-ConvLSTM-UNet. This model uses UNet as its overall framework and includes three main parts: an encoding layer, a bottleneck layer, and a decoding layer. Figure 2As shown, it consists of an IR-Block module, a PS-Block module, a Bi-ConvLSTM module, and a P_C2D module. The IR-Block is used to efficiently extract local spatial structure features, the PS-Block is used to realize the spatial rearrangement and resolution restoration of features in the decoding stage, the Bi-ConvLSTM models the bidirectional temporal dependencies within the sequence, and the P_C2D is used to complete the channel mapping and nonlinear activation of the decoded features. The modules work together to realize the reconstruction and repair of the radar echo sequence.

[0043] To verify the application effect of the deep learning-based weather radar mosaic missing echo repair method, the actual effect of the method of the present invention is evaluated by using the radar echo observation results of a precipitation process (with partially missing radar echoes) and its repair results. Figure 3 This refers to radar echo observations of a single precipitation event. Figure 3 a) and the corresponding radar echoes with partial echo loss ( Figure 3 b) and the radar echo repaired by the method of the present invention ( Figure 3 c).

[0044] like Figure 3 As shown, this was a significant precipitation event. Compared to... Figure 3 Complete radar echo observation results in a, Figure 3 A large area of ​​radar echo was missing in the upper right of b. After repair using the method of this invention, a clear radar echo was successfully recovered from the missing area, and the shape and texture of the radar echo were very close to the actual observation, and the echo intensity was also basically consistent. At the same time, the method of this invention also showed good performance for radar echoes in the edge areas and isolated echoes in some local areas, and the overall repair effect was satisfactory.

[0045] Similarly, the performance of the proposed method and other model methods in radar echo restoration was further evaluated on the test dataset. Table 1 shows the evaluation results of the deep learning-based weather radar mosaic missing echo restoration model Bi-ConvLSTM-UNet and other model methods on the test dataset. As shown in Table 1, compared with ConvLSTM, DSA_Unet, and optical flow methods, the Bi-ConvLSTM-UNet model demonstrates significant advantages in both PSNR and SSIM metrics, indicating that it has better radar echo restoration accuracy and restoration degree. In addition, for radar echoes of different intensities (20dBZ, 30dBZ, 40dBZ), the Bi-ConvLSTM-UNet model also shows the best restoration performance, and its critical success index (CSI) is also the maximum value.

[0046] Table 1 This embodiment focuses on an actual precipitation event. Using weather radar mosaic data from a city area from March to September 2022-2023, and randomly selecting a loss dataset composed of missing radar echo regions, the deep learning-based weather radar mosaic missing echo repair model Bi-ConvLSTM-UNet was trained and optimized. The actual performance of the Bi-ConvLSTM-UNet model in repairing missing echoes was evaluated using real-world radar echo observations as a comparison. The preliminary conclusions are as follows: (1) Compared with the traditional interpolation method, the weather radar mosaic missing echo repair method based on the deep learning model Bi-ConvLSTM and UNet fully leverages the outstanding ability of deep learning technology to solve complex problems and nonlinear modeling. It not only has the characteristics of fast operation, flexible adjustment and high accuracy, but also shows better repair performance when radar echoes are missing in multiple consecutive time periods.

[0047] (2) The method for repairing missing echoes in weather radar mosaics based on deep learning can effectively solve the problem of missing radar echoes that often occurs in radar mosaic data business applications, further improve the accuracy and reliability of radar mosaic data, and can be applied to the post-processing of existing radar mosaic business systems, effectively improving the application level of radar mosaic data in meteorological services.

[0048] The above conclusions show that this invention utilizes the deep learning models Bi-ConvLSTM and UNet to construct a weather radar mosaic missing echo repair model Bi-ConvLSTM-UNet, and combines it with mask technology to achieve effective repair of weather radar mosaic missing echoes based on deep learning technology.

[0049] Example 3 The present invention also provides a weather radar mosaic missing echo repair system based on deep learning. The system is used to implement the method described in Embodiment 1. The system includes: an acquisition module, a preprocessing module, a partitioning module, a construction module, a training module, a discrimination module, and an overlay module. The acquisition module is used to acquire weather radar mosaic data, extract radar composite reflectivity factor data of the area of ​​interest from the weather radar mosaic data, and form a radar echo dataset. The preprocessing module is used to preprocess the radar echo dataset and select multiple radar echo data that are consecutive in time as a group of valid radar data to construct a complete dataset. The partitioning module is used to construct a corresponding loss dataset based on the complete dataset by randomly setting missing radar echo regions; and to jointly partition the complete dataset and the loss dataset into training dataset, validation dataset and test dataset as needed. The module is used to combine the bidirectional convolutional long short-term memory network Bi-ConvLSTM and the U-shaped convolutional neural network U-Net to build a radar missing echo repair model with continuous time series weather radar mosaic data as input and output. The training module is used to train and evaluate radar missing echo repair models based on training and validation datasets, and obtain the best repair model through hyperparameter optimization. The discrimination module is used to combine the output of the best repair model with the radar echo observation results, and generate a radar echo mask based on the mask construction rules based on missing discrimination. The overlay module is used to overlay the radar echo mask onto the output of the optimal repair model and the radar echo observation results to obtain the final weather radar mosaic missing echo repair product.

[0050] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for repairing missing echoes in weather radar mosaics based on deep learning, characterized in that, The method includes: S1: Obtain weather radar mosaic data, extract radar composite reflectivity factor data of the area of ​​interest from the weather radar mosaic data, and form a radar echo dataset; S2: Preprocess the radar echo dataset and select multiple radar echo data that are consecutive in time as a group of valid radar data to construct a complete dataset; S3: Based on the complete dataset, construct the corresponding loss dataset by randomly setting the missing radar echo regions; and divide the complete dataset and the loss dataset into training dataset, validation dataset and test dataset as needed. S4: Combining the bidirectional convolutional long short-term memory network Bi-ConvLSTM and the U-shaped convolutional neural network U-Net, a radar missing echo repair model Bi-ConvLSTM-UNet is constructed with continuous time series weather radar mosaic data as input and output; S5: Based on the training and validation datasets, train and evaluate the radar missing echo repair model, and obtain the best repair model through hyperparameter optimization; S6: Combine the output of the optimal repair model with the radar echo observation results to generate a radar echo mask based on the mask construction rule based on missing discrimination; S7: Overlay the radar echo mask onto the output of the optimal repair model and the radar echo observation results to obtain the final weather radar mosaic missing echo repair product.

2. The method according to claim 1, characterized in that, Methods for preprocessing radar echo datasets include: The radar echo data is normalized, and radar echo data with a precipitation echo ratio lower than a preset threshold are removed. Radar echo values ​​with a precipitation echo area smaller than the threshold are set to 0.

3. The method according to claim 1, characterized in that, Methods for constructing a corresponding loss dataset by randomly setting missing regions in radar echoes include: For each group of valid radar data in the complete dataset, 1 to M radar echo data are randomly selected for loss processing, that is: the location and size of the loss area are randomly determined, and the radar echo value within the loss area is set to 0; wherein, the location and size of each loss area in each group of valid radar data are kept consistent, and a loss dataset is constructed.

4. The method according to claim 1, characterized in that, The radar missing echo repair model Bi-ConvLSTM-UNet, which takes continuous time series weather radar mosaic data as input and output, uses UNet as the overall framework and includes three parts: the coding layer, the bottleneck layer, and the decoding layer. The coding layer consists of three cascaded IR-Block modules, which are used to extract the spatial structure features of radar echoes at different scales step by step. The bottleneck layer consists of IR-Block and Bi-ConvLSTM. IR-Block is used to further regularize and enhance the high-level features from the last level of the coding layer, while Bi-ConvLSTM is used to model the bidirectional temporal dependency of the feature sequence, thereby characterizing the dynamic evolution of radar echoes in the time dimension. The decoding layer consists of alternating PS-Block and IR-Block. PS-Block is used as the upsampling unit, and PixelShuffle is used to restore spatial resolution. Depthwise separable convolution is combined to reshape the upsampled features at the spatial and channel levels. Subsequently, the feature representations of the corresponding encoding and decoding layers are fused through skip connections, and the high-level semantic information and shallow spatial details are further integrated through IR-Block. Finally, the P_C2D module completes the channel mapping and non-linear activation to output the repair result.

5. The method according to claim 1, characterized in that, Methods for training and evaluating radar missing echo repair models based on training and validation datasets include: The radar missing echo repair model was trained using a partitioned training dataset. The trained radar missing echo repair model was evaluated using a partitioned validation dataset. The evaluation metrics were peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). The performance of the model was evaluated by calculating the PSNR and SSIM values ​​of the model repair results and the actual observation results.

6. The method according to claim 1, characterized in that, Methods for generating radar echo mask include: A mask construction rule based on missing data discrimination is adopted, namely: the radar echo repair result output by the obtained best repair model and the input radar echo observation result are subjected to loss discrimination. If the radar echo observation result corresponding to a certain grid point is 0 but the radar echo repair result is not 0, the grid point is marked as 0; otherwise, it is marked as 1, thus obtaining the radar echo mask.

7. The method according to claim 1, characterized in that, Methods for overlaying radar echo masks onto the output of the optimal restoration model and radar echo observations to obtain the final weather radar mosaic missing echo restoration product include: The generated radar echo mask is superimposed on the radar echo repair result and radar echo observation result output by the best repair model. A mask grid point marked as 1 indicates a valid observation, and the radar echo value is based on the radar echo observation result; a mask grid point marked as 0 indicates a missing observation, and the radar echo value is based on the radar echo repair result; finally, the missing echo repair product of the weather radar mosaic is obtained.

8. A deep learning-based weather radar mosaic missing echo repair system, the system being used to implement the method described in any one of claims 1-7, characterized in that, The system includes: an acquisition module, a preprocessing module, a partitioning module, a construction module, a training module, a discrimination module, and an overlay module; The acquisition module is used to acquire weather radar mosaic data, extract radar composite reflectivity factor data of the area of ​​interest from the weather radar mosaic data, and form a radar echo dataset. The preprocessing module is used to preprocess the radar echo dataset and select multiple radar echo data that are consecutive in time as a group of valid radar data to construct a complete dataset. The partitioning module is used to construct a corresponding loss dataset based on the complete dataset by randomly setting radar echo missing regions; and to jointly partition the complete dataset and the loss dataset into training dataset, validation dataset and test dataset as needed. The construction module is used to combine the bidirectional convolutional long short-term memory network Bi-ConvLSTM and the U-shaped convolutional neural network U-Net to construct a radar missing echo repair model Bi-ConvLSTM-UNet with continuous time series weather radar mosaic data as input and output. The training module is used to train and evaluate the radar missing echo repair model based on the training dataset and the validation dataset, and obtain the best repair model through hyperparameter optimization. The discrimination module is used to combine the output of the best repair model with the radar echo observation results, and generate a radar echo mask based on the mask construction rules based on the missing discrimination. The overlay module is used to overlay the radar echo mask onto the output of the optimal repair model and the radar echo observation results to obtain the final weather radar mosaic missing echo repair product.

Images