Method and device for removing clutter from Doppler radar data based on optical flow information

Abstract

The present invention provides a method and device for removing clutter from Doppler radar data based on optical flow information, including: acquiring radar data and extracting detail data from the radar data; performing preprocessing on the extracted Doppler radar image to obtain a radar picture; obtaining an LM-Flow model based on an improved RAFT model, and using the obtained LM-Flow model to analyze continuous radar pictures obtained after preprocessing to extract optical flow information between front and back images; using a motion threshold to filter the obtained optical flow information to generate a binary mask to separate ground clutter; removing ground clutter according to the binary mask, and using a bilinear interpolation method to repair the mistakenly deleted cloud information to obtain a radar data map with ground clutter removed. The present invention reduces the need for manual intervention and the risk of human error through the deployment of a deep learning method, and improves the accuracy of meteorological image processing by accurately processing the optical flow information of Doppler data.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a method and a device for removing clutter from Doppler radar data based on optical flow information. Background Art

[0002] Accurate estimation of precipitation at the grid-centric level is crucial for applications such as flood forecasting, water resource management, and environmental protection. Doppler radar, known for its high spatial and temporal resolution and large range, is a widely used and reliable tool for detecting precipitation patterns, providing instantaneous and continuous observations of precipitation dynamics. However, the effective use of Doppler radar data is often hampered by ground clutter, which can lead to unstable and inaccurate precipitation measurements, thus seriously affecting the accuracy of precipitation estimates.

[0003] Various methods and techniques have been used to overcome the challenges posed by ground clutter and improve the accuracy of precipitation estimation. Initially, notch filters were the preferred solution, which effectively suppress certain frequency components but have the potential to distort the desired signal. To address the shortcomings of notch filters, the Gaussian Model Adaptive Processing (GMAP) algorithm was introduced and showed efficacy in the spectral domain. However, GMAP operates primarily in the spectral domain and is susceptible to uncertainties associated with the choice of time windows, leading to spectral leakage issues.

[0004] In addition, these traditional ground clutter suppression methods rely on techniques such as signal processing, filtering, spectrum analysis, threshold processing, and feature engineering, based on existing knowledge and manually defined data processing rules, and are effective in specific usage scenarios. However, their effectiveness may be limited when applied to different data sets and environmental scenarios. Summary of the invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, an embodiment of the present invention provides a method for removing clutter from Doppler radar data based on optical flow information, aiming to accurately identify and remove ground clutter in radar data, and significantly improve the radar's detection accuracy and reliability of weather phenomena.

[0006] To achieve the above object, an embodiment of the present invention provides a method for removing clutter from Doppler radar data based on optical flow information, the method comprising:

[0007] Acquire radar data, and extract detail data from the radar data, wherein the detail data includes latitude and longitude, resolution, and Doppler radar image;

[0008] performing preprocessing on the extracted Doppler radar image to obtain a radar picture;

[0009] The LM-Flow model is improved based on the RAFT model, and the obtained LM-Flow model is used to analyze the continuous pre-processed radar images to extract the optical flow information between the front and rear images, and the optical flow information represents the movement direction and size of the pixel;

[0010] The optical flow information obtained by motion thresholding is used to generate a binary mask to separate ground clutter.

[0011] The ground clutter is removed according to the binary mask, and the cloud information that was mistakenly deleted is repaired by using a bilinear interpolation method to obtain a radar data map with the ground clutter removed.

[0012] Preferably, the step of performing preprocessing on the extracted Doppler image to obtain a radar picture comprises:

[0013] Normalize Doppler radar images;

[0014] Perform smoothing and filtering on the standardized radar image data;

[0015] Perform edge extraction on the radar image obtained by smoothing and filtering to obtain meteorological edge and structure information in the radar image;

[0016] According to the resolution of radar data, the Doppler image after edge extraction is segmented, the radar edges are filled with 0, the outliers are deleted, and the image is enhanced or weakened.

[0017] Preferably, the LM-Flow model improved based on the RAFT model includes:

[0018] Three-layer feature encoder, depthwise separable convolution, correlation layer model, and gated recurrent unit.

[0019] Preferably, the step of using the obtained LM-Flow model to analyze the continuous pre-processed radar images to extract the optical flow information between the front and rear images specifically includes:

[0020] The three-layer feature encoder receives the input of the two images and performs feature extraction, and extracts the context information of the first image as the initial value of the gated recurrent unit for iterative update of the optical flow;

[0021] The correlation layer model calculates the feature vectors of the features of the two images extracted before and after, and calculates the correlation volume based on the dot product of the feature vectors of the two consecutive images;

[0022] The gated recurrent unit receives the initial value of the iterative update of the optical flow, retrieves the correlation amount and the potential hidden state from the correlation volume, and calculates the updated optimized optical flow information and hidden state. The calculation formula is shown as follows:

[0023] Z t =σ(DSC([h t-1 ,x t ],W z ))

[0024] r t =σ(DSC([h t-1 ,x t ],W r ))

[0025]

[0026]

[0027] Among them, the update gate Z t and reset gate r t It is calculated by deep separation convolution and sigmoid activation function, and the candidate hidden state It is processed by the tanh activation function;

[0028] The final hidden state h t The previous state is updated by merging the new candidate state, and the final output is the optical flow information of the two pictures.

[0029] Preferably, the step of using the optical flow information obtained by motion threshold screening to generate a binary mask to separate ground clutter specifically includes:

[0030] Determine a motion threshold based on the absolute value of pixel motion;

[0031] Classifying pixels using the motion threshold, where pixels below the motion threshold are classified as stationary pixels;

[0032] A motion threshold-based binary mask is generated for the processed optical flow information. The size of the binary mask is the same as the original image, where the pixels of stationary elements on the ground are set to 0 and the pixels of moving clouds are set to 1.

[0033] Preferably, removing ground clutter according to the binary mask and repairing mistakenly deleted cloud information by using bilinear interpolation to obtain a radar data map with ground clutter removed specifically includes:

[0034] Apply the binary mask to the original image to obtain the Doppler radar data image with ground clutter removed;

[0035] Determine the difference between the current image and the previous and next images to determine whether there is a problem with the current image. Perform bilinear interpolation to repair the image using the previous and next images.

[0036] Loop through all images and iterate through them repeatedly to perform bilinear interpolation to repair the images until there are no more problematic images.

[0037] On the other hand, an embodiment of the present invention further provides a device for removing clutter from Doppler radar data based on optical flow information, the device specifically comprising:

[0038] An image acquisition module, used to acquire radar data and extract detail data from the radar data, wherein the detail data includes latitude and longitude, resolution and Doppler radar image;

[0039] A preprocessing module, used for performing preprocessing on the extracted Doppler radar image to obtain a radar picture;

[0040] An optical flow information acquisition module is used to obtain an LM-Flow model based on the RAFT model, and use the obtained LM-Flow model to analyze the continuous pre-processed radar images to extract the optical flow information between the front and rear images, where the optical flow information represents the movement direction and size of the pixel;

[0041] The clutter separation module is used to generate a binary mask to separate ground clutter by using the optical flow information obtained by motion threshold screening;

[0042] The correction module is used to remove ground clutter according to the binary mask and to repair the mistakenly deleted cloud information by using a bilinear interpolation method to obtain a radar data map with the ground clutter removed.

[0043] Preferably, the preprocessing module specifically includes:

[0044] A standardization processing unit, used for performing standardization processing on Doppler radar images;

[0045] A filtering unit, used for performing smoothing filtering on the standardized radar image data;

[0046] An edge extraction unit is used to extract the edge of the radar image obtained by the smoothing filter process to obtain the meteorological edge and structure information in the radar image;

[0047] The image addition and subtraction unit is used to segment the Doppler image after edge extraction according to the resolution of the radar data, fill the radar edge with 0, delete the abnormal value, and enhance or weaken the image.

[0048] Preferably, the LM-Flow model improved based on the RAFT model includes:

[0049] Three-layer feature encoder, depthwise separable convolution, correlation layer model, and gated recurrent unit.

[0050] Preferably,

[0051] The three-layer feature encoder is used to receive the input of the two images and perform feature extraction, and extract the context information of the first image as the initial value of the gated recurrent unit for iterative update of the optical flow;

[0052] The correlation layer model is used to calculate feature vectors of the features of the two consecutive images, and calculate the correlation volume according to the dot product of the feature vectors of the two consecutive images;

[0053] The gated recurrent unit is used to receive the initial value of the iterative update of the optical flow, retrieve the correlation amount and the potential hidden state from the correlation volume, and calculate the updated optimized optical flow information and hidden state. The calculation formula is shown in the following formula:

[0054] Z t =σ(DSC([h t-1 ,x t ],W z ))

[0055] r t =σ(DSC([h t-1 ,x t ],W r ))

[0056]

[0057] Among them, the update gate Z t and reset gate r t It is calculated by deep separation convolution and sigmoid activation function, and the candidate hidden state It is processed by the tanh activation function;

[0058] The final hidden state h t The previous state is updated by merging the new candidate state, and the final output is the optical flow information of the two pictures.

[0059] The embodiment of the present invention calculates optical flow through the LM-Flow model based on the RAFT model, and according to the obtained optical flow information, uses operations such as motion threshold screening, masking and bilinear interpolation to remove clutter in Doppler radar data. The deployment of the deep learning method in the embodiment of the present invention reduces the need for manual intervention, reduces the risk of human error, ensures high-quality results, and by accurately processing the optical flow information of Doppler data, not only the accuracy and efficiency of meteorological image processing are improved, but also the application value and practicality of meteorological data are greatly enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 An overall flow chart of a method for removing Doppler radar data clutter based on deep learning to obtain optical flow information provided by an embodiment of the present invention;

[0062] Figure 2 The overall framework and information flow diagram of the method for removing Doppler radar data clutter based on deep learning to obtain optical flow information provided by an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of the LM-Flow optical flow calculation model proposed in the method for removing Doppler radar data clutter based on deep learning to obtain optical flow information provided by an embodiment of the present invention;

[0064] Figure 4 An example of a data set used in training a deep learning optical flow estimation model in a method for removing Doppler radar data clutter based on deep learning to obtain optical flow information provided by an embodiment of the present invention;

[0065] Figure 5 A performance comparison chart of the deep learning optical flow information calculation model provided by an embodiment of the present invention and other models;

[0066] Figure 6 This is a diagram showing the effect of different optical flow images provided by an embodiment of the present invention on removing ground clutter from Doppler radar data;

[0067] Figure 7 A performance comparison chart of different deep learning optical flow models using different modules provided by an embodiment of the present invention;

[0068] Figure 8 A radar rainfall fitting performance diagram of the optical flow calculation model provided in an embodiment of the present invention after removing ground clutter from Doppler radar data.

[0069] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0070] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0071] Reference Figure 1The embodiment of the present invention provides a method for removing clutter from Doppler radar data based on an optical flow method, the method comprising the following steps:

[0072] S1, acquiring radar data, and extracting detail data from the radar data, wherein the detail data includes latitude and longitude, resolution and Doppler image;

[0073] S2, performing preprocessing on the extracted Doppler image to obtain a radar picture;

[0074] The radar image includes cloud images and will be images, that is, radar data is acquired and parsed, detailed image data information such as region, longitude and latitude, resolution, etc. is obtained from the data, and radar reflection grayscale cloud images with a time interval of 6 minutes in a specific area and quantitative rainfall estimation grayscale images with a time interval of 1 hour are obtained from the image data.

[0075] The preprocessing includes the following:

[0076] S21, standardization processing;

[0077] The embodiment of the present invention can make radar data from different regions have a consistent scale and range by performing standardization processing on Doppler radar images, which helps to eliminate data differences from different radar systems and improve the comparability and fusion of data.

[0078] This is done by scaling the data to a fixed range (usually [0,1]) by using the formula for each image:

[0079]

[0080] Here, x j is the original value, min(x j ) is the minimum value of the feature, and max(x j ) is the maximum value of the feature.

[0081] S22, performing smoothing and filtering processing on the standardized radar data;

[0082] Smoothing filtering can effectively reduce noise and detail changes in the data, making the radar image smoother and more stable. This not only helps to improve the quality and clarity of the radar image, but also reduces errors and interference in subsequent data processing steps.

[0083] In order to perform Gaussian filtering on the standardized radar data, it is first necessary to determine the parameters of the filter, including the window size and standard deviation. The window size is usually chosen to be an odd number, such as 3x3, 5x5, or 7x7, to ensure that there is a center point. The standard deviation (σ) determines the width of the Gaussian function, which affects the strength of the smoothing effect.

[0084] Next, a Gaussian kernel is constructed, which is a weight matrix defined by a Gaussian function, where the center point has the highest weight and the weight decreases as the distance from the center point increases. This Gaussian kernel is then applied to each pixel of the radar data, and the value of the pixel is updated by calculating the weighted average of all points within the window covering the pixel. This process needs to be repeated for each pixel in the image to achieve smoothing of the entire image.

[0085] Finally, the result is a radar image with reduced noise and smoothed detail variations, which helps improve image quality and clarity and reduces errors and interference in subsequent data processing steps.

[0086] S23, performing edge extraction on the obtained radar image to obtain meteorological edge and structure information in the radar image;

[0087] First, a suitable edge detection algorithm is selected. The embodiment of the present invention selects the Canny operator, which identifies edges by calculating the gradient amplitude and direction.

[0088] Apply this algorithm to the Gaussian filtered radar image and set an appropriate threshold to determine significant edges.

[0089] Next, non-maximum suppression is performed to refine the edges to ensure that they are single-pixel wide, and the identification of edges is further optimized through hysteresis threshold processing to obtain meteorological edge and structure information in radar images, such as cloud boundaries and storm systems, providing key data support for weather analysis and prediction, thereby improving the accuracy and effectiveness of forecasts.

[0090] S24. According to the resolution of the radar data, the Doppler image after edge extraction is segmented, the radar edge is filled with 0, the abnormal value is deleted, and the image is enhanced or weakened.

[0091] According to the radar resolution, the extracted Doppler image is divided into 1200*800 images, and the radar boundary is filled with 0, the outliers are deleted, the images with too low intensity are enhanced, and the images with too high intensity are weakened.

[0092] For the extracted radar images, the images were segmented into 1200x800 sizes according to the resolution of the radar system, so that the image data acquired by different radar devices or at different scanning time points were standardized, ensuring the consistency and comparability of subsequent analysis.

[0093] For the incomplete parts of the radar image boundary, 0 is used to fill them to ensure that during image processing and feature analysis, the edges of the image will not introduce errors due to missing data, thereby improving the overall data integrity and processing accuracy.

[0094] Outlier processing and image enhancement identify and delete outliers in radar images, enhance image parts with too low intensity, and weaken parts with too high intensity.

[0095] This step optimizes the visual effect and data quality of the image, helps to more clearly identify and analyze meteorological phenomena, reduces noise and interference in data processing, improves the accuracy of analysis, enhances the application value of data in meteorological analysis, and provides higher quality and more reliable data support for meteorological forecasting and research.

[0096] S3, based on the RAFT model, the LM-Flow model is improved and the LM-Flow model is used to analyze the continuous pre-processed radar images to extract the optical flow information between the front and rear images;

[0097] Reference Figure 3 , a schematic diagram of the LM-Flow optical flow estimation model proposed in a method for removing clutter from Doppler radar data based on the optical flow method provided in an embodiment of the present invention.

[0098] The LM-Flow model is proposed based on the expansion of the RAFT optical flow estimation model. The model is divided into different components: a three-layer feature encoder for feature extraction, a correlation layer model for correlation calculation, and a gated recurrent unit for iterative update of optical flow information.

[0099] In terms of feature extraction, RAFT uses a three-layer feature encoder, each layer consists of two residual blocks, which aims to capture the features of the two images before and after the input, and enhance these features through the CBAM attention mechanism module. In addition to extracting features from the two images before and after, since the optical flow information is similar to the image contour, the model additionally extracts the context information of the first image as the initial value of the optical flow iterative update module.

[0100] The correlation layer model is used to calculate feature vectors for consecutive images, and calculate the 4D correlation volume based on the dot product of the feature vectors of two consecutive images to characterize the correlation between each pixel between the images. Each grid in the 4D correlation volume contains the sum of the feature dot products of all pixels in the two feature maps. The calculation method of the correlation volume is as follows:

[0101] C ijkl =g θ (I 1 )·g θ (I 2 )

[0102] Where C represents the four-dimensional correlation quantity, g θ Represents the input image I 1 and image I 2 The feature extraction function.

[0103] The gated recurrent unit (GRU) is used to solve the temporal dependency problem inherent in continuous data. In this model, the GRU simulates the traditional least squares optimization process. The input of the GRU is the optimized optical flow ΔF output from the previous iteration, the correlation volume retrieved from the correlation volume, and the potential hidden state, while the output is the updated optimized optical flow ΔF and hidden state, where the initial state of ΔF is the context information extracted by the feature extraction module as the initial value. As shown in the following formula:

[0104] Z t =σ(DSC([h t-1 ,x t ],W z ))

[0105] r t =σ(DSC([h t-1 ,x t ],W r ))

[0106]

[0107] Update Gate Z t and reset gate r t It is calculated by deep separable convolution (DSC) and sigmoid activation function, which jointly regulate the continuity and variability of hidden states in time series. is processed by the tanh activation function to provide a latent new state, and the final hidden state h t The previous state is updated by merging it with the new candidate state, and the final output is the optical flow information of the two pictures.

[0108] The LM-Flow model proposed in the embodiment of the present invention is an improved optical flow calculation model based on the RAFT network, which can handle the complex cloud dynamics in the multi-channel radar reflectivity (MCR) image, effectively cope with the challenges brought by the superposition of multiple layers of clouds, and optimize the parameter scale of the model to ensure efficient performance under limited computing resources. For MCR data, the rapid changes of clouds are captured at a time interval of 6 minutes, and the optical flow calculation process is optimized to ensure that the subtle changes in cloud movement can be accurately reflected. This ability is particularly critical for understanding and predicting meteorological phenomena.

[0109] The LM-Flow model uses a deep separable convolution to reconstruct the RAFT model, effectively reducing the required model parameters. This approach not only reduces the computational burden of the model, but also maintains high-precision optical flow estimation, allowing the model to run efficiently even in standard meteorological laboratories with limited GPU resources.

[0110] The LM-Flow model also integrates the CBAM (Convolutional Block Attention Module) attention mechanism. CBAM enhances the model's expressiveness in capturing complex cloud movements by focusing on more critical feature areas, significantly improving the ability to extract cloud features.

[0111] S4, using the optical flow information obtained by motion threshold screening to generate a binary mask to separate ground clutter;

[0112] The optical flow information represents the direction and size of the pixel's movement. A motion threshold is determined based on the absolute value of the pixel's movement to characterize the pixel's motion state. The pixels are classified using the motion threshold, and pixels below the motion threshold are classified as stationary pixels. Setting the threshold too low will result in incomplete removal of ground clutter, while setting it too high will remove cloud information at the same time. Therefore, a suitable threshold can be selected based on the characteristics of a specific data set. If the data set contains a lot of cloud information and you want to retain most of the cloud information, you can select a lower threshold, otherwise select a higher threshold. The specific threshold can be determined based on the fitting effect of the ZI relationship after the data set is removed.

[0113] A motion threshold-based binary mask is generated for the processed optical flow information. The size of the binary mask is the same as the original image, where the pixels of stationary ground elements are set to 0 and the pixels of moving clouds are set to 1.

[0114] In the embodiment of the present invention, the motion state of pixels is classified by accurately setting the motion threshold, and the optical flow information obtained by screening the motion threshold is used to generate a binary mask method, which can effectively distinguish dynamic clouds from static ground clutter, thereby reducing the error caused by ground reflection, providing a clearer dynamic image of clouds, thereby enhancing the reliability and accuracy of meteorological data, simplifying the complexity of the image, accurately removing ground clutter, and reducing false alarms caused by misidentifying clutter as clouds.

[0115] In addition, the embodiments of the present invention not only improve the accuracy and efficiency of meteorological image processing by accurately processing optical flow information, but also greatly enhance the application value and practicality of meteorological data. These improvements provide strong technical support for meteorological forecasting and related research, and help improve the overall effect and responsiveness of meteorological services.

[0116] S5. Removing ground clutter according to the binary mask, and repairing the mistakenly deleted cloud information by using a bilinear interpolation method to obtain a radar data map with ground clutter removed.

[0117] The embodiment of the present invention applies a binary mask to the original image to obtain a Doppler radar data image with ground clutter removed, retains the cloud layer set to 1, and removes the ground layer set to 0, thereby achieving the effect of removing ground clutter.

[0118] The embodiment of the present invention obtains a radar data image with ground clutter removed by applying a binary mask to the original image.

[0119] However, the high pixel similarity unique to radar images means that even small meteorological changes may lead to the loss of important features in the image. This phenomenon is particularly evident when only relying on traditional methods to process continuous radar images, because standard image processing algorithms often have difficulty capturing small but meteorologically significant changes. Optical flow calculations based solely on two consecutive images may lose key pixels due to the high similarity between pixels, which directly affects the accuracy of precise tracking and prediction of dynamic weather events.

[0120] At this time, bilinear interpolation is used to correct the optical flow information. By judging the difference between the current image and the two previous and next images, it is determined whether there is a problem with the current image. For the image with problems, bilinear interpolation is used to repair the image using the two previous and next images. All images are iterated repeatedly to perform bilinear interpolation repair until there are no more problematic images. Finally, the output radar data with ground clutter removed is obtained.

[0121] That is, in order to repair the information lost in the optical flow calculation, bilinear interpolation is performed on the image to be repaired, and the information of the previous and next frames is combined to more accurately reconstruct or correct each pixel in the image. This method uses the values ​​of the four surrounding points to estimate the value of the unknown point, effectively improving the overall quality of the image and the accuracy of the analysis. The time series information of all images is fully utilized through an iterative method. The process of iterative traversal and repeated repair until the number of problematic images in the image set no longer increases ensures the stability and reliability of the final result. Bilinear interpolation utilizes the time series characteristics of the data set and enhances the consistency of image quality.

[0122] The specific implementation of using bilinear interpolation to correct optical flow information is as follows:

[0123] Bilinear interpolation estimates the value of an unknown point based on the four nearest points, thereby achieving a smooth interpolation effect. Linear interpolation is performed first in one direction and then in the other direction to obtain the final interpolation result. The bilinear interpolation process consists of two steps:

[0124] First, in the x direction, Q 11 and Q 21 Linear interpolation is performed between two points to obtain R 1 Similarly, for Q12 and Q 22 Perform linear interpolation to get R 2 .

[0125] The interpolation formula is as follows:

[0126]

[0127] Next, linearly interpolate R_1 and R_2 in the y direction to obtain the final interpolation result P:

[0128]

[0129] For each image in the inpainting queue, perform bilinear interpolation on a series of before and after images;

[0130] Iterate repeatedly until the number of images that need to be repaired no longer increases.

[0131] This iterative process ensures that the quality of the restored images is more stable.

[0132] In a specific implementation environment, the dataset used in one embodiment of the present invention is from the Wuhan area of ​​China, located at 114.0075 east longitude and 30.2225 north latitude, covering an area with a radius of 5 kilometers. It includes the mean composite reflectance (MCR) and quantitative precipitation estimation (QPE) images collected in 2021.

[0133] The MCR images are collected every 6 minutes, while the QPE images are collected every hour. The dataset contains about 65,000 MCR images and 6,000 QPE images.

[0134] Among them, the MCR value is defined as follows:

[0135] MCR=2×dBZ+66

[0136] Here, CMR means minimum detectable reflectivity or comparable meteorological index, and dBZ means radar reflectivity measured in decibels.

[0137] The unit value of precipitation in QPE is 0.1 mm.

[0138] Since convolution operations are used in the LM-Flow optical flow model, the original image of size 1200x800 pixels needs to be modified. The modification process includes cropping the image and resizing it to 288x288 pixels, and then converting it to a three-channel grayscale image. This preprocessing ensures compatibility with subsequent optical flow calculations.

[0139] like Figure 2As shown in the figure, the whole process takes two adjacent images as input, uses the LM-Flow model to calculate the optical flow information, and the optical flow information is applied to the first image to remove clutter.

[0140] Reference Figure 3 , a schematic diagram of the LM-Flow optical flow estimation model of the method for removing clutter from Doppler radar data provided by an embodiment of the present invention.

[0141] Since the time interval of multi-channel radar reflectivity (MCR) data is 6 minutes, it can capture the complex cloud movement in a short time. However, MCR images are uniquely challenging for deep learning models, especially because the superposition of multiple cloud layers makes it difficult to directly extract cloud features.

[0142] Therefore, precise optical flow data is required to accurately represent the subtle dynamics of these cloud motions. In addition, limited GPU computing resources for deep learning in standard meteorological laboratories require effective management of model parameter size to ensure effective generalization and deployment.

[0143] To address these challenges, an LM-Flow model is developed based on the RAFT modeling framework in an embodiment of the present invention.

[0144] In the feature extraction stage, we first extract the continuous image I 1 and I 2 Extract convolution features from

[0145] These features are then enhanced by the CBAM attention mechanism module, which also extracts 1 This step is crucial because the shape of the optical flow image is similar to the original image, effectively adding constraints to the image contour during the iterative optimization process.

[0146] Then, a 4D correlation volume is calculated based on the dot product of the feature vectors of the two images to represent the correlation of each pixel between the images. Each grid in the correlation volume contains the sum of the feature dot products of all pixels in the two feature maps.

[0147] The correlation volume is calculated as follows:

[0148] C ijkl =g θ (I 1 )·g θ (I 2 )

[0149] Among them, C represents the four-dimensional correlation quantity, g θ Represents the input image I 1 and image I 2 The feature extraction function.

[0150] Again, the gated recurrent unit (GRU) is used to solve the temporal dependency problem inherent in continuous data. In this model, GRU simulates the traditional least squares optimization process. The input of GRU is the optimized optical flow, ΔF, the relevant amount retrieved from the relevant volume, and the potential hidden state outputted by the previous iteration, and the output is the updated optimized optical flow, ΔF, and hidden state, where the initial state of ΔF is the context information extracted by the feature extraction module as the initial value, as shown in the following formula:

[0151] Z t =σ(DSC([h t-1 ,x t ],W z ))

[0152] r t =σ(DSC([h t-1 ,x t ],W r ))

[0153]

[0154] Update Gate Z t and reset gate r t It is calculated by deep separable convolution (DSC) and sigmoid activation function, which jointly regulate the continuity and variability of hidden states in time series. is processed by the tanh activation function to provide a latent new state, and the final hidden state h t The previous state is updated by merging it with the new candidate state, and the final output is the optical flow information of the two pictures.

[0155] LM-Flow is based on the RAFT model and integrates the convolutional block attention module (CBAM) and the depth-separable convolution (DSC), reducing model parameters while maintaining high performance. DSC reduces computational complexity, enabling the model to perform deep learning tasks with limited computing resources, while CBAM enhances representation capabilities by sequentially applying channel and spatial attention mechanisms. This integration enables the model to efficiently calculate and accurately process input data.

[0156] The calculation formula is:

[0157] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0158] Where F is the input feature map and σ represents the sigmoid function. Spatial attention focuses on meaningful "places" and is calculated as:

[0159] M s (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))

[0160] Among them, f 7×7 It is a convolution operation applied to the combined average features and max pooling features.

[0161] Depthwise separable convolution aims to reduce the number of parameters and computational load in convolutional neural networks. It is widely used in various neural networks. It decomposes the standard convolution operation into two simpler steps: depthwise convolution and pointwise convolution.

[0162] In depthwise convolution, each input channel is independently convolved with a separate filter. This means that the convolution operation is performed independently for each channel. in Input channels and D out For a convolutional layer with 10 output channels, the depth convolution will use D in filters, each of size K×K, where K is the size of the filter. The output will have D in channels, each channel is formed by convolution of the input channel and its corresponding filter. The depth convolution operation can be expressed as:

[0163]

[0164] in, is the output of the kth channel, X k is the kth input channel, F k is the kth filter of size K times K.

[0165] On the other hand, point convolution takes the output of depthwise convolution as input and combines the channels produced by depthwise convolution to generate the final output channel. out Output channel, point convolution uses D out ×D in Weights, because each output channel is a weighted sum of the input channels. The point convolution operation can be expressed as:

[0166]

[0167] in, is the jth output channel, P j,k is a 1×1 point convolution filter applied to the kth channel.

[0168] Integrating CBAM and DSC in LM-Flow can take advantage of the advantages of both. Although DSC reduces computational complexity and model parameters, it may sacrifice some expressiveness because each input channel is processed independently, while CBAM alleviates this problem by applying an attention mechanism to selectively focus on important features and suppress irrelevant information. This combined approach enables the model to maintain high performance with less computing resources, making it very suitable for deployment on devices with limited capabilities.

[0169] In the LM-Flow model, the input consists of two consecutive images, and the output is a map containing optical flow information. The map maintains the original size of the image, but contains two channels that convey the motion information of each pixel along the X and Y axes respectively. By extracting optical flow data, we can gain a deeper understanding of the dynamics of the image, especially the motion patterns of the clouds.

[0170] In optical flow calculation, a basic assumption is that the grayscale intensity of the pixel remains unchanged, and the optical flow data reflects the change in pixel brightness between two consecutive images. However, due to the high pixel similarity of radar images, using only two adjacent images when calculating optical flow will result in a large number of pixels being lost.

[0171] In order to solve this problem, the embodiment of the present invention utilizes the characteristics of continuous image frames of the data set to adopt bilinear interpolation to correct the optical flow information, thereby significantly improving the image quality. In particular, when there is a large difference between one of the continuous image frames and the two previous and subsequent image frames, the use of bilinear interpolation to correct the optical flow information improves the accuracy of reconstructing the cloud motion trajectory, thereby enhancing the performance and reliability of the LM-Flow results in the ZI relationship fitting task.

[0172] That is, the embodiment of the present invention adopts post-processing technology for optical flow data, detects images that have changed too much compared with the previous frame and the next frame, adds these images that have changed too much compared with the previous and next frames to the repair queue, and performs bilinear interpolation on the images in the queue. If the number of images in the queue stops decreasing after the loop traversal, it means that bilinear interpolation can no longer further repair the images, and the loop traversal can be stopped.

[0173] In addition, the embodiment of the present invention uses the absolute value of the pixel motion to determine an empirical threshold for classifying the motion state of the pixel, and pixels below the empirical threshold are classified as stationary pixels.

[0174] In reality, if the threshold is set too low, ground clutter will not be completely removed, while if it is set too high, cloud information will be removed at the same time. Therefore, the embodiment of the present invention selects a suitable threshold according to the characteristics of the specific data set.

[0175] The processed optical flow information is used to generate a threshold-based binary mask. The size of the binary mask is the same as the original image, where pixels representing stationary ground elements are set to 0 and pixels representing moving clouds are set to 1. This eliminates the interference of stationary ground objects while maintaining the integrity of the clouds.

[0176] The improved image provides a more reliable basis for the subsequent ZI relationship fitting.

[0177] The embodiment of the present invention adopts a statistical method to fit the ZI relationship, substitutes the radar reflectivity Z and the estimated precipitation intensity I into the equation, and uses the least squares method to estimate the values ​​of a and b. The main equation is as follows:

[0178] Z=aI b

[0179] Here, coefficient a is a proportionality constant, while b is an exponential coefficient that varies with precipitation type, development stage, and geographic location.

[0180] Based on the ZI relationship, mathematical statistics are used to convert the reflection coefficient measured by the radar into precipitation intensity. In order to accurately fit the ZI relationship, the embodiment of the present invention adopts a mathematical statistics method, substitutes the numerical value into the equation, and estimates the parameters using the least squares method. This method is simple and reliable, meets the requirements of model verification, highlights the effectiveness of precipitation estimation, and can be used as an indicator of the efficiency of ground clutter removal technology.

[0181] Due to the large amount of data, the embodiment of the present invention also uses the RANSAC method to eliminate the interference of outliers and ensure the stability of the results by obtaining fitting results that conform to most of the data. To ensure the accuracy of the fitting results, 90% of the data is used for fitting and the remaining 10% is used for verification. In order to comprehensively evaluate the quality and accuracy of images processed by different methods, two indicators are introduced: structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR).

[0182] It is worth noting that there is a time difference between QPE and MCR. Therefore, the embodiment of the present invention also conducted additional experiments to study the impact of different time delays on the accuracy of the $ZI$ relationship. The results show that when the quantitative precipitation estimate (QPE) is delayed by 5 images (equivalent to half an hour) relative to the mean composite reflectivity (MCR), the fitting effect of the ZI relationship is best. This is because the measurement of cloud thickness is based on Doppler radar measurement, while the measurement of precipitation is carried out on the ground, and it takes a certain amount of time for clouds to form precipitation and reach the ground.

[0183] Reference Figure 4, a method for removing clutter from Doppler radar data by obtaining optical flow information based on deep learning provided by an embodiment of the present invention, an example of a data set used in training a deep learning optical flow estimation model, and a performance comparison chart between the deep learning optical flow estimation model LM-Flow provided by an embodiment of the present invention and other models.

[0184] The present invention involves the training and comparison of several model architectures, including the traditional filter-based feature clutter removal method, the Farneback dense optical flow method, FlowNet2, the basic RAFT model, and the LM-Flow model. To ensure the consistency of the ratings, the present invention uses the "Chair" open source dataset to train the deep learning model.

[0185] Figure 4 An example from this dataset is shown. Training was performed on the same machine with an upper limit of 120,000 epochs. An early stopping criterion was used during training, whereby training was stopped if no improvement in validation loss was observed for 15 consecutive hours. All training iterations met this criterion, so it was not necessary to reach the upper limit of 120,000 iterations. In addition, a learning rate scheduler was used that reduced the learning rate to one-tenth of its previous value if validation loss did not improve within four epochs. The initial learning rate was set to 0.0003, and the widely used Adam optimizer was chosen for the optimization process.

[0186] The entire training sequence was executed on an NVIDIA 4090 graphics card with 24GB of video memory to ensure efficient handling of the computational load. After training the three deep learning models, they were used to evaluate performance metrics on the test set.

[0187] Figure 5 It well demonstrates the computing power of LM-Flow in the field of optical flow. The embodiment of the present invention uses various indicators such as the number of iterations, computing time and parameter size to evaluate it, and evaluates the optical flow computing performance on the "chair" dataset. It can be seen that the FlowNet2 model is characterized by a wide parameter set, the longest running time, and unsatisfactory performance. In contrast, the original RAFT model is in the middle in terms of parameter size and has superior performance. On the other hand, LM-Flow has the least number of parameters, a shorter running time, and performance close to the RAFT model. Compared with the larger FlowNet2, RAFT has only one-third of the latter's parameters, but its performance is superior.

[0188] LM-Flow, developed from RAFT, introduces a major change, which is to replace traditional convolution with deep separable convolution, which effectively reduces the number of parameters in the RAFT model by half. At the same time, it also cleverly adds an efficient attention mechanism, which greatly improves the feature extraction ability of the model despite a slight increase in the number of parameters. This strategic optimization enables LM-Flow to achieve a delicate balance, optimizing the number of parameters and running time, so as to approach the efficiency of the original model without affecting performance.

[0189] Figure 6 This is a diagram showing the effects of removing ground clutter from Doppler radar data using different optical flow images provided by an embodiment of the present invention.

[0190] Figure 7 The performance comparison chart of different deep learning optical flow models using different modules provided by the embodiment of the present invention shows the impact of different modules on model parameters and performance. The image sequence from left to right is the original image, the filtered image with ground clutter removed, the Farneback dense optical flow, the Flownet2 optical flow estimation model, the original RAFT model, and the LM-Flow result with ground clutter removed.

[0191] Figure 8 A radar rainfall fitting performance diagram of the optical flow estimation model provided in an embodiment of the present invention after removing ground clutter from Doppler radar data.

[0192] The ground clutter removal technique used in the embodiments of the present invention significantly improves the accuracy of the $ZI$ relationship fitting. The RAFT model performs outstandingly in terms of root mean square error (RMSE) and has excellent performance, while the LM-Flow has only half the number of parameters of the RAFT model and has almost the same performance. In terms of the structural similarity index (SSIM), the difference between the two methods is minimal, but LM-Flow ranks the highest. Interestingly, the traditional filtering methods show a higher peak signal-to-noise ratio (PSNR), which may be because they are able to retain more cloud information while effectively reducing ground clutter.

[0193] In the method and device for removing clutter from Doppler radar data based on optical flow information used in the present invention, the performance of the traditional Farneback dense optical flow method is relatively general, but the running time is very short, only 0.016 seconds. This speed makes it suitable for platforms with limited computing resources and scenarios with low precision requirements. In the field of deep learning optical flow methods, Flownet2 is very effective in removing ground clutter, but at the cost of removing some cloud information, resulting in unsatisfactory performance. In contrast, the RAFT model shows superior performance in this area. LM-Flow only retains half of the parameters of the original RAFT model and achieves similar results, but with higher computational efficiency. In the post-processing of the image, bilinear interpolation is used because it is simple and efficient. This technology makes up for the loss in the optical flow calculation process, thereby significantly improving the overall effect.

[0194] The embodiment of the present invention further provides a device for removing clutter from Doppler radar data based on optical flow information, and the device specifically includes:

[0195] An image acquisition module, used to acquire radar data and extract detail data from the radar data, wherein the detail data includes latitude and longitude, resolution and Doppler radar image;

[0196] A preprocessing module, used for performing preprocessing on the extracted Doppler radar image to obtain a radar picture;

[0197] An optical flow information acquisition module is used to obtain an LM-Flow model based on the RAFT model, and use the obtained LM-Flow model to analyze the continuous pre-processed radar images to extract the optical flow information between the front and rear images, where the optical flow information represents the movement direction and size of the pixel;

[0198] The clutter separation module is used to generate a binary mask to separate ground clutter by using the optical flow information obtained by motion threshold screening;

[0199] The correction module is used to remove ground clutter according to the binary mask and to repair the mistakenly deleted cloud information by using a bilinear interpolation method to obtain a radar data map with the ground clutter removed.

[0200] Wherein, the preprocessing module specifically includes:

[0201] A standardization processing unit, used for performing standardization processing on Doppler radar images;

[0202] A filtering unit, used for performing smoothing filtering on the standardized radar image data;

[0203] An edge extraction unit is used to extract the edge of the radar image obtained by the smoothing filter process to obtain the meteorological edge and structure information in the radar image;

[0204] The image addition and subtraction unit is used to segment the Doppler image after edge extraction according to the resolution of the radar data, fill the radar edge with 0, delete the abnormal value, and enhance or weaken the image.

[0205] The LM-Flow model improved based on the RAFT model includes:

[0206] Three-layer feature encoder, depthwise separable convolution, correlation layer model, and gated recurrent unit.

[0207] in,

[0208] The three-layer feature encoder is used to receive the input of the two images and perform feature extraction, and extract the context information of the first image as the initial value of the gated recurrent unit for iterative update of the optical flow;

[0209] The correlation layer model is used to calculate feature vectors of the features of the two consecutive images, and calculate the correlation volume according to the dot product of the feature vectors of the two consecutive images;

[0210] The gated recurrent unit is used to receive the initial value of the iterative update of the optical flow, retrieve the correlation amount and the potential hidden state from the correlation volume, and calculate the updated optimized optical flow information and hidden state. The calculation formula is shown in the following formula:

[0211] Z t =σ(DSC([h t-1 ,x t ],W z ))

[0212] r t =σ(DSC([h t-1 ,x t ],W r ))

[0213]

[0214] Among them, the update gate Z t and reset gate r t It is calculated by deep separation convolution and sigmoid activation function, and the candidate hidden state It is processed by the tanh activation function;

[0215] The final hidden state h t The previous state is updated by merging the new candidate state, and the final output is the optical flow information of the two pictures.

[0216] The implementation method of the device for removing Doppler radar data clutter based on deep learning to obtain optical flow information of the present invention and the implementation method of the method for removing Doppler radar data clutter of the aforementioned embodiment will not be described in detail here.

[0217] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for removing clutter from Doppler radar data based on optical flow information, characterized in that: The method comprises: Acquire radar data, and extract detail data from the radar data, wherein the detail data includes latitude and longitude, resolution, and Doppler radar image; performing preprocessing on the extracted Doppler radar image to obtain a radar picture; The LM-Flow model is improved based on the RAFT model. The LM-Flow model is used to analyze the continuous pre-processed radar pictures to extract the optical flow information between the front and rear images. The optical flow information represents the movement direction and size of the pixel. The LM-Flow model improved based on the RAFT model includes: Three-layer feature encoder, depthwise separable convolution, correlation layer model, and gated recurrent unit; The method of using the obtained LM-Flow model to analyze the continuous pre-processed radar images to extract the optical flow information between the front and rear images specifically includes: The three-layer feature encoder receives the input of the two images and performs feature extraction, and extracts the context information of the first image as the initial value of the gated recurrent unit for iterative update of the optical flow; The correlation layer model calculates the feature vectors of the features of the two images extracted before and after, and calculates the correlation volume based on the dot product of the feature vectors of the two consecutive images; The gated recurrent unit receives the initial value of the iterative update of the optical flow, retrieves the correlation amount and the potential hidden state from the correlation volume, and calculates the updated optimized optical flow information and hidden state. The calculation formula is shown as follows: Z t =σ(DSC([h t-1 ,x t ],W z )) r t =σ(DSC([h t-1 ,x t ],W r )) Among them, the update gate Z t and reset gate r t It is calculated by deep separation convolution and sigmoid activation function, and the candidate hidden state It is processed by the tanh activation function; The final hidden state h t The previous state is updated by merging the new candidate state, and the final output is the optical flow information of the two pictures; The optical flow information obtained by motion thresholding is used to generate a binary mask to separate ground clutter. The ground clutter is removed according to the binary mask, and the cloud information that was mistakenly deleted is repaired by using a bilinear interpolation method to obtain a radar data map with the ground clutter removed.

2. The method for removing clutter from Doppler radar data based on optical flow information as claimed in claim 1, characterized in that: The performing preprocessing on the extracted Doppler image to obtain a radar picture comprises: Normalize Doppler radar images; Perform smoothing and filtering on the standardized radar image data; Perform edge extraction on the radar image obtained by smoothing and filtering to obtain meteorological edge and structure information in the radar image; According to the resolution of radar data, the Doppler image after edge extraction is segmented, the radar edges are filled with 0, the outliers are deleted, and the image is enhanced or weakened.

3. The method for removing clutter from Doppler radar data based on optical flow information as claimed in claim 1, characterized in that: The method of using the optical flow information obtained by screening the motion threshold to generate a binary mask to separate the ground clutter specifically includes: Determine a motion threshold based on the absolute value of pixel motion; Classifying pixels using the motion threshold, where pixels below the motion threshold are classified as stationary pixels; A motion threshold-based binary mask is generated for the processed optical flow information. The size of the binary mask is the same as the original image, where the pixels of stationary elements on the ground are set to 0 and the pixels of moving clouds are set to 1.

4. The method for removing clutter from Doppler radar data based on optical flow information as claimed in claim 3, characterized in that: The method of removing ground clutter according to the binary mask and repairing the mistakenly deleted cloud information by using the bilinear interpolation method to obtain the radar data map with the ground clutter removed specifically includes: Apply the binary mask to the original image to obtain the Doppler radar data image with ground clutter removed; Determine the difference between the current image and the previous and next images to determine whether there is a problem with the current image. Perform bilinear interpolation to repair the image using the previous and next images. Loop through all images and iterate repeatedly to perform bilinear interpolation to repair the images until there are no more problematic images.

5. A device for removing clutter from Doppler radar data based on optical flow information, characterized in that: The device specifically comprises: An image acquisition module, used to acquire radar data and extract detail data from the radar data, wherein the detail data includes latitude and longitude, resolution and Doppler radar image; A preprocessing module, used for performing preprocessing on the extracted Doppler radar image to obtain a radar picture; An optical flow information acquisition module is used to obtain an LM-Flow model based on the RAFT model, and use the obtained LM-Flow model to analyze the continuous pre-processed radar images to extract the optical flow information between the front and rear images, where the optical flow information represents the movement direction and size of the pixel; The LM-Flow model improved based on the RAFT model includes: Three-layer feature encoder, depthwise separable convolution, correlation layer model, and gated recurrent unit; The three-layer feature encoder is used to receive the input of the two images and perform feature extraction, and extract the context information of the first image as the initial value of the gated recurrent unit for iterative update of the optical flow; The correlation layer model is used to calculate feature vectors of the features of the two consecutive images, and calculate the correlation volume according to the dot product of the feature vectors of the two consecutive images; The gated recurrent unit is used to receive the initial value of the iterative update of the optical flow, retrieve the correlation amount and the potential hidden state from the correlation volume, and calculate the updated optimized optical flow information and hidden state. The calculation formula is shown in the following formula: Z t =σ(DXC([h t-1 ,x t ],W z )) r t =σ(DSC([h t-1 ,x t ],W r )) Among them, the update gate Z t and reset gate r t It is calculated by deep separation convolution and sigmoid activation function, and the candidate hidden state It is processed by the tanh activation function; The final hidden state h t The previous state is updated by merging the new candidate state, and the final output is the optical flow information of the two pictures; The clutter separation module is used to generate a binary mask to separate ground clutter by using the optical flow information obtained by motion threshold screening; The correction module is used to remove ground clutter according to the binary mask and to repair the mistakenly deleted cloud information by using a bilinear interpolation method to obtain a radar data map with the ground clutter removed.

6. The device for removing clutter from Doppler radar data based on optical flow information as claimed in claim 5, characterized in that: The preprocessing module specifically includes: A standardization processing unit, used for performing standardization processing on Doppler radar images; A filtering unit, used for performing smoothing filtering on the standardized radar image data; An edge extraction unit is used to extract the edge of the radar image obtained by the smoothing filter process to obtain the meteorological edge and structure information in the radar image; The image addition and subtraction unit is used to segment the Doppler image after edge extraction according to the resolution of the radar data, fill the radar edge with 0, delete the abnormal value, and enhance or weaken the image.

Images