A millimeter wave cloud radar echo classification method and system based on physical guidance deep learning

Abstract

The application discloses a millimeter wave cloud radar echo classification method and system based on physical guidance deep learning, comprising: constructing a multi-source fusion dataset, aligning, standardizing and sliding window sampling preprocessing millimeter wave cloud radar base data and co-sited ground observation data; extracting thermodynamic state information from the ground observation data through a physical constraint gating module, generating a clear sky echo probability graph to adaptively weight the radar initial features; based on an encoder-decoder backbone network, the weighted radar features are jointly convolved and extracted, and an ECH regression branch is used to fuse the thermodynamic features and the radar features to predict the effective clutter height; based on the predicted effective clutter height sequence, an adaptive height filtering region is constructed, and morphological opening operation and intersection operation of dynamic height constraint are used to realize fine echo classification post-processing, so that the height balance of meteorological echo retention rate and clear sky echo suppression rate is realized, and the data quality of the millimeter wave cloud radar is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of meteorological radar data processing and artificial intelligence, and in particular to a field of millimeter-wave cloud radar echo recognition and quality control technology based on physical-guided deep learning. Technical Background

[0002] Millimeter-wave cloud radar possesses high spatiotemporal resolution and sensitive detection capabilities of cloud particle microphysical structures, making it a core device for scenarios such as nowcasting, cloud precipitation physics research, and weather modification. However, this high sensitivity makes it extremely susceptible to interference from clear-sky echoes caused by insects, dust, and turbulence, severely limiting the performance of inversion algorithms and the application of downstream products.

[0003] Existing clear-sky echo identification schemes mainly rely on empirical rules and shallow machine learning methods. Fixed threshold methods typically use radar reflectivity factors and linear depolarization ratios for filtering, which, while computationally efficient, are highly sensitive to changes in atmospheric conditions. Morphological filtering uses convolutional kernels designed based on echo geometry, but performs poorly when dealing with complex layered clutter associated with meteorological echoes. In recent years, deep learning technology has been increasingly applied to the meteorological field. Echo identification can be transformed into a semantic segmentation problem. For example, using a purely data-driven deep learning model, radar data is fed into a neural network to segment echoes, eliminating the need for manual feature selection.

[0004] The existing technologies described above have the following problems: First, clear-sky echoes are highly dynamic, and their generation and dissipation are affected by atmospheric thermodynamics. In complex scenarios where meteorological echoes and clear-sky echoes have highly overlapping characteristics, models lacking awareness of the environmental background may learn spurious correlations, producing physically unreasonable results. Second, while purely data-driven models have strong feature extraction capabilities, they face the challenge of insufficient generalization when processing clear-sky echoes, exhibiting significant performance fluctuations under different seasons and geographical conditions. Third, existing methods fail to fully utilize the physical coupling relationship between atmospheric thermodynamic structure and the vertical range of clear-sky echoes, resulting in a lack of physical interpretability in the models and an inability to achieve an effective balance between clear-sky echo suppression and meteorological echo preservation. Summary of the Invention

[0005] To address the shortcomings of existing cloud radar echo classification methods, such as insufficient generalization ability in complex weather scenarios, poor physical interpretability, and an imbalance between clear-sky echo suppression and meteorological echo preservation performance, this invention provides a millimeter-wave cloud radar echo classification method based on physical-guided deep learning. This method aims to provide high-quality base data for the operational applications and product development of millimeter-wave cloud radar. Specifically, the technical solution of this invention includes the following key steps:

[0006] Step 1: Multi-source data preprocessing: Acquire millimeter-wave cloud radar base data and ground meteorological observation data (including temperature, relative humidity, precipitation, etc.) from the same site and time, and calculate the corresponding lifting condensation height (LCL); perform time alignment and height truncation standardization on the aforementioned data, and construct the model input sample sequence through sliding window sampling.

[0007] Step 2: Physically Constrained Thermodynamic Feature Extraction: Ground meteorological observation data is input into the Physically Constrained Gated Module (PCGB). Thermodynamic features that evolve over time are extracted using a one-dimensional convolutional neural network and a position code based on trigonometric functions is injected. The extracted thermodynamic features, position codes, and lifting condensation height are then stitched together to generate a clear-sky echo prior probability map. This map is used to weight and modulate the initial millimeter-wave cloud radar data, adaptively suppressing low-confidence clutter interference.

[0008] Step 3, Radar Spatiotemporal Feature Encoding and Echo Classification: The modulated radar features are input into an encoder-decoder backbone network containing multiple masked modulation convolutional units. Spatiotemporal features are extracted through pseudo-3D convolution on the time-height plane. In each feature extraction, a normalization factor is synchronously calculated based on the weighted mask, and the extracted features are numerically compensated by division. The preliminary echo classification results are output pixel by pixel.

[0009] Step 4: Effective clutter height regression: At the bottleneck of network feature extraction, an effective clutter height (ECH) regression branch is introduced. Thermodynamic features, radar spatiotemporal features and lifting condensation height are fused and input into the multilayer sensor to predict the maximum vertical range of clear-sky echoes within the boundary layer, i.e., the effective clutter height. The spatiotemporal structure is used to capture the dynamic inertia of the atmosphere and correct the theoretical bias of nighttime.

[0010] Step 5: Dynamic post-processing based on effective clutter height (ECH): An adaptive height filtering region is constructed based on the predicted effective clutter height sequence. A candidate noise set is extracted through global morphological opening operation. At the same time, the predicted effective clutter height sequence is mapped back to the physical height limit. Only when the candidate noise is below the predicted effective clutter height is it determined to be clutter and filtered out.

[0011] Based on the same inventive concept, embodiments of the present invention also provide a millimeter-wave cloud radar echo classification system based on physical-guided deep learning, a computer-readable storage medium, and an electronic device.

[0012] The present invention has the following technical effects:

[0013] First, it significantly improved the accuracy and robustness of echo classification. After introducing the physical constraint gating module, the meteorological echo recognition rate increased to 97.75%, and the false alarm rate decreased to 3.62%. After further adding the effective clutter height regression branch, the meteorological echo recognition rate increased to 98.61%, and the critical success index (CSI) reached 0.9657.

[0014] Secondly, it possesses extremely high clear-sky echo suppression capability. After dynamic post-processing based on effective clutter height, the meteorological echo false alarm rate is further reduced to 1.18%, and the clear-sky echo recognition rate reaches 95.87%, effectively solving the clutter interference problem in complex scenarios.

[0015] Third, it possesses excellent spatiotemporal generalization capabilities. Performance analysis throughout the year shows that this invention maintains a high recognition rate across different seasons with no significant seasonal performance fluctuations; simultaneously, it demonstrates stable performance in tests across different regions, overcoming the impact of environmental differences on echo recognition, thus providing high-quality baseline data support for the operational applications and product development of millimeter-wave cloud radar. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the process for a millimeter-wave cloud radar echo classification method based on physics-guided deep learning.

[0017] Figure 2 Workflow diagram for data preprocessing and sample construction;

[0018] Figure 3 This is a schematic diagram of the internal structure of the physical constraint gating module.

[0019] Figure 4 A schematic diagram of the topology of the backbone network for radar spatiotemporal feature extraction;

[0020] Figure 5 This is a schematic diagram of the internal structure and data flow of a masked modulation convolutional unit.

[0021] Figure 6 A schematic diagram of the effective clutter height regression branch;

[0022] Figure 7 This is a schematic diagram of a dynamic post-processing strategy based on effective clutter height. Detailed Implementation

[0023] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. Those skilled in the art should understand that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0024] Reference Figure 1 The method provided in this embodiment of the invention mainly includes the following steps:

[0025] Step S1: Data Preprocessing and Sample Construction. Acquire cloud radar base data and surface meteorological observation data from the same location and time as the cloud radar. The cloud radar base data includes radar reflectivity factor, radial velocity, spectral width, and linear depolarization ratio, while the surface meteorological observation data includes surface temperature, relative humidity, and precipitation. To meet network input requirements, time alignment and standardization are performed on the multi-source data, and the minute-by-minute lifting condensation height (LCL) is calculated. See [link to details] for further information. Figure 2 :

[0026] Sub-step S11 involves truncating or filling the millimeter-wave cloud radar data in the vertical direction with a fixed range, and strictly matching the radar data with the ground observation data in the time dimension with a one-minute reference.

[0027] Sub-step S12 involves using linear interpolation to fill in short-term missing ground observation data, discarding corresponding radar samples for long-term missing data, and simultaneously calculating the lifting condensation height.

[0028] Sub-step S13 involves slicing the processed daily data into sliding windows with a preset window length and step size to generate a sample sequence with a fixed time length.

[0029] Step S2: Input the ground meteorological observation data into the physical constraint gating module, extract the ground thermodynamic evolution characteristics and generate a clear sky echo prior probability map, and use the probability map to perform feature modulation on the cloud radar base data to suppress low confidence clutter and generate modulated radar features.

[0030] In step S2, to address the lack of environmental awareness in purely data-driven models, thermodynamic information is introduced through a physical constraint gating module. (Refer to...) Figure 3 :

[0031] Sub-step S21 involves extracting the temporal evolution features of the thermodynamic state from the aligned ground meteorological observation data using a two-layer cascaded one-dimensional convolutional neural network. The first convolutional layer has a kernel size of 7 and padding of 3; the second convolutional layer has a kernel size of 5 and padding of 2. Each convolutional layer is sequentially processed by the ReLU activation function and batch normalization, ultimately outputting a physical feature embedding vector of the same length as the input sequence.

[0032] Sub-step S22 converts the radar range library index into a position code based on sine and cosine functions to solve the spatial dimension mismatch problem caused by the lack of vertical height information in ground data;

[0033] Sub-step S23 involves expanding the condensation height in the vertical dimension to match the radar height dimension. The expanded condensation height, the physical feature embedding vector output in step S21, and the position code output in step S22 are then concatenated along the channel dimension to generate a thermodynamic spatial coupling feature map.

[0034] Sub-step S24 performs a nonlinear transformation on the coupled feature map using a confidence header composed of three layers of pointwise convolutions. The first layer maps the number of input channels to 64 and activates it with ReLU, the second layer maps it to 32 and activates it with ReLU, and the third layer maps it to 1 and activates it with Sigmoid. The output is a prior probability map of clear sky echoes with values ​​ranging from zero to one.

[0035] Sub-step S25: The cloud radar base data is binarized (valid value is 1, invalid value is 0) and multiplied element-wise with the complement of the prior probability map to generate the weighted mask.

[0036] Step S3 involves performing physically guided radar spatiotemporal feature extraction and echo classification. This step is accomplished through an encoder-decoder backbone network containing multiple "mask modulation convolutional units".

[0037] Among them, reference Figure 5 The internal execution logic of the "mask modulation convolutional unit" specifically includes the following steps:

[0038] Step S301: Expand the input weighted mask to the same number of channels as the input radar features, and multiply the two element-wise to generate soft-gated modulated radar features. Perform convolution operations on the modulated radar features to initially extract high-dimensional features.

[0039] Step S302: Simultaneously perform convolution on the weighted mask using independent convolution kernels to calculate the normalization factor within the current receptive field. Divide the initially extracted high-dimensional features by this normalization factor for numerical compensation, and output the effective radar features.

[0040] Step S303: If the current cell contains a downsampling operation, then perform synchronous max-pooling scaling on the weighted mask; otherwise, leave it as is. The updated weighted mask, along with the output valid radar features, enters the next network layer.

[0041] Reference Figure 4 The specific macroscopic calling process in the feature extraction backbone network is as follows:

[0042] In sub-step S31, the original radar features and the initial weighted mask are synchronously input into the encoder. After passing through multiple layers of the "mask modulation convolutional unit" and downsampling operation, deep spatial semantic features containing physical prior guidance are extracted layer by layer, and finally enter the bottleneck layer.

[0043] In sub-step S32, the bottleneck layer features, after upsampling, are concatenated with the retained features at the corresponding scale from the encoding stage. Simultaneously, the maximum value of the deep and shallow masks at the corresponding scale is selected as the fusion mask. The concatenated features and the fusion mask are then input together into the "mask modulation convolutional unit" in the decoding stage for feature restoration and refinement.

[0044] In sub-step S33, after the decoder restores the pixel to its original size, pointwise convolution is used to output the probability distribution of each pixel belonging to three categories: clear sky echo, weather echo, and invalid value.

[0045] Step S4: Perform effective clutter height regression, which is accomplished by the parallel ECH regression branch. (See reference...) Figure 6 :

[0046] Sub-step S41 involves performing global average pooling on the radar features of the encoder bottleneck layer, flattening them into a one-dimensional vector that represents the high-dimensional abstract morphological texture of the radar echo.

[0047] Step S42: The one-dimensional vector, the physical feature embedding vector extracted in step S21, and the height of the condensation are spliced ​​together in the channel dimension.

[0048] In sub-step S43, the stitched features are input into a multilayer perceptron for regression calculation to predict the effective clutter height. This process corrects theoretical biases that rely solely on the height of condensation by learning the nonlinear mapping between thermodynamic and radar features.

[0049] Step S5: Perform pixel-level classification on the radar features to obtain preliminary classification results, and perform dynamic post-processing on the preliminary classification results according to the effective clutter height to generate the final cloud radar echo classification results.

[0050] In step S5, the classification results are post-processed based on the predicted effective clutter height. (Refer to...) Figure 7 Specifically, it includes:

[0051] In sub-step S51, the network output head performs classification mapping on the radar features to obtain preliminary classification results including meteorological echoes and clear sky echoes.

[0052] Sub-step S52 involves performing a global morphological opening operation on the preliminary classification result and subtracting the results before and after the operation to obtain a candidate noise set. This noise set contains isolated and tiny spots, which have not yet been distinguished as clear-sky echoes or meteorological echoes.

[0053] Sub-step S53: Determine the dynamic filtering region based on the predicted effective clutter height sequence. The normalized effective clutter height is denormalized to the physical height, which serves as the upper limit for filtering at each time step. The region below this height is the dynamic filtering region.

[0054] Sub-step S54 involves performing a spatial intersection operation between the candidate noise set and the dynamic activity region to obtain the final noise set. That is, only candidate noise located below the effective clutter height is identified as clear-sky echo and discarded.

[0055] In sub-step S55, based on the final noise mask, the cloud radar base data values ​​at the corresponding locations are set to invalid values, while the remaining data are retained as real meteorological echoes, generating high-quality cloud radar echo classification results.

[0056] Furthermore, this invention employs a joint loss function for end-to-end training. A joint loss function incorporating focal loss and mean squared error is constructed and jointly trained using expert-annotated cloud radar data. The classification loss optimizes the classification task to address the sample imbalance problem, while the regression loss optimizes the prediction of the effective clutter height. Specifically, the true effective clutter height is obtained by extracting connected components from the radar reflectivity factor sequence and using the heights of discontinuities that disrupt vertical continuity and have a spacing greater than 500 meters as the boundary of the true effective clutter height at that moment.

[0057] In this embodiment of the invention, the model is trained using the Adam optimizer in conjunction with a cosine annealing learning rate scheduling strategy, with an initial learning rate of 0.001. The training data employs a non-uniform sampling strategy based on sample difficulty weighting. The sample difficulty score is calculated by comprehensively evaluating factors such as class balance, vertical envelope overlap, and temporal instability. The sampling probability is adjusted in conjunction with a seasonal factor to increase the frequency of high-difficulty samples during training, thereby improving the model's robustness to complex scenarios.

Claims

1. A millimeter-wave cloud radar echo classification method based on physics-guided deep learning, characterized in that, The method includes: Multi-source data preprocessing: Millimeter-wave cloud radar base data and co-located ground observation data are time-aligned and height-truncated for standardization, and model input sample sequences are constructed through sliding window sampling; Extracting physical constraint thermodynamic features: The boundary layer thermodynamic state evolution embedding vector is extracted from the aligned ground temperature, relative humidity and precipitation time series data through a one-dimensional convolutional encoder. Combined with sine and cosine position coding and lifting condensation height, a thermodynamic-spatial coupling feature map is generated, and a clear sky echo prior probability map is output through a confidence head. Radar spatiotemporal feature coding and echo classification: The prior probability map is multiplied with the initial radar effective value mask to generate a weighted mask. The radar reflectivity factor, radial velocity, spectral width and linear depolarization ratio are jointly convolutionally encoded and decoded under the weighted mask modulation using pseudo-3D convolutional blocks, and the echo classification results are output pixel by pixel. Effective clutter height regression: The encoder bottleneck layer features are concatenated with the thermodynamic embedding vector and the lifting condensation height after global average pooling, and the normalized predicted value of the effective clutter height at each time step is output by the multilayer perceptron. Dynamic post-processing based on effective clutter height: The predicted effective clutter height sequence is denormalized to a physical height limit. A global morphological opening operation is performed on the preliminary classification results to obtain a candidate noise set. The final noise set is determined by the spatial intersection operation between the candidate noise set and the dynamic active region and then filtered out.

2. The millimeter-wave cloud radar echo classification method based on physics-guided deep learning according to claim 1, characterized in that, The preprocessing of multi-source data includes the following sub-steps: The millimeter-wave cloud radar data is truncated or filled into a fixed range database in the vertical direction, and the radar data is strictly matched with the ground observation data in the time dimension with a one-minute benchmark. Linear interpolation is used to fill in the missing ground observation data for short periods, and the corresponding radar samples are discarded for missing data for long periods. At the same time, the height of the condensation is calculated. The processed daily data is sliced ​​using a sliding window with a preset window length and step size to generate a sample sequence of fixed time length.

3. The millimeter-wave cloud radar echo classification method based on physics-guided deep learning according to claim 1, characterized in that, The extraction of physically constrained thermodynamic features includes the following sub-steps: Minute-level observation sequences of ground temperature, relative humidity, and precipitation are input into a two-layer cascaded one-dimensional convolutional neural network. The first layer has a kernel size of 7 and padding of 3, and the second layer has a kernel size of 5 and padding of 2. Each layer is activated by ReLU and batch normalized in sequence, and the output is a physical feature embedding vector of the same length as the sequence. A fixed position encoding matrix is ​​generated based on sine and cosine functions, and discrete distance library indices are mapped to high-dimensional vertical relative position information vectors. The height of the condensation is extended in the vertical dimension and concatenated with the previously output physical feature embedding vector and position code along the channel dimension to generate a thermodynamic-spatial coupled feature map. The coupled feature map is nonlinearly transformed by a confidence head consisting of three layers of pointwise convolutions, and then activated by Sigmoid to output a prior probability map of clear sky echo with values ​​ranging from zero to one.

4. The millimeter-wave cloud radar echo classification method based on physics-guided deep learning according to claim 1, characterized in that, The radar spatiotemporal feature encoding and echo classification are performed by an encoder-decoder backbone network containing multiple masked modulation convolutional units. The internal execution steps of each masked modulation convolutional unit include: Feature modulation: The input weighted mask is multiplied element-wise with the input radar features to generate soft-gated modulated radar features. Convolution operation is performed on the modulated radar features to initially extract high-dimensional features. Confidence compensation: Convolve simultaneously on the weighted mask using independent convolution kernels, calculate the normalization factor, divide the initially extracted high-dimensional features by the normalization factor for numerical compensation, and output effective radar features; Mask passing: If the current network layer contains a downsampling operation, the weighted mask is synchronously max-pooled and scaled, and the updated weighted mask is passed to the next network layer along with the effective radar features.

5. The millimeter-wave cloud radar echo classification method based on physics-guided deep learning according to claim 1, characterized in that, The specific process of effective clutter height regression is as follows: The encoder bottleneck layer features are globally averaged and pooled along the vertical height dimension to compress them into a one-dimensional time series vector. This vector is then concatenated along the channel dimension with the thermodynamic embedding vector from the physical encoder and the normalized lift-condensation height sequence. The input is a multilayer perceptron containing one-dimensional convolution and ReLU activation. After Sigmoid activation, the output is a normalized prediction sequence of effective clutter height with values ​​ranging from zero to one.

6. The millimeter-wave cloud radar echo classification method based on physics-guided deep learning according to claim 1, characterized in that, The specific process of the dynamic post-processing based on effective clutter height is as follows: The initial classification result output by the model is converted into a binary mask. A morphological opening operation is performed on the binary mask, and the difference between the opening operation result and the original binary mask is defined as the candidate noise set. The effective clutter height normalization sequence predicted by the model is denormalized into a physical height limit sequence in units of distance library, and a dynamic active region is defined at each time step, which contains all pixels below the height limit. Perform spatial intersection operation between the candidate noise set and the dynamic active region. Only candidate noise points located below the effective clutter height at the current time are identified as clutter and filtered out. Candidate noise points located above the effective clutter height are retained as meteorological echoes.

7. The millimeter-wave cloud radar echo classification method based on physics-guided deep learning according to claim 1, characterized in that, The method employs a joint loss function to simultaneously train both the echo classification task and the effective clutter height regression task. The joint loss function is a weighted sum of the classification loss and the regression loss. The classification loss uses focus loss to address the class imbalance problem of echo samples, while the regression loss uses mean squared error loss to calculate the error between the predicted height and the true height.

8. The millimeter-wave cloud radar echo classification method based on physics-guided deep learning according to claim 1, characterized in that, The true value label of the effective clutter height is calculated from manually labeled classification labels by the main connected component extraction method. The specific process is as follows: on the reflectivity factor profile at each time, the first break point of the clear sky echo label in the vertical direction is found as the truncation index of the main connected layer. The break criterion is that the height distance between adjacent clear sky echo labels exceeds a preset threshold. The specified quantile height of the connected component below the truncation is taken as the true value of the effective clutter height at that time.

9. A millimeter-wave cloud radar echo classification system based on physics-guided deep learning, characterized in that, The system includes: The multi-source data preprocessing module is configured to perform time alignment and height truncation standardization on cloud radar base data and co-located ground observation data, and construct the model input sample sequence through sliding window sampling. The physical constraint gating module is configured to extract thermodynamic state evolution features from ground temperature, relative humidity and precipitation time series data, and generate a clear sky echo prior probability map by combining location coding and lifting condensation height. The radar feature encoding and classification module is configured to use pseudo-3D convolution to perform joint spatiotemporal convolutional encoding and decoding of radar multi-channel features under prior probability map weighted mask modulation, and output pixel-by-pixel echo classification results and effective clutter height prediction values. The dynamic post-processing module is configured to construct a dynamic active region based on the predicted effective clutter height sequence, and to perform adaptive noise filtering on the preliminary classification results through the intersection operation of morphological opening operation and dynamic height constraint.

10. The millimeter-wave cloud radar echo classification system based on physics-guided deep learning according to claim 9, characterized in that, After the prior probability map is generated, the physical constraint gating module multiplies the probability map with the initial radar effective value mask through a soft gating mechanism to generate a weighted mask. In each mask modulation convolutional unit of the backbone network, the weighted mask is propagated forward in coordination with the radar features and undergoes numerical compensation and size scaling simultaneously, thereby realizing continuous guidance of the thermodynamic prior for radar feature extraction at various scales of encoding, downsampling, upsampling, and decoding.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the millimeter-wave cloud radar echo classification method based on physical guided deep learning as described in any one of claims 1-8.

12. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the millimeter-wave cloud radar echo classification method based on physical guided deep learning as described in any one of claims 1-8.

Images