Millimeter wave cloud radar sidelobe echo filtering method based on clustering algorithm

Abstract

The invention provides a millimeter wave cloud radar sidelobe echo filtering method based on a clustering algorithm, and the method comprises the steps: carrying out the quality control of cloud radar data, and obtaining a quality control data set; analyzing the quality control data set through an image neighborhood analysis method to obtain a cloud layer data block; screening the cloud layer data blocks through a gradient detection method to obtain cloud echo data; the cloud echo data comprises side lobe cloud echo data and side lobe-free cloud echo data; calculating the optimal clustering number of the cloud echo data with side lobes; constructing a sidelobe echo recognition model according to the optimal clustering number; and performing data rejection operation on the cloud echo data through the sidelobe echo recognition model to obtain sidelobe echo filtered data. According to the method, filtering of millimeter wave cloud radar sidelobe echoes is realized through a clustering algorithm, and the filtering effect of the cloud radar sidelobe echoes is greatly improved.

Description

Technical Field

[0001] The invention relates to the technical field of cloud sidelobe echo filtering, and in particular to a millimeter wave cloud radar sidelobe echo filtering method based on a clustering algorithm. Background Art

[0002] At present, most millimeter-wave cloud radars use a vertical pointing detection method to continuously detect the vertical profiles of various non-precipitation clouds and weak precipitation clouds in the overhead space, and use pulse compression technology to improve the detection capability of solid-state cloud radars. However, pulse compression technology will bring certain "side effects", that is, it will cause range side lobes. At present, improving the signal processing technology after pulse compression is a way to eliminate sidelobe signals. The existing technology uses the partial correlation method to eliminate sidelobe echoes. The existing technology also uses a modified pseudo-random code to improve the success rate of sidelobe echo elimination, but at the same time generates redundant signals. The existing technology also uses pseudo-random code phase modulation technology to treat the range sidelobe as systematic interference, and by deducting this systematic interference, the purpose of eliminating sidelobe echoes is achieved. In terms of radar data, sidelobe echoes are generally judged by comparing the signal or echo strength of different range libraries. The existing technology usually uses 30dB as the judgment threshold, and judges by detecting whether the echo signal difference within the sidelobe influence range exceeds the threshold. The existing technology also uses power spectrum data and adopts fusion algorithms of different observation modes to filter out range sidelobe echoes. The prior art also sets a judgment range and a threshold value through actual radar parameters to perform sidelobe filtering.

[0003] However, although the above-mentioned related technologies have a certain effect on filtering sidelobe echoes, there are still problems such as different radar models and different threshold parameter settings, which lead to poor filtering effects. Therefore, it is very necessary to design a millimeter-wave cloud radar sidelobe echo filtering method based on clustering algorithm. Summary of the invention

[0004] The purpose of the present invention is to provide a millimeter-wave cloud radar sidelobe echo filtering method based on a clustering algorithm, so as to realize the filtering of the millimeter-wave cloud radar sidelobe echo through the clustering algorithm, so as to greatly improve the cloud radar sidelobe echo filtering effect.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for filtering sidelobe echoes of a millimeter-wave cloud radar based on a clustering algorithm comprises the following steps:

[0007] Perform quality control on cloud radar data to obtain a quality control data set;

[0008] The quality control data set is analyzed by image neighborhood analysis method to obtain cloud layer data blocks;

[0009] The cloud layer data blocks are screened by the gradient detection method to obtain cloud echo data; the cloud echo data includes cloud echo data with side lobes and cloud echo data without side lobes;

[0010] Calculate the optimal number of clusters for echo data with sidelobe clouds;

[0011] Construct a sidelobe echo recognition model based on the optimal clustering number;

[0012] The cloud echo data is removed through the sidelobe echo recognition model to obtain the sidelobe echo filtered data.

[0013] Optionally, quality control is performed on the cloud radar data to obtain a quality control data set, and the specific steps are: quality control is performed on the echo intensity characteristics in the cloud radar data by a reflectivity factor-linear depolarization ratio double threshold method to obtain a radar reflectivity factor data set.

[0014] Optionally, the quality control data set is analyzed by an image neighborhood analysis method to obtain a cloud layer data block, and the specific steps include:

[0015] Perform binarization assignment operation on pixel points with echo in the quality control data set to obtain binary pixel points;

[0016] Perform adjacent pixel detection on the binary pixel to obtain the detection result;

[0017] Adjacent pixels are integrated according to the detection results to obtain cloud data blocks.

[0018] Optionally, the cloud layer data block is screened by a gradient detection method to obtain cloud echo data, and the specific steps are: calculating the Z value gradient difference of the data points in the cloud layer data block, and dividing the cloud layer data block into sidelobe cloud echo data and sidelobe-free cloud echo data according to the Z value gradient difference.

[0019] Optionally, the specific steps of calculating the optimal number of clusters for the sidelobe cloud echo data are: calculating the average value of the silhouette coefficients of the sidelobe cloud echo data to obtain the average silhouette coefficient, and taking the number of clusters corresponding to the average silhouette coefficient as the optimal number of clusters; the calculation formula of the silhouette coefficient is: Among them, s(i) is the silhouette coefficient of the i-th data point in the sidelobe cloud echo data, b(i) is the minimum average distance between the i-th data point and the data points of different categories in the sidelobe cloud echo data, a(i) is the average distance between the data points of the same category in the sidelobe cloud echo data, and max{} is the maximum value operation.

[0020] Optionally, a sidelobe echo recognition model is constructed according to the optimal clustering number, and the specific steps include:

[0021] Select multiple category centers according to the optimal number of clusters;

[0022] Classify each data point in the cloud echo data with side lobes according to the distance between each data point and the category center point to obtain a cloud category set;

[0023] Recalculate the category center point according to the cloud category set until the category center point no longer changes, and obtain a new category center point;

[0024] The sidelobe echo recognition model is obtained according to the center point of the new category.

[0025] Optionally, the distance between each data point in the sidelobe cloud echo data and the category center point is calculated as follows: Among them, x is the data point, x j is the value of the data point on the jth feature, d is the number of features of the data point, μ i is the center point of the i-th category, μ i,j μ i The value of the j-th feature.

[0026] Optionally, the category center point is recalculated according to the cloud category set until the category center point no longer changes, and a new category center point is obtained. The specific steps are: the sum of squared distances of the loss function is calculated according to the distances between each data point in the sidelobe cloud echo data and the category center point, until the sum of squared distances of the loss function reaches a minimum value; the calculation formula of the sum of squared distances of the loss function is: Among them, SSE is the sum of squared distances of the loss function, K is the number of categories, and C i is the i-th category.

[0027] A millimeter wave cloud radar sidelobe echo filtering system based on clustering algorithm, comprising:

[0028] Data preprocessing module: used to classify cloud radar data and obtain cloud echo data;

[0029] Model training module: used to calculate the optimal clustering number of sidelobe cloud echo data and build a sidelobe echo recognition model based on the optimal clustering number;

[0030] Classification prediction module: used to perform data elimination operation on cloud echo data through the sidelobe echo recognition model to obtain sidelobe echo filtered data.

[0031] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the millimeter-wave cloud radar sidelobe echo filtering method based on the clustering algorithm provided by the present invention comprises: performing quality control on the cloud radar data to obtain a quality control data set; analyzing the quality control data set by an image neighborhood analysis method to obtain a cloud layer data block; screening the cloud layer data block by a gradient detection method to obtain cloud echo data; the cloud echo data comprises cloud echo data with side lobes and cloud echo data without side lobes; calculating the optimal number of clusters for the cloud echo data with side lobes; constructing a sidelobe echo recognition model according to the optimal number of clusters; performing data elimination operation on the cloud echo data by the sidelobe echo recognition model to obtain sidelobe echo filtered data. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 This is a flow chart of a method for filtering sidelobe echoes of a millimeter-wave cloud radar based on a clustering algorithm according to an embodiment of the present invention;

[0034] Figure 2 is a flow chart of an image neighborhood analysis method according to an embodiment of the present invention;

[0035] Figure 3 A cloud echo data classification diagram according to an embodiment of the present invention;

[0036] Figure 4 A flow chart of constructing a sidelobe echo recognition model according to an embodiment of the present invention;

[0037] Figure 5 This is a clustering result diagram of an embodiment of the present invention;

[0038] Figure 6 Graph showing the sidelobe echo filtering result of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1 As shown, an embodiment of the present invention provides a method for filtering sidelobe echoes of a millimeter-wave cloud radar based on a clustering algorithm, comprising the following steps:

[0042] Step 100: Perform quality control on the cloud radar data to obtain a quality control data set.

[0043] Specifically, the cloud echo intensity in the cloud radar data is first obtained, and then the cloud echo intensity is quality controlled by using a median filter or a reflectivity factor-linear depolarization ratio double threshold method to obtain a radar reflectivity factor (Z) data set after quality control. This embodiment preferably uses a reflectivity factor-linear depolarization ratio double threshold method.

[0044] Step 200: Analyze the quality control data set by the image neighborhood analysis method to obtain the cloud layer data block. The specific steps are: identify the connected domain composed of pixels with the same value and adjacent positions in the quality control data set. The implementation process is as follows Figure 2 As shown:

[0045] Step 201: performing a binarization assignment operation on pixel points with echoes in the quality control data set to obtain binary pixel points.

[0046] Specifically, the entire quality control data set is binarized and assigned a value of 1 to the pixel points where there is an echo, and a value of 0 to the pixel points where there is no echo. The binarization process avoids the influence of the fluctuation of the Z value on the extraction of the connected domain.

[0047] Step 202: perform adjacent pixel detection on the binary pixel to obtain a detection result.

[0048] Specifically, the 8-neighborhood detection method is used to determine whether pixels are adjacent. The 8-neighborhood detection method refers to detecting whether there are adjacent pixels in eight positions around a pixel. This improves the recognition range of scattered pixels around cloud echoes.

[0049] Step 203: Integrate adjacent pixel points according to the detection results to obtain a cloud layer data block.

[0050] Specifically, a pixel point and its adjacent pixel points are combined into a connected domain, multiple combinations are performed to obtain multiple connected domains, and the connected domains are regarded as cloud layer data blocks.

[0051] Step 300: Filter the cloud layer data block by gradient detection method to obtain cloud echo data. The cloud echo data includes cloud echo data with side lobes and cloud echo data without side lobes.

[0052] Specifically, the radial and temporal Z value gradients of the data points in each cloud layer data block are calculated (respectively dz h and dz t The cloud layer data blocks are screened by the Z value gradient difference in each cloud layer data block. If the Z value gradient difference reaches the threshold M, the cloud layer data block is marked as cloud echo data with side lobes, otherwise it is marked as cloud echo data without side lobes. h 、dz t A training data set of cloud echoes with sidelobe features: H, H and t.

[0053] This embodiment selects the observation data of the air above the Fujian Pinghe radar from 20:00 on May 19, 2019 to 04:00 on May 20, 2019 for analysis. Figure 3 As shown, the observation data include: low-altitude stratus clouds, cumulus clouds and high-altitude cirrocumulus clouds.

[0054] Step 400: Calculate the optimal clustering number of the sidelobe cloud echo data.

[0055] Specifically, the number of cloud echo categories K is first determined, and K is evaluated using the silhouette coefficient. The calculation formula of the silhouette coefficient is:

[0056]

[0057] Among them, s(i) is the silhouette coefficient of the i-th data point in the cloud echo data with side lobes, b(i) is the minimum average distance between the i-th data point and the data points of different categories in the cloud echo data with side lobes, a(i) is the average distance of the data points of the same category in the cloud echo data with side lobes, and max{} is the maximum value operation. Then, the overall average silhouette coefficient is obtained by calculating the average value of the silhouette coefficients of all data points. The value range of the average silhouette coefficient is [-1, 1]. The closer the distance between the cloud echo data points in the same category, the farther the distance between the cloud echo data points in different categories, and the larger the average silhouette coefficient value, indicating that the clustering effect is better. Different Ks correspond to different average silhouette coefficients. The K corresponding to the maximum value of the average silhouette coefficient is the optimal number of clusters.

[0058] Step 500: Construct a sidelobe echo recognition model based on the optimal cluster number. Figure 4 As shown, including:

[0059] Step 501: Randomly select K initial cloud echo category center points.

[0060] Step 502: allocating each data point in the cloud echo data with sidelobe to the cloud category with the closest distance according to the distance between each data point in the cloud echo data with sidelobe and the category center point, to obtain a cloud category set.

[0061] Specifically, the calculation formula for the distance between each data point in the sidelobe cloud echo data and the category center point is:

[0062]

[0063] Among them, x is the data point, x j is the value of the data point on the jth feature, d is the number of features of the data point, μ i is the center point of the i-th category, μ i,j μ i The value of the j-th feature.

[0064] Step 503: recalculate the category center point according to the cloud category set, that is, calculate the mean of all data points in the cloud category, and continuously iterate steps 501 to 503 until the position of the category center point no longer changes, thereby obtaining a new category center point.

[0065] Specifically, the criterion for determining whether the category center point no longer changes is that the sum of the squares of the loss function distances between each data point in the sidelobe cloud echo data and the category center point reaches the minimum value. The calculation formula for the sum of the squares of the loss function distances is:

[0066]

[0067] Among them, SSE is the sum of squared distances of the loss function, K is the number of categories, and C i is the i-th category.

[0068] Step 504: Obtain a sidelobe echo recognition model according to the new category center point.

[0069] Specifically, Figure 5 As shown, the data points in the cloud echo data with sidelobe are clustered through the sidelobe echo recognition model to obtain the cloud echo classification label to which each data point belongs, and the data points corresponding to the sidelobes of the image in the reflectivity factor data set are manually identified and marked. Then, the proportion of the sidelobe data corresponding to the marked data points in the K cloud echo categories is counted. If the proportion of the sidelobe data reaches the threshold N, the category is assigned a sidelobe echo class label, otherwise it is assigned a sidelobe echo class label. The model is then trained through the labeling process to obtain a sidelobe echo recognition model. Among them, the category with a sidelobe echo class label is a false sidelobe echo.

[0070] Step 600: performing a data elimination operation on the cloud echo data through the sidelobe echo recognition model to obtain sidelobe echo filtered data.

[0071] Specifically, a training data set of sidelobe cloud echoes including five feature quantities, namely, Z, dzh, dzt, H and t, is input into the sidelobe echo recognition model, and the data points with sidelobe echo class labels are removed, and then the removed data is spliced ​​with the non-sidelobe cloud echo data to generate the final sidelobe echo filtered data. The sidelobe echo filtering result of this embodiment is shown in FIG. Figure 6 shown.

[0072] The embodiment of the present invention further provides a millimeter wave cloud radar sidelobe echo filtering system based on a clustering algorithm, comprising:

[0073] Data preprocessing module: used to classify cloud radar data and obtain cloud echo data;

[0074] Model training module: used to calculate the optimal clustering number of sidelobe cloud echo data and build a sidelobe echo recognition model based on the optimal clustering number;

[0075] Classification prediction module: used to perform data elimination operation on cloud echo data through the sidelobe echo recognition model to obtain sidelobe echo filtered data.

[0076] The beneficial effects of the present invention are as follows:

[0077] 1) A new cloud radar sidelobe echo filtering method based on unsupervised learning is proposed;

[0078] 2) The efficiency of sidelobe echo filtering is improved by assigning labels and identifying labels;

[0079] 3) The data quality of cloud radar and the inversion accuracy of data parameters are improved by classifying cloud radar data.

[0080] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0081] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for filtering sidelobe echoes of millimeter-wave cloud radar based on clustering algorithm, characterized in that: The steps include: Perform quality control on cloud radar data to obtain a quality control data set; Analyzing the quality control data set by an image neighborhood analysis method to obtain a cloud layer data block; The cloud layer data block is screened by a gradient detection method to obtain cloud echo data; The cloud echo data includes cloud echo data with side lobes and cloud echo data without side lobes; Calculating the optimal number of clusters of the sidelobe cloud echo data; Constructing a sidelobe echo recognition model according to the optimal clustering number; The cloud echo data is subjected to a data elimination operation through the sidelobe echo recognition model to obtain sidelobe echo filtered data.

2. The method for filtering sidelobe echoes of millimeter-wave cloud radar based on clustering algorithm according to claim 1, characterized in that: The cloud radar data is quality controlled to obtain a quality control data set. The specific steps are: the echo intensity characteristics in the cloud radar data are quality controlled by a reflectivity factor-linear depolarization ratio double threshold method to obtain a radar reflectivity factor data set.

3. The method for filtering sidelobe echoes of millimeter-wave cloud radar based on clustering algorithm according to claim 1, characterized in that: The quality control data set is analyzed by an image neighborhood analysis method to obtain a cloud layer data block, and the specific steps include: Performing a binarization operation on the pixel points with echoes in the quality control data set to obtain binarized pixel points; Performing adjacent pixel detection on the binary pixel to obtain a detection result; Adjacent pixel points are integrated according to the detection result to obtain the cloud layer data block.

4. The method for filtering sidelobe echoes of millimeter wave cloud radar based on clustering algorithm according to claim 1, characterized in that: The cloud layer data block is screened by a gradient detection method to obtain cloud echo data, and the specific steps are: calculating the Z value gradient difference of the data points in the cloud layer data block, and dividing the cloud layer data block into the sidelobe cloud echo data and the sidelobe-free cloud echo data according to the Z value gradient difference.

5. The millimeter wave cloud radar sidelobe echo filtering method based on clustering algorithm according to claim 1, characterized in that: The specific steps of calculating the optimal number of clusters of the sidelobe cloud echo data are: calculating the average value of the silhouette coefficient of the sidelobe cloud echo data to obtain the average silhouette coefficient, and taking the number of clusters corresponding to the average silhouette coefficient as the optimal number of clusters; the calculation formula of the silhouette coefficient is: Among them, s(i) is the silhouette coefficient of the i-th data point in the sidelobe cloud echo data, b(i) is the minimum average distance between the i-th data point and data points of different categories in the sidelobe cloud echo data, a(i) is the average distance between data points of the same category in the sidelobe cloud echo data, and max{} is the maximum value operation.

6. The method for filtering sidelobe echoes of millimeter wave cloud radar based on clustering algorithm according to claim 1, characterized in that: A sidelobe echo recognition model is constructed according to the optimal clustering number, and the specific steps include: Selecting multiple category center points according to the optimal number of clusters; Classifying each data point in the cloud echo data with side lobes according to the distance between each data point and the category center point to obtain a cloud category set; Recalculating the category center point according to the cloud category set until the category center point no longer changes, thereby obtaining a new category center point; The sidelobe echo recognition model is obtained according to the new category center point.

7. The method for filtering sidelobe echoes of millimeter wave cloud radar based on clustering algorithm according to claim 6, characterized in that: The calculation formula for the distance between each data point in the sidelobe cloud echo data and the category center point is: Among them, x is the data point, x j is the value of the data point on the jth feature, d is the number of features of the data point, μ i is the center point of the i-th category, μ i,j μ i The value of the j-th feature.

8. The method for filtering sidelobe echoes of millimeter wave cloud radar based on clustering algorithm according to claim 7, characterized in that: The category center point is recalculated according to the cloud category set until the category center point no longer changes, and a new category center point is obtained. The specific steps are: the sum of squared distances of the loss function is calculated according to the distances between each data point in the sidelobe cloud echo data and the category center point, until the sum of squared distances of the loss function reaches a minimum value; the calculation formula of the sum of squared distances of the loss function is: Among them, SSE is the sum of squared distances of the loss function, K is the number of categories, and C i is the i-th category.

9. A millimeter-wave cloud radar sidelobe echo filtering system based on clustering algorithm, applied to the millimeter-wave cloud radar sidelobe echo filtering method based on clustering algorithm described in any one of claims 1 to 8, characterized in that: include: Data preprocessing module: used for classifying the cloud radar data to obtain the cloud echo data; Model training module: used for calculating the optimal clustering number of the sidelobe cloud echo data, and constructing a sidelobe echo recognition model according to the optimal clustering number; Classification prediction module: used for performing data elimination operation on the cloud echo data through the sidelobe echo recognition model to obtain sidelobe echo filtered data.

Images