Cloud radar clear sky echo recognition method based on deep learning and visual fusion

Abstract

The invention discloses a cloud radar clear sky echo identification method based on deep learning and visual fusion, and belongs to the field of meteorological radar detection, and the method comprises the steps: obtaining a millimeter wave cloud radar observation data set, carrying out the preprocessing of the millimeter wave cloud radar observation data set, and obtaining a preprocessing data set; constructing a recognition network based on a one-dimensional convolutional neural network and the preprocessed data set, and optimizing the recognition network through an isolated clutter noise point removal method based on connected region analysis to obtain a one-dimensional U-Net clear sky and meteorological echo intelligent classification model; and processing real-time data based on the one-dimensional U-Net clear sky and meteorological echo intelligent classification model to obtain an identification result. The invention provides an effective data post-processing method so as to filter clear sky clutters and continuous suspension clutters of a surface layer, reserve meteorological echoes related to cloud and rainfall and improve the quality of radar data.

Description

Technical Field

[0001] The present invention belongs to the field of meteorological radar detection, and in particular relates to a cloud radar clear sky echo recognition method based on deep learning and visual fusion. Background Art

[0002] In the field of meteorological radar detection, millimeter-wave cloud radar is a key technology that has attracted attention for its ability to provide cloud structure information with high temporal and spatial resolution. Millimeter-wave cloud radar detects meteorological targets with high precision by emitting high-frequency electromagnetic waves, and is particularly suitable for the refined detection of meteorological targets such as precipitation, clouds and fog. This technology uses the backscattering of electromagnetic waves by meteorological particle groups to obtain the echo intensity, radial velocity, velocity spectrum width and polarization information of meteorological targets, and then inverts them to obtain cloud characteristic data products.

[0003] However, millimeter-wave cloud radar is designed to be sensitive to fine particles, which makes it advantageous in detecting clouds and precipitation, but also susceptible to interference from non-meteorological targets. This structural feature leads to the generation of clutter, and algorithm limitations: Existing clutter identification and filtering algorithms often rely on fixed thresholds that may not be applicable to all regions and meteorological conditions, resulting in limited accuracy and universality of clutter identification. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a cloud radar clear sky echo recognition method based on deep learning and visual fusion, comprising:

[0005] Acquire a millimeter-wave cloud radar observation data set, and preprocess the millimeter-wave cloud radar observation data set to obtain a preprocessed data set;

[0006] A recognition network is constructed based on a one-dimensional convolutional neural network and the preprocessed data set, and the recognition network is optimized by an isolated clutter noise point removal method based on connected area analysis to obtain a one-dimensional U-Net clear sky and weather echo intelligent classification model;

[0007] The real-time data is processed based on the one-dimensional U-Net clear sky and weather echo intelligent classification model to obtain calculation results.

[0008] Preferably, the process of obtaining the preprocessed data set includes:

[0009] Plotting five radar observation variables of the millimeter wave cloud radar, and selecting some of the meteorological and non-meteorological echo observation cases determined based on the radar observation variables;

[0010] Manually calibrate the radar bins of the meteorological and non-meteorological echo observation cases, convert the calibration results into one-dimensional variables, and generate pre-training samples;

[0011] Building a decision tree-based additive classification model based on the pre-trained samples, inputting the millimeter-wave cloud radar observation data set into the additive classification model, and generating a calculation result;

[0012] The calculation result is restored into a two-dimensional matrix containing height and time information to generate a preliminary judgment result;

[0013] A Bayesian classifier and a low-pass filter are constructed based on the preliminary discrimination result, and the preprocessed data set is acquired based on the Bayesian classifier and the low-pass filter.

[0014] Preferably, the five radar observation variables include: radar reflectivity factor data, radial velocity data, velocity spectrum width data, linear depolarization ratio data, and signal-to-noise ratio data.

[0015] Preferably, the process of acquiring the preprocessed data set based on the Bayesian classifier and the low-pass filter includes:

[0016] Remove background noise and meteorological echo from the actual radar observation data to obtain cleaned data;

[0017] Obtaining a multidimensional probability density function distribution of the cleaned data points based on the preliminary discrimination results, training a Bayesian classifier based on the multidimensional probability density function distribution, and constructing a low-pass filter at the same time;

[0018] The cleaned data is identified based on the Bayesian classifier and the low-pass filter to obtain the preprocessed data set.

[0019] Preferably, the process of constructing a recognition network based on a one-dimensional convolutional neural network and the preprocessed data set includes: converting a U-Net network into a one-dimensional convolutional structure, inputting the preprocessed data set as a multi-channel value into the one-dimensional convolutional structure, and obtaining the recognition model.

[0020] Preferably, the training parameters of the one-dimensional convolution structure include: setting a decreasing learning rate, using a cross entropy loss function, setting the batch size to 32, and completing the algorithm construction after 50 iterations on a 160-core CPU.

[0021] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a cloud radar clear sky echo recognition method based on deep learning and visual fusion when executing the computer program.

[0022] On the other hand, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements a cloud radar clear sky echo recognition method based on deep learning and visual fusion.

[0023] Compared with the prior art, the present invention has the following advantages and technical effects:

[0024] 1. Improve the accuracy of clutter identification: By developing new algorithms, we can reduce the interference of clutter generated by non-meteorological targets (such as insects, aerosols and turbulence) on cloud radar signals, ensuring that radar data can more accurately reflect actual meteorological conditions.

[0025] 2. Enhanced data quality control: Provide an effective data post-processing method to filter out clear sky clutter and continuous suspension clutter in the near-ground layer, while retaining meteorological echoes related to clouds and precipitation, improving the quality of radar data.

[0026] 3. Optimize the universality and adaptability of the algorithm: Develop a set of clutter identification and filtering algorithms that can adapt to different regions and meteorological conditions, improve the universality and adaptability of the algorithm, and enable it to maintain high accuracy and efficiency in different regions and climate conditions.

[0027] 4. Improve the application value of radar data: By reducing the impact of clutter, the application value of radar data can be improved so that it can better serve fields such as weather, climate research and environmental monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 This is a flow chart of a method for identifying clear sky echoes from cloud radar based on deep learning and visual fusion according to an embodiment of the present invention;

[0030] Figure 2 Schematic diagram of a one-dimensional U-Net convolutional neural network structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0032] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Embodiment 1

[0034] like Figure 1-2 As shown, this embodiment provides a cloud radar clear sky echo recognition method based on deep learning and visual fusion, including:

[0035] Acquire a millimeter-wave cloud radar observation data set, and preprocess the millimeter-wave cloud radar observation data set to obtain a preprocessed data set;

[0036] A recognition network is constructed based on a one-dimensional convolutional neural network and the preprocessed data set, and the recognition network is optimized by an isolated clutter noise point removal method based on connected area analysis to obtain a one-dimensional U-Net clear sky and weather echo intelligent classification model;

[0037] The real-time data is processed based on the one-dimensional U-Net clear sky and weather echo intelligent classification model to obtain the recognition result.

[0038] The training data set involved in this embodiment is composed of millimeter wave cloud radar THI scanning observation data, which includes five radar variables, namely radar reflectivity factor (Z), radial velocity (V), velocity spectrum width (SW), linear depolarization ratio (LDR), and signal-to-noise ratio (SNR). Among them, the radar reflectivity factor, usually in dBZ, is a measure of the ability of clouds and precipitation particles to reflect electromagnetic waves emitted by radar. It reflects the density and size distribution of cloud and precipitation particles and is closely related to precipitation intensity. The larger the reflectivity factor, the stronger the echo signal, which usually means that there are more and larger particles in the cloud layer or precipitation area, and the precipitation intensity is also greater. Radial velocity refers to the speed of the meteorological target relative to the radar, that is, the velocity component of the target along the direction of the radar beam. It is usually in m / s. The radial velocity can be positive (indicating that the target moves toward the radar) or negative (indicating that the target is away from the radar). This parameter is very important for analyzing the motion characteristics of wind fields and precipitation particles. Velocity spectrum width refers to the variance of the Doppler velocity, which reflects the width of the Doppler velocity distribution, that is, the discreteness of the particle velocity. It can provide information about the size distribution and shape of cloud and precipitation particles. A larger spectral width may indicate the presence of particles of different sizes and falling speeds. The linear depolarization ratio is a parameter that describes the scattering characteristics of cloud and precipitation particles to electromagnetic waves of different polarization states. It is defined as the logarithm of the ratio of orthogonal polarization to co-polarization signals, usually expressed in dB. LDR is very useful for distinguishing different types of cloud and precipitation particles (such as raindrops, ice crystals, and snowflakes) because particles of different shapes and structures have different scattering characteristics for electromagnetic waves. The signal-to-noise ratio refers to the ratio of the received target signal intensity to the background noise intensity, which is used to measure the strength and detectability of the target signal. In millimeter-wave radar, a high signal-to-noise ratio means that the intensity of the target signal is greater relative to the background noise, and the target is easier to detect. The signal-to-noise ratio is one of the key indicators for evaluating the performance of a radar system. The technical solution of this embodiment is included in Figure 1 , specifically including:

[0039] True value calibration of millimeter wave cloud radar observation dataset:

[0040] This embodiment first preprocesses the millimeter-wave cloud radar observation data set, including analyzing the radar characteristics of all samples in the training set, and manually selecting clear sky echo samples and meteorological echo samples based on the radar echo characteristics. Secondly, the selected samples are randomly sorted in the time series and expanded into pre-training samples for each radar bin, and then these manually calibrated training samples are used to construct a decision tree-based additive classification model. The trained model is used to classify and distinguish the radar bin observations of all observation samples one by one, and the discrimination results are restored to a two-dimensional matrix containing height and time information, and finally these two-dimensional matrices are used as the pre-calibrated true value results. The specific steps are:

[0041] Step 1: Plot the five radar observation variables of the millimeter-wave cloud radar to understand the radar echo characteristics of clouds and clutter. By comparison, it can be found that compared with meteorological echoes such as clouds and precipitation, clear sky clutter has a lower radar reflectivity factor, a lower velocity spectrum width, and a higher linear depolarization ratio. These three obvious characteristics can be used to distinguish some meteorological echoes from non-meteorological echoes. However, when these two types of echoes are similar in radar observation echo results, especially for weak echo signals with a radar reflectivity factor less than -20dBZ, a fixed single threshold cannot effectively separate the two types of echoes. In addition, the height of the clutter distribution is mainly concentrated between 0-3.5km.

[0042] Step 2: Based on radar reflectivity factor, velocity spectrum width, linear depolarization ratio and other information, select some confirmed meteorological and non-meteorological echo observation cases manually, calibrate the radar bins containing these cases manually, expand these calibrated results into one-dimensional variables, and then use the one-dimensional calibration results of each radar bin without time and height information to train the decision tree-based additive classification model. After the model training is completed, the model is used to further distinguish all radar observation samples, and the discrimination results are restored to a two-dimensional matrix containing time and height information to achieve preliminary judgment of meteorological targets and non-meteorological targets.

[0043] Step 3: Train the Bayesian classifier and combine it with low-pass filtering to achieve accurate clutter identification. First, remove the background noise and non-low cloud (height greater than 3.5km) meteorological echoes such as precipitation, melt layer, and high-level cloud from the actual radar observations; then use the preliminary discrimination results of step 2 to mark the low cloud and clutter signals observed by the radar, and divide the marked sample points into different times, heights, and seasons, and obtain their multidimensional probability density function distribution based on radar reflectivity factor, velocity spectrum width, and linear depolarization ratio; use the obtained multidimensional probability density function to train the Bayesian classifier so that it can capture the characteristics of low cloud and clutter signals under different conditions; the trained Bayesian classifier can be used to separate low cloud signals and clutter when only radar observations are input. Although the multidimensional probability density function provides a more detailed and comprehensive description of the difference between low cloud and clutter, the Bayesian classifier can only distinguish low cloud signals from clutter when all radar input discriminants are available. However, in practice, not all radar discriminants may be available in the range library of a certain radar. For example, the depolarization ratio can only be calculated when both the horizontal polarization and vertical polarization reflectivity factors are observed, which makes some radar distance libraries that lack observations unable to be judged by the Bayesian classifier. Therefore, considering that clouds appear continuously in time and space, and clutter is more chaotic than cloud signals, a low-pass filter is finally used to identify these radar distance libraries that are not judged by the Bayesian classifier. In addition, the preliminary calibration results obtained in step 2 have poor recognition effects in some weak cloud and clutter mixed areas, and some artificial judgments are added here to remove these effects.

[0044] Train the neural network model to achieve intelligent echo classification:

[0045] After obtaining reasonably accurate cloud and clutter true value samples, in order to avoid the dependence on manual calibration when new observations appear, a more objective and intelligent cloud and clutter classification method needs to be designed. Here, the U-structure Neural Network (U-Net) convolutional neural network architecture in the semantic segmentation model is modified to a one-dimensional structure (see Figure 2 ), compared with the previous radar bin-by-radar bin training method, this method can further consider the height information of the entire radar profile, and compared with the two-dimensional U-Net network results, the modified model can use the training data more efficiently. The specific steps are as follows:

[0046] Step 1: Construction of one-dimensional convolutional neural network and algorithm training scheme. Based on the above high-precision label information, the classic U-structure Neural Network (U-Net) convolutional neural network architecture is modified to a one-dimensional structure to consider the recognition results of the entire radar profile and efficiently utilize the training data. The five radar observation variable information in the training data set is input into the model as multi-channel values. By setting a decreasing learning rate, using the cross entropy loss function, and setting the batch size to 32, the algorithm construction is completed after 50 iterations on a 160-core CPU. The final recognition algorithm can output the corresponding recognition results in units of profiles. It should be noted that we did not input profile information with an altitude greater than 3.5km because there is almost no clutter above this altitude.

[0047] Step 2: Isolated clutter noise removal method based on connected area analysis. Based on the constructed one-dimensional U-Net clear sky and weather echo intelligent classification model, it was found that there were certain isolated clutter noise points during the test. In order to solve this problem, the discrimination results of the model were further post-processed. By analyzing the connected area of ​​the discrimination results, isolated clutter noise points with an area less than 30 were removed.

[0048] This embodiment has the following advantages and positive effects:

[0049] Improved accuracy of clutter identification: By developing new algorithms, the interference of clutter generated by non-meteorological targets (such as insects, aerosols and turbulence) on cloud radar signals is reduced, ensuring that radar data can more accurately reflect actual meteorological conditions.

[0050] Enhanced Data Quality Control: An effective data post-processing method is provided to filter out clear-air clutter and continuous suspension clutter near the ground layer, while retaining meteorological echoes associated with clouds and precipitation, improving the quality of radar data.

[0051] Optimize the universality and adaptability of the algorithm: Develop a set of clutter identification and filtering algorithms that can adapt to different regions and meteorological conditions, improve the universality and adaptability of the algorithm, and enable it to maintain high accuracy and efficiency in different regions and climate conditions.

[0052] Improving the application value of radar data: By reducing the impact of clutter, the application value of radar data is improved, enabling it to better serve fields such as weather warning, climate research and environmental monitoring.

[0053] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A cloud radar clear sky echo recognition method based on deep learning and visual fusion, characterized in that: include: Acquire a millimeter-wave cloud radar observation data set, and preprocess the millimeter-wave cloud radar observation data set to obtain a preprocessed data set; A recognition network is constructed based on a one-dimensional convolutional neural network and the preprocessed data set, and the recognition network is optimized by an isolated clutter noise point removal method based on connected area analysis to obtain a one-dimensional U-Net clear sky and weather echo intelligent classification model; The real-time data is processed based on the one-dimensional U-Net clear sky and weather echo intelligent classification model to obtain the recognition result.

2. The method according to claim 1, characterized in that The process of obtaining the preprocessed data set includes: Plotting five radar observation variables of the millimeter wave cloud radar, and selecting some of the meteorological and non-meteorological echo observation cases determined based on the radar observation variables; Manually calibrate the radar bins of the meteorological and non-meteorological echo observation cases, convert the calibration results into one-dimensional variables, and generate pre-training samples; Building a decision tree-based additive classification model based on the pre-trained samples, inputting the millimeter-wave cloud radar observation data set into the additive classification model, and generating a calculation result; The calculation result is restored into a two-dimensional matrix containing height and time information to generate a preliminary judgment result; A Bayesian classifier and a low-pass filter are constructed based on the preliminary discrimination result, and the preprocessed data set is acquired based on the Bayesian classifier and the low-pass filter.

3. The method according to claim 2, characterized in that The five radar observation variables include: radar reflectivity factor data, radial velocity data, velocity spectrum width data, linear depolarization ratio data, and signal-to-noise ratio data.

4. The method according to claim 2, characterized in that: The process of obtaining the preprocessed data set based on the Bayesian classifier and the low-pass filter includes: Remove background noise and meteorological echo from the actual radar observation data to obtain cleaned data; Obtaining a multidimensional probability density function distribution of the cleaned data points based on the preliminary discrimination results, training a Bayesian classifier based on the multidimensional probability density function distribution, and constructing a low-pass filter at the same time; The cleaned data is identified based on the Bayesian classifier and the low-pass filter to obtain the preprocessed data set.

5. The method according to claim 1, characterized in that The process of constructing a recognition network based on a one-dimensional convolutional neural network and the preprocessed data set includes: converting a U-Net network into a one-dimensional convolutional structure, inputting the preprocessed data set as a multi-channel value into the one-dimensional convolutional structure, and obtaining the recognition model.

6. The method according to claim 5, characterized in that The training parameters of the one-dimensional convolution structure include: setting a decreasing learning rate, using a cross entropy loss function, setting the batch size to 32, and completing the algorithm construction after 50 iterations on a 160-core CPU.

7. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 6 is implemented.

Images