Meteorological radar electromagnetic interference data quality control method based on machine learning

Abstract

The invention discloses a meteorological radar electromagnetic interference data quality control method based on machine learning, and belongs to the technical field of meteorological data processing. Characteristic parameters related to electromagnetic interference in echo signal data are extracted, an interference identification model is constructed and deployed to a meteorological radar data processing flow, electromagnetic interference data are identified, the interference type and the interference mode of the electromagnetic interference data are classified and identified, an external interference source and an internal interference source are identified, and the interference identification accuracy is improved. An interference identification model is embedded into a meteorological radar data processing flow in a modularization mode, real-time discrimination and automatic marking of interference data are achieved, classification accuracy is effectively improved by identifying and marking interference types and mode characteristics of the electromagnetic interference data, comparison is carried out according to a known type library, accurate identification of typical interference types is achieved, and the identification accuracy is improved. And by identifying an external interference source and an internal interference source, a bidirectional traceability mechanism improves the operation and maintenance efficiency and the abnormal response capability of the radar station, and reduces the false alarm rate and the missing report rate.

Description

Technical Field

[0001] The present invention relates to the technical field of meteorological data processing, and in particular to a method for quality control of meteorological radar electromagnetic interference data based on machine learning. Background Art

[0002] As an important device for obtaining information on the three-dimensional structure of the atmosphere and precipitation, meteorological radar is widely used in fields such as weather monitoring, weather forecasting, and disaster warning. The quality of its observation data is directly related to the scientificity and accuracy of meteorological analysis and decision-making. However, with the frequent occurrence of electromagnetic activities such as wireless communications, power transmission, and radar navigation, the electromagnetic environment in which the meteorological radar system is located is becoming increasingly complex, and various electromagnetic interference events occur frequently, which seriously affects the accuracy and stability of radar echo data, and may even lead to missed or false reports of key weather processes.

[0003] Traditional radar electromagnetic interference identification methods mainly rely on rule setting and manual experience judgment, such as identifying interference by setting signal strength thresholds, spectrum change characteristics or time series anomalies. Such methods are usually applicable to interference situations with known patterns, but have limited ability to identify atypical or highly concealed interference, and are difficult to adapt to the increasingly complex reality of electromagnetic interference forms. In addition, although some methods have introduced automatic detection mechanisms, they lack further classification and traceability analysis of interference types and sources, resulting in incomplete data labeling and difficulty in forming a closed-loop quality control system. Summary of the invention

[0004] The purpose of the present invention is to provide a weather radar electromagnetic interference data quality control method based on machine learning to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for quality control of electromagnetic interference data of meteorological radar based on machine learning, comprising: Based on the echo signal data collected by the meteorological radar, the characteristic parameters related to electromagnetic interference are extracted through multi-dimensional analysis, and an interference identification model is constructed and trained. The interference identification model is deployed in the meteorological radar data processing flow to identify electromagnetic interference data. The characteristic parameters extracted from the radar signal are received as input for interference discrimination, and the judgment result of whether each frame or each group of data contains electromagnetic interference data is output, and the electromagnetic interference data is explicitly marked. The interference types and interference patterns of electromagnetic interference data are classified and identified, interference source tracing analysis is performed on the identified different types of electromagnetic interference data, and external and internal interference sources are identified.

[0006] Furthermore, based on the original echo signal data collected by the weather radar, characteristic parameters related to electromagnetic interference are extracted through multi-dimensional analysis, including: The meteorological radar system collects and preprocesses raw echo signal data in a conventional detection working mode, and extracts characteristic parameters in multiple dimensions based on the preprocessed echo signal data, wherein the dimensions include time domain, frequency domain and statistical dimensions; In the time domain dimension, the extracted characteristic parameters include signal duration, amplitude mutation rate and intermittent distribution characteristics; in the frequency domain dimension, the echo signal is converted from the time domain to the frequency domain, and the extracted characteristic parameters include spectral energy distribution, spectrum mutation point and bandwidth broadening degree; in the statistical dimension, the characteristic parameters extracted based on the sliding window include extreme value anomaly rate, variance change amplitude, skewness and kurtosis.

[0007] Furthermore, an interference recognition model is constructed and trained, including: Constructing a sample set for model training, wherein the sample set includes normal echo data and abnormal echo data containing electromagnetic interference; The normal echo data is selected from the radar observation data segment confirmed to be free of interference by manual or expert system; the abnormal echo data is selected from the echo data including the interference events marked in the historical records; An interference recognition model is built based on a convolutional neural network. The interference recognition model is trained using a sample set. After the training process is completed, the model parameters are solidified and saved. Based on an independent validation set that does not overlap with the sample set, the performance of the trained interference recognition model is evaluated. During the performance evaluation process, a variety of classification indicators are used to comprehensively analyze the model effect. The classification indicators include accuracy, recall rate, false alarm rate and F1-score.

[0008] Furthermore, the interference identification model is deployed into the weather radar data processing flow to identify electromagnetic interference data, including: The interference identification model that has been trained and verified is embedded in the data processing flow of the meteorological radar system in a modular form. During the model deployment process, a data interaction interface for receiving characteristic parameters is constructed; Input the characteristic parameters into the interference recognition model, perform interference discrimination operation, and output the interference recognition result corresponding to each frame or each group of echo signal data, wherein the interference recognition result includes at least one binary judgment label, and when the label value of the binary judgment label is "1", it indicates that the echo signal data has electromagnetic interference data, and when it is "0", it indicates that the echo signal data is normal data; The echo signal data determined by the model to contain electromagnetic interference data is explicitly marked, and the explicit marking includes the following marking information: label information of whether interference exists, the frame number range in which the interference occurs, the duration of the interference, the interference intensity level and the affected beam or elevation layer.

[0009] Furthermore, the electromagnetic interference data determines the quality level of the meteorological radar electromagnetic interference data, including: Extract the frame number range and duration of interference corresponding to each electromagnetic interference data; Extracting the interference frame number range where the interference signal strength exceeds the preset interference signal strength threshold from the frame number range where the interference occurs; Normalizing the interference frame number range and duration of interference corresponding to each electromagnetic interference data, and obtaining interference frame number range parameters and duration parameters after normalization; Performing difference processing on the interference frame number range parameter and the duration parameter after the normalization processing to obtain the difference between the interference frame number range parameter and the duration parameter after the normalization processing; Comparing the difference between the normalized interference frame number range parameter and the duration parameter with a preset difference threshold; Filter out electromagnetic interference data whose difference between the interference frame number range parameter and the duration parameter after normalization is lower than the difference threshold as the first electromagnetic interference data set; Filter out electromagnetic interference data whose difference between the interference frame number range parameter and the duration parameter after normalization is not less than the difference threshold as the second electromagnetic interference data set; The quality level of the meteorological radar electromagnetic interference data is determined according to the first electromagnetic interference data and the second electromagnetic interference data.

[0010] Further, determining the quality level of the meteorological radar electromagnetic interference data according to the first electromagnetic interference data and the second electromagnetic interference data includes: Extracting an impact beam or elevation layer corresponding to each first electromagnetic interference data and second electromagnetic interference data in the first electromagnetic interference data set and the second electromagnetic interference data set; Normalizing the affected beam using a standard beam to obtain normalized beam parameters corresponding to each first electromagnetic interference data and the second electromagnetic interference data; or, normalizing the elevation layer corresponding to each first electromagnetic interference data and the second electromagnetic interference data using the elevation range corresponding to the standard elevation layer to obtain normalized elevation parameters corresponding to each first electromagnetic interference data and the second electromagnetic interference data; Marking the beam parameters and elevation parameters corresponding to the first electromagnetic interference data as spatial coefficients, and obtaining a first spatial coefficient standard deviation corresponding to the first electromagnetic interference data set according to the spatial coefficients corresponding to the first electromagnetic interference data; Marking the beam parameters and elevation parameters corresponding to the second electromagnetic interference data as spatial coefficients, and obtaining a second spatial coefficient standard deviation corresponding to the second electromagnetic interference data set according to the spatial coefficients corresponding to the second electromagnetic interference data; Extracting the average value of the difference between the interference frame number range parameter and the duration parameter after normalization corresponding to the first electromagnetic interference data set; Extracting the average value of the difference between the interference frame number range parameter and the duration parameter after normalization corresponding to the second electromagnetic interference data set; Obtaining an interference index corresponding to the overall electromagnetic interference data by using the first space coefficient standard deviation and the second space coefficient standard deviation in combination with a difference average corresponding to the first electromagnetic interference data set and a difference average corresponding to the second electromagnetic interference data set; Comparing the interference index corresponding to the overall electromagnetic interference data with a preset index threshold; When the interference index corresponding to the overall electromagnetic interference data exceeds the preset index threshold, the quality level of the meteorological radar electromagnetic interference data is determined to be high, and an interference abnormality alarm is issued; When the interference index corresponding to the overall electromagnetic interference data does not exceed the preset index threshold, the quality level of the meteorological radar electromagnetic interference data is determined to be general, and the interference type and interference mode of the electromagnetic interference data are classified and identified.

[0011] Furthermore, the interference type and interference mode of the electromagnetic interference data are classified and identified, including: Based on the marked electromagnetic interference data, extract and organize the characteristic parameter vector set corresponding to the electromagnetic interference data; An interference type classification library is constructed, wherein the interference type classification library includes reference feature distribution ranges and category labels of interference types, and the interference types include periodic interference, broadband interference, pulse interference, and frequency modulation interference.

[0012] Furthermore, the interference type and interference mode of the electromagnetic interference data are classified and identified, which specifically includes: For the extracted electromagnetic interference data, an unsupervised clustering method is used to perform interference pattern aggregation analysis. The electromagnetic interference data is divided into several clusters based on feature similarity. Each cluster corresponds to an interference category. Based on the established interference type classification library, the interference category of each cluster is identified and the support vector machine classification model is used to complete the labeling. The classified and identified interference type information is added to the original echo signal data to form a triple labeling structure of "observation data-interference existence-interference type".

[0013] Furthermore, interference source tracing analysis is performed on the identified different types of electromagnetic interference data, including: Conduct correlation analysis on the detection time, spatial location and radar operation status corresponding to the electromagnetic interference data, and collect heterogeneous information including radar system operation logs, site geographic information, electromagnetic environment monitoring data, and meteorological condition records; The radar system operation log includes radar power status, beam pointing, scanning mode, equipment self-test records and alarm events; the electromagnetic environment monitoring data includes electromagnetic spectrum scanning results around the radar site, distribution of radio transmitters and operating status of nearby equipment; Data fusion is used to align and map heterogeneous information on the time axis and spatial coordinates to construct a basic data set for interference analysis; Based on the basic data set for interference analysis, a geographic information system is used to construct a spatiotemporal distribution map of interference events. The spatiotemporal distribution map is centered on the radar station, marking the time period when the interference event occurred, the azimuth and elevation layers of the corresponding beam, and the distribution density and change trend of the interference in space are represented by heat maps and trajectory maps.

[0014] Furthermore, interference source tracing analysis is performed on the identified different types of electromagnetic interference data, including: According to the interference type, frequency and intensity, statistics are compiled on the frequency of each type of interference in different time periods, the duration of interference and the distribution of intensity levels, and statistical charts are output; Based on the time-space distribution map, the path of each type of interference is traced and the potential interference radiation source area is inferred. The known external electromagnetic interference source is matched with the potential interference radiation source area. If the spatial distance, interference frequency band and interference type are consistent, it is marked as an external interference source. Identify the interference caused by abnormal equipment inside the weather radar system. Based on the equipment status change trend recorded in the operation log, determine whether there is an internal system abnormality that highly overlaps with the interference, and mark the identified abnormal equipment inside the weather radar system as an internal interference source.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention realizes real-time discrimination and automatic marking of interference data by modularly embedding the interference identification model into the meteorological radar data processing flow, significantly improving the intelligence level and data processing efficiency of the system. During operation, the model can output a clear interference existence label for each frame or each group of echo data, and mark the key parameters of the interference in a structured manner, so that subsequent data users can clearly identify the contaminated areas, avoid misuse of abnormal data, and improve data credibility.

[0016] 2. The present invention further identifies the type and pattern characteristics of interference by clustering analysis and classification identification of labeled electromagnetic interference data, thereby achieving more fine-grained interference control. The interference type classification library is based on the characteristic statistical data of a large number of interference samples, and combines clustering algorithms with support vector machine classifiers to distinguish unknown interference samples, effectively improving the classification accuracy. The interference samples are automatically divided into several pattern categories through an unsupervised clustering algorithm, and then compared with the known type library to achieve accurate identification of typical interference types, providing support for the system to further identify interference sources and formulate response strategies.

[0017] 3. The present invention integrates multi-source heterogeneous data such as radar system operation log, electromagnetic environment monitoring data and geographic information system, and performs source tracing analysis on the identified electromagnetic interference data, so as to achieve accurate positioning of external interference sources, and can also identify interference caused by internal anomalies of the system, greatly improving the system's comprehensive ability to distinguish the causes of interference. Based on the spatiotemporal label information of the interference data, combined with the azimuth, elevation layer and radar beam scanning trajectory of the radar site, the interference distribution heat map and trajectory map are drawn using the geographic information system. By comparing with parameters such as the known interference source location and electromagnetic spectrum, rapid matching and identification of external interference sources are achieved. Combined with data such as beam control, power status, and alarm records in the equipment operation log, highly overlapping internal anomalies are checked to identify possible internal interference sources. The two-way tracing mechanism improves the radar site operation and maintenance efficiency and abnormal response capability, and reduces the false alarm and missed alarm rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic flow chart of the meteorological radar electromagnetic interference data quality control method of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] See also Figure 1 , the present invention provides the following technical solutions: The method for quality control of electromagnetic interference data of meteorological radar based on machine learning includes: Based on the echo signal data collected by the meteorological radar, the characteristic parameters related to electromagnetic interference are extracted through multi-dimensional analysis, and an interference identification model is constructed and trained. The interference identification model is deployed in the meteorological radar data processing flow to identify electromagnetic interference data. The characteristic parameters extracted from the radar signal are received as input for interference discrimination, and the judgment result of whether each frame or each group of data contains electromagnetic interference data is output, and the electromagnetic interference data is explicitly marked. The interference types and interference patterns of electromagnetic interference data are classified and identified, interference source tracing analysis is performed on the identified different types of electromagnetic interference data, and external and internal interference sources are identified.

[0021] Based on the original echo signal data collected by the meteorological radar, characteristic parameters related to electromagnetic interference are extracted through multi-dimensional analysis, including: The meteorological radar system collects and preprocesses raw echo signal data in a conventional detection working mode, and extracts characteristic parameters in multiple dimensions based on the preprocessed echo signal data, wherein the dimensions include time domain, frequency domain and statistical dimensions; In the time domain dimension, the extracted characteristic parameters include signal duration, amplitude mutation rate and intermittent distribution characteristics; in the frequency domain dimension, the echo signal is converted from the time domain to the frequency domain, and the extracted characteristic parameters include spectral energy distribution, spectrum mutation point and bandwidth broadening degree; in the statistical dimension, the characteristic parameters extracted based on the sliding window include extreme value anomaly rate, variance change amplitude, skewness and kurtosis.

[0022] In the above embodiment, by extracting multi-dimensional feature parameters from the original echo signal data collected by the meteorological radar, the system's recognition accuracy and anti-interference ability for electromagnetic interference are effectively improved, and typical interference features are extracted in the time domain, frequency domain, and statistical dimensions, respectively, so that the system can understand the abnormal performance of interference signals from different signal representation levels. For example, time domain features such as signal mutations and intermittency reflect the suddenness and discontinuity of interference, frequency domain features such as abnormal spectral energy distribution reveal the abnormal occupation of frequency resources, and the statistical dimension reveals the overall distribution law of interference signals and the difference from normal signals. This multi-dimensional, all-round feature parameter extraction method avoids the recognition blind spots and misjudgment risks brought about by reliance on single-dimensional signal features, and enhances the model's ability to discriminate complex interference samples.

[0023] Build and train the interference recognition model, including: Constructing a sample set for model training, wherein the sample set includes normal echo data and abnormal echo data containing electromagnetic interference; The normal echo data is selected from the radar observation data segment confirmed to be free of interference by manual or expert system; the abnormal echo data is selected from the echo data including the interference events marked in the historical records; An interference recognition model is built based on a convolutional neural network. The interference recognition model is trained using a sample set. After the training process is completed, the model parameters are solidified and saved. Based on an independent validation set that does not overlap with the sample set, the performance of the trained interference recognition model is evaluated. During the performance evaluation process, a variety of classification indicators are used to comprehensively analyze the model effect. The classification indicators include accuracy, recall rate, false alarm rate and F1-score.

[0024] In the above embodiment, the interference recognition model based on the convolutional neural network is constructed and trained, which not only effectively realizes the automatic recognition and classification of electromagnetic interference data, but also breaks through the technical bottleneck of low recognition accuracy and weak generalization ability of the traditional rule-based or threshold method. By introducing manually annotated normal and interference samples to build a high-quality training set, and combining non-overlapping verification sets for performance evaluation, the scientificity and robustness of model training can be guaranteed. In particular, based on the convolutional neural network (CNN) structure, the model has the ability to efficiently learn local feature structures, can quickly locate the characteristic area of ​​electromagnetic interference in large-scale echo signals, and achieve high-precision classification and discrimination. In addition, the comprehensive use of multiple index evaluation standards such as accuracy, recall rate, false alarm rate and F1-score during model training helps to comprehensively analyze the performance of the model and facilitate subsequent optimization and adjustment. The model parameters obtained by the final training are solidified and saved, which provides a stable and reliable technical guarantee for actual deployment, and effectively supports the real-time judgment requirements of the radar system for electromagnetic interference data in actual operation.

[0025] Deploy interference identification models into the weather radar data processing flow and identify electromagnetic interference data, including: The interference identification model that has been trained and verified is embedded in the data processing flow of the meteorological radar system in a modular form. During the model deployment process, a data interaction interface for receiving characteristic parameters is constructed; Input the characteristic parameters into the interference recognition model, perform interference discrimination operation, and output the interference recognition result corresponding to each frame or each group of echo signal data, wherein the interference recognition result includes at least one binary judgment label, and when the label value of the binary judgment label is "1", it indicates that the echo signal data has electromagnetic interference data, and when it is "0", it indicates that the echo signal data is normal data; The echo signal data determined by the model to contain electromagnetic interference data is explicitly marked, and the explicit marking includes the following marking information: label information of whether interference exists, the frame number range in which the interference occurs, the duration of the interference, the interference intensity level and the affected beam or elevation layer.

[0026] In the above embodiment, by modularly embedding the interference identification model into the weather radar data processing flow, real-time discrimination and automatic marking of interference data are achieved, which significantly improves the intelligence level and data processing efficiency of the system. During operation, the model can output a clear interference presence label for each frame or each group of echo data, and mark the key parameters of the interference in a structured manner, such as the duration of the interference, the intensity level, and the affected beam range, so that subsequent data users can clearly identify the contaminated area, avoid misusing abnormal data, and improve data credibility. In addition, the explicitly marked interference information also provides the necessary basis for subsequent interference classification and identification and source tracing analysis, forming a complete interference identification closed loop, greatly reducing the workload of manual interference identification, and improving the automation and intelligence level of the radar system.

[0027] Specifically, electromagnetic interference data determines the quality level of meteorological radar electromagnetic interference data, including: Extract the frame number range and duration of interference corresponding to each electromagnetic interference data; Extracting the interference frame number range where the interference signal strength exceeds the preset interference signal strength threshold from the frame number range where the interference occurs; Normalizing the interference frame number range and duration of interference corresponding to each electromagnetic interference data, and obtaining interference frame number range parameters and duration parameters after normalization; Performing difference processing on the interference frame number range parameter and the duration parameter after the normalization processing to obtain the difference between the interference frame number range parameter and the duration parameter after the normalization processing; Comparing the difference between the normalized interference frame number range parameter and the duration parameter with a preset difference threshold; Filter out electromagnetic interference data whose difference between the interference frame number range parameter and the duration parameter after normalization is lower than the difference threshold as the first electromagnetic interference data set; Filter out electromagnetic interference data whose difference between the interference frame number range parameter and the duration parameter after normalization is not less than the difference threshold as the second electromagnetic interference data set; The quality level of the meteorological radar electromagnetic interference data is determined according to the first electromagnetic interference data and the second electromagnetic interference data.

[0028] The technical effect of the above technical solution is: by extracting the frame number range and duration of interference corresponding to each electromagnetic interference data, and further screening out the interference frame number range whose interference signal strength exceeds the preset dry signal strength threshold, the effective interference data range can be determined more accurately, and some interference data with low interference intensity and small impact on meteorological radar data can be excluded, so that the interference data processed later is more representative and accurate, thereby improving the accuracy of electromagnetic interference data of meteorological radar. The interference frame number range and duration are normalized so that the two parameters of different magnitudes are on the same comparable scale, eliminating the influence of different dimensions and numerical ranges on the original data, providing a more scientific basis for subsequent difference processing, which is conducive to improving the accuracy of data processing and analysis, and thus improving the reliability of quality assessment of electromagnetic interference data of meteorological radar. By comparing the difference between the interference frame number range parameter and the duration parameter after normalization with the preset difference threshold, the electromagnetic interference data is screened into a first electromagnetic interference data set (the difference is lower than the difference threshold) and a second electromagnetic interference data set (the difference is not lower than the difference threshold), thereby achieving effective classification of electromagnetic interference data. This classification method helps to more clearly understand the characteristics of electromagnetic interference data of different quality levels, and facilitates the subsequent targeted measures based on the characteristics of different sets of data. For example, the data of the first electromagnetic interference data set may be more trusted and used for more important analysis, while the data of the second electromagnetic interference data set may need further verification or supplementation, etc., which improves the efficiency and pertinence of meteorological radar data processing. The quality level of meteorological radar electromagnetic interference data is determined based on the first electromagnetic interference data and the second electromagnetic interference data. Based on the previous precise data extraction, standardized processing and reasonable data classification, this quality level determination method is more scientific and reasonable, and can more accurately reflect the quality of meteorological radar electromagnetic interference data, providing a reliable basis for performance evaluation, interference analysis and possible improvement measures of the meteorological radar system, and ultimately helping to improve the overall performance and reliability of the meteorological radar system.

[0029] Specifically, determining the quality level of the meteorological radar electromagnetic interference data according to the first electromagnetic interference data and the second electromagnetic interference data includes: Extracting an impact beam or elevation layer corresponding to each first electromagnetic interference data and second electromagnetic interference data in the first electromagnetic interference data set and the second electromagnetic interference data set; Normalizing the affected beam using a standard beam to obtain normalized beam parameters corresponding to each first electromagnetic interference data and the second electromagnetic interference data; or, normalizing the elevation layer corresponding to each first electromagnetic interference data and the second electromagnetic interference data using the elevation range corresponding to the standard elevation layer to obtain normalized elevation parameters corresponding to each first electromagnetic interference data and the second electromagnetic interference data; Marking the beam parameters and elevation parameters corresponding to the first electromagnetic interference data as spatial coefficients, and obtaining a first spatial coefficient standard deviation corresponding to the first electromagnetic interference data set according to the spatial coefficients corresponding to the first electromagnetic interference data; Marking the beam parameters and elevation parameters corresponding to the second electromagnetic interference data as spatial coefficients, and obtaining a second spatial coefficient standard deviation corresponding to the second electromagnetic interference data set according to the spatial coefficients corresponding to the second electromagnetic interference data; Extracting the average value of the difference between the interference frame number range parameter and the duration parameter after normalization corresponding to the first electromagnetic interference data set; Extracting the average value of the difference between the interference frame number range parameter and the duration parameter after normalization corresponding to the second electromagnetic interference data set; Obtaining an interference index corresponding to the overall electromagnetic interference data by using the first space coefficient standard deviation and the second space coefficient standard deviation in combination with a difference average corresponding to the first electromagnetic interference data set and a difference average corresponding to the second electromagnetic interference data set; The interference index corresponding to the overall electromagnetic interference data is obtained by the following formula: ; Where Q represents the interference index corresponding to the overall electromagnetic interference data; k 01 and k 02 Respectively represent the first space coefficient standard deviation and the second space coefficient standard deviation; C 01 and C 02 They represent the difference averages corresponding to the first electromagnetic interference data set and the difference averages corresponding to the second electromagnetic interference data set respectively; C represents a preset difference threshold, which can be set according to the actual application and combined with the historical data experience value; specifically, k 01 and k 02 Reflects the discrete degree of interference in space, C 01 , C 02 It reflects the average deviation of the difference of time-related parameters (interference frame range and duration), and C is the preset reference standard. The purpose of multiplying the standard deviation by their ratio is to combine the spatial interference characteristics (standard deviation) and the temporal interference characteristics (difference average, difference threshold) in one calculation link. This can fully reflect the impact of the comprehensive characteristics of electromagnetic interference in the space-time dimension on the interference index, because the electromagnetic interference of meteorological radar is not simply determined by a certain dimension of space or time, but the result of the joint action of the two. The smaller the value of this logarithmic part, the greater the degree of interference dispersion (k) of the first electromagnetic interference data set in space. 01 ) is relatively small, and the average value of the difference between the frame number range parameter and the duration parameter in time interference (C 01) and the preset difference threshold (C) are also small. That is, if the interference in the first electromagnetic interference data set is relatively stable and regular in time and space, the data quality corresponding to the first electromagnetic interference data set is better; conversely, the larger the value of the logarithmic part, the worse the data quality. The smaller the value of the logarithmic part is, the smaller the second electromagnetic interference data set is in the spatial interference dispersion degree (k 02 ) and the degree of deviation of temporal interference characteristics (given by C 02 The relationship between C and EMI is better, the interference is relatively stable and orderly in time and space, and the data quality corresponding to the second electromagnetic interference data set is better; the larger the value of the logarithmic part, the worse the data quality. The two logarithmic functions represent the quantitative results of the comprehensive interference characteristics of the first and second electromagnetic interference data sets respectively. The subtraction operation is to compare the differences in the temporal and spatial interference characteristics of the two types of data sets. By subtracting, the differences between the two in terms of the discrete degree of spatial interference and the degree of deviation of temporal interference characteristics can be highlighted, so as to more clearly understand the differences in the interference of different types of electromagnetic interference data sets on meteorological radars, and provide a basis for the comprehensive evaluation of the overall electromagnetic interference data quality of meteorological radars. There are some common factors (such as environmental background interference, etc.) in the electromagnetic interference of meteorological radars, which may act on both types of electromagnetic interference data sets at the same time. The subtraction of logarithmic functions can eliminate the influence of these common factors to a certain extent, and focus more on the differences in the unique interference characteristics of the two types of data sets, so that the interference index can more accurately reflect the difference in the impact of different data sets on the quality of meteorological radar data. The smaller the difference after subtraction (i.e., Q value), the smaller the difference in temporal and spatial interference characteristics between the first electromagnetic interference data set and the second electromagnetic interference data set, and overall, the interference is relatively stable and regular in time and space, and the quality of the meteorological radar electromagnetic interference data is better; the larger the difference, the greater the difference, which indicates that the two types of data sets have great differences in temporal and spatial interference characteristics, the interference situation is complex and changeable, and the quality of the meteorological radar electromagnetic interference data is worse.

[0030] Comparing the interference index corresponding to the overall electromagnetic interference data with a preset index threshold; When the interference index corresponding to the overall electromagnetic interference data exceeds the preset index threshold, the quality level of the meteorological radar electromagnetic interference data is determined to be high, and an interference abnormality alarm is issued; When the interference index corresponding to the overall electromagnetic interference data does not exceed the preset index threshold, the quality level of the meteorological radar electromagnetic interference data is determined to be general, and the interference type and interference mode of the electromagnetic interference data are classified and identified.

[0031] The technical effect of the above technical solution is: extract the affected beam or elevation layer corresponding to the interference data, and perform normalization processing to obtain the beam parameters and elevation parameters, and unify them into spatial coefficients. This enables the characteristics of electromagnetic interference data to be accurately measured from the spatial dimension. Taking into account the interference conditions of different beams and elevation layers, it helps to comprehensively and accurately evaluate the interference effects on meteorological radars at different spatial locations, improve the accuracy of the analysis of the spatial distribution characteristics of interference, and thus improve the reliability of meteorological radar data quality assessment. Calculate the standard deviation of the spatial coefficients corresponding to the first and second electromagnetic interference data sets. The standard deviation can reflect the degree of discreteness of the data. Through this operation, the fluctuation of the spatial coefficients in the two data sets can be quantitatively understood, that is, the degree of dispersion or concentration of interference in the spatial dimension. This provides a quantitative indicator for evaluating the stability of interference in space, making the analysis of electromagnetic interference conditions of meteorological radars more scientific, and can assist in judging the degree of influence of interference on the spatial detection performance of radars. Extract the average value of the difference between the normalized interference frame number range parameter and the duration parameter in the two data sets. The average value reflects the average difference between the two types of interference data in terms of time-related characteristics (frame range and duration), which helps to comprehensively analyze the characteristics of interference data from the time dimension. Combined with the standard deviation of the spatial coefficient, it can more comprehensively grasp the characteristics of electromagnetic interference data from the time and space dimensions, and improve the accuracy of the interference assessment of the meteorological radar. The interference index corresponding to the overall electromagnetic interference data is obtained by using the standard deviation of the spatial coefficient and the average value of the difference. By integrating multiple key indicators to obtain the interference index, the severity of the electromagnetic interference suffered by the meteorological radar can be quantitatively characterized as a whole, providing a concise and comprehensive evaluation index, which is convenient for quickly and intuitively judging the interference situation, providing a clear basis for subsequent quality level determination and response measures, and improving the efficiency and effectiveness of the interference assessment of the meteorological radar. The interference index is compared with the preset index threshold to determine the quality level of the electromagnetic interference data of the meteorological radar, and different response strategies (interference abnormality alarm or interference type, pattern classification and identification) are adopted. This hierarchical judgment and targeted processing method enables the meteorological radar to make a reasonable response according to the severity of the interference, timely alarm and prompt attention for high-level interference, and further analyze the type and mode of general-level interference, which helps to improve the ability of the meteorological radar system to deal with electromagnetic interference and ensure the quality of radar data and stable operation of the system.

[0032] On the other hand, the first spatial coefficient standard deviation k is included in the formula 01 , the second spatial coefficient standard deviation k 02 , the difference average value C corresponding to the first electromagnetic interference data set 01 Interference frame number range parameter, difference average value C corresponding to the second electromagnetic interference data set 02and the preset difference threshold C. These factors are integrated through logarithmic operations, and the discrete degree (standard deviation) of interference in the spatial dimension and the average difference (difference average) of the time-related dimension are fully considered, so that the interference index can comprehensively reflect the various characteristics of electromagnetic interference of meteorological radar, avoid the one-sidedness of single factor evaluation, and improve the accuracy and comprehensiveness of the interference index in describing the interference situation. The logarithmic function is used for calculation. The nonlinear characteristics of the logarithmic function can perform nonlinear transformation on the input parameters, compress large values ​​and stretch small values ​​to a certain extent, so that parameters of different magnitudes can be comprehensively calculated on a relatively balanced scale. At the same time, the calculation form of each parameter in the formula and the preset difference threshold C is similar to a normalization method, which converts parameters of different sources and magnitudes into comparable and comprehensively calculable forms, which helps to eliminate the influence of dimension and magnitude differences between parameters, making the calculated interference index more reasonable and comparable, and can measure the degree of interference more scientifically. The formula highlights the difference between the first and second electromagnetic interference data sets in the standard deviation of the spatial coefficient and the average difference value in the form of logarithmic subtraction. The logarithmic subtraction operation makes the relative differences between the corresponding parameters of the two data sets more obvious, which can effectively capture the differences in the temporal and spatial characteristics of the two types of interference data, thereby making the interference index more sensitive to the differences between the meteorological radar electromagnetic interference data sets, providing a more powerful quantitative basis for accurately evaluating the interference situation.

[0033] Classify and identify the interference types and interference patterns of electromagnetic interference data, including: Based on the marked electromagnetic interference data, extract and organize the characteristic parameter vector set corresponding to the electromagnetic interference data; Constructing an interference type classification library, the interference type classification library includes reference feature distribution ranges and category labels of interference types, the interference types including periodic interference, broadband interference, pulse interference and frequency modulation interference; For the extracted electromagnetic interference data, an unsupervised clustering method is used to perform interference pattern aggregation analysis. The electromagnetic interference data is divided into several clusters based on feature similarity. Each cluster corresponds to an interference category. Based on the established interference type classification library, the interference category of each cluster is identified and the support vector machine classification model is used to complete the labeling. The classified and identified interference type information is added to the original echo signal data to form a triple labeling structure of "observation data-interference existence-interference type".

[0034] In the above embodiment, by clustering and classifying the marked electromagnetic interference data, the system can further identify the type and pattern characteristics of the interference, thereby achieving more fine-grained interference control. The interference type classification library is based on the characteristic statistical data of a large number of interference samples, combined with clustering algorithms and support vector machine classifiers to distinguish unknown interference samples, effectively improving the classification accuracy. The interference samples are automatically divided into several pattern categories through an unsupervised clustering algorithm, and then compared with the known type library, which can achieve accurate identification of typical interference types such as periodic, pulsed, broadband and frequency modulation interference, providing support for the system to further identify the source of interference and formulate response strategies. In addition, the classification and recognition results are attached to the original observation data to form a triple structure of "observation data-interference existence-interference type", which not only improves the integrity and systematicness of data annotation, but also helps to build a high-quality interference training data set for continuous optimization of future models.

[0035] Conduct interference source tracing analysis on the identified different types of electromagnetic interference data, including: Conduct correlation analysis on the detection time, spatial location and radar operation status corresponding to the electromagnetic interference data, and collect heterogeneous information including radar system operation logs, site geographic information, electromagnetic environment monitoring data, and meteorological condition records; The radar system operation log includes radar power status, beam pointing, scanning mode, equipment self-test records and alarm events; the electromagnetic environment monitoring data includes electromagnetic spectrum scanning results around the radar site, distribution of radio transmitters and operating status of nearby equipment; Data fusion is used to align and map heterogeneous information on the time axis and spatial coordinates to construct a basic data set for interference analysis; Based on the basic data set of interference analysis, a geographic information system is used to construct a spatiotemporal distribution map of interference events. The spatiotemporal distribution map is centered on the radar station, marking the time period when the interference event occurred, the azimuth and elevation layers of the corresponding beam, and the distribution density and change trend of the interference in space are represented by heat maps and trajectory maps. According to the interference type, frequency and intensity, statistics are compiled on the frequency of each type of interference in different time periods, the duration of interference and the distribution of intensity levels, and statistical charts are output; Based on the time-space distribution map, the path of each type of interference is traced and the potential interference radiation source area is inferred. The known external electromagnetic interference source is matched with the potential interference radiation source area. If the spatial distance, interference frequency band and interference type are consistent, it is marked as an external interference source. Identify the interference caused by abnormal equipment inside the weather radar system. Based on the equipment status change trend recorded in the operation log, determine whether there is an internal system abnormality that is highly coincident with the interference. Mark the identified abnormal equipment inside the weather radar system as an internal interference source. In the above embodiment, by integrating multi-source heterogeneous data such as radar system operation log, electromagnetic environment monitoring data and geographic information system, the identified electromagnetic interference data is traced and analyzed, which not only realizes the precise positioning of external interference sources, but also can identify the interference caused by internal anomalies of the system, greatly improving the system's comprehensive judgment ability on the cause of interference. Based on the spatiotemporal label information of the interference data, combined with the azimuth, elevation layer and radar beam scanning trajectory of the radar site, the interference distribution heat map and trajectory map are drawn using the geographic information system to visualize the evolution trend of interference in time and space, effectively assisting interference diagnosis and decision-making, and by comparing with parameters such as the known interference source location and electromagnetic spectrum, rapid matching and identification of external interference sources are achieved; at the same time, combined with the beam control, power supply status, alarm record and other data in the equipment operation log, highly overlapping internal anomalies are checked to identify possible internal interference sources. This two-way tracing mechanism improves the radar site operation and maintenance efficiency and abnormal response capability, reduces the false alarm and missed alarm rate, and provides strong support for ensuring the quality of radar observation data.

[0036] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for quality control of electromagnetic interference data of meteorological radar based on machine learning, characterized in that: include: Based on the echo signal data collected by the meteorological radar, the characteristic parameters related to electromagnetic interference are extracted through multi-dimensional analysis, and an interference identification model is constructed and trained. The interference identification model is deployed in the meteorological radar data processing flow to identify electromagnetic interference data. The characteristic parameters extracted from the radar signal are received as input for interference discrimination, and the judgment result of whether each frame or each group of data contains electromagnetic interference data is output, and the electromagnetic interference data is explicitly marked. The interference types and interference patterns of electromagnetic interference data are classified and identified, interference source tracing analysis is performed on the identified different types of electromagnetic interference data, and external and internal interference sources are identified.

2. The method for controlling the quality of electromagnetic interference data of a meteorological radar based on machine learning according to claim 1, characterized in that: Based on the original echo signal data collected by the meteorological radar, characteristic parameters related to electromagnetic interference are extracted through multi-dimensional analysis, including: The meteorological radar system collects and preprocesses raw echo signal data in a conventional detection working mode, and extracts characteristic parameters in multiple dimensions based on the preprocessed echo signal data, wherein the dimensions include time domain, frequency domain and statistical dimensions; In the time domain dimension, the extracted characteristic parameters include signal duration, amplitude mutation rate and intermittent distribution characteristics; in the frequency domain dimension, the echo signal is converted from the time domain to the frequency domain, and the extracted characteristic parameters include spectral energy distribution, spectrum mutation point and bandwidth broadening degree; in the statistical dimension, the characteristic parameters extracted based on the sliding window include extreme value anomaly rate, variance change amplitude, skewness and kurtosis.

3. The method for controlling the quality of electromagnetic interference data of a meteorological radar based on machine learning as claimed in claim 1, characterized in that: Build and train the interference recognition model, including: Constructing a sample set for model training, wherein the sample set includes normal echo data and abnormal echo data containing electromagnetic interference; The normal echo data is selected from the radar observation data segment confirmed to be free of interference by manual or expert system; the abnormal echo data is selected from the echo data including the interference events marked in the historical records; An interference recognition model is built based on a convolutional neural network. The interference recognition model is trained using a sample set. After the training process is completed, the model parameters are solidified and saved. Based on an independent validation set that does not overlap with the sample set, the performance of the trained interference recognition model is evaluated. During the performance evaluation process, a variety of classification indicators are used to comprehensively analyze the model effect. The classification indicators include accuracy, recall rate, false alarm rate and F1-score.

4. The method for controlling the quality of electromagnetic interference data of a weather radar based on machine learning according to claim 1, characterized in that: Deploy interference identification models into the weather radar data processing flow and identify electromagnetic interference data, including: The interference identification model that has been trained and verified is embedded in the data processing flow of the meteorological radar system in a modular form. During the model deployment process, a data interaction interface for receiving characteristic parameters is constructed; Input the characteristic parameters into the interference recognition model, perform interference discrimination operation, and output the interference recognition result corresponding to each frame or each group of echo signal data, wherein the interference recognition result includes at least one binary judgment label, and when the label value of the binary judgment label is "1", it indicates that the echo signal data has electromagnetic interference data, and when it is "0", it indicates that the echo signal data is normal data; The echo signal data determined by the model to contain electromagnetic interference data is explicitly marked, and the explicit marking includes the following marking information: label information of whether interference exists, the frame number range in which the interference occurs, the duration of the interference, the interference intensity level and the affected beam or elevation layer.

5. The method for quality control of meteorological radar electromagnetic interference data based on machine learning as claimed in claim 4, characterized in that: Electromagnetic interference data determines the quality level of meteorological radar electromagnetic interference data, including: Extract the frame number range and duration of interference corresponding to each electromagnetic interference data; Extracting the interference frame number range where the interference signal strength exceeds the preset interference signal strength threshold from the frame number range where the interference occurs; Normalizing the interference frame number range and duration of interference corresponding to each electromagnetic interference data, and obtaining interference frame number range parameters and duration parameters after normalization; Performing difference processing on the interference frame number range parameter and the duration parameter after the normalization processing to obtain the difference between the interference frame number range parameter and the duration parameter after the normalization processing; Comparing the difference between the normalized interference frame number range parameter and the duration parameter with a preset difference threshold; Filter out electromagnetic interference data whose difference between the interference frame number range parameter and the duration parameter after normalization is lower than the difference threshold as the first electromagnetic interference data set; Filter out electromagnetic interference data whose difference between the interference frame number range parameter and the duration parameter after normalization is not less than the difference threshold as the second electromagnetic interference data set; The quality level of the meteorological radar electromagnetic interference data is determined according to the first electromagnetic interference data and the second electromagnetic interference data.

6. The method for controlling the quality of electromagnetic interference data of a meteorological radar based on machine learning as claimed in claim 5, characterized in that: Determining the quality level of the meteorological radar electromagnetic interference data according to the first electromagnetic interference data and the second electromagnetic interference data includes: Extracting an impact beam or elevation layer corresponding to each first electromagnetic interference data and second electromagnetic interference data in the first electromagnetic interference data set and the second electromagnetic interference data set; Normalizing the affected beam using a standard beam to obtain normalized beam parameters corresponding to each first electromagnetic interference data and the second electromagnetic interference data; or, normalizing the elevation layer corresponding to each first electromagnetic interference data and the second electromagnetic interference data using the elevation range corresponding to the standard elevation layer to obtain normalized elevation parameters corresponding to each first electromagnetic interference data and the second electromagnetic interference data; Marking the beam parameters and elevation parameters corresponding to the first electromagnetic interference data as spatial coefficients, and obtaining a first spatial coefficient standard deviation corresponding to the first electromagnetic interference data set according to the spatial coefficients corresponding to the first electromagnetic interference data; Marking the beam parameters and elevation parameters corresponding to the second electromagnetic interference data as spatial coefficients, and obtaining a second spatial coefficient standard deviation corresponding to the second electromagnetic interference data set according to the spatial coefficients corresponding to the second electromagnetic interference data; Extracting the average value of the difference between the interference frame number range parameter and the duration parameter after normalization corresponding to the first electromagnetic interference data set; Extracting the average value of the difference between the interference frame number range parameter and the duration parameter after normalization corresponding to the second electromagnetic interference data set; Obtaining an interference index corresponding to the overall electromagnetic interference data by using the first space coefficient standard deviation and the second space coefficient standard deviation in combination with a difference average corresponding to the first electromagnetic interference data set and a difference average corresponding to the second electromagnetic interference data set; Comparing the interference index corresponding to the overall electromagnetic interference data with a preset index threshold; When the interference index corresponding to the overall electromagnetic interference data exceeds the preset index threshold, the quality level of the meteorological radar electromagnetic interference data is determined to be high, and an interference abnormality alarm is issued; When the interference index corresponding to the overall electromagnetic interference data does not exceed the preset index threshold, the quality level of the meteorological radar electromagnetic interference data is determined to be general, and the interference type and interference mode of the electromagnetic interference data are classified and identified.

7. The method for controlling the quality of electromagnetic interference data of a weather radar based on machine learning according to claim 1, characterized in that: Classify and identify the interference types and interference patterns of electromagnetic interference data, including: Based on the marked electromagnetic interference data, extract and organize the characteristic parameter vector set corresponding to the electromagnetic interference data; An interference type classification library is constructed, wherein the interference type classification library includes reference feature distribution ranges and category labels of interference types, and the interference types include periodic interference, broadband interference, pulse interference, and frequency modulation interference.

8. The method for controlling the quality of electromagnetic interference data of a meteorological radar based on machine learning as claimed in claim 7, characterized in that: Classify and identify the interference type and interference mode of electromagnetic interference data, including: For the extracted electromagnetic interference data, an unsupervised clustering method is used to perform interference pattern aggregation analysis. The electromagnetic interference data is divided into several clusters based on feature similarity. Each cluster corresponds to an interference category. Based on the established interference type classification library, the interference category of each cluster is identified and the support vector machine classification model is used to complete the labeling. The classified and identified interference type information is added to the original echo signal data to form a triple labeling structure of "observation data-interference existence-interference type".

9. The method for controlling the quality of electromagnetic interference data of a weather radar based on machine learning according to claim 1, characterized in that: Conduct interference source tracing analysis on the identified different types of electromagnetic interference data, including: Conduct correlation analysis on the detection time, spatial location and radar operation status corresponding to the electromagnetic interference data, and collect heterogeneous information including radar system operation logs, site geographic information, electromagnetic environment monitoring data, and meteorological condition records; The radar system operation log includes radar power status, beam pointing, scanning mode, equipment self-test records and alarm events; the electromagnetic environment monitoring data includes electromagnetic spectrum scanning results around the radar site, distribution of radio transmitters and operating status of nearby equipment; Data fusion is used to align and map heterogeneous information on the time axis and spatial coordinates to construct a basic data set for interference analysis; Based on the basic data set for interference analysis, a geographic information system is used to construct a spatiotemporal distribution map of interference events. The spatiotemporal distribution map is centered on the radar station, marking the time period when the interference event occurred, the azimuth and elevation layers of the corresponding beam, and the distribution density and change trend of the interference in space are represented by heat maps and trajectory maps.

10. The method for controlling the quality of electromagnetic interference data of a weather radar based on machine learning according to claim 9, characterized in that: Conduct interference source tracing analysis on the identified different types of electromagnetic interference data, including: According to the interference type, frequency and intensity, statistics are compiled on the frequency of each type of interference in different time periods, the duration of interference and the distribution of intensity levels, and statistical charts are output; Based on the time-space distribution map, the path of each type of interference is traced and the potential interference radiation source area is inferred. The known external electromagnetic interference source is matched with the potential interference radiation source area. If the spatial distance, interference frequency band and interference type are consistent, it is marked as an external interference source. Identify the interference caused by abnormal equipment inside the weather radar system. Based on the equipment status change trend recorded in the operation log, determine whether there is an internal system abnormality that highly overlaps with the interference, and mark the identified abnormal equipment inside the weather radar system as an internal interference source.

Images