A multi-source fusion UAV intelligent detection system

By integrating multi-source intelligent drone detection systems with radio frequency, radio wave, radar, and optoelectronic technologies, the problem of low accuracy in identifying multiple drones has been solved. This enables accurate identification of drone models and anti-interference capabilities, making it suitable for locations such as prisons and airports.

CN115508821BActive Publication Date: 2024-11-15CHENGDU TIANZONG YUANHANG INTELLIGENT TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202210728659.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-24
Publication Date
2024-11-15
Estimated Expiration
2042-06-24

AI Technical Summary

Technical Problem

Existing drone detection technologies have low accuracy in identifying multiple drones and are susceptible to interference, making it difficult to accurately identify drone models and manufacturers.

Method used

The UAV intelligent detection system adopts multi-source fusion, combining radio frequency fingerprinting, radio detection, radar detection and photoelectric recognition technologies. The signal processor processes the UAV signals and images, and the feature fusion module makes decisions to achieve weighted fusion of multi-source datasets.

Benefits of technology

It improves the accuracy and anti-interference ability of drone identification, can accurately identify drone models in the case of multiple drones, has all-weather working capability, and has a wide range of application scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115508821B_ABST
    Figure CN115508821B_ABST
Patent Text Reader

Abstract

The present invention claims a multi-source fusion UAV intelligent detection system, wherein the radio frequency fingerprint recognition module is used to collect radio frequency signals of UAV targets in the environment; the radio detection module is used to detect radio signals of UAVs in the environment; the radar detection module is used to scan the signals of UAVs in the environment through electromagnetic waves; the photoelectric image detection module is used to capture the image of UAVs in the environment; the signal processor is used to perform signal processing on the radio frequency signals of UAV targets, the radio signals of UAVs, the scanning signals of UAVs in the environment by electromagnetic waves, and the target images of UAVs; the feature fusion module is used to perform decision-making layer feature fusion on the above recognition results; the target is always tracked by the photoelectric device; the UAV intelligent detection system module is used to display signal data and processing results, feature fusion results, and real-time data of UAV tracking and positioning. The present invention realizes accurate, effective and reliable detection of UAVs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle detection, and in particular to a multi-source fusion unmanned aerial vehicle intelligent detection system. Background Art

[0002] In order to meet the detection needs of drones in various complex situations, drone detection technologies include radio frequency fingerprint, radio, radar, optoelectronics and other technical categories. Radio frequency fingerprint recognition technology passively detects drone signals through radio frequency antennas, processes radio frequency signals with relevant algorithms to extract radio frequency fingerprints and identify specific drone models. Its effective range can reach more than 3km, and it can be applied in most scenarios, but the recognition accuracy is not high in the case of multiple drones; radio detection technology detects drone signals through radio detection equipment, and can perform sideways and positioning of drone signals, but this technology is susceptible to interference from other radio signals; radar detection measures the direction of drones by sending electromagnetic waves and detecting echoes reflected from drones. Radar has a wide effective range, but is expensive and difficult to detect small drones; optoelectronic detection collects drone images and uses trajectory discrimination and optical flow characteristics to identify drone targets, but the effective range is small and is easily affected by weather, so the recognition rate is low for small drones. The above drone detection and identification technologies all have their limitations. It is difficult to accurately, effectively and reliably detect drones using a single technology.

[0003] The existing system aims to improve the detection accuracy and anti-interference ability. When facing multiple drones, the efficiency and recognition rate are still insufficient. At the same time, there is no relevant technology that can identify the specific model and manufacturer of the drone. Radio frequency fingerprint technology has advantages in this regard. The differences in electronic hardware during the production process are ultimately reflected in the radio frequency signal, namely the radio frequency fingerprint. Even the electronic hardware produced in the same batch will be different. Therefore, the use of radio frequency fingerprint technology can specifically identify the drone model and its manufacturer. For the identification of multiple drones, the present invention adopts multi-source fusion, and adopts radio frequency fingerprint recognition, radio detection, radar detection and optoelectronic recognition. Multiple identification methods solve the problem of low feature recognition rate in a single technology, and combine multiple feature data sets to achieve efficient and reliable identification of drones. Summary of the invention

[0004] The present invention aims to solve the above problems of the prior art. A multi-source fusion UAV intelligent detection system is proposed. The technical solution of the present invention is as follows:

[0005] A multi-source fusion UAV intelligent detection system, which includes: a radio frequency fingerprint recognition module, a radio detection module, a radar detection module, an optoelectronic image detection module, a signal processor, an optoelectronic image information processing module, a feature fusion module, a UAV tracking and positioning module, and a UAV intelligent detection system module, wherein

[0006] The radio frequency fingerprint recognition module is used to collect radio frequency signals of drone targets in the environment;

[0007] The radio detection module is used to detect radio signals of drones in the environment;

[0008] The radar detection module is used to scan the signals of drones in the environment through electromagnetic waves;

[0009] The photoelectric image detection module is used to capture images of drones in an environment;

[0010] The signal processor is used to perform signal processing on the radio frequency signal of the drone target, the radio signal of the drone, the scanning signal of the electromagnetic wave on the drone in the environment, and the target image of the drone, so as to obtain the recognition result of the drone radio frequency fingerprint, the recognition and positioning result of the drone radio signal, the recognition and positioning result of the drone radar detection, and the recognition result of the drone image;

[0011] The feature fusion module is used to perform decision-level feature fusion on the recognition results of the drone radio frequency fingerprint, the recognition and positioning results of the drone radio signal, the recognition and positioning results of the drone radar detection, and the recognition results of the drone image;

[0012] The UAV tracking and positioning module includes two modules: tracking and positioning. In the UAV tracking stage, the photoelectric image detection module obtains the position of the previous frame of the image. Then the photoelectric device always tracks the target and follows the rotation of the UAV. In the UAV positioning stage, radar detection is the main method, supplemented by radio detection. The radar detection module feeds back the direction, distance and height of the target, and the radio detection feeds back the direction of the target.

[0013] The UAV intelligent detection system module is used to display UAV radio frequency signal data and processing results, radio detection information data and processing results, radar detection information data and processing results, photoelectric image data and processing results, feature fusion results and real-time data of UAV tracking and positioning.

[0014] Furthermore, the signal processor includes a radio frequency signal processing module, a radio detection information processing module, and a radar detection information processing module, wherein:

[0015] The radio frequency signal processing module is used to perform corresponding preprocessing, denoising, feature extraction and recognition algorithm on the collected radio frequency signals to identify the drone, and obtain the recognition result of the radio frequency fingerprint of the drone;

[0016] The radio detection information processing module is used to grayscale the spatial distribution of the radio signal intensity formed by the detection, obtain the gradient histogram features, identify the target UAV, and obtain the identification and positioning results of the UAV radio signal;

[0017] The radar detection information processing module is used to process the results of electromagnetic wave scanning using a wavelet threshold denoising method, extract the target micro-Doppler characteristics, and obtain the recognition and positioning results of the drone radar detection;

[0018] The photoelectric image information processing module is used to perform target detection on the collected drone images, and after detecting the moving targets, the drone targets are screened out by using trajectory discrimination and optical flow characteristic discrimination, and feature recognition is performed on the drone targets to obtain the recognition results of the drone images;

[0019] Furthermore, the radio frequency fingerprint recognition module specifically includes the following steps:

[0020] S1: The radio frequency signal acquisition device passively detects the radio frequency signal in the 2.4 GHz frequency band and generates a signal waveform diagram;

[0021] S2: Preprocess the signal waveform and continuously downsample the signal using wavelet transform;

[0022] S3: The processed signal enters a two-stage detection system. The first stage distinguishes whether the RF signal is a signal or noise. The separated RF signal enters a second stage. In the second stage, the RF signal is distinguished as a drone signal and other IoT RF signals. The separated drone RF signal is subsequently processed.

[0023] S4: When the UAV RF signal is detected, short-time Fourier transform is used and then features are extracted;

[0024] S5: Using the recognition algorithms including KNN, SVM or NN on the extracted features, and comparing them with the existing RF fingerprint dataset, the RF signals not in the dataset library are judged as illegal drones, and the system gives corresponding prompts in the RF fingerprint recognition module;

[0025] S6: When data collection is performed, the collected radio frequency fingerprint features are stored in the radio frequency fingerprint feature library for subsequent radio frequency fingerprint identification;

[0026] S7: The detection results are passed to the multi-source feature fusion module for further identification.

[0027] Furthermore, the detection steps of the two-machine detection system in step S3 specifically include:

[0028] A1. The detected signal is input into the two-stage detection system;

[0029] A2. In the first stage, the detector uses Bayesian hypothesis testing to determine whether the captured signal is an RF signal or noise. If an RF signal is detected, the second-stage detector is activated;

[0030] A3. In the second stage, the detector determines whether the captured RF signal is from an interference source or a drone. The interference source includes WiFi or Bluetooth devices. The detector uses bandwidth analysis and modulation-based features to detect interference. First, the bandwidth of WIFI is significantly higher than that of drones and Bluetooth signals, so the WIFI signal can be separated by bandwidth identification. Most mobile Bluetooth devices use Gaussian frequency shift keying (GFSK / FSK) modulation. In the present invention, two GFSK / FSK modulation characteristics, namely frequency deviation and symbol duration, will be used to distinguish Bluetooth signals. Therefore, if the detected RF signal is not from a WiFi or Bluetooth interference source, it is assumed that it is a signal transmitted by a drone; therefore, the detected signal is passed to the next stage feature extraction classification system to identify the drone.

[0031] Furthermore, the detected signal is transmitted to the next stage feature extraction and classification system to identify the drone, which specifically includes:

[0032] B1. The detected drone signal is transmitted to the feature extraction module for extraction;

[0033] B2. For the representation of RF signal in energy time-frequency domain, the spectrum of the signal is obtained by short-time Fourier transform SIFT. The following is the spectrum calculation formula

[0034]

[0035] Among them, y T [n] is the preprocessed signal captured by the detection system, m is the discrete time, ω is the frequency, k is the discrete variable, and ω[n] is the sliding window function used as a filter;

[0036] B3. The energy trajectory is calculated by taking the maximum energy value along the time axis in the acquired spectrum, and the energy transient is estimated by finding the maximum mutation of the mean or variance of the normalized energy trajectory. The energy transient characteristic defines the transient characteristics of the signal in the energy domain;

[0037] B4. When energy transients are detected, feature values ​​with higher weights are selected to extract radio frequency fingerprints, which are statistical moment features that characterize energy transients.

[0038] Furthermore, the radio detection module specifically includes the following steps:

[0039] S21: After the radio scanning device performs omnidirectional scanning, the collected signals are plotted to obtain a radio signal strength distribution diagram;

[0040] S22: performing data correction and grayscale processing on the drawn signal intensity distribution graph;

[0041] S23: Obtain the gradient and direction from the image, divide the image into blocks, normalize the divided sub-blocks, merge the histograms of each sub-block, obtain the directional gradient histogram feature vector on the image, and obtain the orientation of the drone at the same time;

[0042] S24: The extracted features are identified and matched with the drone radio feature data set. Unmatched drones will sound an alarm in the radio detection module and the drone is judged to be a suspected illegal drone.

[0043] S25: When data collection is performed, the collected UAV radio signal features are stored in a UAV radio feature library for subsequent UAV radio identification;

[0044] S26: The detection results are transmitted to the multi-source feature fusion module for further accurate recognition.

[0045] Furthermore, the radar detection module specifically includes the following steps:

[0046] S31: The radar detection device emits electromagnetic waves to perform an all-round scan of the surrounding air environment. When the target UAV is scanned by the radar, an echo signal is generated;

[0047] S32: After the radar receives the echo signal sent back, the wavelet threshold denoising method is used to denoise the noisy drone radar echo signal;

[0048] S33: Amplify the echo signal, extract the micro-Doppler feature, and use the time it takes for the electromagnetic wave to reach the target UAV and return from the target UAV to the radar antenna to measure the distance of the UAV, and use the directional characteristics of the radar to determine the height and azimuth information of the UAV;

[0049] S34: Identify the extracted features and match them with the signals in the radar micro-Doppler feature data set library. Unmatched drones are determined to be suspected illegal drones, and an alarm is issued in the radar detection module;

[0050] S35: When performing data collection, the collected features are stored in a radar micro-Doppler feature data set library for subsequent identification;

[0051] S36: The detection results are transmitted to the multi-source feature fusion module for further accurate recognition.

[0052] Furthermore, the feature fusion module specifically includes the following steps:

[0053] S51: transferring the radio frequency fingerprint recognition decision result, the radio frequency recognition decision result, the radar recognition decision result and the photoelectric image recognition decision result to the feature fusion module;

[0054] S52: fusing the radio frequency data, radio detection data, radar detection data and optoelectronic image detection data based on the weighting principle in hierarchical decision making;

[0055] S53: The fused data is transmitted to the fusion identification module. When an illegal UAV is detected, the system generates an alarm and tracks and locates the target UAV in real time;

[0056] S54: According to the similarity with each feature template after fusion, the specific type of the UAV target can be finally determined.

[0057] Furthermore, the specific steps of the S52 weighting method are as follows:

[0058] C1. Use Hamming distance to calculate the similarity between the RF fingerprint recognition decision results, radio recognition decision results, radar recognition decision results, and optoelectronic recognition decision results and the UAV feature library HD a , HD b , HD c and HD d , where the Hamming distance algorithm calculates the similarity between the decision result and different feature values ​​in the feature library.

[0059] C2. The weighted fusion unit fuses the RF fingerprint recognition decision results, radio recognition decision results, radar recognition decision results and optoelectronic recognition decision results. The fusion formula is:

[0060] FS=w a ×(1-HD a )+w b ×(1-HD b )+w c ×(1-HD c )+w d ×(1-HD d )

[0061] Where FS represents the similarity between the fused recognition result and the UAV feature library, and w a 、w b 、w c and w d represents the weight coefficient of RF fingerprint, radio, radar and optoelectronic recognition results, and w a +w b +w c +w d =1, the weight coefficient can be determined according to the actual situation, and the type of the UAV target can be determined eventually.

[0062] Furthermore, the drone tracking and positioning module is used to process the obtained drone location information. When the system generates an alarm, the illegal drone identified by the feature fusion recognition module is tracked and positioned in real time. The specific steps are as follows:

[0063] S61: The system generates an alarm, and the radar positioning device, radio detection device and optoelectronic image tracking device of the system start to operate;

[0064] S62: Real-time positioning of the target UAV based on the direction, distance, and altitude of the target obtained from the radar detection module and the positioning information obtained from the radio detection module;

[0065] S63: After confirming that the target is a drone, the position of the previous frame image detected by the optoelectronic image detection module is obtained, and then the optoelectronic device always tracks the target;

[0066] S64: Perform joint network pre-training based on the established drone image dataset, perform Kalman filter prediction on the previous frame and update the current frame according to the detection result, and perform multiple matching in combination with the motion and appearance features retained during initialization, and finally output the confirmed state trajectory, and adjust the optoelectronic lens to track the target drone in real time;

[0067] S65: When the target UAV flies out of the no-fly zone, the tracking ends, and the UAV tracking and positioning module ends its operation.

[0068] The advantages and beneficial effects of the present invention are as follows:

[0069] Compared with other detection systems, the present invention adopts radio frequency fingerprint recognition, radio recognition, radar recognition, photoelectric recognition and multi-source fusion technology, which can identify the specific model of drones, and still has a high recognition rate when facing multiple drones; and solves the limitations of drone detection with a single technology, and has the ability to work around the clock; for other wireless signals, the radio frequency fingerprint recognition module in the system uses the bandwidth and modulation characteristics of wireless signals to distinguish between WIFI signals, Bluetooth signals and drone signals, and has stronger anti-interference ability and wider application scenarios; by establishing drone radio frequency fingerprints, radio signals, radar micro-Doppler features and photoelectric image data sets, the weighted method is used to make decisions on the recognition results of each module, so as to achieve accurate, effective and reliable recognition of drone models. The final results can be applied to prisons, airports and important activity venues, with rich market prospects and value. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a block diagram of a radio frequency fingerprint recognition module according to a preferred embodiment of the present invention;

[0071] Figure 2 This is a block diagram of the radio detection and identification module of the present invention;

[0072] Figure 3 This is a block diagram of the radar detection and identification module of the present invention;

[0073] Figure 4 This is a block diagram of the photoelectric identification module of the present invention;

[0074] Figure 5 This is a block diagram of a multi-source feature fusion module of the present invention;

[0075] Figure 6 This is a block diagram of the multi-source fusion UAV intelligent detection system of the present invention. DETAILED DESCRIPTION

[0076] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0077] The technical solution of the present invention to solve the above technical problems is:

[0078] like Figure 6 As shown, the present invention is a multi-source fusion UAV intelligent detection system, including a radio frequency fingerprint recognition module, a radio detection module, a radar detection module, a photoelectric detection module, a feature fusion module, a UAV tracking and positioning module, and a UAV intelligent detection platform.

[0079] The radio frequency fingerprint recognition module is as follows Figure 1 , specifically including the following steps:

[0080] S1: The radio frequency signal acquisition device passively detects the radio frequency signal in the 2.4 GHz frequency band and generates a signal waveform diagram;

[0081] S2: Preprocess the signal waveform and continuously downsample the signal using wavelet transform;

[0082] S3: The processed signal enters a two-stage detection system. The first stage distinguishes whether the RF signal is a signal or noise. The separated RF signal enters a second stage. In the second stage, the RF signal is distinguished as a drone signal and other IoT RF signals. The separated drone RF signal is subsequently processed.

[0083] S4: When the UAV RF signal is detected, short-time Fourier transform is used and then features are extracted;

[0084] S5: Using a recognition algorithm on the extracted features, the recognition algorithm includes but is not limited to KNN, SVM or NN, and comparing with the existing RF fingerprint data set, the RF signal that is not in the database is determined to be an illegal drone, and the system gives a corresponding prompt in the RF fingerprint recognition module;

[0085] S6: When data collection is performed, the collected radio frequency fingerprint features are stored in the radio frequency fingerprint feature library for subsequent radio frequency fingerprint identification;

[0086] S7: The detection results are passed to the multi-source feature fusion module for further accurate recognition.

[0087] Furthermore, the step S3 of the two-machine detection system specifically includes:

[0088] 1. The detected signal is input into the two-stage detection system;

[0089] 2. In the first stage, the detector uses Bayesian hypothesis testing to determine whether the captured signal is an RF signal or noise. If an RF signal is detected, the second stage detector is activated;

[0090] 3. In the second stage, the detector decides whether the captured RF signal is from an interference source (WiFi or Bluetooth device) or a drone. The detector uses bandwidth analysis and modulation-based features for interference detection, so if the detected RF signal is not from a WiFi or Bluetooth interference source, it is assumed to be a signal transmitted by a drone. Therefore, the detected signal is passed to the next stage feature extraction classification system to identify the drone.

[0091] The step S3 of the two-machine detection system specifically includes:

[0092] 1. The detected drone signal is passed to the feature extraction module for extraction.

[0093] 2. For the representation of RF signal in energy time-frequency domain, the spectrum of the signal is obtained by short-time Fourier transform (STFT). The following is the spectrum calculation formula

[0094]

[0095] Among them, y T [n] is the preprocessed signal captured by the detection system, m is the discrete time, k is the discrete variable, ω is the frequency, and ω[n] is the sliding window function used as a filter.

[0096] 3. The energy trajectory is calculated by taking the maximum energy value along the time axis in the acquired spectrum. The energy transient is estimated by finding the maximum mutation of the mean or variance of the normalized energy trajectory. The energy transient characteristic defines the transient characteristics of the signal in the energy domain.

[0097] 4. When an energy transient is detected, the RF fingerprint (a set of 15 statistical features, and the eigenvalues ​​with higher weights can be selected) will be extracted, and the four eigenvalues ​​with the highest weights will be selected. These RF fingerprints are the statistical moment characteristics that characterize the energy transient.

[0098] The radio detection module is as follows Figure 2 , specifically including the following steps:

[0099] S21: After the radio scanning device performs omnidirectional scanning, the collected signals are plotted to obtain a radio signal strength distribution diagram;

[0100] S22: performing data correction and grayscale processing on the drawn signal intensity distribution graph;

[0101] S23: Obtain the gradient and direction from the image, divide the image into blocks, normalize the divided sub-blocks, merge the histograms of each sub-block, obtain the directional gradient histogram feature vector on the image, and obtain the orientation of the drone at the same time;

[0102] S24: The extracted features are identified and matched with the drone radio feature data set. Unmatched drones will sound an alarm in the radio detection module and the drone is judged to be a suspected illegal drone.

[0103] S25: When data collection is performed, the collected UAV radio signal features are stored in a UAV radio feature library for subsequent UAV radio identification;

[0104] S26: The detection results are transmitted to the multi-source feature fusion module for further accurate recognition.

[0105] The radar detection module is as follows Figure 3 , specifically including the following steps:

[0106] S31: The radar detection device emits electromagnetic waves to perform an all-round scan of the surrounding air environment. When the target UAV is scanned by the radar, an echo signal is generated;

[0107] S32: After the radar receives the echo signal sent back, the wavelet threshold denoising method is used to denoise the noisy drone radar echo signal;

[0108] S33: Amplify the echo signal, extract the micro-Doppler feature, and use the time it takes for the electromagnetic wave to reach the target UAV and return from the target UAV to the radar antenna to measure the distance of the UAV, and use the directional characteristics of the radar to determine the height, direction and other information of the UAV;

[0109] S34: Identify the extracted features and match them with the signals in the radar micro-Doppler feature data set library. Unmatched drones are determined to be suspected illegal drones, and an alarm is issued in the radar detection module;

[0110] S35: When performing data collection, the collected features are stored in a radar micro-Doppler feature data set library for subsequent identification;

[0111] S36: The detection results are transmitted to the multi-source feature fusion module for further accurate recognition.

[0112] The photoelectric detection module is as follows Figure 4 , specifically including the following steps:

[0113] S41: Use a photoelectric detection device to detect surrounding aerial targets and collect images in real time;

[0114] S42: The target detection module receives the video data captured by the optoelectronic camera, detects whether there is a moving target within the monitoring range, and when a moving target is detected, sends the target motion trajectory and the area information to the target recognition module;

[0115] S43: After the target detection module detects the moving target, it preliminarily screens the target through trajectory discrimination and optical flow characteristic discrimination, and excludes some non-UAV targets;

[0116] S44: extracting features from the selected drone images, and storing the collected features in a drone photoelectric image feature library during data collection for subsequent identification;

[0117] S45: The extracted UAV photoelectric image features are transmitted to the target recognition system and matched with the UAV photoelectric image feature data set. Unmatched UAVs are identified as illegal UAVs, and the UAV photoelectric detection module issues an alarm.

[0118] S46: The detection results are transmitted to the multi-source feature fusion module for further accurate recognition.

[0119] The feature fusion module is as follows Figure 5 , specifically including the following steps:

[0120] S51: transferring the radio frequency fingerprint recognition decision result, the radio frequency recognition decision result, the radar recognition decision result and the photoelectric image recognition decision result to the feature fusion module;

[0121] S52: In hierarchical decision making, based on the weighted principle, RF data, radio detection data, radar detection data and optoelectronic image detection data are integrated to reduce the demand for computing resources for multi-source fusion and improve the environmental applicability and recognition accuracy of the UAV identification and positioning system;

[0122] S53: The fused data is transmitted to the fusion identification module. When an illegal UAV is detected, the system generates an alarm and tracks and locates the target UAV in real time;

[0123] S54: According to the similarity with each feature template after fusion, the specific type of the UAV target can be finally determined.

[0124] The specific steps of the S52 weighting method are as follows:

[0125] 1. Use Hamming distance to calculate the similarity between the RF fingerprint recognition decision results, radio recognition decision results, radar recognition decision results, and optoelectronic recognition decision results and the UAV feature library HD a , HD b , HD c and HD d , where the Hamming distance algorithm calculates the similarity between the decision result and different feature values ​​in the feature library.

[0126] 2. The weighted fusion unit fuses the RF fingerprint recognition decision results, radio recognition decision results, radar recognition decision results and photoelectric recognition decision results. The following is the fusion formula

[0127] FS=w a ×(1-HD a )+w b ×(1-HD b )+w c ×(1-HD c )+w d ×(1-HD d )

[0128] Where FS represents the similarity between the fused recognition result and the UAV feature library, and w a 、w b 、w c and w d represents the weight coefficient of RF fingerprint, radio, radar and optoelectronic recognition results, and w a +w b +w c +w d = 1. The weight coefficient can be set according to the actual situation, and the type of the drone target can be determined eventually.

[0129] The drone tracking and positioning module is used to process the obtained drone location information. When the system generates an alarm, the illegal drone identified by the feature fusion recognition module is tracked and positioned in real time. The specific steps are as follows:

[0130] S61: The system generates an alarm, and the radar positioning device, radio detection device and optoelectronic image tracking device of the system start to operate;

[0131] S62: Real-time positioning of the target UAV based on the direction, distance, and height of the target obtained from the radar detection module and the positioning information obtained from the radio detection module;

[0132] S63: After confirming that the target is a drone, the position of the previous frame image detected by the optoelectronic image detection module is obtained, and then the optoelectronic device always tracks the target.

[0133] S64: Perform joint network pre-training based on the established drone image dataset, perform Kalman filter prediction on the previous frame and update the current frame based on the detection results, and perform multiple matches based on the motion and appearance features retained during initialization. Finally, output the confirmed trajectory and adjust the optoelectronic lens to track the target drone in real time.

[0134] S65: When the target UAV flies out of the no-fly zone, the tracking ends, and the UAV tracking and positioning module ends its operation.

[0135] The UAV intelligent detection system has the following specific manifestations:

[0136] S71: The system controls each module to cooperate with each other. When the system is turned on, the radio frequency signal detection module, the radio signal detection module, the radar detection module and the photoelectric detection module start to operate;

[0137] S72: When the RF model detection module starts to operate, the UAV RF fingerprint display module displays the waveform of the UAV RF signal transmitted by the detection module, the waveform after the signal is downsampled by wavelet transform, the spectrum after short-time Fourier transform, and the RF fingerprint feature data;

[0138] S73: When the radio detection module starts to operate, the detection data is transmitted to the system, and the radio detection display module displays the distribution diagram of the UAV radio signal strength and the UAV radio characteristic data;

[0139] S74: When the radar detection module starts to operate, the drone radar detection display module displays the drone echo data scanned by the radar and the micro-Doppler characteristic data;

[0140] S75: When the photoelectric detection module starts to operate, the photoelectric detection display module displays the drone target detection image and the drone photoelectric image feature data in real time;

[0141] S76: Display the multi-source feature fusion recognition results, determine the specific type of the drone target, generate an alarm and perform positioning and tracking;

[0142] S77: To locate illegal drones, radar detection and positioning is mainly used to display the drone's direction, distance, altitude and other information, and radio is used as a supplement to display the drone's signal strength distribution map, that is, the drone's direction, and track the target in real time based on the first frame image position of the optoelectronic image detection module.

[0143] S78: When the illegal drone leaves the target area, the tracking ends, the drone tracking system ends, and the detection module detects the surrounding aerial targets. When the system ends, each module stops running one after another.

[0144] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0145] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0146] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A multi-source fusion UAV intelligent detection system, characterized by: include: Radio frequency fingerprint recognition module, radio detection module, radar detection module, photoelectric image detection module, signal processor, photoelectric image information processing module, feature fusion module, drone tracking and positioning module and drone intelligent detection system module, among which The radio frequency fingerprint recognition module is used to collect radio frequency signals of drone targets in the environment; The radio detection module is used to detect radio signals of drones in the environment; The radar detection module is used to scan the signals of drones in the environment through electromagnetic waves; The photoelectric image detection module is used to capture images of drones in the environment; The signal processor is used to process the radio frequency signal of the drone target, the radio signal of the drone, the electromagnetic wave scanning signal of the drone in the environment, and the target image of the drone to obtain the recognition result of the drone radio frequency fingerprint, the recognition and positioning result of the drone radio signal, the recognition and positioning result of the drone radar detection, and the recognition result of the drone image; The feature fusion module is used to perform decision-level feature fusion on the recognition results of the drone's radio frequency fingerprint, the recognition and positioning results of the drone's radio signal, the recognition and positioning results of the drone's radar detection, and the recognition results of the drone's image; The UAV tracking and positioning module includes two modules: tracking and positioning. During the UAV tracking phase, the photoelectric image detection module obtains the position of the previous frame of the image. Subsequently, the photoelectric device always tracks the target and follows the rotation of the UAV. During the UAV positioning phase, radar detection is the main method, supplemented by radio detection. The radar detection module feedbacks the target's direction, distance and altitude, while the radio detection feedbacks the target's direction. The UAV intelligent detection system module is used to display UAV radio frequency signal data and processing results, radio detection information data and processing results, radar detection information data and processing results, photoelectric image data and processing results, feature fusion results and real-time data of UAV tracking and positioning; The radio detection module specifically includes the following steps: S21: After the radio scanning device performs an omnidirectional scan, the collected signals are plotted to obtain a radio signal strength distribution map; S22: performing data correction on the drawn signal intensity distribution graph and performing grayscale processing; S23: Obtain the gradient and direction from the image, divide the image into blocks, normalize the divided sub-blocks, merge the histograms of each sub-block, obtain the directional gradient histogram feature vector on the image, and obtain the orientation of the drone; S24: The extracted features are identified and matched with the drone radio feature dataset. Unmatched drones will sound an alarm in the radio detection module and be judged as suspected illegal drones. S25: When performing data collection, the collected UAV radio signal features are stored in a UAV radio feature library for subsequent UAV radio identification; S26: The detection results are passed to the multi-source feature fusion module for further accurate recognition; The signal processor includes a radio frequency signal processing module, a radio detection information processing module, and a radar detection information processing module, wherein: The radio frequency signal processing module is used to perform corresponding preprocessing, denoising, feature extraction and recognition algorithm on the collected radio frequency signals to identify the drone and obtain the recognition result of the radio frequency fingerprint of the drone; The radio detection information processing module is used to grayscale the spatial distribution of the radio signal intensity formed by the detection, obtain gradient histogram features, identify the target drone, and obtain the identification and positioning results of the drone radio signal; The radar detection information processing module is used to process the results of electromagnetic wave scanning using a wavelet threshold denoising method, extract the target micro-Doppler characteristics, and obtain the identification and positioning results of the UAV radar detection; The photoelectric image information processing module is used to perform target detection on the collected drone images. After detecting the moving target, the drone target is screened out using trajectory discrimination and optical flow characteristic discrimination, and the feature recognition of the drone target is performed to obtain the recognition result of the drone image.

2. The multi-source fusion UAV intelligent detection system according to claim 1 is characterized in that: The radio frequency fingerprint recognition module specifically includes the following steps: S1: The radio frequency signal acquisition device passively detects the 2.4GHz frequency band radio frequency signal and generates a signal waveform diagram; S2: Preprocess the signal waveform and continuously downsample the signal using wavelet transform; S3: The processed signal enters a two-stage detection system. The first stage distinguishes whether the RF signal is a noise signal. The separated RF signal enters the second stage, where the RF signal is distinguished as a drone signal and other IoT RF signals. The separated drone RF signal is then processed. S4: When the UAV RF signal is detected, short-time Fourier transform is applied and then features are extracted; S5: Using a recognition algorithm such as KNN, SVM, or NN on the extracted features, the system compares them with the existing RF fingerprint dataset. If the RF signal is not in the dataset, it is determined to be an illegal drone, and the system will give a corresponding prompt in the RF fingerprint recognition module. S6: When performing data collection, the collected radio frequency fingerprint features are stored in the radio frequency fingerprint feature library for subsequent radio frequency fingerprint identification; S7: The detection results are passed to the multi-source feature fusion module for further recognition.

3. The multi-source fusion UAV intelligent detection system according to claim 2 is characterized in that: The detection steps of the two-machine detection system in step S3 specifically include: A1. The detected signal is input into the two-stage detection system; A2. In the first stage, the detector uses Bayesian hypothesis testing to determine whether the captured signal is RF or noise. If an RF signal is detected, the second-stage detector is activated. A3. In the second stage, the detector determines whether the captured RF signal comes from an interference source or a drone. Interference sources include WiFi or Bluetooth devices. The detector uses bandwidth analysis and modulation-based features to detect interference. First, WiFi has a significantly higher bandwidth than drone and Bluetooth signals, so WiFi signals can be separated through bandwidth identification. Mobile Bluetooth devices use Gaussian frequency shift keying (GFSK / FSK) modulation and use two GFSK / FSK modulation characteristics, namely frequency deviation and symbol duration, to distinguish Bluetooth signals. Therefore, if the detected RF signal does not come from a WiFi or Bluetooth interference source, it is assumed to be a signal transmitted by a drone. Therefore, the detected signal is passed to the next stage, the feature extraction and classification system, to identify the drone.

4. The multi-source fusion UAV intelligent detection system according to claim 3 is characterized in that: The detected signal is transmitted to the next stage feature extraction and classification system to identify the drone, specifically including: B1. The detected drone signal is transmitted to the feature extraction module for extraction; B2. For the representation of RF signals in the energy time-frequency domain, the spectrum of the signal is obtained through short-time Fourier transform (SIFT). The following is the spectrum calculation formula Among them, y T [k] is the preprocessed signal captured by the detection system, m is the discrete time, ω is the frequency, k is the discrete variable, and ω[km] is the sliding window function used as a filter; B3. Calculate the energy trajectory by taking the maximum energy value along the time axis of the acquired spectrum. Estimate the energy transient by finding the maximum change in the mean or variance of the normalized energy trajectory. The energy transient characteristic defines the transient characteristics of the signal in the energy domain. B4. When an energy transient is detected, eigenvalues ​​with higher weights are selected to extract radio frequency fingerprints. These radio frequency fingerprints are statistical moment features that characterize the energy transient.

5. The multi-source fusion UAV intelligent detection system according to claim 1 is characterized in that: The radar detection module specifically includes the following steps: S31: The radar detection device transmits electromagnetic waves to perform an all-around scan of the surrounding air environment. When the target UAV is scanned by the radar, an echo signal is generated; S32: After the radar receives the echo signal, the wavelet threshold denoising method is used to denoise the noisy drone radar echo signal; S33: Amplify the echo signal, extract the micro-Doppler feature, and use the time it takes for the electromagnetic wave to reach the target UAV and return from the target UAV to the radar antenna to measure the distance of the UAV. Use the directional characteristics of the radar to determine the altitude and azimuth information of the UAV; S34: Identify the extracted features and match them with the signals in the radar micro-Doppler feature dataset library. Unmatched drones are identified as suspected illegal drones, and an alarm is issued in the radar detection module. S35: When performing data collection, the collected features are stored in a radar micro-Doppler feature data set library for subsequent identification; S36: The detection results are passed to the multi-source feature fusion module for further accurate recognition.

6. The multi-source fusion UAV intelligent detection system according to claim 1 is characterized in that: The feature fusion module specifically includes the following steps: S51: The radio frequency fingerprint recognition decision result, the radio frequency recognition decision result, the radar recognition decision result, and the photoelectric image recognition decision result are transmitted to the feature fusion module; S52: In hierarchical decision making, the radio frequency data, radio detection data, radar detection data, and photoelectric image detection data are integrated based on the weighted method principle; S53: The fused data is transmitted to the fusion recognition module. When an illegal drone is detected, the system generates an alarm and tracks and locates the target drone in real time; S54: Based on the similarity between the fused image and each feature template, the specific type of the drone target can be finally determined.

7. The multi-source fusion UAV intelligent detection system according to claim 6 is characterized in that: The specific steps of the S52 weighting method are as follows: C1. Use Hamming distance to calculate the similarity HD between the RF fingerprint recognition decision results, radio recognition decision results, radar recognition decision results, and optoelectronic recognition decision results and the UAV feature library a 、HD b 、HD c and HD d , where the Hamming distance algorithm calculates the similarity between the decision result and different feature values ​​in the feature library; C2. The weighted fusion unit fuses the radio frequency fingerprint recognition decision results, radio recognition decision results, radar recognition decision results, and photoelectric recognition decision results. The fusion formula is: FS=w a ×(1-HD a )+w b ×(1-HD b )+w c ×(1-HD c )+w d ×(1-HD d ) Where FS represents the similarity between the fused recognition result and the UAV feature library, w a 、w b 、w c and w d represents the weight coefficient of RF fingerprint, radio, radar and optoelectronic recognition results, and w a +w b +w c +w d =1, the weight coefficient can be determined according to the actual situation, and the type of the UAV target can be determined eventually.

8. The multi-source fusion UAV intelligent detection system according to claim 1 is characterized in that: The drone tracking and positioning module is used to process the obtained drone location information. When the system generates an alarm, it tracks and locates the illegal drone identified by the feature fusion recognition module in real time. The specific steps are as follows: S61: The system generates an alarm, and the system's radar positioning device, radio detection device, and optoelectronic image tracking device begin to operate; S62: Real-time positioning of the target UAV based on the direction, distance, and altitude of the target obtained from the radar detection module and the positioning information obtained from the radio detection module; S63: After confirming that the target is a drone, the position of the previous frame of image detected by the photoelectric image detection module is obtained, and then the photoelectric device always tracks the target; S64: Perform joint network pre-training based on the established drone image dataset, perform Kalman filter prediction on the previous frame and update the current frame based on the detection results, and perform multiple matching based on the motion and appearance features retained during initialization. Finally, output the confirmed trajectory and adjust the optoelectronic lens to track the target drone in real time; S65: When the target UAV flies out of the no-fly zone, the tracking ends and the UAV tracking and positioning module ends its operation.

Citation Information

Patent Citations

  • Detection system and detection method for low-altitude slow small targets

    CN108957445A

  • Unmanned aerial vehicle target detection method based on multi-sensor information fusion

    CN112068111A

  • High-precision multi-target intelligent identifying, positioning and tracking method and system based on unmanned aerial vehicle

    CN113538585A

  • Radar monitoring early warning system based on unmanned aerial vehicle

    CN208298351U

  • Synchronic positional tracking method using radar of multi unmanned aerial vehicles

    KR102298950B1