An unmanned aerial vehicle detection method and system based on multi-modal fusion

Abstract

This invention relates to the field of target detection and identification technology, and discloses a UAV detection method and system based on multimodal fusion. It aims to solve the problem of low decision-making accuracy caused by the lack of effective information fusion and decision-making mechanisms when multiple detection technologies are combined. This invention achieves this through the following methods: simultaneously acquiring data from radar, radio frequency, photoelectric, and acoustic units and extracting features; associating multi-source features with a target position sequence formed from radar data to create a unified target profile; acquiring real-time environmental parameters to dynamically adjust the weights of each feature; and finally, using evidence theory to fuse the weighted features, calculating the confidence level, and determining the threat level. Cross-validation of multi-source information reduces the false alarm rate; dynamic weight adjustment improves adaptability to environmental changes; and graded response based on the fused confidence level improves detection reliability and handling efficiency.

Description

Technical Field

[0001] This invention relates to the field of target detection and recognition technology, and in particular to a UAV detection method and system based on multimodal fusion. Background Technology

[0002] Currently, with the widespread adoption of small drone technology, unauthorized drones pose an increasingly serious threat to the security of critical areas such as airports, important facilities, and large event venues. To address this, the industry has developed various drone detection technologies, with mainstream methods including radar detection, radio frequency signal monitoring, photoelectric image recognition, and acoustic feature analysis. However, these individual detection methods all have inherent limitations in practical applications: radar detection is ineffective against low-altitude, slow-moving, and small targets with very small radar cross-sections and is easily interfered with by clutter such as birds; photoelectric detection is greatly affected by environmental factors such as lighting and weather, and has a limited effective range; radio frequency monitoring is a passive detection method and cannot detect autonomously flying drones that have not activated radio communication; and acoustic detection has a short effective range and is easily drowned out by environmental noise. Therefore, no single detection technology can achieve reliable monitoring in all weather conditions and all airspace.

[0003] To compensate for the limitations of single sensors, existing technologies have attempted to combine multiple detection methods. However, this simple superposition has failed to fundamentally solve the problem; instead, it has introduced deeper and more challenging technical difficulties. In heterogeneous systems composed of sensors based on different principles, how to establish an effective information fusion and joint decision-making mechanism is crucial. When different sensors provide contradictory or ambiguous detection results for the same target (e.g., radar identifies the target as a drone, while photoelectric identification identifies it as a bird), and when the reliability of each sensor fluctuates dynamically due to environmental changes (such as heavy fog or strong electromagnetic interference), existing methods generally lack a scientific arbitration and balancing mechanism, making it difficult to reach a final decision more accurate and reliable than any single sensor. Therefore, there is an urgent need in this field for a new drone detection method that not only integrates the information advantages of multiple sensors but, more importantly, intelligently handles the uncertainties and conflicts of multi-source information, thereby truly achieving a balance between high detection probability and low false alarm rate in complex and ever-changing environments. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, to address the problem of low accuracy in decision-making under information conflicts and environmental changes caused by the lack of an effective information fusion and joint decision-making mechanism when combining multiple existing UAV detection technologies, this invention provides a UAV detection method and system based on multimodal fusion.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a UAV detection method based on multimodal fusion, which includes the following steps: S1. Synchronously acquire raw data containing a unified time stamp obtained by radar, radio frequency monitoring unit, photoelectric unit and acoustic unit; S2. Preprocess the raw data and extract radar features, radio frequency features, photoelectric features and acoustic features from the preprocessed data respectively; S3. Based on the radar features, a target position sequence continuously detected by the radar is formed, and the target position sequence is used to predict the spatial position range of the target in the next moment. The radio frequency features, the photoelectric features and the acoustic features whose detection direction points to the spatial position range are associated to form a unified target file. S4. Obtain real-time environmental parameters, and dynamically adjust the weights of various features in decision fusion based on the real-time environmental parameters; based on the weights, use evidence theory to fuse various features in the unified target file, and calculate the target value as the final confidence level of the UAV. S5. Determine the threat level of the target based on the comparison result between the final confidence level and the preset threshold.

[0007] As a preferred embodiment of the UAV detection method based on multimodal fusion described in this invention, the specific steps for simultaneously acquiring raw data containing a unified time stamp obtained by radar, radio frequency monitoring unit, photoelectric unit, and acoustic unit are as follows: The GPS clock module provides a unified standard time for the radar, the radio frequency monitoring unit, the photoelectric unit, and the acoustic unit; and when each unit collects the raw data, it adds a synchronization time stamp with millisecond-level precision corresponding to the standard time to the raw data.

[0008] As a preferred embodiment of the UAV detection method based on multimodal fusion described in this invention, the original data is preprocessed, and radar features, radio frequency features, photoelectric features, and acoustic features are extracted from the preprocessed data, respectively. The specific steps are as follows: The radar echo signal is processed to generate a micro-Doppler spectrum that shows the frequency change of the signal over time. The radar features are extracted based on the presence of stable high-frequency harmonic spectral lines generated by the UAV rotor rotation in the micro-Doppler spectrum. The signal acquired by the radio frequency monitoring unit is analyzed to identify the unique encoding and transmission rules of the signal, and these rules are used as the radio frequency features. Moving object regions are separated from the images acquired by the photoelectric unit, and the moving object regions are analyzed using a target recognition network to obtain the target's contour features, which are then used as the photoelectric features. Finally, the audio signal acquired by the acoustic unit is subjected to spectral analysis to extract the unique combination of sound frequencies generated by the UAV propellers as the acoustic features.

[0009] As a preferred embodiment of the UAV detection method based on multimodal fusion described in this invention, the method involves: forming a target position sequence continuously detected by radar based on the radar features; predicting the target's spatial position range at the next moment based on the target position sequence; and associating the features among the radio frequency features, the photoelectric features, and the acoustic features whose detection direction points to the spatial position range to form a unified target profile. The specific steps are as follows: The specific details of predicting the spatial location range of the target at the next moment include: making a prediction based on the target's current position and flight speed as reflected in the target position sequence, and dynamically adjusting the size of the spatial location range according to the radar's measurement error and the target's maneuvering state.

[0010] As a preferred embodiment of the UAV detection method based on multimodal fusion described in this invention, the method includes: acquiring real-time environmental parameters; dynamically adjusting the weights of various features in decision fusion based on the real-time environmental parameters; and fusing various features in the unified target profile using evidence theory based on the weights to calculate the target value as the final confidence level of the UAV. The specific steps are as follows: The step of dynamically adjusting the weights specifically includes: reducing the weight of the photoelectric feature when the atmospheric visibility in the real-time environmental parameters is detected to be lower than a preset value; reducing the weight of the radio frequency feature when the environmental electromagnetic noise level in the real-time environmental parameters is detected to be higher than a preset value; and reducing the weight of the acoustic feature when the wind speed in the real-time environmental parameters is detected to be higher than a preset value. The step of using evidence theory fusion specifically includes: for a pre-set identification framework that includes the conclusion that the target is a drone and the conclusion that the target is a bird, generating corresponding evidence confidence assignments based on the features in the unified target file; and then combining the evidence confidence assignments according to their corresponding weights based on the combination rules of the evidence theory to calculate the final confidence level.

[0011] As a preferred embodiment of the UAV detection method based on multimodal fusion described in this invention, the threat level of the target is determined based on the comparison result between the final confidence level and a preset threshold, and the specific steps are as follows: When the threat level is determined to be high, a high-level alarm is issued and the target information in the unified target file is pushed to the countermeasure system; and when the threat level is determined to be medium, the photoelectric unit is automatically instructed to continuously track the target for manual verification by the operator.

[0012] Secondly, the present invention provides a UAV detection system based on multimodal fusion, comprising: The multimodal sensing module includes radar, radio frequency monitoring unit, optoelectronic unit and acoustic unit; A synchronization module connected to the multimodal sensing module is used to attach a unified time stamp to the raw data collected by the multimodal sensing module; The data processing module, connected to the multimodal sensing module, is used to preprocess the raw data and extract radar features, radio frequency features, photoelectric features and acoustic features from the preprocessed data; The target association module, connected to the data processing module, is used to form a target location sequence based on the radar features, predict the spatial location range of the target, and associate the target based on the spatial location range to form a unified target file. The fusion decision module, connected to the target association module, is used to acquire real-time environmental parameters, dynamically adjust the weights of the features, and fuse the features in the unified target file using evidence theory to calculate the final confidence level of the target, and then determine the threat level of the target based on the final confidence level.

[0013] As a preferred embodiment of the UAV detection system based on multimodal fusion described in this invention, the data processing module is configured to: process the radar echo signal from the radar to generate a micro-Doppler spectrum that can show the frequency change of the signal over time; and extract the radar features based on whether there are stable high-frequency harmonic spectral lines generated by the rotation of the UAV rotor in the micro-Doppler spectrum.

[0014] As a preferred embodiment of the UAV detection system based on multimodal fusion according to the present invention, the fusion decision module further includes an environmental perception unit for acquiring the real-time environmental parameters; and the fusion decision module is configured to: reduce the weight of the photoelectric features when the environmental perception unit detects that the atmospheric visibility is lower than a preset value; reduce the weight of the radio frequency features when the environmental electromagnetic noise level is higher than a preset value; and reduce the weight of the acoustic features when the wind speed is higher than a preset value.

[0015] The beneficial effects of this invention are: The system simultaneously acquires raw data from radar, radio frequency monitoring units, optoelectronic units, and acoustic units, extracting their respective features. Based on radar features, it forms a target location sequence and predicts the spatial range to correlate multi-source features, creating a unified target profile. Real-time environmental parameters are acquired, and the weights of various features in decision-making are dynamically adjusted based on these parameters. Finally, evidence theory is used to fuse the weighted features, calculate the target's final confidence level, and determine its threat level. Through multi-source information cross-validation, false alarms caused by complex background factors are effectively reduced. The dynamic weight adjustment mechanism enables the system to adapt to environmental changes such as weather and electromagnetic interference, improving detection reliability. The complementarity of multimodal information increases the detection probability of "low, slow, and small" UAV targets. Furthermore, it can perform graded responses based on confidence levels, improving handling efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is the overall flowchart;

[0018] Figure 2 Here is a flowchart of the multidimensional feature vector extraction process in step S2;

[0019] Figure 3 This is a flowchart of step S3, cross-modal association and documentation.

[0020] Figure 4 The flowchart for step S4, dynamic weight adaptive fusion decision-making, is shown. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0023] Secondly, the term "one embodiment" or "example" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the invention. An embodiment appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0024] Example 1

[0025] Reference Figures 1-4 This is the first embodiment of the present invention, which provides a UAV detection method based on multimodal fusion, including the following steps:

[0026] S1. Synchronously acquire raw data containing a unified time stamp obtained by radar, radio frequency monitoring unit, photoelectric unit and acoustic unit. The specific operation steps are as follows:

[0027] The GPS clock module provides a unified standard time for the radar, radio frequency monitoring unit, optoelectronic unit, and acoustic unit; and when each unit collects raw data, it adds a synchronization time stamp with millisecond-level precision corresponding to the standard time to the raw data.

[0028] It should be noted that the synchronous acquisition involves raw data with a unified time stamp obtained by the radar, radio frequency monitoring unit, optoelectronic unit, and acoustic unit. This step is achieved through a centralized synchronization module, specifically: using the Global Positioning System (GPS) clock module, a unified standard time reference is provided for all sensors deployed within the detection area, namely the radar, radio frequency monitoring unit, optoelectronic unit, and acoustic unit; and when each sensor unit captures its own raw data packet (e.g., data from radar echo cycles, video frames, radio frequency signal segments, or an audio stream), a synchronization time stamp with millisecond-level accuracy generated by the standard time is immediately appended to it.

[0029] S2. Preprocess the raw data, and extract radar features, radio frequency features, photoelectric features, and acoustic features from the preprocessed data. The specific operation steps are as follows:

[0030] The radar echo signal is processed to generate a micro-Doppler spectrum that shows the frequency change of the signal over time. Radar features are extracted based on the presence of stable high-frequency harmonic spectral lines generated by the UAV rotor rotation in the micro-Doppler spectrum. The signals collected by the radio frequency monitoring unit are analyzed to identify the unique encoding and transmission rules of the signals, and these rules are used as radio frequency features. Moving object regions are separated from the images collected by the photoelectric unit, and the moving object regions are analyzed using a target recognition network to obtain the contour features of the targets, which are then used as photoelectric features. Finally, the audio signals collected by the acoustic unit are subjected to spectral analysis to extract the unique combination of sound frequencies generated by the UAV propellers as acoustic features.

[0031] It should be noted that the extraction process for various features is as follows:

[0032] Radar Feature Extraction: The system processes the raw echo signal received by the radar to generate a micro-Doppler spectrum that displays the frequency components of the signal over time in a two-dimensional manner. Due to the high-speed, uniform rotation of its rigid rotor, UAVs exhibit multiple clear, stable, and high-frequency harmonic spectral lines on the spectrum, resembling a set of parallel lines. In contrast, non-rigid targets such as birds, due to the reciprocating flapping of their wings, produce lower-frequency, dispersed, and chaotic spectral bands. The system uses algorithms to identify the presence of such stable high-frequency harmonic spectral line structures in the spectrum and uses them as the core radar feature characterizing the kinematic properties of the UAV.

[0033] Radio Frequency (RF) Feature Extraction: The system demodulates and analyzes signals captured by the RF monitoring unit in specific frequency bands (such as 2.4GHz and 5.8GHz). Its core function is to identify the unique encoding and transmission rules of the signals, essentially parsing the signal's "communication protocol." Different drone manufacturers use different communication protocols, which differ significantly in data frame structure, frequency hopping mode, and modulation method. The system identifies the type of the captured signal by comparing its rules with a built-in library of known drone protocols, such as "DJI OcuSync protocol (proprietary high-definition digital image transmission technology)" or "standard Wi-Fi protocol," and uses this identification result as the RF feature.

[0034] Photoelectric Feature Extraction: The system first performs background modeling and subtraction on the high-definition visible light image sequence acquired by the photoelectric unit to accurately separate the moving object regions in the image. Then, it calls a deep learning target recognition network (such as YOLOv5) pre-trained on tens of thousands of UAV images to analyze the moving object regions. The network can not only determine whether the region belongs to a UAV, but also output a bounding box. Based on the bounding box, the system further calculates and extracts the target's contour features, such as the target's aspect ratio, compactness, and Hu moment invariants. These features together constitute an accurate description of the target's geometric shape.

[0035] Among them, the so-called Hu moment invariant is a value calculated from the outline of an object using a specific mathematical method. Its core advantage is that no matter how the target rotates in the air or how large or small it appears in the picture, this set of values ​​remains basically constant, providing a basis for the system to identify targets under complex dynamic conditions.

[0036] Acoustic Feature Extraction: The system performs a Fast Fourier Transform (FFT) on the audio signal acquired by the acoustic unit (usually a microphone array) to obtain its spectrum. By analyzing the spectrum, the system focuses on finding unique combinations of sound frequencies with a specific fundamental frequency and its integer multiples of harmonics, generated by the drone propellers cutting through the air. This sound feature is similar to a "humming sound," and its spectral structure is significantly different from common environmental noises (such as wind noise and traffic noise), serving as an effective auxiliary identification feature.

[0037] S3. Based on radar characteristics, a sequence of target positions continuously detected by radar is formed. Based on this sequence, the spatial range of the target's location at the next moment is predicted. Features from radio frequency, photoelectric, and acoustic characteristics whose detection direction points to the spatial range are correlated to form a unified target profile. The specific operation steps are as follows:

[0038] The specific steps for predicting the spatial location range of a target in the next moment include: making a prediction based on the target's current position and flight speed as reflected in the target's position sequence, and dynamically adjusting the size of the spatial location range according to radar measurement errors and the target's maneuvering state.

[0039] It should be noted that after obtaining multi-source features, it was confirmed that these features from different sensors point to the same target. The specific implementation steps are as follows:

[0040] 1. Establishment of the reference track: The system uses radar data with the most stable detection performance as the reference. When the radar detects the same target in consecutive scanning cycles, these temporally consecutive position points constitute the target's trajectory, i.e., the target position sequence.

[0041] 2. Dynamic Generation of Correlation Gates: The system employs trajectory tracking algorithms such as the Kalman filter to predict the spatial range in which the target is most likely to appear in the next time moment, based on the target's current position, flight speed, and acceleration as reflected in the target position sequence. The size of this range is not fixed but intelligently adjusted: if the target's flight is stable, the range is smaller to improve correlation accuracy; if the target is performing high-speed maneuvers or radar measurements show fluctuations, the range will be appropriately enlarged to ensure the target does not miss.

[0042] 3. Cross-modal information association: The system unifies the directional information provided by other sensors (such as the direction finding results of the radio frequency unit, the sound source localization direction of the acoustic unit, and the target coordinates of the photoelectric unit in the image) into a three-dimensional coordinate system centered on the radar. It then determines whether these direction vectors intersect with the predicted spatial location range. If an intersection exists, it is considered that these features and the radar track point to the same physical target.

[0043] In order to determine whether directional observation information provided by non-radar sensors (such as radio frequency or photoelectric sensors) is associated with the target track established by the radar, this embodiment uses Mahalanobis distance as the criterion. The calculation process for association determination is as follows:

[0044]

[0045] : Represents the square of the Mahalanobis distance. is a "normalized distance" that takes into account uncertainty, which can be understood as the correlation cost or mismatch degree. The smaller its value, the better the match between the observed information and the predicted trajectory.

[0046] : Represents the observation vector provided by a non-radar sensor. For example, for an optoelectronic unit, It can be a two-dimensional vector containing the target azimuth and elevation angles. .

[0047] This represents the target state prediction vector given by a radar tracker (such as a Kalman filter). It typically includes information such as the target's position and velocity. .

[0048] : Represents the observation matrix. It is the state prediction vector. Mapped to the observation vector The same space. For example, three-dimensional coordinates. Converted into predicted azimuth and elevation angles.

[0049] This indicates a theoretical observation value predicted based on radar tracks. For example, It refers to the system predicting the azimuth and elevation angles that the target should appear from the perspective of the photoelectric unit.

[0050] : Represents the covariance matrix of the innovation. Represents the size and shape of the "spatial location range" (i.e., the correlation gate), quantifying the uncertainty of the prediction. If radar measurements are stable, The smaller the element values ​​of the matrix, the smaller the correlation gate; conversely, if the target is maneuvering, The larger the element values ​​of the matrix, the larger the correlation gate.

[0051] and : These represent the transpose and inverse operations of a matrix, respectively.

[0052] Calculate observations Compared with theoretical predictions The residuals between Using the inverse of the covariance matrix This residual is then weighted. The significance of this weighting process is that if the uncertainty in the prediction in a certain direction is large ( If the corresponding element value is large, then even if the residual is large, it will affect the final result. The contribution will also be reduced. This is achieved by comparing it with a preset threshold. Comparison, (Association threshold) is selected based on the chi-square distribution. For example, for two-dimensional observations (azimuth and elevation), if a 99% confidence level is desired, then a threshold of [value missing] can be selected. =9.21, if If so, the association is considered successful.

[0053] 4. Formation of a Unified Profile: Once the association is successful, the system creates a structured data record, namely the "Unified Target Profile." This profile is indexed by a unique target ID and stores all information related to that target: the three-dimensional track provided by radar, micro-Doppler characteristics, radio frequency protocol type, electro-optical profile characteristics, acoustic spectrum characteristics, and their respective timestamps. By establishing the profile, scattered intelligence is integrated into a complete description of a single target.

[0054] S4. Obtain real-time environmental parameters and dynamically adjust the weights of various features in the decision fusion based on these parameters. Using evidence theory, fuse and unify the features in the target profile, and calculate the target value as the final confidence level of the UAV. The specific steps are as follows:

[0055] The steps for dynamically adjusting weights specifically include: reducing the weight of photoelectric features when atmospheric visibility in real-time environmental parameters is lower than a preset value; reducing the weight of radio frequency features when environmental electromagnetic noise levels in real-time environmental parameters are higher than a preset value; and reducing the weight of acoustic features when wind speed in real-time environmental parameters is higher than a preset value. The steps for using evidence theory fusion specifically include: for a pre-defined identification framework that includes the conclusions of "target is a drone" and "target is a bird," generating corresponding evidence confidence assignments based on various features in the unified target file; and then, according to the combination rules of evidence theory, combining the confidence assignments of various pieces of evidence according to their corresponding weights to calculate the final confidence level.

[0056] It should be noted that the dynamic adaptive adjustment of weights is the key difference between this invention and traditional fixed-weight fusion methods. The system has a built-in environmental sensing unit that continuously monitors environmental parameters that affect the performance of each sensor. For example, when the illuminance meter in the photoelectric unit detects that the ambient light level is below 1... (At night), or when the visibility sensor reports that the atmospheric visibility is less than 500 meters (in foggy or hazy weather), the system determines that the reliability of the photoelectric sensor has decreased significantly and will automatically reduce the weighting coefficient of its photoelectric characteristics from the default 1.0 to 0.2.

[0057] When the radio frequency monitoring unit analyzes and finds that the average power spectral density of the background electromagnetic noise is higher than -65 In a strong interference environment, the system will correspondingly reduce the weight of the radio frequency characteristics.

[0058] When an external weather sensor shows that the current wind speed exceeds 15 m / s (strong wind weather), the weight of the acoustic features will also be reduced.

[0059] In order to dynamically adjust the weights of various features based on real-time environmental parameters, this embodiment uses an exponentially decaying function to calculate the weights. Taking the weight calculation of photoelectric features as an example:

[0060]

[0061] : Represents the final weight of the features provided by the photoelectric unit.

[0062] : Represents the natural exponential function.

[0063] : Represents the sensitivity coefficient for weight decay. It is an adjustable parameter. The larger the value, the faster the weight decreases as the environment deteriorates. The value is a positive number between 0.5 and 5.0, determined based on the actual system debugging results.

[0064] : Represents the real-time atmospheric visibility value measured by the environmental sensing unit.

[0065] : Indicates the ideal or optimal atmospheric visibility value for the operation of the photoelectric unit.

[0066] It should also be noted that after determining the weights of each piece of evidence, the system begins the fusion calculation. For a pre-defined identification framework containing only the conclusions "target is a drone" and "target is a bird," the system generates a corresponding "confidence score assignment" based on each feature in the unified target file using a pre-defined function. For example, a strong micro-Doppler feature might generate an assignment of (drone: 0.9, bird: 0.05, uncertain: 0.05). Following the combination rules of evidence theory, the system iteratively merges all these "confidence score assignments" with different weights. This process is a more rigorous mathematical reasoning process that handles uncertainty, ultimately outputting a single, final confidence score value that integrates all positive and negative evidence.

[0067] To integrate weighted evidence from multiple sensors, this embodiment employs evidence theory combination rules to iteratively merge all weighted "evidence confidence assignments." The calculation process can be broken down into the following two formulas:

[0068] The first step is to calculate the degree of conflict between two different pieces of evidence (e.g., from sensor 1 and sensor 2). :

[0069] (Confidence level of sensor 1 supporting conclusion X * confidence level of sensor 2 supporting conclusion Y)

[0070] Summation here ( This only applies to cases where all conclusions X and Y conflict with each other (e.g., X is "drone" and Y is "bird").

[0071] The second step is to calculate the final support for any conclusion A (e.g., "drone") after fusion. :

[0072] (Confidence level of sensor 1 supporting conclusion B * confidence level of sensor 2 supporting conclusion C)

[0073] Summation here ( This only applies to the case where all combinations of conclusions B and C can lead to conclusion A.

[0074] Here, confidence level refers to the numerical value of support for a certain conclusion (such as "drone," "bird," or "uncertain") in the confidence level allocation of various pieces of evidence. Degree of Conflict Normalization factor is used to measure the degree of "disagreement" between two pieces of evidence. This is the "calibration" coefficient, whose function is to proportionally redistribute the confidence levels "wasted" due to disagreement in the final allocation to the undisputed conclusions. Final Support This refers to the system's new and more credible judgment on conclusion A after integrating two pieces of evidence.

[0075] The entire fusion process can be understood as a rigorous reasoning process. Based on the formula in the first step, the system identifies all directly contradictory parts between the two pieces of evidence and calculates the total degree of contradiction. According to the formula in the second step, the system identifies all mutually corroborating or non-contradictory parts of the two pieces of evidence, multiplies their support levels, and sums them. The sum is then multiplied by a "calibration" coefficient to obtain the final cross-validated decision for a given conclusion. In practice, this fused decision serves as new, stronger evidence, iteratively fused with evidence from the next sensor until all evidence is adopted, ultimately yielding a global final confidence level. The core advantage of this method lies in its ability to scientifically handle information uncertainty and conflict, thereby arriving at conclusions more reliable than any single piece of evidence.

[0076] S5. Based on the comparison between the final confidence level and the preset threshold, determine the threat level of the target. The specific steps are as follows:

[0077] When the threat level is determined to be high, a high-level alarm is issued and the target information in the unified target file is pushed to the countermeasure system; and when the threat level is determined to be medium, the photoelectric unit is automatically instructed to continuously track the target for manual verification by the operator.

[0078] It should be noted that the system sets two key thresholds: a high-level threshold (e.g., 0.90) and a medium-level threshold (e.g., 0.60).

[0079] If the final confidence level is greater than 0.90, the system classifies the target's threat level as "high," indicating that the system has a very high degree of confidence in its assessment that the target is a drone. The system will immediately execute a high-level response: issuing the highest-priority audible and visual alarm on the operating terminal and marking the target's precise trajectory with a flashing red icon on the electronic map interface; simultaneously, it will push a structured data packet containing information such as the target ID, real-time 3D coordinates, and velocity vector to the associated downstream countermeasures systems (such as radio frequency jamming equipment and laser weapons) in real time via the network interface, providing them with precise guidance.

[0080] If the final confidence level is between 0.60 and 0.90, the system will classify the threat level as "Medium." This indicates that the target is highly suspicious, but a degree of uncertainty remains. The system will execute a medium-level response: issuing a warning alarm and automatically instructing the photoelectric unit to align its turntable with the target's current position, and activating automatic zoom and tracking functions to display a real-time image of the target on the operator's screen for final manual review and decision-making.

[0081] If the final confidence level is below 0.60, the system determines that it does not pose a threat and simply monitors it silently in the background or removes it directly from the current target list, thus avoiding interference from irrelevant information.

[0082] Example 2

[0083] This invention provides a UAV detection system based on multimodal fusion, which is the second embodiment of this invention, comprising:

[0084] The multimodal sensing module includes radar, radio frequency monitoring unit, optoelectronic unit and acoustic unit;

[0085] The synchronization module, connected to the multimodal sensing module, is used to attach a unified time stamp to the raw data collected by the multimodal sensing module.

[0086] The data processing module, connected to the multimodal sensing module, is used to preprocess the raw data and extract radar features, radio frequency features, photoelectric features and acoustic features from the preprocessed data;

[0087] The target association module, connected to the data processing module, is used to generate target location sequences based on radar features, predict the spatial location range of the targets, and associate them according to the spatial location range to form a unified target file.

[0088] The fusion decision module, connected to the target association module, is used to acquire real-time environmental parameters, dynamically adjust the weights of features, and use evidence theory to fuse and unify the features in the target file to calculate the final confidence level of the target, and then determine the threat level of the target based on the final confidence level.

[0089] Example 3

[0090] To further clarify the technical solution of the present invention, the following is a third embodiment of the present invention, using a specific scenario of detecting illegally intruding drones in an area of ​​critical infrastructure as an example, to provide a more detailed description of the technical solution of the present invention.

[0091] In this embodiment, the UAV detection system of the present invention was deployed within the airspace protection zone of a major transportation hub, such as a large international airport. One afternoon, an unauthorized small quadcopter UAV intruded into the protected area from a distance at low altitude.

[0092] Step S1: Synchronized acquisition of multimodal data

[0093] Once the drone entered the detection range, the system's multimodal perception module was triggered. The radar initially detected a faint moving point with a small radar cross-section (RCS) at the X-coordinate; the radio frequency monitoring unit also detected a weak signal operating in the 2.4 GHz band, exhibiting frequency hopping characteristics; a blurry moving pixel appeared in the wide-angle camera image of the optoelectronic unit; and the acoustic unit received a faint abnormal noise from that direction. All these initial findings from different sensors were assigned a unified time stamp, accurate to milliseconds and originating from the same GPS clock, ensuring absolute synchronization of all clues in time.

[0094] Step S2: Extraction of multidimensional feature vectors

[0095] The system then performs parallel feature extraction on the various types of raw data collected:

[0096] For the points captured by the radar, the system generates a micro-Doppler spectrum image through processing. The image shows several stable and high-frequency parallel harmonic spectral lines generated by the high-speed rotation of the UAV rotor. Based on this, the system extracts the "UAV kinematics" radar features.

[0097] For the detected 2.4GHz signal, the system analyzes its encoding and transmission rules to identify the communication protocol commonly used by DJI drones, and uses this identification result as the decisive radio frequency feature.

[0098] The moving object separated from the image of the photoelectric unit is analyzed by the target recognition network. Its aspect ratio is calculated to be 1.1 and its compactness is calculated to be 0.85. At the same time, the shape recognition code, which is not sensitive to attitude, is highly consistent with the benchmark value of similar quadcopter UAVs in the database. These together constitute the photoelectric features.

[0099] Spectral analysis of the audio signal also revealed harmonic peaks that highly matched the acoustic model of the small UAV propeller, forming an effective acoustic feature.

[0100] Step S3: Cross-modal association and establishment of a unified profile

[0101] Based on the most stable target position sequence generated by radar, the system predicts that the UAV will enter an ellipsoidal spatial range in the next instant, according to its flight speed. The system determines that the azimuth of the OcuSync signal reported by the radio frequency unit and the coordinates of the object captured by the optoelectronic unit in the image both point to this range. Based on strong correlation, the system confirms that all these features from different dimensions point to the same physical target. At this point, a unified target profile with a unique ID is successfully created, which includes all the UAV's radar, radio frequency, optoelectronic, and acoustic features.

[0102] Step S4: Adaptive Fusion Decision Based on Dynamic Weights

[0103] Entering the decision-making phase, assuming it is daytime with light fog and visibility of approximately 1500 meters, and some background electromagnetic interference around the airport, the system automatically adjusts the weight of photoelectric features from the initial 1.0 to 0.85, and adjusts the weight of radio frequency features to 0.9, while maintaining the weights of radar and acoustic features at a relatively high level.

[0104] When fusing the evidence theories, the strong micro-Doppler characteristics of the radar gave a support score as high as 0.9 for the "UAV conclusion"; the OcuSync protocol from radio frequency identification gave an extremely high support score of 0.95; while the photoelectric evidence affected by fog gave a more conservative support score of 0.65. After iterative fusion calculation of all weighted evidence, the system finally concluded that the confidence level of the target was 97.8%.

[0105] Step S5: Threat Level Determination and Response Classification

[0106] Since the final confidence level of 97.8% is much higher than the preset high-level threshold of 90%, the system immediately determined the threat level of the drone to be "high level".

[0107] The system immediately issued the highest-level audible and visual alarm on the airspace monitoring screen in the control tower, and highlighted the drone's icon in flashing red. A real-time data stream containing the drone's precise three-dimensional coordinates, speed, and heading was pushed to the downstream countermeasure system via an application programming interface. In this embodiment, this system is specifically a directional radio frequency jamming system deployed at the airport, providing precise targeting and suppression guidance.

[0108] In summary, the system simultaneously collects raw data from radar, radio frequency monitoring units, optoelectronic units, and acoustic units, extracting their respective features. Based on radar features, a target location sequence is formed, and the spatial location range is predicted to correlate multi-source features, creating a unified target profile. Real-time environmental parameters are acquired, and the weights of various features in the decision-making process are dynamically adjusted based on these parameters. Finally, evidence theory is used to fuse the weighted features, calculate the target's final confidence level, and determine its threat level. Cross-validation of multi-source information effectively reduces false alarms caused by complex background factors. The dynamic weight adjustment mechanism enables the system to adapt to environmental changes such as weather and electromagnetic interference, improving detection reliability. The complementarity of multimodal information increases the detection probability of "low, slow, and small" UAV targets. Furthermore, the system can perform graded responses based on confidence levels, improving handling efficiency.

[0109] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A UAV detection method based on multimodal fusion, characterized in that, Includes the following steps: S1. Synchronously acquire raw data containing a unified time stamp obtained by radar, radio frequency monitoring unit, photoelectric unit and acoustic unit; S2. Preprocess the raw data and extract radar features, radio frequency features, photoelectric features and acoustic features from the preprocessed data respectively; S3. Based on the radar features, a target position sequence continuously detected by the radar is formed, and the target position sequence is used to predict the spatial position range of the target in the next moment. The radio frequency features, the photoelectric features and the acoustic features whose detection direction points to the spatial position range are associated to form a unified target file. S4. Obtain real-time environmental parameters, and dynamically adjust the weights of various features in decision fusion based on the real-time environmental parameters; based on the weights, use evidence theory to fuse various features in the unified target file, and calculate the target value as the final confidence level of the UAV. S5. Determine the threat level of the target based on the comparison result between the final confidence level and the preset threshold.

2. The UAV detection method based on multimodal fusion as described in claim 1, characterized in that, The specific steps of the synchronous acquisition include: using the Global Positioning System clock module to provide a unified standard time for the radar, the radio frequency monitoring unit, the photoelectric unit, and the acoustic unit; and when each unit acquires the raw data, adding a synchronization time stamp with millisecond-level precision corresponding to the standard time to the raw data.

3. The UAV detection method based on multimodal fusion as described in claim 1, characterized in that, The steps for extracting radar features, radio frequency features, photoelectric features, and acoustic features specifically include: processing the radar echo signal to generate a micro-Doppler spectrum diagram that shows the frequency change of the signal over time, and extracting the radar features based on whether there are stable high-frequency harmonic spectral lines generated by the rotation of the UAV rotor in the micro-Doppler spectrum diagram; analyzing the signal collected by the radio frequency monitoring unit, identifying the unique encoding and transmission rules of the signal, and using the encoding and transmission rules as the radio frequency features; separating the moving object region in the image collected by the photoelectric unit, and using a target recognition network to analyze the moving object region to obtain the contour shape features of the target, and using the contour shape features as the photoelectric features; and performing spectrum analysis on the audio signal collected by the acoustic unit to extract the unique combination of sound frequencies generated by the UAV propeller as the acoustic features.

4. The UAV detection method based on multimodal fusion as described in claim 1, characterized in that, The step of predicting the spatial location range of the target at the next moment specifically includes: making a prediction based on the target's current position and flight speed as reflected in the target position sequence, and dynamically adjusting the size of the spatial location range according to the radar's measurement error and the target's maneuvering state.

5. The UAV detection method based on multimodal fusion as described in claim 1, characterized in that, The step of dynamically adjusting the weights specifically includes: reducing the weight of the photoelectric feature when the atmospheric visibility in the real-time environmental parameters is detected to be lower than a preset value; reducing the weight of the radio frequency feature when the environmental electromagnetic noise level in the real-time environmental parameters is detected to be higher than a preset value; and reducing the weight of the acoustic feature when the wind speed in the real-time environmental parameters is detected to be higher than a preset value.

6. The UAV detection method based on multimodal fusion as described in claim 1, characterized in that, The steps of using evidence theory fusion specifically include: for a pre-defined identification framework that includes the conclusion that the target is a drone and the conclusion that the target is a bird, generating corresponding evidence confidence assignments based on the features in the unified target file; and then, according to the combination rules of the evidence theory, combining the evidence confidence assignments according to their corresponding weights to calculate the final confidence score.

7. The UAV detection method based on multimodal fusion as described in claim 1, characterized in that, When the threat level is determined to be high, a high-level alarm is issued and the target information in the unified target file is pushed to the countermeasure system. And when the threat level is determined to be medium, the photoelectric unit is automatically instructed to continuously track the target for manual verification by the operator.

8. A UAV detection system based on multimodal fusion, characterized in that, include: The multimodal sensing module includes radar, radio frequency monitoring unit, optoelectronic unit and acoustic unit; A synchronization module connected to the multimodal sensing module is used to attach a unified time stamp to the raw data collected by the multimodal sensing module; The data processing module, connected to the multimodal sensing module, is used to preprocess the raw data and extract radar features, radio frequency features, photoelectric features and acoustic features from the preprocessed data; The target association module, connected to the data processing module, is used to form a target location sequence based on the radar features, predict the spatial location range of the target, and associate the target based on the spatial location range to form a unified target file. The fusion decision module, connected to the target association module, is used to acquire real-time environmental parameters, dynamically adjust the weights of the features, and fuse the features in the unified target file using evidence theory to calculate the final confidence level of the target, and then determine the threat level of the target based on the final confidence level.

9. The UAV detection system based on multimodal fusion as described in claim 8, characterized in that, The data processing module is configured to: process the radar echo signal from the radar to generate a micro-Doppler spectrum that shows the frequency change of the signal over time; and extract the radar features based on whether there are stable high-frequency harmonic spectral lines generated by the rotation of the UAV rotor in the micro-Doppler spectrum.

10. The UAV detection system based on multimodal fusion as described in claim 8, characterized in that, The fusion decision module further includes an environmental sensing unit for acquiring the real-time environmental parameters; and the fusion decision module is configured to: reduce the weight of the photoelectric features when the environmental sensing unit detects that the atmospheric visibility is lower than a preset value; reduce the weight of the radio frequency features when the environmental electromagnetic noise level is higher than a preset value; and reduce the weight of the acoustic features when the wind speed is higher than a preset value.

Images