Unmanned aerial vehicle detection and tracking method based on multi-sensor fusion and deep learning

Abstract

The invention discloses an unmanned aerial vehicle detection and tracking system based on multi-sensor fusion and a deep learning technology. The unmanned aerial vehicle detection and tracking system comprises a data collection layer, a data processing layer, an extended Kalman filter (EKF) estimation layer and a deep learning layer. The system firstly judges an unmanned aerial vehicle needing to be tracked, then obtains target distance, speed, direction, image information and the like through a data collection layer (a radar, a camera and an RF signal collection device), and the data processing layer completes time synchronization, denoising, data standardization and data enhancement processing. The EKF estimation layer performs real-time estimation on target motion states (position, speed, acceleration and the like), and the deep learning layer can adopt a target detection and classification model method to realize detection, classification and tracking of the unmanned aerial vehicle in combination with EKF output and image features. The system can detect and track a single unmanned aerial vehicle in a target airspace and a key unmanned aerial vehicle of an unmanned aerial vehicle cluster, and is suitable for the field of airspace management and security.

Description

Technical Field

[0001] The present invention relates to the field of drone detection and tracking, and specifically designs a drone detection and tracking system based on multi-sensor fusion, extended Kalman filter (EKF) and deep learning technology. Background Art

[0002] In the wave of modern automation and intelligent development, drone systems have shown their unique value and potential in many areas. Drones can perform a variety of tasks by integrating advanced sensor technology. Due to the widespread use of drones, drone detection and positioning has become a popular research direction. The positioning of drones requires not only real-time perception and analysis of the surrounding environment, but also accurate detection of the target.

[0003] With the advancement of technology, the original single reliance on GPS can no longer meet the current drone detection needs. Most drone detection systems use wireless signals such as Bluetooth signals, WiFi signals, and wireless radio frequency signal lights to extract features for detection. This method requires complex communication signal processing and is difficult to implement.

[0004] Therefore, the UAV detection system that combines multiple sensors and advanced technologies can better adapt to existing needs. Although multi-sensor fusion technology has made great progress, autonomous unmanned systems still face many challenges in practical applications. For example, dynamic changes in the environment require sensors to have higher learning and adaptability, and the fusion processing of sensor data also requires more efficient algorithm support. Research on these issues has given rise to the integration of emerging technologies, including deep learning, large model technology, and so on. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a drone detection and tracking system based on multi-sensor fusion and deep learning to solve the current problems of drone detection. By introducing EKF state estimation and deep learning technology, the system can be effectively provided with better detection accuracy and tracking efficiency.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] The drone detection and tracking method proposed in this patent includes the following steps:

[0008] Step 1: Select the target drone that needs to be detected and tracked, and collect its basic information. The data collection layer collects multi-dimensional information of the target in the airspace through multiple sensors, including the distance, speed, direction and image data of the target. These data can be used for subsequent preliminary state estimation;

[0009] Step 2: Preprocess the collected target information. The data processing layer is used to perform time synchronization, noise removal, data standardization and data enhancement on the data collected by the data collection layer to form high-quality input in a unified format, and map it to the local world coordinate system to improve the efficiency of subsequent use of data and locate the target drone more quickly.

[0010] Step 3: Preliminary estimation and update of the target UAV state. The EKF estimation layer fuses the sensor data based on the input provided by the data processing layer, and attempts to estimate the motion state of the target UAV, including dynamic information such as position, velocity, and acceleration, and then updates and corrects the UAV position and other information;

[0011] Step 4: Predict and update the state of the target UAV. The deep learning layer combines the state estimation information output by the EKF and the images collected by the sensor, and uses the target detection and classification model to achieve real-time detection and classification of the UAV.

[0012] As a drone detection and tracking method based on multi-sensor fusion and deep learning, the patent belongs to a detection and tracking scheme that includes four levels: data collection layer, data processing layer, EKF estimation layer, and deep learning layer.

[0013] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, this patent belongs to. Before collecting data, the detected drone needs to be confirmed. If it is a single drone, it can be detected and tracked directly. If it is a drone cluster, it is necessary to identify the key drones and then detect and track them.

[0014] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, to which this patent belongs, the data collection layer adopts radar equipment, cameras, and RF signal collection equipment. The radar equipment is used to obtain the distance and speed information of the target, the camera is used to capture the image of the drone target, and the RF radio frequency signal collection equipment is used to capture the control signal characteristics of the drone. The information captured by the above different devices is relative information, which needs to be converted into absolute information for subsequent use.

[0015] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, to which this patent belongs, the data processing layer needs to pre-process the data collected by the sensor layer to facilitate subsequent drone detection. The functions of the data processing layer include time synchronization module, denoising module, data standardization module and data enhancement processing module.

[0016] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, to which this patent belongs, the time synchronization module of the data processing layer is used to align the timestamps of the data collected by the sensor to ensure that the data time used subsequently is synchronized. The denoising module is used to select a suitable filtering algorithm for the source of the data to remove the environmental noise in the signal and improve the subsequent data utilization efficiency. The data standardization module is used to unify the format and unit of the sensor output, and map the collected drone information to the local world coordinate system, so as to facilitate the subsequent input to the EKF estimation layer as the EKF. The data enhancement module is used to expand the diversity and robustness of the sensor data.

[0017] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, to which this patent belongs, the time synchronization module described in the data processing layer aligns the sensor data through a timestamp alignment method; the denoising module removes noise through Gaussian filtering, frequency filtering and other methods; the data standardization module unifies the data standard through formatting processing, and maps the drone information to the local world coordinate system; the data enhancement module improves the efficiency and robustness of the subsequent EKF estimation layer by extracting features from the data.

[0018] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, the working steps of the EKF estimation layer are as follows:

[0019] Step 3.1, the state prediction module uses the motion state equation of the UAV system and the data detected by the sensor to predict the next state of the target (including position, speed and acceleration);

[0020] Step 3.2, the state update module corrects the state prediction value through the data collected by the sensor, and uses the Kalman gain to adjust the impact of the sensor data on the state update.

[0021] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, the specific working steps of the state prediction module are as follows:

[0022] First, define the state transfer equation:

[0023] x k =Fx k-1 +Bu k

[0024] Where F is the state transfer matrix, which represents the linear change of the state over time, B is the control matrix, which represents the influence of the control input on the state, and u k is the control input; for complex nonlinear motion, the state vector is defined as a six-tuple, x k =[xyzvx v y v z ] T , where x, y, and z all represent the three-dimensional distance based on the detection device, and v x 、v y 、v z It represents the three-dimensional velocity of the target. According to the state transfer equation, the corresponding Jacobian matrix is ​​also required:

[0025]

[0026] Where f(x, u) is the state transfer function, which can be obtained through the state transfer equation, and then the prediction error covariance P needs to be calculated k :

[0027]

[0028] Here P k|k-1 represents the error covariance of predicting time k based on time k-1, F k represents the Jacobian matrix at time k, Q k Represents the process noise covariance matrix; After the above process, the position and motion state of the UAV can be preliminarily predicted. Even if no update or correction is performed at this time, a reasonable prediction can still be made based on the target's motion model and the state estimate at the previous moment.

[0029] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, the specific working steps of the status update module are as follows:

[0030] By calculating the predicted observed value z^ k|k-1 and the innovation vector y k To calculate the Kalman gain K k , the predicted state of the drone is updated through the Kalman gain, and the updated value is used for the position prediction at the next moment. It can also slowly correct the errors of the state estimation value and state covariance matrix obtained in the prediction stage to achieve higher accuracy. The Kalman gain K k The calculation formula is:

[0031]

[0032] Where K k Represents the Kalman gain, which determines the weight of the observed value and the predicted value in the state update. Of course, before this, it is necessary to calculate the predicted value of the observation, that is, to use the observation function h(x) and the predicted state to estimate x^ k∣k-1 Calculate the prediction of the current observation value, that is, the ideal observation value, for the subsequent innovation vector y kThe calculation steps are as follows:

[0033]

[0034]

[0035] where z^ k|k-1 Represents the predicted observation value, which is calculated using the observation function. h(x) represents the observation function, which describes the nonlinear relationship between the state and the observation value, where z k represents the actual observation value provided by the sensor, z^ k|k-1 Represents the predicted observation value; after calculating the Kalman gain, the state of the target drone can be updated. The next update requires updating the state estimate and the error covariance.

[0036] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, the deep learning layer needs to detect and classify drones based on the input data. The target drone is detected and tracked through the deep learning model to achieve the detection and monitoring functions of the drone, and combined with the EKF estimation layer to enhance the detection efficiency and robustness of the system.

[0037] As a drone detection and tracking method based on multi-sensor fusion and deep learning technology, this patent belongs to the following: first, the YOLO model is selected in the deep learning layer, which can detect drone targets in real time and has high detection speed and accuracy; secondly, the ResNet model is selected to classify the types of drones; then, the target motion state output by the EKF estimation layer and the sensor data are used to build a model to predict the future motion trajectory of the target; finally, the EKF estimation layer and the deep learning layer are combined, and the target state information output by the EKF and the image features are jointly input into the deep learning model to optimize the target detection and classification performance.

[0038] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0039] The present invention adopts a UAV detection and tracking method based on multi-sensor fusion and deep learning technology, which improves the detection accuracy through multi-sensor fusion and introduces EKF to realize efficient state estimation. It can not only effectively detect the target UAV but also perform continuous tracking. In addition, the joint optimization of deep learning and EKF effectively overcomes the problem of insufficient adaptability of traditional detection methods. This joint optimization effectively solves the problems of low UAV detection accuracy and tracking efficiency, and provides an intelligent and efficient innovative solution for the field of airspace management and security. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1This is the overall flow chart of the embodiment of the present invention. The four modules work together to complete

[0041] Figure 2 This is a specific implementation diagram of an embodiment of the present invention, including the work completed by each module and the equipment required

[0042] Figure 3 Specific implementation flow chart of the EKF estimation layer of the embodiment of the present invention DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0044] like Figure 1 The system architecture of the present invention is shown. First, determine the drone that needs to be detected and tracked, and then enter the data collection layer. The data includes radar equipment, cameras, and RF signal collection equipment, which respectively collect the distance, speed, direction, and image data of the drone. These data are sent to the data processing layer, and after time synchronization, denoising, data standardization, and data enhancement processing, high-quality input data is formed. The EKF estimation layer receives the processed data, performs state prediction and update, and estimates the motion state of the drone. The deep learning layer combines the output of the EKF and the image features of the sensor layer camera input, uses the YOLO model for target detection, and the ResNet model for type classification, and then combines with the output of the EKF estimation layer to optimize the drone detection state and update state, and finally continuously detects and tracks the target drone.

[0045] like Figure 2 The figure shows the equipment required by each module, the work completed, and how to collaborate for input and output. The following is a detailed description of each module and the work completed.

[0046] First, determine whether the target drone is a single drone or a drone cluster. A single drone can be directly detected and tracked. A drone cluster needs to identify the key drones before detection and tracking. Next, enter the data collection layer.

[0047] The first layer, data collection layer:

[0048] Different data information of the target drone is detected and input through different devices, which respectively cover the position, relative distance, movement direction and other information of the target drone. Radar equipment, camera, RF signal collection equipment, radar equipment is used to obtain the distance and speed information of the target, camera is used to capture the image of the drone target, and RF radio frequency signal analysis equipment is used to capture the control signal characteristics of the drone.

[0049] The second layer, data processing layer:

[0050] The data processing layer includes a time synchronization module, a denoising module, a data standardization module, and a data enhancement processing module; the time synchronization module is used to align the timestamps of the data collected by the sensor to ensure that the data time used later is synchronized, the denoising module is used to select a suitable filtering algorithm for the source of the data to remove the environmental noise in the signal and improve the efficiency of subsequent data use, the data standardization module is used to unify the format and unit of the sensor output to facilitate the subsequent input to the EKF estimation layer as EKF, and the data enhancement module is used to expand the diversity and robustness of the sensor data. Through the above processing, the data availability and efficiency input to the next layer of EKF estimation layer will be higher.

[0051] Optionally, the time synchronization module aligns the sensor data through a timestamp alignment method; the denoising module removes noise through Gaussian filtering, frequency filtering and other methods; the data standardization module unifies the data standard through formatting processing and maps it to a unified local world coordinate system; the data enhancement module improves the efficiency and robustness of the subsequent EKF estimation layer by extracting features from the data.

[0052] The steps for mapping data to the local absolute coordinate system are as follows: First, establish an absolute three-dimensional coordinate system for the target airspace. Here, we take the distance coordinate as an example. After obtaining the data input from the radar and camera, it can be converted to the world coordinate system according to the following formula:

[0053] P w =R stw ·P s +T stw

[0054] Where P w Represents the position of the world coordinate system, P s Represents the position in the sensor coordinate system, that is, the relative position measured by the sensor, R stw Represents the rotation matrix from the sensor coordinate system to the world coordinate system, describing the rotation relationship between the sensor coordinate system and the world coordinate system, T stw Indicates the position of the sensor center point in the world coordinate system, that is, the translation vector; if the sensor input is polar coordinates (for radar equipment), it can be converted to polar coordinates: P s =[rcosθ,rsinθ]

[0055] Since the detection is aimed at the target airspace, the position and direction of the sensor are fixed, that is, T stw Matrices and R stw The content of the matrix, P s It can be obtained based on sensor measurements.

[0056] The third layer, EKF estimation layer:

[0057] The EKF estimation layer consists of two modules: a state prediction module and a state update module. The state prediction module uses the motion state equation of the drone system and the data detected by the sensor to predict the next state of the target (including position, velocity and acceleration); the state update module corrects the state prediction value through the data collected by the sensor and uses the Kalman gain to adjust the influence of the sensor data on the state update; Figure 3 The specific working steps and principles of the two modules are explained:

[0058] The parameters used and their descriptions are shown in the following table:

[0059]

[0060]

[0061] The two modules of the EKF estimation layer are described below:

[0062] State estimation module: First, determine that the UAV motion system is a typical nonlinear system. For a nonlinear system, use the following equation to describe it: k =f(x k-1 ,u k )+w k (1)

[0063] z k =h(x k )+v k (2)

[0064] Where (1) is the state equation, x k is the state vector of the system at time k (including position, velocity, acceleration), f(·) is the nonlinear state transfer function, describing the state change of the system from k-1 to k, u k is the control input, w k is process noise, derived from the motion model or actual data, reflecting the uncertainty of state transfer, obeying zero-mean Gaussian distribution, and covariance Q k , where (2) is the observation equation, z k The observation value vector of the system at time k, h(·) is a nonlinear observation function that describes the relationship between the state and the observation value, v k Observation noise, determined by sensor characteristics or experimental calibration, describes measurement error and follows a zero-mean Gaussian distribution with covariance R k , where the process noise covariance Q k and the observation noise covariance R kThese are two very important parameters, which are related to the subsequent calculation of the extended Kalman gain. According to the actual test environment settings, the initial value should be larger and will be gradually reduced by process iteration until it is appropriate.

[0065] For a nonlinear motion system function, its state transfer equation is defined as follows: k =Fx k-1 +Bu k (3)

[0066] Where F is the state transfer matrix, which represents the linear change of the state over time, and B is the control matrix, which represents the influence of the control input on the state. For complex nonlinear motion, the state vector is defined as a six-tuple, x k =[xyzv x v y v z ] T , assuming that the drone flies along a circular trajectory, its state transfer function is:

[0067]

[0068] Where R is the trajectory radius, ω is the angular velocity, φ is the initial phase, and Δt is the time interval. The corresponding state transfer matrix F can be obtained by calculating the Jacobian matrix of f(x, u):

[0069] Specifically, it can be decomposed into the Jacobian matrix A of the state transfer k and the observed Jacobian matrix H K , and its calculation formula is as follows:

[0070]

[0071] Now to initialize the drone state, we need to initialize the state estimate x 0 And initialize the error covariance matrix P 0 , if the target has been observed before, it is called the prior state x k , the input can be calculated according to the following formula (3). On the contrary, if there is no observation experience before observation, it is called the posterior state x 0 At this time, you can manually set its state according to the system or manually set it reasonably according to the first frame observed by the sensor, for example, the initial speed v x etc. are set to 0; in addition, the initial error covariance matrix P 0 You can set it based on historical data or manually based on experience. It should be noted that if you are not sure about the position, the initial setting still needs to be larger to facilitate the subsequent estimation and measurement of the position. The matrix is ​​defined as:

[0072]

[0073] The parameters are set according to sensor accuracy parameters or experience, such as σ x =10m means the position uncertainty is 10 meters.

[0074] Next, we can predict the target. The equation for state prediction is:

[0075]

[0076] where x^ k∣k-1 It represents the predicted value based on the state at the previous moment, and the state covariance prediction equation is:

[0077]

[0078] Where P k∣k-1 Represents the uncertainty of the predicted state, the data comes from the covariance matrix of the previous moment, F k It is obtained by finding the Jacobian matrix of f(x,u), referring to formula (5), Q k represents the process noise covariance matrix.

[0079] After the above process, the position and motion state of the drone can be preliminarily predicted. Even if no update or correction is performed at this time, a reasonable prediction can still be made based on the target's motion model and the state estimate of the previous moment. The main work of this stage is to use the state transfer equation to push the state of the system to the next moment, that is, to provide a reasonable initialization prediction value input for subsequent state updates. By setting the initial state x 0 and P 0 , combined with the kinematic model, the Jacobian matrix is ​​calculated to obtain the initial state through the state transfer function, and then the target position is updated and predicted.

[0080] State update module: First, the Kalman gain is calculated, which determines the weights of the observed value and the predicted value in the correction model. The expression formula is as follows:

[0081]

[0082] Where K k Represents the Kalman gain, which determines the weight of the observed value and the predicted value in the state update. It can weigh the credibility of the predicted value and the observed value. For example, the observed value with less noise will be given a higher weight. k|k-1 represents the predicted state covariance matrix, R krepresents the observation noise covariance matrix, which describes the statistical characteristics of the sensor measurement error. Before calculating the Kalman gain, it is necessary to calculate the predicted value of the observation, that is, to use the observation function h(·) and the predicted state estimate x^ k∣k-1 Calculate the prediction of the current observation value, that is, the ideal observation value, for the subsequent innovation vector y k The calculation steps are as follows:

[0083]

[0084] where z^ k|k-1 Represents the predicted observation value, which is calculated using the observation function. It is an important bridge between the prediction stage and the update stage, and is used to measure the actual observation value z k The deviation between the predicted observation and the predicted observation, h(·) represents the observation function, describing the nonlinear relationship between the state and the observation;

[0085] Then calculate the innovation vector y k , which is used to represent the deviation between the sensor observation value and the observation prediction value. It does not directly participate in the calculation of the extended Kalman gain, but is used through the innovation covariance H k P k|k-1 H k T +R k Indirectly affects the calculation of Kalman gain:

[0086]

[0087] where z k represents the actual observation value provided by the sensor, z^ k|k-1 Represents the predicted observation value; after calculating the extended Kalman gain, the state can be updated. The state estimate and state covariance need to be updated. The update formula is as follows:

[0088]

[0089] P k|k =(IK k H k ) k|k-1 (15)

[0090] Formula (14) represents the observed value x^ based on the influence of time k-1 on time k k∣k-1 and the extended Kalman gain K k , innovation vector y k To update the state estimate; Formula (15) uses the covariance matrix P of the influence of time k-1 on time k k|k-1 , the identity matrix I and the correction term K k H kto update.

[0091] Through the above steps, the predicted stage position of the target can be updated, and then the updated value can be used for the position prediction at the next moment. The errors of the state estimation value and the state covariance matrix obtained in the prediction stage can also be slowly corrected to achieve higher accuracy. The updated data also needs to be sent back to the previous module for estimation at the next moment.

[0092] Fourth layer: Deep learning layer:

[0093] The target drone is detected and tracked through a deep learning model to achieve the detection and tracking functions of the drone, and combined with the EKF estimation layer to enhance the detection efficiency and robustness of the system.

[0094] The image information collected by the camera of the sensor layer and the EKF state information (position, speed, etc.) of the EKF estimation layer are used as input, and the YOLO model is used internally for detection. Its strong real-time characteristics are very suitable for the rapid detection of drone targets; then the ResNet model is used to classify the detected drones, such as fixed-rotor and multi-rotor; then the updated content of the EKF estimation layer is combined with the sensor data to predict the next trajectory and behavior pattern of the drone; finally, the EKF estimation layer and the deep learning layer are combined, and the deep learning layer feeds back the content after detecting the target drone to the EKF estimation layer to optimize the state estimation.

[0095] In particular, the EKF estimation layer and the deep learning layer are jointly optimized. The deep learning layer detects the drone and then feeds the output back to the EKF estimation layer, which can better optimize the efficiency of drone detection. By detecting the state of the drone through deep learning, the EKF estimation algorithm can be better used to correct the drone state, and the prediction results are more accurate.

[0096] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A drone detection and tracking system based on multi-sensor fusion and deep learning technology, characterized in that: include: S1, determine the detection object. If it is a single drone, directly detect and track it. If it is a drone cluster, it is necessary to determine the key drone; S2, the data collection layer is used to collect multi-dimensional information of the target in the target airspace, including the distance, speed, direction and image data of the target; S3, the data processing layer performs time synchronization, noise removal, data standardization and data enhancement on the data collected by the data collection layer to form high-quality input in a unified format; S4, the EKF estimation layer fuses the sensor data based on the input provided by the data processing layer, and attempts to estimate the current state of the target UAV, including dynamic information such as position, velocity, and acceleration; S5, the deep learning layer combines the dynamic state output by EKF and the images collected by the sensor, and uses the target detection and classification model to achieve real-time detection and tracking of drones.

2. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 1 is characterized in that: In S1, the method for determining the key UAV is to analyze the traffic passing through the UAV through the forwarded data packets. The UAV with the highest forwarding rate in the UAV cluster is considered to be the key UAV, and it is detected and tracked.

3. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 1, characterized in that: In S2, radar equipment, cameras, RF signal collection equipment, etc. are used to collect different data information of the drone. This information covers the relative position, relative distance, movement direction, etc. of the target drone.

4. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 1, characterized in that: In S3, the data processing layer includes a time synchronization module, a noise removal module, a data standardization module, and a data enhancement processing module; The time synchronization module is used to align timestamps to ensure that the data used subsequently is synchronized. The denoising module is used to select a suitable filtering algorithm to remove environmental noise in the signal and improve the efficiency of subsequent data use. The data standardization module is used to unify the format and unit of sensor output and map it to a local world coordinate system to facilitate subsequent unified input to the EKF estimation layer. The data enhancement module is used to expand the diversity and robustness of sensor data.

5. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 4 is characterized in that: In S3, the method for mapping to the local world coordinate system is to determine the rotation matrix Rstw from the sensor coordinate system to the world coordinate system and the position matrix Tstw of the sensor center point in the world coordinate system according to the position and orientation of the placed sensor, and calculate the position Pw of the target in the world coordinate system according to the data transmitted by the sensor. The formula is as follows: P w =R stw ·P s +T stw。 6. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 1, characterized in that: In S4, the estimation process of the EKF estimation layer is as follows: S3-1, state prediction module, uses the motion state equation of the UAV system and the data detected by the sensor to predict the next state of the target (including position, velocity and acceleration); S3-2: The state update module corrects the state prediction value through the data collected by the sensor and uses the Kalman gain to adjust the impact of the sensor data on the state update.

7. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 6, characterized in that: In S4, the specific working steps of the state prediction module are as follows: First, the model prediction is performed to determine the state transfer model of the unmanned nonlinear system to predict the next moment, the UAV motion model is linearized, the Jacobian matrix is ​​used to calculate the nonlinear function, and then based on the prior state x^ k And the predicted error covariance matrix P k Simple identification and inference are performed. The main work of this stage is to use the state transfer equation to push the state of the system to the next moment, that is, to provide a reasonable initialization prediction value input for subsequent state updates. By setting the initialization states x0 and P0, combined with the kinematic model, the Jacobian matrix is ​​calculated to obtain the initial state through the state transfer function, and then the target position is updated and predicted. The state transfer equation is: x k =Fx k-1 +Bu k Where F is the state transfer matrix, which represents the linear change of the state over time, and B is the control matrix, which represents the influence of the control input on the state.

8. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 6, characterized in that: In S4, the specific working steps of the status update module are as follows: By calculating the predicted observed value z^ k|k-1 and the innovation vector y k To calculate the Kalman gain K k , the predicted state of the drone is updated through the Kalman gain, and the updated value is used for the position prediction at the next moment. It can also slowly correct the errors of the state estimation value and state covariance matrix obtained in the prediction stage to achieve higher accuracy. The Kalman gain K k The calculation formula is: Where K k Represents the Kalman gain, which determines the weight of the observed value and the predicted value in the state update. It can weigh the credibility of the predicted value and the observed value. For example, the observed value with less noise will be given a higher weight. k|k-1 represents the predicted state covariance matrix, R k represents the observation noise covariance matrix, which describes the statistical characteristics of the sensor measurement error.

9. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 1, characterized in that: In S5, the target drone is detected and tracked through a deep learning model, and combined with the EKF estimation layer to enhance the detection efficiency and robustness of the system.

10. The method for detecting and tracking unmanned aerial vehicles based on multi-sensor fusion and deep learning technology according to claim 9, characterized in that: In S5, the YOLO model is first selected, which can detect drone targets in real time with high detection speed and accuracy. Secondly, the ResNet model is selected to classify the types of drones. Then, the target motion state output by the EKF estimation layer and the sensor data are used to model and predict the future motion trajectory of the target. Finally, the EKF estimation layer and the deep learning layer are combined, and the target state information output by the EKF and the image features are jointly input into the deep learning model to optimize the target detection and classification performance.

Images