Unmanned aerial vehicle swarm interception system and method based on multi-modal fusion and visual navigation

Abstract

An unmanned aerial vehicle swarm interception system and method based on multi-modal fusion and visual navigation comprises a command control system, a multi-modal detection and identification system and a kinetic energy interception unmanned aerial vehicle cluster system, and the command control system is connected with the multi-modal detection and identification system and the kinetic energy interception unmanned aerial vehicle cluster system through an Ethernet TCP / IP network protocol; according to the system, advanced technologies such as multi-sensor data fusion, vision-inertial navigation fusion guidance, distributed task allocation and online incremental reinforcement learning are utilized to improve the interception efficiency, accuracy and stability, and compared with an existing system, the system is high in interception success rate, high in environmental adaptability, low in interception cost and high in interception stability.

Description

Technical Field

[0001] The present invention belongs to the field of drone swarm defense and intelligent decision-making technology, and specifically relates to a drone swarm interception system and method based on multimodal fusion and visual navigation. Background Art

[0002] With the rapid development of drone technology, drone swarms are increasingly being used in both military and civilian fields. However, they also pose a potential security threat. Intercepting drone swarms has become a hot and challenging issue in the current drone countermeasures field. Current countermeasures against drone swarms include electromagnetic interference (EMI), high-power microwave (HPM), and air defense munitions. Electromagnetic interference (EMI) suffers from slow strike speed and limited effectiveness, while HPM suffers from a short range and high uncertainty in kill effectiveness. Air defense munitions, however, suffer from low interception rates, low cost-to-use ratios, and significant collateral damage. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a drone swarm interception system and method with multimodal fusion and visual navigation. The system uses advanced multi-sensor data fusion, vision-inertial fusion guidance, distributed task allocation, online incremental reinforcement learning and other technologies to improve interception efficiency, accuracy and stability. Compared with the existing system, it has a high interception success rate, strong environmental adaptability, low interception cost and high interception stability.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions: a multimodal fusion and visual navigation drone swarm interception system, comprising a command and control system, a multimodal detection and identification system, and a kinetic interception drone cluster system, wherein the command and control system is connected to the multimodal detection and identification system and the kinetic interception drone cluster system via the Ethernet TCP / IP network protocol;

[0005] The command and control system includes a communication subsystem, a situation display subsystem, a data processing subsystem, and a decision-making control subsystem. The communication subsystem, the situation display subsystem, and the data processing subsystem are all electrically connected to the decision-making control subsystem. The multimodal detection and identification system and the kinetic energy interception drone cluster system are connected to the decision-making control subsystem through the communication subsystem.

[0006] The multimodal detection system mainly includes a radar detection device, a visible light detection device, an infrared detection device, and a multimodal data fusion device. The multimodal data fusion device is connected to the radar detection device, the visible light detection device, and the infrared detection device via Ethernet TCP / IP network protocol communication. The multimodal data fusion device collects data from the radar detection device, the visible light detection device, and the infrared detection device in real time, and uses spatiotemporal registration, multi-sensor data fusion, and multi-target recognition and tracking algorithms to detect and identify invading drone swarms.

[0007] The kinetic energy interception UAV cluster system includes a ground control station and multiple kinetic energy interception UAVs. The kinetic energy interception UAVs include a flight control system, a power system, a visual navigation system, an inertial navigation system, a communication system, and a power supply system. The kinetic energy interception UAV visual navigation system includes a laser radar, a visible light camera, and an infrared camera. The kinetic energy interception UAV communication system includes a communication interface and a wireless communication module. The kinetic energy interception UAV communication system is connected to the ground control station through the wireless communication module, and the ground control station is connected to the command and control system through Ethernet TCP / IP network protocol communication.

[0008] The interception method of the drone swarm interception system based on multimodal fusion and visual navigation includes the following steps:

[0009] Step 1: Multimodal data acquisition: The multimodal detection and identification system detects the invading drone swarm through radar detection equipment, visible light detection equipment, and infrared detection equipment, and transmits the multimodal detection data to the multimodal data fusion device;

[0010] Step 2: Data preprocessing and fusion: The multimodal data fusion equipment performs denoising, correction, and spatiotemporal alignment on the collected data. It also extracts radar RCS characteristics, infrared thermal radiation characteristics, and visual morphological characteristics. An improved DSmT evidence fusion framework is used for decision-level data fusion. The processed data is then transmitted to the data processing subsystem of the command and control system.

[0011] Step 3: Target detection and tracking: The data processing subsystem of the command and control system uses a convolutional neural network algorithm to detect and identify drone swarm targets, and uses a multi-target tracking algorithm to track the detected drone swarms. The update frequency is no less than 10Hz, and the detected data is transmitted to the situation display subsystem for display.

[0012] Step 4: Behavior prediction and mission planning: The command and control system decision-making control subsystem uses a long short-term memory network to predict drone behavior, uses a reinforcement learning algorithm to plan the interception path, and generates interception instructions, which are transmitted to the kinetic interception drone swarm system through the communication subsystem.

[0013] Step 5: Kinetic interception and obstacle avoidance of drone swarms. The intercepting drones of the kinetic interception drone swarm system take off and approach the target drone swarm according to the interception command of the command and control system decision-making control subsystem. When the target enters the range of its own sensors, it relies on its own multimodal sensors to perform multimodal data fusion, target recognition and visual navigation guidance collision interception, and uses reinforcement learning strategies to avoid obstacles, improving interception accuracy and stability.

[0014] Step 6: Intelligent interception control: The command and control system decision-making control subsystem uses a distributed collaborative algorithm to achieve task allocation and coordinated operations among multiple interception drones, reducing repeated interceptions and resource waste;

[0015] Step 7: Dynamic feedback and evaluation: the command and control system data processing subsystem receives interception effect feedback in real time, evaluates the interception effect, and sends the data results to the display control subsystem for display.

[0016] Specifically, in the spatiotemporal alignment of multimodal data in step 2, the cubic spline interpolation algorithm is used for time registration, and the interpolation function is:

[0017]

[0018] where t k is the kth time point that needs to be interpolated, a i is the interpolation coefficient of the i-th subinterval;

[0019] Spatial registration: Multimodal data is uniformly mapped to the same coordinate system, and sensor calibration and coordinate transformation are performed. The radar 3D point cloud data is projected to 2D image coordinates. Finally, the spatial registration is optimized by minimizing the projection error. The reprojection error optimization function is:

[0020]

[0021] Where R is the rotation matrix between sensors, π is the projection function, X i It is a 2D projected coordinate space.

[0022] Specifically, in the multimodal data fusion processing in step 2, the multimodal feature extraction mainly considers the data of radar RCS fluctuation characteristics, infrared thermal radiation characteristics and visual morphological characteristics;

[0023] The radar RCS characteristic is calculated by calculating the micro-Doppler characteristic matrix, and the formula is:

[0024]

[0025] Where w(n) is a window function. The number of rotor blades is extracted by spectrum peak detection. The RCS fluctuation feature is extracted. If the pulse-to-pulse fluctuation rate is greater than 15%, it is judged as a drone.

[0026] The infrared layer extracts the temperature gradient of hot spots and uses the Otsu adaptive threshold method to segment hot spots. The motor temperature is estimated based on the Planck radiation law:

[0027]

[0028] Where h is Planck's constant, c is the speed of light in vacuum, k is the Boltzmann constant, λ is the wavelength of radiation, and M λ (T) is the energy radiated by wavelength λ per unit area and per unit time at the object temperature T;

[0029] The visual layer adopts the improved YOLOv7-tiny network structure, adds the attention module CBAM, and calculates the output feature vector. The formula is:

[0030]

[0031] in, is the weighted eigenvector, is the attention channel weight, H f is the feature map height, W f is the width of the feature map, C f is the image channel, It is a high-order semantic feature map after multi-layer convolution;

[0032] The improved DSmT evidence fusion algorithm is used to process heterogeneous feature data and perform decision-level fusion. The formula for calculating the confidence of data target recognition is:

[0033]

[0034] Where A is the sensor hypothesis set after fusion, B, C, and D correspond to three types of sensor hypothesis sets, and m1, m2, and m3 correspond to the confidence distribution functions of the three types of sensors respectively;

[0035] Specifically, the multimodal data fusion processing and target recognition described in step 5 are consistent with the algorithms used in steps 2 and 3;

[0036] Specifically, in step 5, the visual navigation adopts the improved ORB-SLAM3 framework, integrates IMU data and visual odometry data, performs ORB feature extraction on the front end of the SLAM system architecture, and adopts a joint optimization strategy on the back end. The optimization objective function is:

[0037]

[0038] in, are the residuals of vision, IMU and odometry respectively, ∑ v ,∑ i ,∑o is the corresponding covariance matrix.

[0039] Specifically, in step 6, the distributed collaborative algorithm adopts a distributed game decision framework, establishes an interception game model, defines a payment matrix, and solves the Nash equilibrium solution through a distributed ADMM algorithm.

[0040] Beneficial effects: The present invention has the following advantages:

[0041] 1. The present invention adopts multimodal perception fusion technology to integrate heterogeneous data from radar, visible light, and infrared detection equipment in a layered manner, overcoming the performance degradation problem of a single sensor under adverse conditions such as electromagnetic interference, strong light, haze, and rainy weather, and improving the robustness of drone swarm target detection and recognition.

[0042] 2. The present invention adopts vision-inertial fusion guidance technology, uses the kinetic energy of drone swarms to intercept drone swarms, and can work in GPS-denied environments, greatly reducing the interception cost and improving the system's adaptability to complex electromagnetic environments.

[0043] 3. This invention adopts distributed task allocation technology and online incremental reinforcement learning technology to decouple global planning from local obstacle avoidance, reduce computational complexity, and realize adaptive evolution of interception strategies, which can greatly improve the interception success rate and effectively improve the interception efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 : is a schematic diagram of the system structure of the present invention;

[0045] Figure 2 : It is a schematic diagram of the composition and subsystem connection of the command and control system of the present invention;

[0046] Figure 3 : Schematic diagram of the kinetic energy interception drone cluster system of the present invention;

[0047] Figure 4 : It is a combat step schematic diagram of the present invention. DETAILED DESCRIPTION

[0048] like Figure 1 As shown, a multimodal fusion and visual navigation UAV swarm interception system includes a command and control system, a multimodal detection and identification system, and a kinetic interception UAV cluster system. The command and control system is connected to the multimodal detection and identification system and the kinetic interception UAV cluster system via the Ethernet TCP / IP network protocol;

[0049] like Figure 2As shown, the command and control system includes a communication subsystem, a situation display subsystem, a data processing subsystem, and a decision-making control subsystem. The communication subsystem, the situation display subsystem, and the data processing subsystem are all electrically connected to the decision-making control subsystem. The multimodal detection and identification system and the kinetic energy interception UAV cluster system are connected to the decision-making control subsystem through the communication subsystem.

[0050] The multimodal detection system mainly includes a radar detection device, a visible light detection device, an infrared detection device, and a multimodal data fusion device. The multimodal data fusion device is connected to the radar detection device, the visible light detection device, and the infrared detection device via Ethernet TCP / IP network protocol communication. The multimodal data fusion device collects data from the radar detection device, the visible light detection device, and the infrared detection device in real time, and uses spatiotemporal registration, multi-sensor data fusion, and multi-target recognition and tracking algorithms to detect and identify invading drone swarms.

[0051] The kinetic energy interception UAV cluster system includes a ground control station, multiple kinetic energy interception UAVs, such as Figure 3 As shown, the kinetic energy interception UAV includes a flight control system, a power system, a visual navigation system, an inertial navigation system, a communication system, and a power supply system. The visual navigation system of the kinetic energy interception UAV includes a laser radar, a visible light camera, and an infrared camera. The communication system of the kinetic energy interception UAV includes a communication interface and a wireless communication module. The communication system of the kinetic energy interception UAV is connected to the ground control station through the wireless communication module, and the ground control station is connected to the command and control system through Ethernet TCP / IP network protocol communication.

[0052] like Figure 4 As shown, the drone swarm interception system and method using multimodal fusion and visual navigation includes the following steps:

[0053] Step 1: Multimodal data acquisition: The multimodal detection and identification system detects the invading drone swarm through radar detection equipment, visible light detection equipment, and infrared detection equipment, and transmits the multimodal detection data to the multimodal data fusion device;

[0054] Step 2: Data preprocessing and fusion. The multimodal data fusion equipment denoises, corrects, and aligns the collected data in time and space, and extracts radar RCS characteristics, infrared thermal radiation characteristics, and visual morphological characteristics respectively. The improved DSmT evidence fusion framework is used for decision-level data fusion, and the processed data is transmitted to the data processing subsystem of the command and control system.

[0055] Step 3: Target detection and tracking: The data processing subsystem of the command and control system uses a convolutional neural network (CNN) algorithm to detect and identify drone swarm targets. It uses a multi-target tracking algorithm to track the detected drone swarms with an update frequency of no less than 10Hz. The detected data is transmitted to the situation display subsystem for display.

[0056] Step 4: Behavior prediction and mission planning: The command and control system decision-making control subsystem uses a long short-term memory network (LSTM) to predict drone behavior, uses a reinforcement learning algorithm to plan the interception path, and generates interception instructions, which are transmitted to the kinetic interception drone swarm system through the communication subsystem.

[0057] Step 5: Kinetic interception and obstacle avoidance of drone swarms. The intercepting drones of the kinetic interception drone swarm system take off and approach the target drone swarm according to the interception command of the command and control system decision-making control subsystem. When the target enters the range of its own sensors, it relies on its own multimodal sensors to perform multimodal data fusion, target recognition and visual navigation guidance collision interception, and uses reinforcement learning strategies to avoid obstacles, improving interception accuracy and stability.

[0058] Step 6: Intelligent interception control. The command and control system decision control subsystem uses a distributed collaborative algorithm to achieve task allocation and collaborative operations among multiple interception drones, reducing repeated interceptions and resource waste.

[0059] Step 7: Dynamic feedback and evaluation: the command and control system data processing subsystem receives interception effect feedback in real time, evaluates the interception effect, and sends the data results to the display control subsystem for display.

[0060] Specifically, in the spatiotemporal alignment of multimodal data in step 2, the cubic spline interpolation algorithm is used for time registration, and the interpolation function is:

[0061]

[0062] where t k is the kth time point that needs to be interpolated, a i is the interpolation coefficient of the i-th subinterval.

[0063] Spatial registration: Map the multimodal data to the same coordinate system, perform sensor calibration and coordinate transformation, project the radar 3D point cloud data to 2D image coordinates, and finally optimize the spatial registration by minimizing the projection error. The reprojection error optimization function is:

[0064]

[0065] Where R is the rotation matrix between sensors, π is the projection function, X iIt is a 2D projected coordinate space.

[0066] Specifically, in the multimodal data fusion processing in step 3, the multimodal feature extraction mainly considers the data of radar RCS fluctuation characteristics, infrared thermal radiation characteristics and visual morphological characteristics;

[0067] The radar RCS characteristic is calculated by calculating the micro-Doppler characteristic matrix, and the formula is:

[0068]

[0069] Where w(n) is a window function, and the number of rotor blades is extracted by spectrum peak detection, and the RCS fluctuation feature is extracted (pulse fluctuation rate > 15% is judged as a drone);

[0070] The infrared layer extracts the hot spot temperature gradient and uses the Otsu adaptive threshold method to segment the hot spots (typical temperature rise ΔT of the rotor motor is 8-12°C). The motor temperature is estimated based on Planck's radiation law:

[0071]

[0072] Where h is Planck's constant, c is the speed of light in vacuum, k is the Boltzmann constant, λ is the wavelength of radiation, and M λ (T) is the energy radiated by wavelength λ per unit area and per unit time at the object temperature T.

[0073] The visual layer adopts the improved YOLOv7-tiny network structure and adds a new attention module CBAM to calculate the output feature vector. The formula is:

[0074]

[0075] in, is the weighted eigenvector, is the attention channel weight, H f is the feature map height, W f is the width of the feature map, C f is the image channel, It is a high-order semantic feature map after multi-layer convolution.

[0076] The improved DSmT evidence fusion algorithm is used to process heterogeneous feature data and perform decision-level fusion. The formula for calculating the confidence of data target recognition is:

[0077]

[0078] Where A is the sensor hypothesis set after fusion, B, C, and D correspond to three types of sensor hypothesis sets, and m1, m2, and m3 correspond to the confidence distribution functions of the three types of sensors respectively;

[0079] Specifically, the multimodal data fusion processing and target recognition described in step 5 are consistent with the algorithms used in steps 2 and 3 and will not be described in detail.

[0080] Specifically, in step 5, the visual navigation adopts the improved ORB-SLAM3 framework, integrates IMU data and visual odometry data, performs ORB feature extraction on the front end of the SLAM system architecture, and adopts a joint optimization strategy on the back end. The optimization objective function is:

[0081]

[0082] in, are the residuals of vision, IMU and odometry respectively, ∑ v ,∑ i ,∑ o is the corresponding covariance matrix.

[0083] Specifically, the distributed collaborative algorithm in step 6 adopts a distributed game decision framework, establishes an interception game model, defines a payment matrix, and solves the Nash equilibrium solution through a distributed ADMM algorithm.

[0084] Example:

[0085] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to this embodiment.

[0086] like Figure 1 As shown, a drone swarm interception system and method based on multimodal fusion visual navigation is characterized by including a command and control system, a multimodal detection and identification system, and a kinetic interception drone cluster system. The command and control system is connected to the multimodal detection and identification system and the kinetic interception drone cluster system via the Ethernet TCP / IP network protocol;

[0087] Furthermore, if Figure 2 As shown, the command and control system includes a communication subsystem, a situation display subsystem, a data processing subsystem, and a decision-making control subsystem. The communication subsystem, the situation display subsystem, and the data processing subsystem are all electrically connected to the decision-making control subsystem. The multimodal detection and identification system and the kinetic energy interception UAV cluster system are connected to the decision-making control subsystem through the communication subsystem.

[0088] Furthermore, the multimodal detection system mainly includes a radar detection device, a visible light detection device, an infrared detection device, and a multimodal data fusion device. The multimodal data fusion device is connected to the radar detection device, the visible light detection device, and the infrared detection device via Ethernet TCP / IP network protocol communication. The multimodal data fusion device collects data from the radar detection device, the visible light detection device, and the infrared detection device in real time, and uses spatiotemporal registration, multi-sensor data fusion, and multi-target recognition and tracking algorithms to detect and identify invading drone swarms.

[0089] Further, if Figure 3 As shown, the kinetic energy interception UAV cluster system includes a ground control station and multiple kinetic energy interception UAVs. The kinetic energy interception UAVs include a flight control system, a power system, a visual navigation system, an inertial navigation system, a communication system, and a power supply system. The kinetic energy interception UAV visual navigation system includes a laser radar, a visible light camera, and an infrared camera. The kinetic energy interception UAV communication system includes a communication interface and a wireless communication module. The kinetic energy interception UAV communication system is connected to the ground control station through the wireless communication module, and the ground control station is connected to the command and control system through Ethernet TCP / IP network protocol communication.

[0090] like Figure 4 As shown, the interception method of the drone swarm interception system using multimodal fusion and visual navigation includes the following steps:

[0091] Step 1: Multimodal data acquisition: Six detection nodes are deployed on the ground in a hexagonal grid, with a spacing of 500 meters between each node. Each node integrates a Ku-wave active phased array radar with an operating frequency band of 16 GHz, a bandwidth of 1.5 GHz, and a range resolution of 0.1 m; a dual-band infrared camera with a NETD of less than 50 mK and a resolution of 3-5 μm and 8-12 μm; and a high-speed 4K visible light camera with a resolution of 3840 × 2160 and 120 fps. These cameras detect invading drone swarms and transmit the multimodal detection data to a multimodal data fusion device.

[0092] Step 2: Data preprocessing and fusion. The multimodal data fusion equipment performs denoising, correction, and spatiotemporal alignment on the collected data and transmits the processed data to the data processing subsystem of the command and control system. When processing the spatiotemporal alignment of multimodal data, the cubic spline interpolation algorithm is used for time registration. The interpolation function is:

[0093]

[0094] where t k is the kth time point that needs to be interpolated, a i is the interpolation coefficient of the i-th subinterval.

[0095] Spatial registration: Multimodal data is uniformly mapped to the same coordinate system, and sensor calibration and coordinate transformation are performed. The radar 3D point cloud data is projected to 2D image coordinates. Finally, the spatial registration is optimized by minimizing the projection error. The reprojection error optimization function is:

[0096]

[0097] Where R is the rotation matrix between sensors, π is the projection function, X i is a 2D projected coordinate space;

[0098] The improved DSmT evidence fusion framework fuses and processes multimodal heterogeneous data of radar RCS characteristics, infrared thermal radiation characteristics and visual morphological characteristics;

[0099] The radar layer calculates the micro-Doppler characteristic matrix:

[0100]

[0101] Where w(n) is a window function. The number of rotor blades is extracted by spectrum peak detection. The RCS fluctuation feature is extracted. If the pulse-to-pulse fluctuation rate is greater than 15%, it is judged as a drone.

[0102] The infrared layer extracts the hot spot temperature gradient and uses the Otsu adaptive threshold method to segment the hot spots. The typical temperature rise of the rotor motor is ΔT = 8-12°C. The motor temperature is estimated based on Planck's radiation law:

[0103]

[0104] Where h is Planck's constant, c is the speed of light in vacuum, k is the Boltzmann constant, λ is the wavelength of radiation, and M λ (T) is the energy radiated by wavelength λ per unit area and per unit time at the object temperature T;

[0105] The visual layer adopts the improved YOLOv7-tiny network structure and adds a new attention module CBAM to calculate the output feature vector. The formula is:

[0106]

[0107] in, is the weighted eigenvector, is the attention channel weight, H f is the feature map height, W f is the width of the feature map, C f is the image channel, It is a high-order semantic feature map after multi-layer convolution;

[0108] The improved DSmT data target recognition confidence calculation formula is:

[0109]

[0110] Where A is the sensor hypothesis set after fusion, B, C, and D correspond to three types of sensor hypothesis sets, and m1, m2, and m3 correspond to the confidence distribution functions of the three types of sensors respectively;

[0111] DSmT fusion decision: Define the identification framework Θ = {UAV, bird, false target} and generate the basic probability distribution:

[0112] Radar: m1 (UAV) = 0.85 (RCS fluctuation rate > 18%)

[0113] Infrared: m2 (UAV) = 0.76 (ΔT > 8°C)

[0114] Vision: m3 (drone) = 0.93 (YOLO confidence > 0.9)

[0115] Calculate the joint confidence:

[0116] Dynamic discount factor adjustment: When a sensor is abnormal for five consecutive frames, its γ value is reduced from 0.9 to 0.4.

[0117] Step 3: Target detection and tracking. The data processing subsystem of the command and control system uses a convolutional neural network algorithm to detect and identify drone swarm targets, and uses a multi-target tracking algorithm to track the detected drone swarms. The update frequency is not less than 10Hz, and the detected data is transmitted to the situation display subsystem for display.

[0118] Step 4: Behavior prediction and mission planning. The command and control system decision control subsystem uses a long short-term memory network to predict the behavior of the UAV, uses a reinforcement learning algorithm to plan the interception path, and generates interception instructions, which are transmitted to the kinetic interception UAV cluster system through the communication subsystem.

[0119] Step 5: Kinetic interception and obstacle avoidance of drone swarms. The intercepting drones of the kinetic interception drone swarm system take off and approach the target drone swarm according to the interception command of the command and control system decision-making control subsystem. When the target enters the range of its own sensors, it relies on its own multimodal sensors to perform multimodal data fusion, target recognition and visual navigation guidance collision interception, and uses reinforcement learning strategies to avoid obstacles, improving interception accuracy and stability.

[0120] Visual navigation uses the improved ORB-SLAM3 framework, integrates IMU data, and performs ORB feature extraction on the front end of the SLAM system architecture, extracting 1,000 feature points per frame. The back end adopts a joint optimization strategy, and the optimization objective function is:

[0121]

[0122] in, are the residuals of vision, IMU and odometry respectively, ∑ v ,∑ i ,∑ o is the corresponding covariance matrix;

[0123] Reinforcement Learning Strategy:

[0124] Establish state space: 256×256 depth map + 10-dimensional IMU data (acceleration, angular velocity) action space: 3D velocity command (v x ,v y ,v z )

[0125] Reward function design:

[0126] Step 6: Intelligent interception control. The command and control system decision control subsystem uses a distributed collaborative algorithm to achieve task allocation and collaborative operations among multiple interception drones, reducing repeated interception and resource waste. The distributed collaborative algorithm adopts a distributed game decision framework, establishes an interception game model, and defines the payment matrix P∈R M×N , where the elements Solve the Nash equilibrium solution through the distributed ADMM algorithm;

[0127] (1) Game model construction

[0128] Player collection: M interceptor drones;

[0129] Strategy space: Each drone selects at most K targets out of N, K = 3;

[0130] Profit function:

[0131]

[0132] Among them, δ ij represents the interception of target drone j by drone i, c is the threat coefficient, in this case, δ ij =1, c=0.3;

[0133] (2) ADMM solution

[0134] Local variable update:

[0135]

[0136] Global variable aggregation:

[0137]

[0138] Multiplier update:

[0139]

[0140] Step 7: Dynamic feedback and evaluation: the command and control system data processing subsystem receives interception effect feedback in real time, evaluates the interception effect, and sends the data results to the display control subsystem for display.

Claims

1. A drone swarm interception system using multimodal fusion and visual navigation, characterized by: It includes a command and control system, a multi-modal detection and identification system, and a kinetic energy interception UAV cluster system. The command and control system is connected to the multi-modal detection and identification system and the kinetic energy interception UAV cluster system via the Ethernet TCP / IP network protocol; The command and control system includes a communication subsystem, a situation display subsystem, a data processing subsystem, and a decision-making control subsystem. The communication subsystem, the situation display subsystem, and the data processing subsystem are all electrically connected to the decision-making control subsystem. The multimodal detection and identification system and the kinetic energy interception drone cluster system are connected to the decision-making control subsystem through the communication subsystem. The multimodal detection system includes a radar detection device, a visible light detection device, an infrared detection device, and a multimodal data fusion device. The multimodal data fusion device is connected to the radar detection device, the visible light detection device, and the infrared detection device via Ethernet TCP / IP network protocol communication. The multimodal data fusion device collects data from the radar detection device, the visible light detection device, and the infrared detection device in real time, and uses spatiotemporal registration, multi-sensor data fusion, and multi-target recognition and tracking algorithms to detect and identify invading drone swarms. The kinetic interception drone cluster system includes a ground control station and multiple kinetic interception drones. The kinetic interception drones are connected to the ground control station through a wireless communication module; the ground control station is connected to the command and control system through Ethernet TCP / IP network protocol communication.

2. A method for intercepting a drone swarm interception system using multimodal fusion and visual navigation as described in claim 1, characterized in that: Step 1: Multimodal data acquisition: The multimodal detection and identification system detects the invading drone swarm through radar detection equipment, visible light detection equipment, and infrared detection equipment, and transmits the multimodal detection data to the multimodal data fusion device; Step 2: Data preprocessing and fusion: The multimodal data fusion equipment performs denoising, correction, and spatiotemporal alignment on the collected data. It also extracts radar RCS characteristics, infrared thermal radiation characteristics, and visual morphological characteristics. An improved DSmT evidence fusion framework is used for decision-level data fusion. The processed data is then transmitted to the data processing subsystem of the command and control system. Step 3: Target detection and tracking: The data processing subsystem of the command and control system uses a convolutional neural network algorithm to detect and identify drone swarm targets, and uses a multi-target tracking algorithm to track the detected drone swarms. The update frequency is no less than 10Hz, and the detected data is transmitted to the situation display subsystem for display. Step 4: Behavior prediction and mission planning: The command and control system decision-making control subsystem uses a long short-term memory network to predict drone behavior, uses a reinforcement learning algorithm to plan the interception path, and generates interception instructions, which are transmitted to the kinetic interception drone swarm system through the communication subsystem. Step 5: Kinetic interception and obstacle avoidance of drone swarms. The intercepting drones of the kinetic interception drone swarm system take off and approach the target drone swarm according to the interception command of the command and control system decision-making control subsystem. When the target enters the range of its own sensors, it relies on its own multimodal sensors to perform multimodal data fusion, target recognition and visual navigation guidance collision interception, and uses reinforcement learning strategies to avoid obstacles, improving interception accuracy and stability. Step 6: Intelligent interception control: The command and control system decision-making control subsystem uses a distributed collaborative algorithm to achieve task allocation and coordinated operations among multiple interception drones, reducing repeated interceptions and resource waste; Step 7: Dynamic feedback and evaluation: the command and control system data processing subsystem receives interception effect feedback in real time, evaluates the interception effect, and sends the data results to the display control subsystem for display.

3. The interception method of the multimodal fusion and visual navigation drone swarm interception system according to claim 2, characterized in that: In the spatiotemporal alignment of multimodal data in step 2, a cubic spline interpolation algorithm is used for temporal registration, and the interpolation function is: where t k is the kth time point that needs to be interpolated, a i is the interpolation coefficient of the i-th subinterval; Spatial registration: Multimodal data is uniformly mapped to the same coordinate system, and sensor calibration and coordinate transformation are performed. The radar 3D point cloud data is projected to 2D image coordinates. Finally, the spatial registration is optimized by minimizing the projection error. The reprojection error optimization function is: Where R is the rotation matrix between sensors, π is the projection function, X i It is a 2D projected coordinate space.

4. The interception method of the multimodal fusion and visual navigation drone swarm interception system according to claim 2, characterized in that: In the multimodal data fusion processing in step 2, multimodal feature extraction mainly considers the data of radar RCS fluctuation characteristics, infrared thermal radiation characteristics and visual morphological characteristics; The radar RCS characteristic is calculated by calculating the micro-Doppler characteristic matrix, and the formula is: Where w(n) is a window function. The number of rotor blades is extracted by spectrum peak detection. The RCS fluctuation feature is extracted. If the pulse-to-pulse fluctuation rate is greater than 15%, it is judged as a drone. The infrared layer extracts the temperature gradient of hot spots and uses the Otsu adaptive threshold method to segment hot spots. The motor temperature is estimated based on the Planck radiation law: Where h is Planck's constant, c is the speed of light in vacuum, k is the Boltzmann constant, λ is the wavelength of radiation, and M λ (T) is the energy radiated by wavelength λ per unit area and per unit time at the object temperature T; The visual layer adopts the improved YOLOv7-tiny network structure, adds the attention module CBAM, and calculates the output feature vector. The formula is: in, is the weighted eigenvector, is the attention channel weight, H f is the feature map height, W f is the width of the feature map, C f is the image channel, It is a high-order semantic feature map after multi-layer convolution; The improved DSmT evidence fusion algorithm is used to process heterogeneous feature data and perform decision-level fusion. The formula for calculating the confidence of data target recognition is: Among them, A is the sensor hypothesis set after fusion, B, C, and D correspond to three types of sensor hypothesis sets, and m1, m2, and m3 correspond to the confidence distribution functions of the three types of sensors respectively.

5. The interception method of the multimodal fusion and visual navigation drone swarm interception system according to claim 2, characterized in that: The multimodal data fusion processing and target recognition described in step 5 are consistent with the algorithms used in steps 2 and 3.

6. The interception method of the multimodal fusion and visual navigation drone swarm interception system according to claim 2, characterized in that: In step 5, the visual navigation adopts the improved ORB-SLAM3 framework, integrates IMU data and visual odometry data, performs ORB feature extraction on the front end of the SLAM system architecture, and adopts a joint optimization strategy on the back end. The optimization objective function is: in, are the residuals of vision, IMU and odometry respectively, ∑ v ,∑ i ,∑ o is the corresponding covariance matrix.

7. The interception method of the multimodal fusion and visual navigation drone swarm interception system according to claim 2, characterized in that: In step 6, the distributed collaborative algorithm adopts a distributed game decision framework, establishes an interception game model, defines a payment matrix, and solves the Nash equilibrium solution through a distributed ADMM algorithm.

Images