Unmanned aerial vehicle-unmanned vehicle system cooperative target detection method and system and application thereof

Abstract

The invention discloses an unmanned aerial vehicle-unmanned vehicle system collaborative target detection method and system and application thereof, and the method comprises the steps: integrating the high-altitude multi-modal data of an unmanned aerial vehicle and the ground three-dimensional point cloud / multi-view image of an unmanned vehicle through constructing a collaborative architecture of an unmanned aerial vehicle subsystem, an unmanned vehicle subsystem and a data fusion center; a deep learning algorithm is adopted to extract image features, Bayesian estimation is combined to fuse multi-source data, and posterior probability distribution of a target state is generated. And task allocation and dynamic path optimization are realized based on a path planning algorithm and a Hungary algorithm, and a detection strategy is adjusted through real-time communication. The system disclosed by the invention comprises a high-precision sensor, a communication module and a high-performance data processing center. According to the scheme, the target detection precision, the environmental adaptability and the efficiency in a complex scene are remarkably improved, and the method can be widely applied to urban security and protection, field exploration and logistics distribution.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned system collaborative control and target detection, and specifically relates to a target detection method, system and application of collaborative control between a drone and an unmanned vehicle system, which is particularly suitable for high-precision target detection in complex terrain, urban security and dynamic scenarios. Background Art

[0002] With the rapid development of science and technology, unmanned systems have been widely used in various fields. In target detection tasks, traditional single detection equipment, whether it is ground-based unmanned vehicles or aerial drones, has certain limitations.

[0003] For autonomous vehicles, the ground environment they operate in is complex and ever-changing. In urban environments, the frequent traversal of other vehicles and the presence of tall buildings can easily obstruct target detection by autonomous vehicles. When an autonomous vehicle needs to detect distant targets ahead, if blocked by vehicles or buildings, its onboard sensors, such as cameras and millimeter-wave radars, cannot obtain effective information about the obscured targets, resulting in detection failure or a significant decrease in detection accuracy. For example, when monitoring for illegal vehicles at intersections, if a normally moving vehicle obscures the offending vehicle, the autonomous vehicle will struggle to accurately identify the violation. In complex mountainous environments, the terrain is rugged, and obstacles such as hillsides and boulders can also interfere with the autonomous vehicle's line of sight, making the detection of targets such as people and facilities within mountainous areas extremely challenging.

[0004] While drones offer the advantage of a high-altitude perspective, enabling them to overlook large areas and, to a certain extent, mitigate the obstruction of ground obstacles, they also present numerous challenges. For one thing, their flight altitude limits their ability to detect target details. When flying at high altitudes, the camera captures images of smaller targets, such as small devices on the ground or small animals, with low resolution, making it difficult to accurately identify target features. Furthermore, drones have limited endurance, significantly restricting their ability to operate continuously for extended periods. Furthermore, in adverse weather conditions such as strong winds, heavy rain, and dust storms, drones' flight stability is severely impacted, and they may even be unable to fly normally, leading to interruptions in target detection missions.

[0005] Furthermore, existing target detection technologies often analyze and process data from a single device, failing to fully utilize the diverse information captured by different devices from different perspectives. This makes it difficult to effectively guarantee the reliability and accuracy of target detection in complex environments. As the requirements for target detection accuracy and reliability continue to increase across various industries, developing a collaborative target detection technology that combines the strengths of drones and unmanned vehicles while avoiding their limitations is of great practical significance. Summary of the Invention

[0006] In order to solve the above-mentioned problems in the prior art, the present invention provides a target detection method for a collaborative drone-unmanned vehicle system, its system and application. The present invention overcomes the influence of occlusion by vehicles, buildings and other obstacles by fusing the information obtained by the drone and the unmanned vehicle at different perspectives, thereby significantly improving the reliability and accuracy of target detection.

[0007] The present invention includes the following technical solutions:

[0008] A target detection method for a UAV-UAV system collaboration includes the following steps:

[0009] a. Build a collaborative system architecture consisting of a UAV subsystem, an unmanned vehicle subsystem, and a data fusion and processing center;

[0010] b. The drone subsystem collects aerial images and thermal radiation data of the target area through the high-resolution camera and thermal imager carried by the drone subsystem, and transmits its own position and attitude information in real time through the positioning and navigation equipment;

[0011] c. Collecting ground three-dimensional point cloud data, multi-angle images and obstacle distance information through the lidar, multi-view camera and ultrasonic sensor carried by the unmanned vehicle subsystem;

[0012] d. In the data fusion and processing center, deep learning algorithms are used to extract features from image data collected by drones and unmanned vehicles, perform point cloud segmentation and geometric feature calculations on lidar data, and fuse multi-source data based on Bayesian estimation methods to generate a posterior probability distribution of the target state;

[0013] e. Use path planning algorithms to assign tasks and plan flight and driving paths for drones and unmanned vehicles based on target area characteristics and mission requirements;

[0014] f. During the detection process, the detection strategies of the drone and unmanned vehicle are dynamically adjusted through real-time communication, and the anti-occlusion processing mechanism is triggered when the target is occluded. This includes the drone adjusting the flight angle / altitude to obtain multi-view images, and the unmanned vehicle combining lidar and drone data to reconstruct target features.

[0015] Furthermore, in the above-mentioned target detection method for a UAV-UAV system collaboration, in step d:

[0016] The convolutional neural network (CNN) model is used to extract features of visible light images;

[0017] Specially designed grayscale stretching and convolutional networks are used to extract features from thermal images;

[0018] Statistical filtering and DBSCAN clustering algorithm are used to remove noise and segment targets from lidar point cloud data.

[0019] Furthermore, in the above-mentioned target detection method for collaborative UAV-UV system, the path planning algorithm in step e includes the A* algorithm or the Dijkstra algorithm, and is optimized in combination with the UAV flight altitude constraint and the terrain adaptability of the UVV.

[0020] Furthermore, in the above-mentioned target detection method for collaborative drone-unmanned vehicle system, the task allocation in step e adopts the Hungarian algorithm, and is dynamically allocated according to the detection efficiency of the drone and unmanned vehicle and the complexity of the target area.

[0021] Furthermore, in the above-mentioned target detection method for a UAV-UAV system collaboration, the anti-occlusion processing mechanism in step f includes:

[0022] The drone calculates the optimal flight angle and altitude through trigonometric functions to avoid obstructions in the line of sight, and restores the obscured target information through image stitching algorithms.

[0023] The unmanned vehicle uses ultrasonic sensors and lidar data to determine the shape of obstructions, and combines multi-view images from drones to infer and reconstruct target features.

[0024] Furthermore, in the above-mentioned target detection method for collaborative drone-unmanned vehicle system, the communication between the drone and the unmanned vehicle adopts 4G / 5G wireless transmission technology, and the data transmission protocol includes UDP and TCP / IP.

[0025] The present invention also discloses a UAV-UAV collaborative target detection system, which is used in the above method and comprises:

[0026] Drone subsystem: equipped with a high-resolution camera, thermal imager, GNSS / IMU positioning and navigation module, and 4G / 5G communication module;

[0027] Unmanned vehicle subsystem: equipped with multi-line laser radar, multi-view cameras, ultrasonic sensors and on-board computing units;

[0028] Data fusion and processing center: Deploy high-performance servers, run data fusion algorithms based on deep learning and Bayesian estimation, and realize task scheduling and dynamic collaborative control of drones and unmanned vehicles.

[0029] Furthermore, in the above system, the thermal imager of the drone subsystem is a FLIR Tau2 series, and the laser radar of the unmanned vehicle subsystem is a 16-line or 32-line multi-line laser radar.

[0030] The present invention also discloses the application of the above system or method in complex terrain exploration, which includes the following steps:

[0031] 1) UAVs can quickly scan large areas from a high-altitude perspective and identify potential targets;

[0032] 2) The unmanned vehicle conducts detailed inspections of rugged terrain based on the information fed back by the drone, and generates a three-dimensional environmental model by combining lidar and multi-view camera data.

[0033] The present invention also discloses an application of the above system or method in an urban security scenario, which is characterized by comprising the following steps:

[0034] 1) UAVs monitor urban high-altitude areas in real time and, upon detecting unusual targets, guide unmanned vehicles to avoid traffic obstacles and drive towards the target;

[0035] 2) The unmanned vehicle detects close-range obstacles through ultrasonic sensors and accurately identifies targets based on multi-angle images from drones.

[0036] Preferably, the purpose of the present invention can be further achieved by the following specific system structure and algorithm:

[0037] Compared with the prior art, the present invention has the following outstanding beneficial effects:

[0038] The present invention discloses a target detection method for a UAV-UAV system collaboration, a system and an application thereof, which have the following specific innovations and beneficial effects:

[0039] 1. Innovative Collaborative System Architecture: This system integrates an unmanned aerial vehicle (UAV) subsystem, an unmanned vehicle (UAV) subsystem, and a data fusion and processing center. The UAV subsystem is equipped with high-resolution cameras, thermal imagers, and high-precision positioning and navigation equipment, enabling comprehensive, multimodal data acquisition of the target area from high altitude. The UAV subsystem, equipped with lidar, multi-view cameras, and ultrasonic sensors, focuses on close-range, multi-angle data collection on the ground. Each subsystem has a clear division of labor and closely collaborates with the data fusion and processing center, efficiently integrating data from both. This overcomes the limitations of traditional single-device detection and enables comprehensive, multi-level target detection data acquisition and processing.

[0040] 2. Advanced Data Fusion Algorithms: Deep learning algorithms, such as convolutional neural networks (CNNs), are employed to accurately extract target features such as shape, color, and texture from images captured by drones and unmanned vehicles. For lidar data, specialized point cloud processing algorithms are employed to obtain precise 3D spatial position and shape information of the target. Furthermore, innovative Bayesian estimation-based data fusion methods are employed, fully considering the reliability and confidence levels of data from different sensors. The probability distribution of fused target features is calculated, resulting in more accurate and comprehensive fusion results and significantly improving the precision of target feature description.

[0041] 3. Intelligent Collaborative Inspection Strategy: Before mission execution, scientific task allocation and path planning are performed using algorithms such as A* and Dijkstra, based on the characteristics of the target area, mission requirements, and target type and distribution. During the inspection process, the drone and unmanned vehicle maintain real-time communication and dynamically adjust the inspection strategy. The drone, with its high-altitude, rapid scanning capabilities, detects potential targets and transmits location information to the unmanned vehicle, which then accurately navigates to the target area for detailed inspection. If the unmanned vehicle encounters complex situations such as obstructions, the drone can promptly adjust its flight altitude and angle, conduct further inspections, and transmit data to assist the unmanned vehicle, achieving an intelligent and efficient collaborative inspection process.

[0042] 4. Efficient anti-occlusion processing mechanism: When a target is detected as potentially obscured, the system responds quickly. The drone leverages its maneuverability to accurately calculate and adjust its flight angle and altitude based on positioning information and target location, avoiding obstructions to its line of sight. It then captures images from a new angle and uses an image stitching algorithm to restore the obscured information. The unmanned vehicle uses ultrasonic sensors to sense the location and distance of obstructions, combining lidar data with existing map information to determine the shape and size of the obstructions. It then requests image data from other angles from the drone and uses a data fusion algorithm to infer and reconstruct the features of the obscured target. This effectively addresses the occlusion problem in complex environments and significantly improves the accuracy of target detection in obscured scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Schematic diagram of the system and method of the present invention. DETAILED DESCRIPTION

[0044] like Figure 1 FIG. 1 is a schematic diagram of the system and method of the present invention, which is implemented by the following basic method:

[0045] 1. System construction

[0046] Drone Configuration: Choose a drone with long flight time and stable flight performance. Install a high-resolution visible light camera with a resolution of at least 4K and a frame rate of 60fps or higher to obtain clear images of the target. Also, equip the drone with a thermal imager to detect the target's thermal radiation signature at night or in low-visibility environments. Install high-precision positioning and navigation equipment, such as a global navigation satellite system (GNSS) and an inertial measurement unit (IMU), to ensure the drone can accurately fly to the target area and transmit its position information in real time.

[0047] Unmanned Vehicle Configuration: Choose an unmanned vehicle with excellent off-road performance and stable driving capabilities. Install a multi-line LiDAR (LiDAR), such as a 16- or 32-line LiDAR, to acquire 3D point cloud data of the surrounding environment. Equip the vehicle with multiple cameras, including front, rear, and side cameras, with a 360-degree field of view around the vehicle and a resolution of at least 1080p. Additionally, install ultrasonic sensors for close-range obstacle detection.

[0048] Data Fusion and Processing Center Construction: A high-performance server serves as the data fusion and processing center. Equipped with a multi-core processor, such as an Intel Xeon series processor, the server boasts powerful data computing capabilities. Large-capacity memory (at least 32GB) is installed to ensure data storage and computational speed during data processing. The server runs specially developed data fusion and processing software, which enables real-time reception, storage, and processing of data transmitted by drones and unmanned vehicles.

[0049] 2. Data collection and transmission

[0050] Drone Data Collection: The drone flies over the target area according to a predetermined flight path. During flight, visible light cameras and thermal imagers continuously capture image and video data of the target area. Positioning and navigation equipment records the drone's position, attitude, and other information in real time. The collected data is transmitted to the data fusion and processing center via wireless communication modules, such as 4G or 5G, with high bandwidth and low latency.

[0051] Data Collection for Unmanned Vehicles: Unmanned vehicles drive along a planned path on the ground. LiDAR continuously emits laser beams and receives reflected laser signals, generating 3D point cloud data of the surrounding environment. Cameras capture real-time images of the vehicle's surroundings. Ultrasonic sensors detect the distance to nearby obstacles. This data is also transmitted to the data fusion and processing center via wireless communication modules.

[0052] 3. Data fusion processing

[0053] Image Feature Extraction: In the data fusion and processing center, deep learning frameworks such as TensorFlow or PyTorch are used to extract features from image data transmitted by drones and unmanned vehicles. For visible light images, convolutional neural network (CNN) models such as ResNet and VGG are used to extract target features such as shape, color, and texture. For thermal images, a specialized thermal imaging feature extraction network is used to extract the target's thermal radiation characteristics.

[0054] LiDAR Data Processing: Preprocess the 3D point cloud data acquired by the LiDAR to remove noise and outliers. Then, using a point cloud segmentation algorithm, the target is separated from the background point cloud. By calculating the geometric features of the target point cloud, such as its center of mass, volume, and surface area, the target's 3D spatial position and shape are determined.

[0055] Data fusion: This approach utilizes a Bayesian estimation-based data fusion method. Based on the prior probabilities and likelihood functions of the different sensor data, the posterior probability distribution of the fused target features is calculated. For example, the most likely location of a target is calculated using the Bayesian formula by combining the positioning and navigation data of a drone and the lidar data of an unmanned vehicle. During the fusion process, different sensor data are weighted differently based on their accuracy and reliability to improve the accuracy of the fusion results.

[0056] 4. Collaborative testing execution

[0057] Task Assignment and Path Planning: Before a mission begins, optimal flight and driving paths are planned for drones and autonomous vehicles using path planning algorithms such as A* and Dijkstra's algorithm, based on the target area's map information and mission requirements. Tasks are also assigned to drones and autonomous vehicles based on the type and distribution of targets. For example, for a large target area, a drone will quickly scan it. Upon identifying potential targets, it will transmit the target's location information to the autonomous vehicle, which will then conduct a detailed inspection of the target.

[0058] Real-time collaborative detection: During the detection process, the drone and the unmanned vehicle maintain real-time communication. When the drone detects a potential target, it transmits information such as the target's location and general shape to the unmanned vehicle. Based on this information, the unmanned vehicle adjusts its driving path and quickly drives toward the target area. As the unmanned vehicle approaches the target, the drone can adjust its flight altitude and angle based on the unmanned vehicle's feedback, re-detecting the target from different perspectives and promptly transmitting the newly acquired data to the unmanned vehicle. If the unmanned vehicle encounters an obstruction and cannot obtain target information, the drone uses its maneuverability to capture the target from alternative angles. The captured image data is transmitted to the data fusion and processing center, where data fusion algorithms are used to recover and infer information about the obscured target.

[0059] 5. Anti-occlusion processing implementation

[0060] Drone Anti-Occlusion Operation: When a drone detects a potential obstruction to a target, it first calculates the optimal flight angle and altitude based on its own positioning information and the target's location to avoid obstructions to the target's view. The drone then adjusts its flight attitude and photographs the target area from a new angle. Once completed, the newly acquired image data is transmitted to the Data Fusion and Processing Center. There, an image stitching algorithm is used to stitch together images taken from different angles to attempt to recover information about the obscured portion of the target.

[0061] Unmanned vehicle anti-occlusion operation: The unmanned vehicle uses its own ultrasonic sensors to detect the location and distance of surrounding obstructions. When an obstruction is detected, it uses lidar data and existing map information to determine its shape and size. Simultaneously, the unmanned vehicle sends a request to the data fusion and processing center for drone-generated image data of the target area. Combining its own sensor data with the data transmitted by the drone, a data fusion algorithm is used to infer and reconstruct the features of the obstructed object, thereby achieving accurate detection of the obstructed object.

[0062] The following is a clear and complete description of the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0063] Example 1

[0064] Target detection method for UAV-UV system collaboration and its system and application

[0065] 1. System construction

[0066] Drone configuration

[0067] Choose a drone platform with a flight time of at least 2 hours and high flight stability, such as the DJI M300RTK. Install a high-resolution visible light camera on this drone, such as the Sony IMX477, with a resolution of up to 4K (3840×2160 pixels) and a frame rate of at least 60fps to ensure clear and smooth target images. At the same time, it is equipped with a FLIRTau2 series thermal imager to accurately detect the thermal radiation characteristics of the target at night or in low-visibility environments. To ensure that the drone can accurately fly to the target area and transmit its own position information in real time, install a high-precision global satellite navigation system (GNSS), such as the u-blox NEO-M8N, and an inertial measurement unit (IMU), such as the Bosch BMI088. This combination can provide accurate position, attitude, and velocity data.

[0068] The drone's communication module is optimized, and a module that supports 4G / 5G communication, such as Quectel's RM500Q-GN, is used to achieve high-bandwidth, low-latency data transmission, ensuring that the collected data can be quickly and stably transmitted to the data fusion and processing center.

[0069] Unmanned vehicle configuration

[0070] Select an unmanned vehicle with excellent off-road performance and stable driving capabilities, such as the Clearpath Jackal. Install a 16- or 32-line multi-line lidar, such as the Velodyne VLP-16 or Ouster OS1-32, on the vehicle to obtain high-precision 3D point cloud data of the surrounding environment. Configure multiple cameras, including front-view, rear-view, and side-view cameras. The Hikvision DS-2CD3T47WD-L camera can be used, with a field of view that covers a 360-degree area around the unmanned vehicle and a resolution of at least 1080p (1920×1080 pixels). Additionally, install the HC-SR04 ultrasonic sensor for close-range obstacle detection, with an effective detection range of 2cm-400cm.

[0071] Equipping unmanned vehicles with high-performance on-board computing units, such as NVIDIA Jetson Xavier NX, is responsible for preliminary pre-processing of data collected by sensors, reducing the computing burden of the data fusion and processing center, while ensuring that the unmanned vehicle can perform some simple decision-making and control locally.

[0072] Construction of data fusion and processing center

[0073] A high-performance server serves as the data fusion and processing center. Equipped with an Intel Xeon Platinum 8380 processor and boasting 40 computing cores, it boasts powerful data processing capabilities. 64GB of memory ensures data storage and computational speed during data processing. The server runs specialized data fusion and processing software developed on the Linux operating system. Written in Python and leveraging the TensorFlow and PyTorch deep learning frameworks, the software enables real-time reception, storage, and processing of data transmitted by drones and unmanned vehicles.

[0074] To ensure stable communication between the data fusion and processing center and drones and unmanned vehicles, high-speed wired network connections, such as 10 Gigabit Ethernet, are used and equipped with firewall equipment to ensure the security and stability of data transmission.

[0075] 2. Data collection and transmission

[0076] Drone data collection

[0077] Before takeoff, a drone uses specialized path-planning software (such as Mission Planner) to plan a flight path based on the target area and mission requirements. This flight path takes into account factors such as the target area's terrain, obstacle distribution, and signal strength, ensuring the drone can safely and efficiently cover the target area.

[0078] During flight, the visible light camera and thermal imager continuously capture image and video data of the target area at a set frame rate. The positioning and navigation equipment (GNSS+IMU) records the drone's position (longitude, latitude, altitude), attitude (roll, pitch, and yaw), and other information in real time at a 10Hz frequency. The collected data is packaged and transmitted via the 4G / 5G communication module using the UDP protocol, achieving data transmission rates exceeding 100Mbps, ensuring rapid transmission to the data fusion and processing center.

[0079] Unmanned vehicle data collection

[0080] Before commencing a mission, the autonomous vehicle also loads a pre-planned driving path through its onboard control system. This path is generated using path planning algorithms such as the A* algorithm, combining map information with mission requirements to create an optimal trajectory that avoids obstacles and dangerous areas.

[0081] The LiDAR continuously emits laser beams at a frequency of 10Hz and receives reflected laser signals to generate 3D point cloud data of the surrounding environment. Each camera captures real-time images of the autonomous vehicle's surroundings at a frame rate of 30fps. Ultrasonic sensors detect the distance to nearby obstacles at a frequency of 50Hz. This data is transmitted to the data fusion and processing center via the onboard communication module using the TCP / IP protocol to ensure reliable data transmission.

[0082] 3. Data fusion processing

[0083] Image feature extraction

[0084] At the Data Fusion and Processing Center, a convolutional neural network (CNN) model was built using the TensorFlow framework. For visible light images, the ResNet50 model was used for feature extraction. Visible light image data transmitted by drones and unmanned vehicles undergoes preprocessing, including normalization, converting pixel values ​​from 0-255 to 0-1. This data is then input into the ResNet50 model, which, through a series of convolutional, pooling, and fully connected layers, extracts features such as the target's shape, color, and texture, outputting a feature vector of length 2048.

[0085] For thermal images, a specially designed thermal imaging feature extraction network, built on the PyTorch framework, is used. First, the thermal image is grayscale stretched to enhance image contrast. Then, through multiple convolutional layers and ReLU activation functions, the target's thermal radiation characteristics are extracted, ultimately outputting a feature vector of length 1024.

[0086] LiDAR data processing

[0087] The 3D point cloud data acquired by the LiDAR was preprocessed using a statistical filtering algorithm to remove noise and outliers. The number of neighborhood points was set to 50, and the mean and standard deviation of the distance between each point and its neighbors were calculated. Points with a distance greater than three times the standard deviation were considered noise points and removed.

[0088] The DBSCAN (Density-Based Spatial Clustering of Applications with Noise) point cloud segmentation algorithm was used to separate the target from the background point cloud. The algorithm clustered the point cloud data into distinct target and background regions, setting a neighborhood radius of 0.5m and a minimum point count of 10. For each target cluster, its centroid coordinates, volume, surface area, and other geometric features were calculated to form a LiDAR feature vector of length 7.

[0089] Data fusion

[0090] The data fusion method based on Bayesian estimation is adopted. Assume that the feature vector of the visible light image acquired by the UAV is The thermal imaging image feature vector is The laser radar feature vector obtained by the unmanned vehicle is The target state variable is (including target location, category and other information). According to the Bayesian formula Calculate the posterior probability distribution of the fused target features.

[0091] in, is the prior probability, obtained through historical data statistics. Likelihood function The neural network model is trained on a large number of data samples containing different targets and scenes, and then used for estimation. During the fusion process, different sensor data are assigned different weights based on the accuracy and reliability of the sensors. For example, the weight of visible light image features is 0.4, the weight of thermal image features is 0.3, and the weight of lidar features is 0.3, to improve the accuracy of the fusion results.

[0092] 4. Collaborative testing execution

[0093] Task allocation and path planning

[0094] Before a mission begins, the A* or Dijkstra algorithm is used to plan optimal flight and driving paths for drones and unmanned vehicles based on the target area's map information and mission requirements. Simultaneously, the Hungarian algorithm is used to assign tasks to drones and unmanned vehicles. The target area is divided into multiple subareas, and the cost of inspecting each subarea by a drone or unmanned vehicle is calculated based on their detection capabilities and efficiency. This cost function takes into account factors such as flight or driving distance, inspection time, and energy consumption. For example, large, open subareas are assigned to drones for rapid scanning, while complex, narrow subareas are assigned to unmanned vehicles for detailed inspection.

[0095] During path planning, the drone's altitude limit and the vehicle's terrain adaptability were considered. The drone's flight altitude was set between 50 and 200 meters, avoiding obstacles such as buildings and high-voltage power lines. The vehicle's route avoided impassable areas such as steep slopes and rivers.

[0096] Real-time collaborative detection

[0097] During the inspection process, the drone and the unmanned vehicle maintain real-time communication via a wireless communication module at a frequency of 1Hz. When the drone detects a potential target, it transmits information such as the target's location (longitude, latitude, and altitude) and its approximate shape (preliminarily determined based on image features) to the unmanned vehicle. Based on this information, the unmanned vehicle uses its onboard path planning algorithm to replan its route and quickly drive to the target area.

[0098] As the unmanned vehicle approaches the target, the drone adjusts its altitude and angle based on the vehicle's feedback, such as speed and location, and re-detects the target from different perspectives. For example, if the unmanned vehicle reports that the target is partially obscured, the drone lowers its altitude to 100 meters and adjusts its angle to 45 degrees to obtain a clearer image of the target. The drone then promptly transmits the newly acquired data to the unmanned vehicle.

[0099] 5. Anti-occlusion processing implementation

[0100] UAV anti-occlusion operation

[0101] When the drone detects that the target may be obstructed, it first calculates the optimal flight angle and altitude based on its own positioning information (longitude, latitude, altitude) and the target location information using trigonometric functions to avoid the obstruction of the target's line of sight. Assume that the current position of the drone is (x u ,y u ,z u ), the target position is (x t ,y t ,z t ), the position of the occluder is (x o ,y o ,z o ), by calculating the vector and The optimal flight angle and altitude adjustment value are determined based on the included angle and the flight safety range of the UAV.

[0102] The drone adjusts its flight attitude and photographs the target area from a new angle, capturing 3-5 images from different angles. Once the images are captured, they are transmitted to the data fusion and processing center. There, image stitching algorithms from the OpenCV library, such as the SIFT (Scale-Invariant Feature Transform) algorithm, are used to stitch the images from different angles together and attempt to recover any obscured information about the target.

[0103] Unmanned vehicle anti-occlusion operation

[0104] The autonomous vehicle uses its ultrasonic sensors at a frequency of 50Hz to detect the position and distance of surrounding obstructions. When an obstruction is detected, it uses LiDAR data and existing map information to determine its shape and size using a point cloud matching algorithm. The LiDAR point cloud data is matched with the obstacle point cloud data in the map to determine the specific shape and location of the obstruction.

[0105] At the same time, the unmanned vehicle sends a request to the data fusion and processing center to obtain image data of the target area captured by the drone from other angles. Combining its own sensor data with the data transmitted by the drone, a data fusion algorithm is used to infer and reconstruct the features of the obscured target. For example, by fusing the 3D point cloud features of the lidar with the features of the drone image, a neural network model is used to infer the complete shape and category of the obscured target, thereby achieving accurate detection of the obscured target.

[0106] Example 2

[0107] Examples of preferred structures and algorithms of the present invention

[0108] (1) System architecture design

[0109] The team built an efficient and adaptable drone-autonomous vehicle collaborative system architecture, primarily composed of drone and vehicle subsystems, as well as a data fusion and processing center. This architecture takes into account obstacles, signal interference, and complex terrain in diverse application scenarios, such as urban security, field exploration, and logistics distribution. The design aims to improve the reliability and accuracy of target detection in complex environments.

[0110] 1. UAV subsystem

[0111] Sensor configuration and data acquisition: To adapt to complex application environments, the drone is equipped with a high-resolution visible light camera and a thermal imager. The visible light camera is selected with a resolution of n pixels (n≥4×10 7 pixels), a device with a frame rate of f (f ≥ 60fps), which collects image I v The process can be expressed as:

[0112]

[0113] Where (x, y) is the image pixel coordinate, λ is the wavelength of light, λ1 and λ2 are the lower and upper limits of the visible light wavelength range, S(λ, x, y) is the spectral reflectance of the scene at position (x, y) for wavelength λ, and R(λ) is the response function of the camera to wavelength λ. The thermal imager is used to obtain the target thermal radiation information and generate a thermal image I t , its principle is based on Planck's law. The relationship between the radiation flux density L received by the thermal imager and the temperature T is:

[0114] Where h is Planck's constant, c is the speed of light in a vacuum, k is the Boltzmann constant, λ is the wavelength of thermal radiation, and T is the target temperature. The drone determines its position and attitude through a positioning and navigation module composed of a global satellite navigation system (GNSS) and an inertial measurement unit (IMU). GNSS positioning outputs a position vector IMU measures the attitude angle vector Used to describe the roll, pitch, and yaw angles of a drone.

[0115] Data transmission: The data collected by the drone is transmitted to the data fusion and processing center through wireless communication links. Considering the signal interference problem in complex environments such as cities, the adaptive modulation and coding (AMC) technology is used to dynamically adjust the transmission rate R according to the channel quality index CQI. t The channel quality index CQI is related to the signal-to-noise ratio (SNR) and is calculated using the following formula: CQI = f(SNR)

[0116]

[0117] Among them, P s is the signal power, P n is the noise power. Transmission rate R t It can be expressed as: R t =Blog2(1+γSNR)

[0118] Here, B is the channel bandwidth, and γ is a constant related to the modulation and coding scheme. To ensure reliable data transmission over long distances, such as in the wild, low-Earth orbit (LEO) satellite communications are combined with ground-based relay communications. Satellite communications enable long-distance data transmission. Ground-based relay communications address issues such as terrain obstruction in the communication link between drones and satellites, ensuring stable data transmission to the data fusion and processing center.

[0119] 2. Unmanned vehicle subsystem

[0120] Multi-sensor fusion acquisition: The unmanned vehicle is equipped with a multi-line laser radar, a multi-view camera, and an ultrasonic sensor. The multi-line laser radar emits laser beams and receives reflected light to generate 3D point cloud data. Assume that the laser radar emits m laser beams. In the i-th scan, the distance value measured by the j-th laser beam is d ij , the angle value is α ij , then the coordinates of the point in the laser radar coordinate system (x l ,y l ,z l ) can be converted by the following formula: l =d ij sin(α ij )cos(β j )

[0121] y l =d ij sin(α ij )sin(β j )

[0122] zl =d ij cos(α ij )

[0123] Among them, β j is the vertical angle of the jth laser beam. The multi-view camera collects images from different directions, including the front view image I f 、Rear view image I r , side view image I s Through the principle of stereo vision, the three-dimensional position information of the target can be calculated using triangulation. Ultrasonic sensors are used to detect obstacles at close range. The measured distance D is based on the sound wave propagation time t, and the formula is:

[0124] Among them, v s is the speed of sound.

[0125] Data preprocessing and local computing: The unmanned vehicle performs preliminary preprocessing on the sensor data locally to reduce the amount of data transmission and improve the real-time performance of the system. For the lidar point cloud data, the voxel filtering algorithm is used to remove outliers and noise points. By setting the voxel grid size V size , dividing the point cloud into different voxels, calculating the mean of the points within each voxel as the representative point of that voxel, and achieving data downsampling. For camera image data, edge detection algorithms (such as the Canny algorithm) are used to extract image edge features and reduce redundant information. In the local computing module, field-programmable gate arrays (FPGAs) are used to accelerate computing. For some simple target detection tasks (such as close-range obstacle detection), preliminary detection is completed locally, reducing the computational burden on the data fusion and processing center, while providing more targeted feature information for data fusion.

[0126] 3. Data Fusion and Processing Center

[0127] Hardware platform construction: The data fusion and processing center uses a computing platform based on a high-performance graphics processing unit (GPU) cluster. Each GPU has N computing cores (N≥5000) and high-bandwidth memory M (M≥16GB). Through high-speed network interconnection technology, such as InfiniBand network, low-latency, high-bandwidth data transmission between GPUs and between drones and unmanned vehicles is achieved. The network bandwidth is B net ≥100Gbps. The platform runs an operating system based on a distributed computing framework (such as Apache Spark), which can efficiently manage computing resources and achieve parallel processing of massive data.

[0128] Data fusion algorithm implementation: adopt the data fusion algorithm based on Bayesian network. Assume that the feature vector collected by the UAV is The feature vector collected by the unmanned vehicle is The target state variable is According to Bayes' formula, the posterior probability It can be expressed as:

[0129] in, is the likelihood function, which describes the probability that the sensor observes the corresponding feature under a given target state; It is a prior probability, determined based on historical data or prior knowledge; is the probability of evidence, used for normalization. In actual calculations, a Bayesian network structure is constructed to determine the conditional probability relationships between nodes, and the maximum a posteriori (MAP) method is used to find the optimal estimate of the target state variable $\vec{X}$. To improve computational efficiency, an approximate inference algorithm (such as variational inference) is used to solve the complex Bayesian network. This significantly reduces the computational effort while maintaining a certain level of accuracy, meeting the system's real-time requirements.

[0130] Task Scheduling and Management: The Data Fusion and Processing Center is responsible for the unified scheduling and management of UAV and UGV tasks. Based on task priority, resource usage, and environmental information, task scheduling algorithms (such as priority-based preemptive scheduling algorithms) are used to rationally allocate computing and communication resources. For example, for urgent target detection tasks, more computing resources are prioritized for the relevant data processing processes, and the data transmission priority for this task is increased to ensure timely completion of the task. At the same time, the status of UAVs and UGVs is monitored in real time. When equipment failures or communication anomalies are detected, the task allocation strategy is adjusted promptly, and the flight path of the UAV or the driving route of the UGV are replanned to ensure the continuous and stable operation of the system.

[0131] (2) Data fusion algorithm

[0132] We developed a highly adaptive and precise data fusion algorithm designed to seamlessly integrate diverse and heterogeneous data collected by drones and autonomous vehicles, leveraging the strengths of both to improve the reliability and accuracy of target detection in complex environments. This algorithm, closely aligned with practical application context, considers the characteristics of data in different scenarios, including noise interference, to achieve efficient data fusion.

[0133] 1. Image feature extraction

[0134] Visible light image feature extraction: In urban environments, the targets are diverse and the background is complex. Convolutional neural networks (CNNs) are used to extract visible light image features. Taking the ResNet model as an example, the input image I vThe size is H×W×C (H is the image height, W is the image width, C is the number of channels, and C is 3 for color images in general). It is processed through a series of convolutional layers, batch normalization layers (BN), and activation functions. Let the convolution kernel size of the first layer be K l ×K l , the step size is S l , filled with P l , then the l-th layer outputs the feature map F l for:

[0135]

[0136] Among them, (i, j) is the feature map coordinate, k is the feature map channel index, W l is the convolution kernel weight of the lth layer, b l is the bias term, C l-1 is the number of channels of the l-1th layer feature map. Through multi-layer convolution operations, the edge, texture, shape and other features of the target are gradually extracted, and the final feature vector is output.

[0137] Thermal image feature extraction: Thermal images can provide key information in outdoor environments with low visibility at night. A specially designed thermal image feature extraction network is used. Considering that thermal images reflect differences in target thermal radiation, their features are different from those of visible light images. The network structure contains multiple convolutional layers and pooling layers. Suppose the input thermal image I t Size H t ×W t ×1 (single channel), after N convolution layers, the size of the convolution kernel in the nth layer is The step size is Fill with Output feature map for:

[0138]

[0139] Here σ is the activation function (such as sigmoid function), is the weight of the thermal imaging convolution kernel of the nth layer, is the bias term, is the number of channels of the thermal imaging feature map at layer n-1. Finally, the thermal imaging feature vector is obtained

[0140] 2. LiDAR Data Processing

[0141] In field exploration of complex terrain or urban traffic scenes, the laser radar of the unmanned vehicle obtains a large amount of 3D point cloud data. First, the original point cloud data P = {p1, p2, ..., p N}(pi =(x i ,y i ,z i ) is the coordinate of the three-dimensional space point) for preprocessing, and statistical filtering is used to remove outliers. Suppose point p i The neighborhood point set is N i , calculate the neighborhood point and point p i The mean distance μ i and standard deviation σ i :

[0142]

[0143] If ||p i -p j ||>μ i +kσ i (k is the threshold, usually 2-3), then point p j Then, a clustering algorithm based on Euclidean distance (such as DBSCAN) is used to segment the point cloud and separate the target from the background point cloud. l , calculate its center of mass

[0144] and volume V l and surface area S l Equal geometric features to form the laser radar feature vector

[0145] 3. Data fusion based on Bayesian estimation

[0146] Assume that the feature vector set obtained by the UAV is The feature vector obtained by the unmanned vehicle is The target state variable is (including target location, category, etc.) According to the Bayesian formula, the posterior probability for:

[0147]

[0148] in, is the prior probability, which can be determined based on historical data or prior knowledge. Likelihood function Taking into account the independence assumption of different sensor data (corrected by correlation analysis in practice), it can be decomposed into:

[0149]

[0150] and They represent the probability of visible light image features, thermal imaging image features, and lidar features appearing under a given target state, respectively, and are estimated through training data. is the probability of evidence, used for normalization. In order to improve computational efficiency, the variational inference algorithm is used in practical applications, and the approximate distribution is introduced. To approximate the posterior probability By minimizing the Kullback-Leibler (KL) divergence To solve. KL divergence is defined as:

[0151]

[0152] Through continuous iterative optimization Finally, the target state variable is obtained The optimal estimation of the target image can be realized, and the effective fusion of different sensor data can be achieved, providing a more accurate and comprehensive target feature description to adapt to complex and changing application scenarios.

[0153] (3) Collaborative testing strategy

[0154] Develop an intelligent and flexible UAV-UAV collaborative detection strategy that efficiently utilizes both resources based on the complex characteristics of different application scenarios, improving target detection efficiency and accuracy. This strategy encompasses key aspects such as task allocation, path planning, and dynamic collaborative adjustment.

[0155] 1. Task Allocation

[0156] In urban security scenarios, targets are widely distributed and of various types, and task allocation needs to be done by comprehensively considering the detection capabilities and resource limitations of drones and unmanned vehicles. Let the drone set be U = {u1,u2,...,u m}, the set of unmanned vehicles is G={g1,g2,...,g n}, the target area is divided into multiple sub-areas R = {r1, r2, ..., r k}. Define drone u i For region r j Detection efficiency Its flight speed Sensor coverage And the potential target density d in the area j Related, can be expressed as:

[0157]

[0158] in, It's a drone i Initial position coordinates, (x j ,y j ,z j) is the region r j Center coordinates. Similarly, the unmanned vehicle g l For region r j Detection efficiency The speed of the autonomous vehicle Sensor effective detection distance And the regional complexity c j It can be expressed as:

[0159]

[0160] here, It is an unmanned vehicle l Initial position coordinates. Maximize the total detection efficiency E through task allocation algorithms such as the Hungarian algorithm:

[0161] in, and Assign a decision variable to the task, which takes a value of 0 or 1, indicating whether the UAV $u_i$ or the unmanned vehicle g l Responsible for area r j detection task.

[0162] 2. Path Planning

[0163] In field exploration scenarios, the terrain is complex and there are many obstacles. It is crucial to plan safe and efficient paths for drones and unmanned vehicles. An improved A* algorithm is used to consider the flight altitude constraint of the drone and the terrain adaptability of the unmanned vehicle. For the drone, the starting point is S u , the end point is T u , the cost function f(N) of node N is: f(N)=g(N)+h(N)+λ·p(N)

[0164] Among them, g(N) is the u The actual cost to reach node N is related to the flight distance and energy consumption; h(N) is the cost from node N to the end point T u The estimated cost is Euclidean distance; p(N) represents the terrain penalty term at node N. If there are obstacles such as mountains at this location that make flight difficult, the p(N) value increases; λ is the terrain penalty weight, which is adjusted according to the actual terrain complexity. For unmanned vehicles, path planning needs to consider factors such as terrain slope and obstacle distribution. Let the current position of the unmanned vehicle be S g , the target position is T g , the cost function F(M) of node M is: F(M)=G(M)+H(M)+μ·q(M)

[0165] Here, G(M) is derived from S gThe actual driving cost to node M is related to the driving distance and the increase in energy consumption caused by the terrain slope; H(M) is the cost of traveling from node M to T g The estimated cost is also based on Euclidean distance. q(M) is the obstacle penalty at node M. The presence of obstacles such as boulders increases q(M). μ is the obstacle penalty weight, determined based on the density of obstacles in the environment. By continuously searching for the path node with the lowest cost, the optimal driving path for the drone and unmanned vehicle is planned.

[0166] 3. Dynamic collaborative adjustment

[0167] In logistics and distribution scenarios, the target location may change due to conditions such as cargo handling, requiring real-time dynamic adjustment of the collaborative strategy. When the drone detects a change in the target location, it sets the original target location to P0 = (x0, y0, z0) and the new target location to P1 = (x1, y1, z1), and calculates the position change vector The drone sends this information to the unmanned vehicle, which then uses its current position P g =(x g ,y g ) and the new target position P1, and replans the driving path. At the same time, the UAV adjusts the flight altitude and angle according to the driving status of the unmanned vehicle and the new target position to ensure that it continues to provide effective target information support for the unmanned vehicle. For example, when the unmanned vehicle is traveling at a speed of v g , the direction angle is θ g When the UAV is at a distance d from the unmanned vehicle ug and the relative angle α ug , adjust the flight altitude h u for:

[0168]

[0169] in, is the initial flight altitude of the UAV, and k is the adjustment coefficient, which is determined according to the actual situation. Through this dynamic collaborative adjustment mechanism, the system can quickly adapt to changes in the target and environment, ensuring the smooth progress of the target detection task.

[0170] (4) Anti-occlusion processing mechanism

[0171] Design an intelligent and efficient anti-occlusion processing mechanism to address the challenge of target occlusion in complex application scenarios and ensure the accuracy and completeness of target detection. This mechanism fully utilizes the sensor characteristics and collaborative capabilities of drones and unmanned vehicles, and combines the characteristics of different scenarios for targeted processing.

[0172] 1. Anti-occlusion operation based on drones

[0173] Flight angle and height optimization: In urban environments, building occlusion is a common problem. When the drone detects that the target may be blocked, the current position coordinate of the drone is set as P u =(x u ,y u ,z u ), the target position coordinate is P t =(x t ,y t ,z t ), the location information of the occluder is obtained through the early map construction or real-time sensor data, and the coordinates of the key points of the occluder are set as P o =(x o ,y o ,z o ). To calculate the optimal flight angle and altitude, an objective function J is constructed to maximize the visual angle between the target and the UAV while considering the UAV flight energy consumption and safety constraints. J = ω1·θ vis -ω2·ΔE-ω3·d safe

[0174] Among them, θ vis is the visual angle between the target and the UAV, which is obtained through vector operation, that is,

[0175] ΔE is the energy consumption increased by changing the flight angle and altitude, which is related to the change in flight distance Δs and the energy consumption per unit distance e. ΔE = e·Δs, where Δs can be calculated through new path planning; d safe Is the safe distance between the drone and surrounding obstacles (including obstructions). If it is less than the safety threshold, a penalty term is added; ω1, ω2, and ω3 are weight coefficients, which are adjusted according to the urgency of the mission and the complexity of the environment. The objective function is solved by an optimization algorithm (such as particle swarm optimization algorithm) to obtain the optimal flight angle α opt and height h opt .

[0176] Image stitching and information recovery: After the UAV adjusts its flight attitude, it takes a sequence of images of the target area {I1, I2, ..., I n}. Use the scale-invariant feature transform (SIFT) algorithm to extract and match image features. For image I i and I i+1 , extract the feature point set K i and K i+1 , by calculating the Euclidean distance between the feature point descriptors, we can get the matching point pair set M i,i+1 . Let the matching point pair be (k i,j ,k i+1,l ), where k i,j ∈Ki , k i+1,l ∈K i+1 By calculating the homography matrix H i,i+1 , image I i+1 Transform to I i In the same coordinate system, image stitching is achieved. The homography matrix is ​​solved by minimizing the reprojection error. The reprojection error E reproj Defined as:

[0177] By image stitching, we try to restore the target information of the blocked part and generate the complete target area image I comp .

[0178] 2. Anti-occlusion operation based on unmanned vehicles

[0179] Obstruction perception and analysis: In the wild environment, terrain undulations and vegetation obstructions are common. The unmanned vehicle uses ultrasonic sensors to detect the distance information of surrounding obstructions. Let the mth ultrasonic sensor measure the distance d m , measuring angle β m , then in the unmanned vehicle coordinate system, the coordinates of the blocking object position point (x o,m ,y o,m ) can be expressed as: x o,m =d m cos(β m )

[0180] y o,m =d m sin(β m )

[0181] Combined with the LiDAR point cloud data, the density-based spatial clustering algorithm (DBSCAN) is used to perform cluster analysis on the occluder point cloud to determine the shape and size of the occluder. Let the cluster set be C = {C1, C2, ..., C s}, for each cluster C i , calculate its center of mass Long axis length L i and the minor axis length W i Geometric features such as occluder are used to describe the shape of the occluder.

[0182] Target feature inference and reconstruction: The unmanned vehicle sends a request to the data fusion and processing center to obtain the target area image data taken by the drone from other angles. Combining its own lidar point cloud data and drone image data, the target feature inference network based on deep learning is used. Assume that the input data is the unmanned vehicle lidar feature vector and the drone image feature vector Feature fusion and inference are performed through a multi-layer neural network (such as a fully connected neural network). The network structure is Where W i and b i are the network weights and bias terms, and $f$ is the activation function (such as ReLU). By training a large amount of sample data containing occlusion situations, the network learns the characteristic patterns of the occluded target, thereby inferring and reconstructing the complete features of the occluded target and achieving accurate detection of the occluded target.

[0183] Through the above specific implementation methods, the drone-unmanned vehicle system collaborative target detection technology of the present invention can effectively integrate the information obtained by the drone and the unmanned vehicle from different perspectives, avoid the influence of obstacle occlusion, and provide more reliable and accurate target detection results, which has broad application prospects and practical value.

[0184] The above are only a few preferred embodiments of the present invention, and their description is relatively specific and detailed, but it should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and such modifications and improvements are within the scope of protection of the present invention.

Claims

1. A target detection method for a UAV-UAV system collaboration, characterized in that: The following steps are involved: a. Build a collaborative system architecture consisting of a drone subsystem, an unmanned vehicle subsystem, and a data fusion and processing center; b. The drone subsystem collects aerial images and thermal radiation data of the target area through the high-resolution camera and thermal imager carried by the drone subsystem, and transmits its own position and attitude information in real time through the positioning and navigation equipment; c. The unmanned vehicle subsystem uses lidar, multi-view cameras, and ultrasonic sensors to collect ground three-dimensional point cloud data, multi-angle images, and obstacle distance information; d. In the data fusion and processing center, deep learning algorithms are used to extract features from image data collected by drones and unmanned vehicles, perform point cloud segmentation and geometric feature calculations on lidar data, and fuse multi-source data using Bayesian estimation methods to generate a posterior probability distribution of the target state; e. Use path planning algorithms to assign tasks and plan flight and driving paths for drones and autonomous vehicles based on target area characteristics and mission requirements; f. During the detection process, the detection strategies of the drone and unmanned vehicle are dynamically adjusted through real-time communication. When the target is occluded, the anti-occlusion processing mechanism is triggered. This involves the drone adjusting its flight angle / altitude to obtain multi-view images, and the unmanned vehicle combining lidar and drone data to reconstruct target features.

2. The method according to claim 1, characterized in that In the step d: The convolutional neural network (CNN) model is used to extract features from visible light images; Specially designed grayscale stretching and convolutional networks are used to extract features from thermal images; Statistical filtering and DBSCAN clustering algorithm are used to remove noise and segment targets from lidar point cloud data.

3. The method according to claim 1, characterized in that The path planning algorithm in step e includes the A* algorithm or the Dijkstra algorithm, and is optimized in combination with the UAV flight altitude constraint and the terrain adaptability of the UAV.

4. The method according to claim 1, wherein The task allocation in step e is performed using the Hungarian algorithm, which is dynamically allocated based on the detection efficiency of the UAV and the unmanned vehicle and the complexity of the target area.

5. The method according to claim 1, wherein The anti-occlusion processing mechanism includes: The drone calculates the optimal flight angle and altitude through trigonometric functions to avoid obstructions in the line of sight, and restores the obscured target information through image stitching algorithms. The unmanned vehicle uses ultrasonic sensors and lidar data to determine the shape of obstructions, and combines multi-view images from drones to infer and reconstruct target features.

6. The method according to claim 1, wherein The communication between the UAV and the unmanned vehicle adopts 4G / 5G wireless transmission technology, and the data transmission protocols include UDP and TCP / IP.

7. A target detection system for UAV-UAV collaboration, used in the method according to any one of claims 1 to 6, characterized in that: include: Drone subsystem: equipped with a high-resolution camera, thermal imager, GNSS / IMU positioning and navigation module, and 4G / 5G communication module; Unmanned vehicle subsystem: equipped with multi-line laser radar, multi-view cameras, ultrasonic sensors and on-board computing units; Data fusion and processing center: Deploy high-performance servers, run data fusion algorithms based on deep learning and Bayesian estimation, and realize task scheduling and dynamic collaborative control of drones and unmanned vehicles.

8. The system according to claim 7, characterized in that The thermal imager of the drone subsystem is the FLIRTau2 series, and the laser radar of the unmanned vehicle subsystem is a 16-line or 32-line multi-line laser radar.

9. Application of the method according to any one of claims 1 to 6 in complex terrain exploration, characterized in that: The following steps are involved: 1) UAVs can quickly scan large areas from a high-altitude perspective and identify potential targets; 2) The unmanned vehicle conducts detailed inspections of rugged terrain based on the information fed back by the drone, and generates a three-dimensional environmental model by combining lidar and multi-view camera data.

10. Application of the method according to any one of claims 1 to 6 in urban security scenarios, characterized in that , including the following steps: 1) UAVs monitor urban high-altitude areas in real time and, upon detecting unusual targets, guide unmanned vehicles to avoid traffic obstacles and drive towards the target; 2) The unmanned vehicle detects close-range obstacles through ultrasonic sensors and accurately identifies targets based on multi-angle images from drones.

Images