Multi-unmanned aerial vehicle cooperative tracking method and system based on complex environment

Abstract

The invention relates to the technical field of unmanned aerial vehicle cooperative tracking, in particular to a multi-unmanned aerial vehicle cooperative tracking method and system based on a complex environment. Comprising the following steps: S1, acquiring image data acquired by each unmanned aerial vehicle and flight position data of each unmanned aerial vehicle, and determining a tracking target, a main tracking unmanned aerial vehicle and an auxiliary unmanned aerial vehicle; s2, establishing an appearance feature library for the tracking target according to the image data; an auxiliary unmanned aerial vehicle checks an environment shielding risk in advance through rasterizing a pre-survey point and simulating a target moving track, when shielding occurs, a stop mechanism is triggered, an auxiliary tracking position is optimized, an optimal position is calculated in combination with a direction weight and a distance threshold value, continuity of a tracking process is ensured, and the tracking efficiency is improved. Meanwhile, when the image feature proportion of the auxiliary unmanned aerial vehicle is higher than that of the main tracking machine, identity replacement is automatically triggered, it is ensured that when the view angle of the main tracking machine is limited, continuous recording of the target can still be kept through the auxiliary unmanned aerial vehicle, and the task reliability in the complex environment is improved.

Description

Technical Field

[0001] This invention relates to the field of drone collaborative tracking technology, and more specifically, to a method and system for multi-drone collaborative tracking in complex environments. Background Technology

[0002] In the field of wildlife behavior research and ecological recording, the use of drones for animal tracking and monitoring has become an important technical means. The main purpose of existing technologies is to achieve remote observation of animal activity trajectories and behavioral status through image acquisition equipment carried by drones.

[0003] Currently, most methods use single drones or simple drone swarms to track animals based on preset routes. However, using a fixed tracking distance makes it impossible to dynamically adjust the distance to the target based on the animal species and behavioral state. This can easily cause animals to become alert or escape when drones approach, directly affecting the authenticity and scientific value of the recorded data. Furthermore, there is insufficient adaptability to complex environments (such as forests, mountains, and wetlands), and a lack of comprehensive analysis of the semantic features of terrain obstacles and animal habitats. Target loss is often caused by occlusion and positioning errors, making it difficult to cover diverse tracking scenarios. Therefore, this paper proposes a multi-drone collaborative tracking method and system based on complex environments. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-UAV cooperative tracking method and system based on complex environments, so as to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, one objective of this invention is to provide a multi-UAV cooperative tracking method based on complex environments, comprising the following steps: S1. Acquire image data collected by each drone and flight position data of each drone, and determine the tracking target, the main tracking drone, and the auxiliary drones; S2. Establish an appearance feature database for the tracking target based on the image data, and then use the image data, flight position data and appearance feature database collected by each UAV to locate the target position data of the tracking target. S3. Set the tracking distance threshold, analyze the movement direction and speed of the tracking target, dynamically set the environmental survey distance based on the movement direction and speed, and then control multiple auxiliary drones to simulate the forward movement of the tracking target based on the environmental survey distance; S4. During the forward survey, the image data collected by the auxiliary UAV is combined with the distance between the main tracker and the tracked target to perform a tracking integrity simulation analysis. When environmental occlusion occurs between the auxiliary UAVs during the simulation, the movement is stopped and the auxiliary tracking position analysis is performed. S5. Combine the tracking distance threshold with the auxiliary tracking position to select the position, and control the auxiliary drone to perform fixed-point tracking according to the selected auxiliary tracking position, and dynamically adjust the fixed-point tracking drone according to the target position data; S6. Perform appearance feature ratio analysis on the image data collected by the auxiliary drone and the image data collected by the main tracking drone, and temporarily replace the identities of the auxiliary drone and the main tracking drone based on the ratio analysis results.

[0006] As a further improvement to this technical solution, S1 establishes a drone swarm network module, adding all drones used for collaborative tracking to the drone swarm network module, which then performs unified control over the multiple drones.

[0007] As a further improvement to this technical solution, step S1 is as follows: S1.1 Acquire image data from each UAV and locate the flight position data of each UAV. S1.2. Based on the image data collected by all drones, determine the target to be tracked. In order to maintain the continuity of the documentary, select a main tracking drone from among the multiple drones according to the drone configuration, and then use the other drones as auxiliary drones. The main tracking drone continuously moves according to the location of the target being tracked.

[0008] As a further improvement to this technical solution, step S2 is as follows: S2.1 Extract appearance features based on image data and the determined tracking target, and then integrate the extracted appearance features to establish an appearance feature library; S2.2. Use the image data, flight position data and appearance feature database collected by each UAV to locate the target position data of the tracked target and obtain the target position data of the tracked target.

[0009] As a further improvement to this technical solution, step S3 is as follows: S3.1 Set a tracking distance threshold based on the tracking target, and simultaneously acquire map data of the location of the tracking target. Combine the target location data with the map data and the real-time appearance features of the tracking target to analyze the movement direction and speed, and obtain the subsequent movement direction and speed of the tracking target. S3.2 Dynamically set the environmental survey distance based on the direction and speed of movement; The faster the movement speed, the farther the environmental survey distance; The slower the movement speed, the closer the environmental survey distance; S3.3 Control multiple auxiliary drones to work together and simulate target movement based on the acquired movement direction and environmental survey distance; For each tracking target, two auxiliary drones are assigned to the opposite direction of movement. One auxiliary drone simulates the position movement of the tracking target, and the other auxiliary drone simulates the main tracking drone.

[0010] As a further improvement to this technical solution, step S4 is as follows: S4.1 During the forward survey, the image data collected by the auxiliary drone simulating the main tracking drone is extracted, and then the tracking integrity analysis of the auxiliary drone simulating the tracking target is performed based on the extracted image data. S4.2 When the integrity analysis shows that the auxiliary drone that simulates the target cannot be extracted from the image data, the two auxiliary drones are controlled to stop moving. Then, the image data collected by the two auxiliary drones is extracted for auxiliary tracking position analysis to obtain the complete auxiliary tracking position that can track the simulated target. The tracking position is then sent to S5. S4.3 If the integrity analysis shows that the auxiliary drone that simulates the target being tracked can be extracted from the image data, then continue monitoring.

[0011] As a further improvement to this technical solution, step S5 is as follows: S5.1. Set weights for each movement direction based on the real-time appearance characteristics of the tracked target, then select the auxiliary tracking position by combining the tracking distance threshold, calculate the value of each auxiliary tracking position, and prioritize saving the auxiliary tracking position with the higher value. Furthermore, each direction of movement corresponds to only one auxiliary tracking position; The lower the weight ratio, the less distance is between the auxiliary tracking position and the simulated tracking target. S5.2. The auxiliary tracking position selected in S5.1 is sent to the auxiliary drone, thereby controlling the auxiliary drone to move to the auxiliary tracking position for fixed-point tracking; S5.3 Dynamically adjust the drone for fixed-point tracking based on target location data; When the target being tracked moves in the opposite direction, the auxiliary drone for fixed-point tracking is recalled, causing the auxiliary drone to move to the fixed-point tracking position in the opposite direction.

[0012] As a further improvement to this technical solution, step S6 is as follows: S6.1 Analyze the proportion of appearance features in the image data collected by the auxiliary drone at the fixed tracking position and the image data collected by the main tracking drone; S6.2 When the proportion of the appearance features of the auxiliary drone is greater than that of the main tracking drone, the auxiliary drone shall be used as the temporary main tracking drone for tracking target recording. S6.3 When the proportion of the appearance features of the auxiliary drone is less than that of the appearance features of the main tracking drone, the main tracking drone shall be recorded as the tracking target.

[0013] The second objective of this invention is to provide a multi-UAV collaborative tracking system based on complex environments, including any one of the above-mentioned multi-UAV collaborative tracking methods based on complex environments, comprising a target localization module, an auxiliary tracking module, and a UAV identity adjustment module; The target positioning module is used to acquire image data and flight position data collected by each UAV, determine the tracking target and the main tracking UAV and auxiliary UAV, and at the same time establish an appearance feature library for the tracking target and locate the target position data of the tracking target. The auxiliary tracking module is used to analyze the movement direction and speed of the tracked target, dynamically set the environmental survey distance according to the movement direction and speed, and then control multiple auxiliary drones to simulate the forward movement of the tracked target according to the environmental survey distance. At the same time, the image data collected by the auxiliary drones is combined with the distance between the main tracker and the tracked target to perform a tracking integrity simulation analysis. The drone identity adjustment module is used to select a location by combining the tracking distance threshold with the auxiliary tracking location, and to dynamically adjust the fixed-point tracking drone according to the target location data. Then, it performs appearance feature ratio analysis on the image data collected by the auxiliary drone and the image data collected by the main tracking drone, and temporarily replaces the identities of the auxiliary drone and the main tracking drone according to the ratio analysis results.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. A multi-UAV collaborative tracking method and system based on complex environments. The system uses an auxiliary UAV to pre-survey points using a gridded system and simulates the target's movement trajectory to identify environmental occlusion risks in advance. When occlusion occurs, a stop mechanism is triggered and the auxiliary tracking position is optimized. The optimal position is calculated by combining directional weights and distance thresholds to ensure the continuity of the tracking process. At the same time, when the image feature ratio of the auxiliary UAV is higher than that of the main tracker, an identity replacement is automatically triggered to ensure that the auxiliary UAV can continue to record the target even when the main tracker's field of view is limited, thereby improving the reliability of the task in complex environments.

[0015] 2. A multi-UAV collaborative tracking method and system based on complex environments, which identifies animal types and calls up a habit database, dynamically adjusts the tracking distance threshold in combination with its behavioral state, avoids stress caused by UAVs approaching, ensures that its natural behavior is not disturbed, and thus guarantees the authenticity of the recorded images. At the same time, the auxiliary UAVs can conduct environmental surveys in advance by simulating the target's movement path, and the main tracking UAV maintains a safe distance from the target to reduce direct confrontation. Attached Figure Description

[0016] Figure 1 This is an overall flowchart of the present invention; Figure 2 This is a flowchart illustrating the process of acquiring image data from each drone according to the present invention. Figure 3 This is a flowchart illustrating the process of establishing an appearance feature library by integrating the extracted appearance features in this invention. Figure 4 This is a flowchart illustrating the process of dynamically setting the environmental survey distance based on the direction and speed of movement according to the present invention. Figure 5 This is a flowchart illustrating the process of performing a tracking integrity analysis on an auxiliary UAV that simulates a target based on extracted image data, according to the present invention. Figure 6 The flowchart for prioritizing the saving of auxiliary tracking positions with higher numerical values ​​in this invention; Figure 7 This is a flowchart illustrating how the auxiliary drone is used as a temporary primary tracking drone to record target tracking data according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figures 1-7 As shown, one of the objectives of this invention is to provide a multi-UAV cooperative tracking method based on complex environments, comprising the following steps: S1. Acquire image data collected by each drone and flight position data of each drone, and determine the tracking target, the main tracking drone, and the auxiliary drones; S1 establishes a drone swarm network module, adding all drones used for collaborative tracking to the drone swarm network module, which then performs unified control over multiple drones.

[0019] First, complete the basic construction of the UAV swarm network module, determine the core framework of the module such as communication protocol, data transmission rules, and control command system, and then add each UAV participating in collaborative tracking to the UAV swarm network module in sequence according to the access standards set by the module (such as identity code and communication frequency band adaptation), so that the UAVs can establish a connection with the module and realize information exchange. The steps for S1 are as follows: S1.1 Acquire image data from each UAV and locate the flight position data of each UAV. The drone activates its image acquisition equipment (camera) and captures images of the surrounding environment at a set frequency, while simultaneously acquiring its own latitude, longitude, altitude, and flight attitude position data through the positioning system (GPS).

[0020] S1.2. Based on the image data collected by all drones, determine the target to be tracked. In order to maintain the continuity of the documentary, select a main tracking drone from among the multiple drones according to the drone configuration, and then use the other drones as auxiliary drones. The main tracking drone continuously moves according to the location of the tracked target, and the specific steps are as follows: Target identification: All preprocessed image data from drones are aggregated into the drone swarm network module. Based on the task requirements, targets that meet the criteria are selected from the identification results and used as the objects of this collaborative tracking. Selection of primary tracking drone: Based on the drone configuration parameters (such as stability, endurance, and image resolution) and current status (such as battery level and location), establish an evaluation index system (such as prioritizing drones with long endurance and high image quality), calculate the comprehensive score of each drone, and select the drone with the highest score as the primary tracking drone, which is responsible for continuously locking onto and following the target; Auxiliary UAV allocation: All remaining UAVs are designated as auxiliary UAVs, and different tasks (such as forward reconnaissance, rear cover, and lateral environmental monitoring) are assigned according to their location and performance characteristics to ensure real-time data sharing and command synchronization.

[0021] S2. Establish an appearance feature database for the tracking target based on the image data, and then use the image data, flight position data and appearance feature database collected by each UAV to locate the target position data of the tracking target. The steps for S2 are as follows: S2.1 Extract appearance features based on image data and the determined tracking target, and then integrate the extracted appearance features to establish an appearance feature library; The image data collected by each UAV is preprocessed, and the key features of the tracking target are extracted from the images by the feature extraction algorithm. At the same time, motion features are extracted to enhance the spatiotemporal continuity of the feature library. The extracted multi-dimensional features are fused to form the target feature vector. Cluster analysis is performed on the feature vector to remove redundant features, retain the most representative feature subset, and establish a feature index structure to facilitate fast matching and retrieval, forming a dynamically updatable appearance feature library.

[0022] S2.2. Use the image data, flight position data, and appearance feature database collected by each UAV to locate the target position data of the tracked target, and obtain the target position data of the tracked target. The specific steps are as follows: Multi-source data fusion localization: The current frame image is input into the target detection model, and the appearance feature library is matched to obtain the pixel coordinates of the target in the image. Combined with the UAV's own position data and camera parameters, the world coordinates of the target are calculated through perspective transformation. At the same time, the observation results of multiple UAVs can be fused, and the triangulation principle is used to eliminate single-view error and improve the localization accuracy. Real-time location data optimization: A Kalman filter is applied to smooth the location data of consecutive frames, predicting the target's position at the next moment. Motion constraints (such as maximum acceleration limits) are introduced to filter out anomaly detection results, improving positioning stability. S3. Set the tracking distance threshold, analyze the movement direction and speed of the tracking target, dynamically set the environmental survey distance based on the movement direction and speed, and then control multiple auxiliary drones to simulate the forward movement of the tracking target based on the environmental survey distance; The steps for S3 are as follows: S3.1. Set a tracking distance threshold based on the target (set the tracking distance threshold based on the animal type to prevent the drone from getting too close to the animal, causing the animal to be disturbed and exhibit abnormal behavior, leading to recording errors). At the same time, acquire map data of the target's location. Combine the target location data with the map data and the real-time appearance characteristics of the target to analyze its movement direction and speed, and obtain the target's subsequent movement direction and speed. The specific steps are as follows: Animal type identification and distance threshold setting: Animal species (such as birds, mammals, and reptiles) are identified and tracked using drone image data. A pre-set animal behavior database (containing alert distances and stress response thresholds for different species) is accessed, and the tracking distance is dynamically adjusted based on the animal's current behavioral state (such as resting, foraging, or migrating). For example, migratory birds have a lower alert distance, so a closer threshold can be set; for predators in their brooding period, a larger safety distance is required. The specific formula is as follows: ; Among them, D th For the final tracking distance threshold, D T D is based on the species-specific baseline warning distance (e.g., 50 meters for wolves). BThe distance is adjusted for behavior (e.g., increasing by 20 meters under stress), with α1 and α2 being weighting coefficients. Habitat map data acquisition and semantic annotation: Use remote sensing equipment (such as infrared cameras and lidar) on drones to collect three-dimensional maps of animal habitats, perform semantic segmentation on the maps, and identify possible animal activity areas (such as grassland foraging areas and forest nests) and obstacles (such as cliffs and rivers). Fusion analysis of animal appearance features and movement status: Extract real-time appearance features of animals (such as bird wing frequency and mammal limb movements), combine them with posture estimation algorithms to determine movement intention (such as preparation for takeoff or running precursors), calculate the current movement speed and direction based on continuous frame position data, enhance the smoothness of movement trajectory through optical flow field analysis, and filter out positioning noise caused by sudden animal movements. By overlaying the animal's current location with map data, analyzing the constraints of habitat terrain on movement (e.g., deer herds tend to move along valley paths to avoid open areas), and combining animal behavior (e.g., migration routes, foraging cycles) and historical trajectory data, a probabilistic model is used to predict subsequent movement paths, outputting the most likely speed range and turning probability.

[0023] S3.2 Dynamically set the environmental survey distance based on the direction and speed of movement; The faster the movement speed, the farther the environmental survey distance; The slower the movement speed, the closer the environmental survey distance; Ensure the effectiveness of environmental surveys and avoid over-surveying.

[0024] S3.3 Control multiple auxiliary drones to work together and simulate target movement based on the acquired movement direction and environmental survey distance; For each tracking target, two auxiliary drones are assigned to the opposite direction of movement. One auxiliary drone simulates the position movement of the tracking target, and the other auxiliary drone simulates the main tracking drone.

[0025] The target's movement direction vector is combined with the dynamically calculated environmental survey distance to determine the pre-survey points in each possible movement direction. The pre-survey points are then rasterized, and priority areas are divided according to environmental complexity and target movement uncertainty. Assign a pair of auxiliary drones (A and B) for each direction of movement; UAV A moves ahead of time to the pre-survey point according to the predicted trajectory and speed of the target, and maintains its relative position to the target. UAV B moves synchronously to a position where it can clearly observe UAV A, simulating the perspective of the main tracking drone, and collects image data to assess the occlusion situation.

[0026] UAV B will transmit the collected image data back to the control center in real time to analyze whether UAV A (the virtual target) is obstructed by obstacles.

[0027] S4. During the forward survey, the image data collected by the auxiliary UAV is combined with the distance between the main tracker and the tracked target to perform a tracking integrity simulation analysis. When environmental occlusion occurs between the auxiliary UAVs during the simulation, the movement is stopped and the auxiliary tracking position analysis is performed. The steps for S4 are as follows: S4.1 During the forward survey, the image data collected by the auxiliary drone simulating the main tracking drone is extracted, and then the tracking integrity analysis of the auxiliary drone simulating the tracking target is performed based on the extracted image data. S4.2 When the integrity analysis shows that the auxiliary drone that simulates the target cannot be extracted from the image data, the two auxiliary drones are controlled to stop moving. Then, the image data collected by the two auxiliary drones is extracted for auxiliary tracking position analysis to obtain the complete auxiliary tracking position that can track the simulated target. The tracking position is then sent to S5. S4.3 If the integrity analysis shows that the auxiliary drone simulating the target being tracked can be extracted from the image data, continue monitoring. The specific steps are as follows: Tracking integrity quantification analysis: Feature points (such as corners and edges) of UAV A are extracted from the preprocessed image. The detection rate and distribution uniformity of the feature points are calculated. At the same time, the pixel ratio, contour integrity, and whether UAV A is occluded by obstacles (such as tree or building shadows) in the image are analyzed. The formula is as follows: ; Where S is the integrity score (0~1), Nd is the number of detected feature points, Nt is the total number of preset feature points, Ao is the pixel area of ​​the target being occluded, At is the total pixel area of ​​the target, Cv is the distance from the center of the target to the center of the image, Cm is the image radius, and w1, w2, and w3 are weight coefficients.

[0028] Integrity threshold judgment and decision: Set an integrity score threshold (e.g., feature point detection rate > 70%, occlusion area < 30% is considered complete). If the score is ≥ the threshold, the tracking is considered complete and UAV B maintains the current monitoring state; if the score is < the threshold, the stop mechanism is triggered. Stopping mechanism: When integrity is insufficient, both UAVs A and B are controlled to hover simultaneously, stop moving forward, collect image data of the current positions of the two UAVs, and analyze the distribution of obstacles and feasible airspace in the surrounding environment; Assisted tracking position optimization calculation: Based on environmental data, multiple candidate positions are generated, and the feasibility of tracking UAV A at each position is evaluated (such as unobstructed view and distance to the target).

[0029] S5. Combine the tracking distance threshold with the auxiliary tracking position to select the position, and control the auxiliary drone to perform fixed-point tracking according to the selected auxiliary tracking position, and dynamically adjust the fixed-point tracking drone according to the target position data; The steps for S5 are as follows: S5.1. Set weights for each movement direction based on the real-time appearance characteristics of the tracked target, then select the auxiliary tracking position by combining the tracking distance threshold, calculate the value of each auxiliary tracking position, and prioritize saving the auxiliary tracking position with the higher value. Furthermore, each direction of movement corresponds to only one auxiliary tracking position; The lower the weight ratio, the less distance is between the auxiliary tracking position and the simulated tracking target. Extract key appearance features from real-time images of the tracked target and establish a mapping relationship between appearance features and movement direction; If the target's head is facing a certain direction, the weight of that direction increases; if a rapid turning action occurs, the weight of the corresponding direction surges. For each candidate position, a comprehensive score is calculated by combining directional weights, distance from the target, and environmental factors. Only the position with the highest score in each direction is retained as the final auxiliary tracking position. Check whether the distance between the retained auxiliary tracking position and the simulated tracking target is less than the tracking distance threshold. If it is less than the threshold, reduce the scoring weight of the position (multiply by the attenuation coefficient) and prioritize positions with moderate distance to reduce the interference of the drone on the tracking target. ; Among them, P d f is the weighting coefficient for direction d. pose (d) is the orientation preference function based on the target attitude, f motion (d) represents the directional probability based on historical motion trajectories, f env (d) represents the environmental suitability score, where β1, β2, and β3 are weighting coefficients; ; Where s(p) is the comprehensive score of the auxiliary tracking position p, and d(p, P) is the overall score of the auxiliary tracking position p. A () represents the position p to the simulated tracking target P. A The distance is given by occ(p), where occ(p) is the obstacle occupancy rate at position p, e is the natural constant, and λ is the environmental impact coefficient (e.g., λ=2). ; Where Z is the corrected final score, and K is the attenuation coefficient (1 when the auxiliary tracking position and the simulated tracking position are greater than the tracking distance threshold, and 1 when the auxiliary tracking position and the simulated tracking position are less than the tracking distance threshold, with the smaller the value, the lower the value, and the minimum is 0.1).

[0030] S5.2. The auxiliary tracking position selected in S5.1 is sent to the auxiliary drone, thereby controlling the auxiliary drone to move to the auxiliary tracking position for fixed-point tracking; The calculated optimal auxiliary tracking position coordinates (latitude, longitude, and altitude) are packaged into control commands and sent to the corresponding auxiliary drones through the drone swarm network module. After receiving the commands, the auxiliary drones start the navigation system to calculate the deviation between the current position and the target position, generate an obstacle avoidance route, control the drones to move along the optimal path to the auxiliary tracking position, and keep hovering.

[0031] S5.3 Dynamically adjust the drone for fixed-point tracking based on target location data; When the target being tracked moves in the opposite direction, the auxiliary drone for fixed-point tracking is recalled, causing the auxiliary drone to move to the fixed-point tracking position in the opposite direction.

[0032] The system continuously acquires real-time position data of the target and calculates the relative displacement change between the target position and the auxiliary drone. When the target displacement exceeds a preset threshold (such as a 180° change in movement direction or a displacement distance exceeding 50% of the tracking distance threshold), a dynamic adjustment mechanism is triggered. If the target is detected to be moving in the opposite direction, a recall command is sent to the currently stationary tracking auxiliary drone, causing it to return to the safe area or designated rendezvous point along the original path. Based on the target's new direction of movement, the auxiliary tracking position in the opposite direction is recalculated, a new control command is generated and sent to the drone, causing it to move to the new position to continue tracking.

[0033] S6. Perform appearance feature ratio analysis on the image data collected by the auxiliary drone and the image data collected by the main tracking drone, and temporarily replace the identities of the auxiliary drone and the main tracking drone based on the ratio analysis results.

[0034] The steps for S6 are as follows: S6.1 Analyze the proportion of appearance features in the image data collected by the auxiliary drone at the fixed tracking position and the image data collected by the main tracking drone; S6.2 When the proportion of the appearance features of the auxiliary drone is greater than that of the main tracking drone, the auxiliary drone shall be used as the temporary main tracking drone for tracking target recording. S6.3 When the proportion of the appearance features of the auxiliary drone is less than that of the appearance features of the main tracking drone, the main tracking drone shall be recorded as the tracking target.

[0035] When the switch is triggered, the control authority of the auxiliary drone is upgraded to "temporary master tracker", and its position and attitude data are synchronized to the master tracking system. The original master tracking drone is downgraded to an auxiliary role and adjusts its flight path according to the instructions of the new master tracker to ensure the continuity of the tracking mission.

[0036] The second objective of this invention is to provide a multi-UAV collaborative tracking system based on complex environments, including any one of the above-mentioned methods for multi-UAV collaborative tracking in complex environments, comprising a target localization module, an auxiliary tracking module, and a UAV identity adjustment module; The target localization module is used to acquire image data and flight position data collected by each UAV, determine the tracking target, the main tracking UAV, and the auxiliary UAV, and at the same time, establish an appearance feature library for the tracking target and locate the target position data. The auxiliary tracking module is used to analyze the movement direction and speed of the tracked target, dynamically set the environmental survey distance based on the movement direction and speed, and then control multiple auxiliary drones to simulate the forward movement of the tracked target based on the environmental survey distance. At the same time, the image data collected by the auxiliary drones is combined with the distance between the main tracker and the tracked target to perform a simulation analysis of the tracking integrity. The drone identity adjustment module is used to select a location by combining the tracking distance threshold with the auxiliary tracking location, and to dynamically adjust the fixed-point tracking drone based on the target location data. Then, it performs appearance feature ratio analysis on the image data collected by the auxiliary drone and the image data collected by the main tracking drone, and temporarily replaces the identities of the auxiliary drone and the main tracking drone based on the ratio analysis results.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-UAV cooperative tracking method based on complex environments, characterized in that: The steps include: S1. Acquire image data collected by each drone and flight position data of each drone, and determine the tracking target, the main tracking drone, and the auxiliary drones; S2. Establish an appearance feature database for the tracking target based on the image data, and then use the image data, flight position data and appearance feature database collected by each UAV to locate the target position data of the tracking target. S3. Set the tracking distance threshold, analyze the movement direction and speed of the tracking target, dynamically set the environmental survey distance based on the movement direction and speed, and then control multiple auxiliary drones to simulate the forward movement of the tracking target based on the environmental survey distance; S4. During the forward survey, the image data collected by the auxiliary UAV is combined with the distance between the main tracker and the tracked target to perform a tracking integrity simulation analysis. When environmental occlusion occurs between the auxiliary UAVs during the simulation, the movement is stopped and the auxiliary tracking position analysis is performed. S5. Combine the tracking distance threshold with the auxiliary tracking position to select the position, and control the auxiliary drone to perform fixed-point tracking according to the selected auxiliary tracking position, and dynamically adjust the fixed-point tracking drone according to the target position data; S6. Perform appearance feature ratio analysis on the image data collected by the auxiliary drone and the image data collected by the main tracking drone, and temporarily replace the identities of the auxiliary drone and the main tracking drone based on the ratio analysis results.

2. The multi-UAV cooperative tracking method based on complex environments according to claim 1, characterized in that: S1 establishes a drone swarm network module, adding all drones used for collaborative tracking to the drone swarm network module, which then performs unified control over the multiple drones.

3. The multi-UAV cooperative tracking method based on complex environments according to claim 1, characterized in that: The steps in S1 are as follows: S1.1 Acquire image data from each UAV and locate the flight position data of each UAV. S1.

2. Based on the image data collected by all drones, determine the target to be tracked. In order to maintain the continuity of the documentary, select a main tracking drone from among the multiple drones according to the drone configuration, and then use the other drones as auxiliary drones. The main tracking drone continuously moves according to the location of the target being tracked.

4. The multi-UAV cooperative tracking method based on complex environments according to claim 1, characterized in that: The steps in S2 are as follows: S2.1 Extract appearance features based on image data and the determined tracking target, and then integrate the extracted appearance features to establish an appearance feature library; S2.

2. Use the image data, flight position data and appearance feature database collected by each UAV to locate the target position data of the tracked target and obtain the target position data of the tracked target.

5. The multi-UAV cooperative tracking method based on complex environments according to claim 1, characterized in that: The steps in S3 are as follows: S3.1 Set a tracking distance threshold based on the tracking target, and simultaneously acquire map data of the location of the tracking target. Combine the target location data with the map data and the real-time appearance features of the tracking target to analyze the movement direction and speed, and obtain the subsequent movement direction and speed of the tracking target. S3.2 Dynamically set the environmental survey distance based on the direction and speed of movement; The faster the movement speed, the farther the environmental survey distance; The slower the movement speed, the closer the environmental survey distance; S3.3 Control multiple auxiliary drones to work together and simulate target movement based on the acquired movement direction and environmental survey distance; For each tracking target, two auxiliary drones are assigned to the opposite direction of movement. One auxiliary drone simulates the position movement of the tracking target, and the other auxiliary drone simulates the main tracking drone.

6. The multi-UAV cooperative tracking method based on complex environments according to claim 1, characterized in that: The steps in S4 are as follows: S4.1 During the forward survey, the image data collected by the auxiliary drone simulating the main tracking drone is extracted, and then the tracking integrity analysis of the auxiliary drone simulating the tracking target is performed based on the extracted image data. S4.2 When the integrity analysis shows that the auxiliary drone that simulates the target cannot be extracted from the image data, the two auxiliary drones are controlled to stop moving. Then, the image data collected by the two auxiliary drones is extracted for auxiliary tracking position analysis to obtain the complete auxiliary tracking position that can track the simulated target. The tracking position is then sent to S5. S4.3 If the integrity analysis shows that the auxiliary drone that simulates the target being tracked can be extracted from the image data, then continue monitoring.

7. The multi-UAV cooperative tracking method based on complex environments according to claim 1, characterized in that: The steps in S5 are as follows: S5.

1. Set weights for each movement direction based on the real-time appearance characteristics of the tracked target, then select the auxiliary tracking position by combining the tracking distance threshold, calculate the value of each auxiliary tracking position, and prioritize saving the auxiliary tracking position with the higher value. Furthermore, each direction of movement corresponds to only one auxiliary tracking position; The lower the weight ratio, the less distance is between the auxiliary tracking position and the simulated tracking target. S5.

2. The auxiliary tracking position selected in S5.1 is sent to the auxiliary drone, thereby controlling the auxiliary drone to move to the auxiliary tracking position for fixed-point tracking; S5.3 Dynamically adjust the drone for fixed-point tracking based on target location data; When the target being tracked moves in the opposite direction, the auxiliary drone for fixed-point tracking is recalled, causing the auxiliary drone to move to the fixed-point tracking position in the opposite direction.

8. The multi-UAV cooperative tracking method based on complex environments according to claim 1, characterized in that: The steps in S6 are as follows: S6.1 Analyze the proportion of appearance features in the image data collected by the auxiliary drone at the fixed tracking position and the image data collected by the main tracking drone; S6.2 When the proportion of the appearance features of the auxiliary drone is greater than that of the main tracking drone, the auxiliary drone shall be used as the temporary main tracking drone for tracking target recording. S6.3 When the proportion of the appearance features of the auxiliary drone is less than that of the appearance features of the main tracking drone, the main tracking drone shall be recorded as the tracking target.

9. A multi-UAV cooperative tracking system based on complex environments, used to implement the multi-UAV cooperative tracking method based on complex environments as described in any one of claims 1-8, characterized in that: This includes a target localization module, an auxiliary tracking module, and a drone identity adjustment module; The target positioning module is used to acquire image data and flight position data collected by each UAV, determine the tracking target and the main tracking UAV and auxiliary UAV, and at the same time establish an appearance feature library for the tracking target and locate the target position data of the tracking target. The auxiliary tracking module is used to analyze the movement direction and speed of the tracked target, dynamically set the environmental survey distance according to the movement direction and speed, and then control multiple auxiliary drones to simulate the forward movement of the tracked target according to the environmental survey distance. At the same time, the image data collected by the auxiliary drones is combined with the distance between the main tracker and the tracked target to perform a tracking integrity simulation analysis. The drone identity adjustment module is used to select a location by combining the tracking distance threshold with the auxiliary tracking location, and to dynamically adjust the fixed-point tracking drone according to the target location data. Then, it performs appearance feature ratio analysis on the image data collected by the auxiliary drone and the image data collected by the main tracking drone, and temporarily replaces the identities of the auxiliary drone and the main tracking drone according to the ratio analysis results.

Images