Cooperative multi-target tracking method for rotor unmanned aerial vehicle cluster

Abstract

The invention provides a cooperative multi-target tracking method for a rotor unmanned aerial vehicle cluster. The cooperative multi-target tracking method is applied to the field of task allocation, motion planning and control. In an unknown environment, each unmanned aerial vehicle constructs and updates a rolling scale local map in real time through self-positioning and a depth map so as to obtain surrounding environment information. And the unmanned aerial vehicle performs target detection by using the information, captures moving target information, and shares the information through a broadcast mechanism to realize data exchange in a cluster. And based on the collected target information, the unmanned aerial vehicle cluster executes task allocation, and after respective tracking targets are determined, spatio-temporal trajectory planning is performed, including front-end path search and rear-end trajectory optimization. In order to avoid collision, the unmanned aerial vehicles exchange planning trajectories and adjust flight routes. According to the method, each unmanned aerial vehicle member can autonomously perceive the environment, map and position without depending on a GPS, real-time cooperative target tracking can be completed only by using a visual sensor, and it is ensured that the trajectory avoids static obstacles and cluster internal collision.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cluster unmanned aerial vehicle task allocation, motion planning and control, in particular to a method for cooperative target tracking of a cluster of rotor unmanned aerial vehicles. BACKGROUND

[0002] Cluster and intelligence have become the frontier and research hotspot in the field of unmanned systems. Improving the cooperative perception and interaction, autonomous cooperative decision-making, planning and control capability of autonomous unmanned systems, and improving the cluster intelligence level of unmanned systems are undoubtedly important development directions. Among the many potential application scenarios of unmanned system clusters, cooperative tracking of multiple targets is undoubtedly one of the most practical and important application scenarios. Under the premise of no prior information about the environment, the unmanned aerial vehicle cluster system needs to perceive the surrounding environmental obstacle information when performing target tracking tasks, and on this basis, real-time target detection and real-time task planning and decision-making are needed to achieve highly autonomous cooperative target tracking.

[0003] For cluster cooperative target tracking, in addition to considering the target tracking of individual unmanned aerial vehicles within the cluster, target allocation of unmanned aerial vehicle clusters and transmission of target information between unmanned aerial vehicles also need to be considered. Therefore, how to perform real-time cooperative target tracking is a problem that needs to be solved for cluster cooperative target tracking. SUMMARY

[0004] The present application provides a method for cooperative multi-target tracking of a cluster of rotor unmanned aerial vehicles, which can perform cluster cooperative target tracking for multiple targets. The following technical solutions are adopted:

[0005] In a first aspect, the unmanned aerial vehicle cluster is a cluster composed of at least two rotor unmanned aerial vehicles, and the same cooperative tracking method is run on each rotor unmanned aerial vehicle. The cooperative tracking method is executed in a loop, and each execution cycle includes:

[0006] S1, acquiring point cloud data of the environment in which the rotor unmanned aerial vehicle is located by using an onboard depth camera, and constructing a local environment map around the rotor unmanned aerial vehicle, the local environment map being a scale map representing the environment around the rotor unmanned aerial vehicle; the local environment map is in the form of a probability grid map, which divides the environment into small grids, each grid has two states of occupancy and vacancy, and uses an occupancy probability to represent the possibility of the obstacle being occupied;

[0007] S2, acquiring image data of the environment in which the rotor unmanned aerial vehicle is located by using an onboard camera, and constructing a target detection method with the image data as input to obtain the position and velocity information of the target, the target detection method being a method for extracting target-related information from perception information;

[0008] S3, based on the local environment map around the rotor unmanned aerial vehicle, an Euclidean distance field around the rotor unmanned aerial vehicle is constructed, the Euclidean distance field is a method for recording the Euclidean distance of each point in space to the surface of the nearest object in the form of a scalar and distinguishing the internal and external relations by symbols;

[0009] S4, based on the target detection and the target motion information obtained by the internal communication of the cluster, a centralized cluster allocation scheme is constructed to allocate a corresponding target to each rotor unmanned aerial vehicle cluster individual; for the motion information of a single specific target, a target motion prediction model of the rotor unmanned aerial vehicle is constructed, the target prediction method is a method for predicting the motion information of a dynamic target in a future period of time based on the historical motion information of the dynamic target;

[0010] S5, a space-time trajectory planning of a flight trajectory is performed, the space-time trajectory planning of the flight trajectory is a method for planning a path that does not collide with obstacles according to the constructed local environment map, optimizing the flight trajectory according to the planned path and the constructed Euclidean distance field, so that the optimized flight trajectory satisfies the collision-free condition with obstacles and other rotor unmanned aerial vehicle cluster individuals in space and satisfies the shortest total time of the planned flight trajectory within the dynamic constraint range of the rotor unmanned aerial vehicle in time; then the trajectory can be sent to the flight controller of the rotor unmanned aerial vehicle to complete the trajectory tracking of the unmanned aerial vehicle;

[0011] S6, the optimization results of the flight trajectory of the cluster members, the detected target position and speed information are transmitted in the cluster through a broadcast communication mechanism.

[0012] In a second aspect, the present application provides a rotor unmanned aerial vehicle cluster multi-target tracking system for a cooperative multi-target tracking task of a rotor unmanned aerial vehicle cluster, which comprises a data transmission module, a perception module, a mapping module, an assignment module, a planning module and a control module; the data transmission mainly involves receiving and sending of observed target position information and planned trajectory information between each rotor unmanned aerial vehicle; the perception module mainly involves obtaining depth pictures or laser point cloud data of the surrounding environment by a depth camera or a laser radar and converting them into environmental point cloud data and target observation information, and obtaining self-positioning information of the environmental point cloud data based on a visual or laser radar odometer; the mapping module mainly converts the environmental point cloud data in the perception module into a probability grid map; the assignment module mainly involves assigning the existing target observation information to a single rotor unmanned aerial vehicle; the planning module mainly involves generating a smooth flight trajectory that is safe, dynamically feasible and meets the tracking task constraints according to the current self-state of the rotor unmanned aerial vehicle obtained by the perception module, the probability grid information obtained by the mapping module and the assigned target observation position information obtained by the assignment module; and the control module mainly involves converting the flight trajectory obtained by the planning module into motor control instructions according to the self-dynamics characteristics of the rotor unmanned aerial vehicle to realize control of the unmanned aerial vehicle. Considering the relatively complex relationship between the modules, the structural relationship between the modules is shown in FIG. 1 for more intuitive presentation of the relationship between the modules. Figure 14

[0013] The perception module is configured to obtain point cloud data and position information of an environment in which the rotor unmanned aerial vehicle is located by an on-board depth camera and a positioner, and obtain target observation position information by a target detection method.

[0014] The mapping module is configured to construct a local environment map and an Euclidean distance field of the rotor unmanned aerial vehicle, wherein the local environment map is a scale map representing the environment around the rotor unmanned aerial vehicle, and the Euclidean distance field is a map representing the closest distance of the surrounding environment to an obstacle.

[0015] The assignment module is configured to perform task assignment on the observed cluster targets, so that each target can be reasonably assigned to each cluster member.

[0016] The planning module is configured to plan a flight trajectory, which comprises a flight position trajectory and a flight attitude trajectory, and optimize the flight trajectory according to the planned path and the previously constructed Euclidean distance field.

[0017] The control module is configured to control the flight of the multi-rotor unmanned aerial vehicle according to the optimized flight trajectory.

[0018] The data transmission module is configured to broadcast the optimized flight trajectory and the observed target information in the cluster.

[0019] ​The cooperative multi-target tracking method for a rotor unmanned aerial vehicle cluster of the application, when the cluster system is in an unknown environment, a local environment map is constructed and updated in real time through unmanned aerial vehicle positioning and a depth image, and a Euclidean distance field around the unmanned aerial vehicle is constructed in real time; the position information of a target is obtained based on an image obtained through a depth camera and by using a target detection method; after the target position information is collected, a reasonable tracking target is allocated to each multi-rotor unmanned aerial vehicle cluster member through task allocation; space-time trajectory planning is performed in the constructed environment, front-end path search and rear-end trajectory optimization are implemented; communication is performed between cluster members, the trajectories planned by each member and the observed target information are exchanged, collision avoidance is performed, and finally the flight trajectory of each member is obtained. Each member in the application has the ability of autonomous perception and autonomous planning, can only rely on a visual sensor and does not rely on a GPS signal for positioning; an unknown environment is perceived and mapped; target position information is identified and obtained from the perception information; task allocation is performed according to the observed target position information; real-time cooperative target tracking is performed, and the flight trajectory planned in the tracking process can avoid obstacles and avoid inter-aircraft collision.

[0020] The application has the advantages and beneficial effects that, compared with the traditional unmanned aerial vehicle target tracking method, the application can be applied to a multi-target tracking scene; the application adopts a software architecture combining a centralized target allocation method and a distributed target tracking method, which combines the characteristics of simple structure, small calculation complexity and closer to the optimal solution of the defined problem of the centralized target allocation scheme, and combines the characteristics of strong robustness, high calculation efficiency and real-time operation of the distributed target tracking method; a trajectory generation method for joint optimization of space-time trajectories is proposed in the application, which can optimize the spatial position, yaw angle and flight time, and in addition to considering the safety constraints and dynamic feasibility constraints necessary for autonomous navigation, the distance constraints and target visibility constraints in the target tracking process are also considered, so that the method has better performance in the target tracking scene. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The design flowchart of the application.

[0022] Figure 2 The target detection flowchart principle diagram in the application.

[0023] Figure 3 The target detection network structure diagram in the application.

[0024] Figure 4 The YOLOv5 model structure diagram used for target detection in the application.

[0025] Figure 5 The space-time trajectory planning structure diagram of the application.

[0026] Figure 6 A schematic diagram of the local environment map construction process of the present application.

[0027] Figure 7 A schematic diagram of the Euclidean distance field map construction process of the present application.

[0028] Figure 8 A schematic diagram of the search of the A* search algorithm considering dynamics of the present application.

[0029] Figure 9 A schematic diagram of the target tracking method for a single target of the present application.

[0030] Figure 10 A schematic diagram of the cooperative target tracking method for a single target of the present application.

[0031] Figure 11 , 12 A schematic diagram of the cooperative target tracking method for multiple targets of the present application.

[0032] Figure 12 A schematic diagram of the finite state machine switching logic of the present application.

[0033] Figure 13 A schematic diagram of the relationship between the modules in the present application. DETAILED DESCRIPTION

[0034] The design idea of the present embodiment is mainly to design a flight unmanned aerial vehicle cluster system capable of intelligent cooperative tracking. The system can process data transmitted by different positioning sources, and the unmanned aerial vehicle can use multiple data for self-state estimation, implement a perception method and a cooperative target tracking method in an unknown environment, and achieve efficient cooperation. The design includes a positioning data transmission module, a perception module, a mapping module, an allocation module, a planning module, and a control module, as well as data interaction interfaces between the modules. The stable operation of the unmanned aerial vehicle cluster system is ensured through the writing of software modules and the building of the overall operation framework. Finally, the performance of the method is evaluated through the actual flight verification results of the system.

[0035] ​This invention provides a cooperative multi-target tracking method for a swarm of rotary-wing UAVs. The method is used in a swarm consisting of at least two rotary-wing UAVs, with each UAV running a cooperative target tracking method that is executed cyclically. The rotary-wing UAVs in the swarm system are equipped with sensors such as a flight controller, onboard computer, binocular depth camera, and visual tracking camera. For example, a PixHawk4 is selected as the underlying controller for the rotary-wing UAVs; data from RTK-GPS / NOKOV motion capture system / UWB ultra-wideband wireless communication system / Intel Realsense D435i binocular camera is used to provide external position estimation for the rotary-wing UAVs; and an Intel NUC 12WSKi7 is used as the onboard computer for the UAVs. The swarm UAV system software modules include the software design of a positioning data transmission module, a perception module, a mapping module, an allocation module, a planning module, and a control module, as well as the data interaction interfaces between these modules. The data transmission module mainly includes software functions such as reading, processing, filtering, and sending positioning data. The perception module mainly includes functions for mapping the surrounding environment using received location and image data, as well as for detecting and locating targets. The allocation module mainly assigns all detected targets to each cluster member. The planning module mainly implements the spatiotemporal trajectory planning method for the system, enabling collaborative target tracking within the cluster system. The control module mainly converts the received instructions from the planning module into control quantities for the UAV motors, thereby controlling the UAVs to track a predetermined flight trajectory. This collaborative target tracking method is adaptable to rotary-wing UAVs of various wheelbases and supports the cluster UAV system to fly in environments with weak GPS signals through multi-source positioning fusion.

[0036] like Figure 14 As shown, each execution cycle includes:

[0037] S1. Use an airborne depth camera to acquire point cloud data of the environment in which the rotorcraft drone is located, and construct a local environment map around the rotorcraft drone.

[0038] The local environment map is a scale map representing the environment around the rotor unmanned aerial vehicle. In this embodiment, the scale map is selected as the local environment map around the unmanned aerial vehicle because it can be indexed and queried and can represent a dense map. The local environment map adopts the form of a probability grid map, which divides the environment into small grids, each grid has two states of occupancy and vacancy, and uses an occupancy probability to represent the possibility of being occupied by an obstacle. Specifically, the local environment map can be perceived and constructed by an environment perception method. For example, the depth map and other image information perceived by a depth camera are used to obtain the obstacle point cloud of the environment where the rotor unmanned aerial vehicle is located and to update it in real time. The scale map is further constructed based on the obstacle point cloud, and the position and attitude information of the unmanned aerial vehicle at this time is obtained by combining a visual real-time positioning method.

[0039] S2, acquiring image data of the environment where the rotor unmanned aerial vehicle is located by an on-board camera, and constructing a target detection method taking the image data as input to obtain the position and speed information of the target;

[0040] The target detection method is a method for extracting target-related information from perception information. This part uses a target detection method based on the YOLOv5 algorithm. The overall structure of the YOLOv5 algorithm includes an input end, a backbone network end, a neck network end, and a prediction end.

[0041] S3, constructing an Euclidean distance field around the rotor unmanned aerial vehicle based on the local environment map around the rotor unmanned aerial vehicle. The Euclidean distance field is a method for recording the Euclidean distance from each point in space to the surface of the nearest object in the form of a scalar and distinguishing the internal and external relationships by symbols.

[0042] S4, based on the target motion information obtained by target detection and intra-cluster communication, assigning each rotor unmanned aerial vehicle cluster individual a corresponding target by constructing a centralized cluster assignment scheme. For single specific target motion information, a target motion prediction model of the rotor unmanned aerial vehicle is constructed. The target prediction method is a method for predicting the motion information of a dynamic target in a future period of time based on the historical motion information of the dynamic target.

[0043] S5, performing space-time trajectory planning of the flight trajectory. The space-time trajectory planning of the flight trajectory is a method for planning a path that does not collide with obstacles based on the constructed local environment map, optimizing the flight trajectory based on the planned path and the constructed Euclidean distance field, so that the optimized flight trajectory satisfies the collision-free condition with obstacles and other rotor unmanned aerial vehicle cluster individuals in space and satisfies the condition that the total time of the planned flight trajectory is the shortest within the dynamic constraint range of the rotor unmanned aerial vehicle in time. Then the trajectory can be sent to the flight controller of the rotor unmanned aerial vehicle to complete the trajectory tracking of the unmanned aerial vehicle.

[0044] S6, the optimization result of the cluster member flight trajectory, the detected target position and speed information are transmitted in the cluster through a broadcast communication mechanism.

[0045] In the embodiment, each unmanned aerial vehicle member in the cluster has a distributed asynchronous planner and a target detector, can independently observe target information and plan its own path. After the cluster member completes target detection, it sends the observed target position information to the task planning center node, and the center node adjusts the paths of the cluster members according to the observed target position information. After the cluster member completes space-time trajectory planning, the cluster members exchange the planned results, and adjust their respective paths to avoid collision between the unmanned aerial vehicle cluster, and finally exchange the adjusted paths, and finally complete the trajectory planning of the unmanned aerial vehicle. The distributed asynchronous planner of the cluster member independently plans its own flight trajectory, ensuring that the unmanned aerial vehicle cluster can fly cooperatively. After obtaining the trajectories of other unmanned aerial vehicles, the distributed asynchronous planner takes into account factors such as target tracking, unmanned aerial vehicle dynamics, formation shape, and establishes a space-time trajectory optimization model for the unmanned aerial vehicle to optimize the flight trajectory, and broadcasts the optimized result to the entire cluster network. The distributed asynchronous planner iteratively optimizes the trajectory in real time, achieving safe and collision-free flight between the unmanned aerial vehicle cluster. The distributed asynchronous planner does not require all unmanned aerial vehicles to simultaneously plan space-time trajectories, and each unmanned aerial vehicle independently plans the state without considering the planning state of other unmanned aerial vehicles.

[0046] The embodiment is suitable for multi-rotor unmanned aerial vehicle cooperative target tracking cluster system decision strategy, flight data analysis and method performance evaluation in unknown environment. In actual application, a finite state machine can be used as the decision strategy of the unmanned aerial vehicle cluster system, different trigger conditions are used to ensure that the unmanned aerial vehicle performs different actions, and thus the efficient cooperative tracking of the unmanned aerial vehicle cluster system is ensured. The cluster unmanned aerial vehicle built can be used as a test platform for cooperative target tracking methods, and the flight log data generated can be imported into flight analysis software to analyze the position, attitude, speed, acceleration and other information of the cluster unmanned aerial vehicle when tracking the target in unknown environment, and further evaluate the reliability and real-time performance of the planning method.

[0047] In the embodiment, in S1, after constructing the local environment map of the rotor unmanned aerial vehicle, the probability of each occupancy grid in the local environment map is updated in real time, wherein the occupancy probability at time t is wherein n i represents the probability that the grid numbered i in the local environment map is occupied, and t represents time. For example, the state of each grid in the scale map is initialized as unknown, and after initialization, each grid can have two states, occupied and free. For a certain grid n on the map, we use p(n i) represents the probability that it is occupied (called the occupancy probability), denoted by . This represents the probability of being idle. This represents the logical NOT operator, for example, p(n) i ) represents grid n i The probability of being occupied, then n i The probability of being unoccupied, i.e., idle; therefore, For a grid cell that has not yet been observed, its initial occupied and idle probabilities are both set to 0.5. Considering p(n i The probability of a grid being occupied is intuitive and easy to understand, but it is not easy to process. Therefore, its expression is modified. First, the ratio of the two probabilities is taken as p. r , for p r Take the logarithm, denoted as l: l(n i ) = log(p r (n i Therefore, l(n) is used. i To represent the scale n i This represents the probability of a grid being occupied. Assuming that at time t-1, the probability of a grid being occupied is l. t-1 (n i Then, a new observation z is generated for the grid. Under this observation, the probability that the scale is occupied is denoted as p(n). i |z), is a probability between 0 and 1, the specific value of which depends on the sensor properties. After this, the grid occupancy probability is... By overlaying the current observations with historical observations, the state of each grid at this point is finally determined. Figure 1 This refers to a local environmental map rendered using a scaled map format.

[0048] In this embodiment, as described in S2, image data of the environment in which the rotary-wing UAV is located is acquired through an airborne camera, and a target detection method using the image data as input is constructed to obtain the target's position and velocity information. The construction process of the target detection method is as follows: Figure 6 A schematic diagram of the target detection method used is shown in the figure. Figure 2 , Figure 3 ;

[0049] The target detection part process mainly includes the following steps: data collection, using a camera carried by a UAV to collect target image information under different scenes and different lighting conditions; using a labeling tool such as LabelImg to label the collected images, labeling the class and position information of the target; performing rotation, scaling, flipping, etc. on the labeled images to expand the dataset and improve the model generalization ability, and then dividing the dataset into training set, validation set and test set in the ratio of 7:2:1; YOLOv5 model as the basic model, load the pre-training weight. Model training: use the training set to train the model, set appropriate hyperparameters; evaluate the model performance on the validation set, adjust the hyperparameters according to the validation results, model optimization: according to the validation results, adjust and optimize the model, for example: adjust the network structure, loss function; export the trained model into a format that can be used on the UAV platform; model deployment, deploy the exported model to the UAV platform to realize real-time target detection, which ends the entire UAV target detection process;

[0050] In the target image information collection stage, the resolution needs to be at least 1080p, and the frame rate needs to be no less than 30fps to ensure image clarity and dynamic capture capability. The collection scene should cover different terrains and different lighting conditions to enhance the diversity of the dataset;

[0051] In the image labeling stage using LabelImg labeling tool, the accuracy of labeling needs to be ensured. Specifically, a bounding box is drawn for each target instance, and its class label is marked, such as person, vehicle, animal, etc. During the labeling process, the bounding box needs to be tightly fitted to the target, avoiding too large background area or missing part of the target, and the consistency and accuracy of the labeling need to be ensured;

[0052] In the data augmentation stage of the dataset, the labeled images are rotated at multiple angles (0°, 90°, 180°, 270°), scaled at multiple scales (0.8 times to 1.2 times), flipped (horizontal flip, vertical flip), etc. In addition, random cropping and color jittering methods can be used to further expand the dataset. After completing data augmentation, the dataset is randomly divided in the ratio of 7:2:1 to ensure the independence and representativeness of the training, validation and test data;

[0053] The overall structure of YOLOv5 algorithm includes input end, backbone network, neck network and prediction end;

[0054] The input end mainly consists of three parts: mosaic image enhancement, adaptive anchor frame calculation, and adaptive picture scaling. The mosaic enhancement is achieved by randomly scaling, randomly cropping, and randomly arranging four pictures for splicing, which can enrich the background and small target information of the detected object. The adaptive anchor frame calculation automatically calculates the optimal anchor frame parameters of the input image through learning without manual setting, which can improve the accuracy and robustness of target detection. The adaptive picture scaling can maintain the aspect ratio of the picture unchanged and scale it to the appropriate size, which can greatly preserve the original information in the picture and avoid information loss.

[0055] The adaptive anchor frame calculation calculates anchor frame parameters on feature maps of different scales to better adapt to targets of different sizes. The specific steps include:

[0056] S2-1. Feature map extraction. In the backbone network, multiple scale feature maps are extracted at different stages of the CSPNet structure. Suppose three scale feature maps are extracted, denoted as F1, F2, and F3, which correspond to different resolutions of the original image.

[0057] Anchor frame parameter calculation. For each scale feature map, anchor frame parameters are calculated independently. Anchor frame parameters usually include the width and height of the anchor frame.

[0058] For the i-th scale feature map F i , the calculation formula of the process of calculating anchor frame parameters a i is:

[0059] a i =Clustering(F i ,K), where Clustering is a K-means clustering algorithm used to find the optimal anchor frame size, and K is the number of cluster centers, i.e., the number of anchor frames.

[0060] S2-3. Multi-scale anchor frame parameter fusion. A fusion strategy is designed to weight and fuse anchor frame parameters of different scales. The fusion formula is:

[0061] a=w1·a1+w2·a2+w3·a3

[0062] where w1, w2, and w3 are the weights of the three scales, a1, a2, and a3 are the corresponding anchor frame parameters, and a is the fused anchor frame parameter.

[0063] The weight w i can be dynamically adjusted according to the target size and the resolution of the feature map, and the formula is:

[0064]

[0065] where s iis the average target size of the i-th feature map, s target is the average size of the target, and σ is a parameter that controls the smoothness of the weight;

[0066] S2-4, bounding box regression during training, using the fused anchor parameters for bounding box regression training; the bounding box regression loss function formula is:

[0067]

[0068] where N is the number of anchor boxes, b i is the real bounding box, and CIoU is the complete Intersection over Union (IoU) loss function;

[0069] S2-5. Target detection at the prediction end, in the prediction end part, using the fused anchor parameters for target detection;

[0070] Through the above improvement of calculating anchor parameters on feature maps of different scales, the unmanned aerial vehicle target detection method improves the detection ability of the model for targets of different scales, especially for small size targets, which can achieve more accurate positioning, and at the same time enhances the generalization ability of the model in complex scenes, which can better adapt to diversified target distribution, optimizes the calculation process of anchor parameters, reduces the workload of manual adjustment of anchor parameters, and improves the automation degree of the model. Through multi-size anchor parameter fusion, the precision of the detection frame is improved, and the overall performance of the unmanned aerial vehicle target detection is further improved. In practical applications, it is usually inclined to divide the product object marking frame by the square root of the length-width product of the entire image as the basis for judgment, and if its value is less than 3%, it is called a small target.

[0071] The backbone network part is mainly used for extracting image features. First, the Focus structure is introduced to periodically extract pixels from the input image with higher resolution, and the four subgraphs are spliced and reconstructed into low-resolution images with smaller pixel numbers, realizing the downsampling and feature compression of the input feature map. Then, the CSP structure is introduced to divide the input feature Figure 4 into two, and after cross-stage processing, it is spliced and combined as the input of the next layer, which significantly reduces the parameters and calculation amount of the network, and improves the efficiency of feature extraction, thereby speeding up the training and inference speed of the model.

[0072] The neck network part further improves the feature extraction capability. The neck network adopts the structure of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN), which uses the FPN structure to pass down strong semantic features, uses the PAN module to build a feature pyramid structure, and passes up strong positioning features from bottom to top, realizing the feature fusion between different layers.

[0073] The prediction information in the prediction end mainly consists of boundary box position information, category, and confidence. Among them, the Bounding Box loss function adopts CIoU (Complete Intersection over Union) loss function, and the specific calculation formula is as shown below, which is mainly used to measure the difference between the predicted boundary box and the real boundary box. NMS (Non-Maximum Suppression) is used for the selection of the prediction box.

[0074]

[0075] wherein A and B represent different boxes, respectively represent the width and height of the predicted box , w and h represent the width and height of the real box b respectively, and p represents the Euclidean distance between the centers of the two anchor boxes. b c represents the diagonal distance of the smallest closed region containing the real anchor box and the predicted anchor box.

[0076] In the model training stage, the pre-trained YOLOv5 weight is used as the initial weight, the initial learning rate is set to 0.01, and the learning rate decay strategy (such as StepLR, CosineAnnealingLR, etc.) is used according to the training progress. According to the memory capacity, the training batch size is set to 16 or 32. The training period is determined according to the size of the data set and the convergence of the model, usually between 100 and 300 training periods.

[0077] In the model deployment stage, the exported model is integrated into the unmanned aerial vehicle platform, combined with the depth camera, airborne computer, and graphics processing unit (GPU) to form a real-time target detection system. Compatibility, power consumption, heat dissipation, and other issues need to be considered during deployment. After actual deployment, on-site testing is required to ensure that the model can run stably under different environments and conditions to achieve real-time detection. At the same time, on-site data is collected to provide a basis for further optimization of the model.

[0078] After completing all steps of the target detection process, a complete target detection module is realized. This feature achieves a balance between extraction efficiency and detection accuracy and has good generalization ability, and can be applied to actual application scenarios.

[0079] In this embodiment, in S3, the local environment map around the rotor-wing unmanned aerial vehicle is used to construct the Euclidean distance field around the rotor-wing unmanned aerial vehicle, including: constructing the Euclidean distance field in real time on the basis of the obstacle information in the known local environment map. The Euclidean distance field is used to represent the distance from any point in space to the surface of the nearest obstacle and contains the positional relationship. The calculation of the Euclidean distance field is generally divided into the Euclidean distance field of the free area relative to the obstacle and the Euclidean distance field of the obstacle area relative to the free area. The Euclidean distance field of the obstacle area relative to the free area is inverted and then superimposed on the Euclidean distance field of the free area relative to the obstacle to obtain the complete Euclidean distance field. The value of each point in the Euclidean distance field is obtained through Euclidean distance transform (EDT), and the specific representation of two-dimensional and three-dimensional EDT is as follows:

[0080]

[0081] wherein x, y, and z respectively represent the coordinates corresponding to the three-dimensional space position vector on the x, y, and z axes, and x o ,y o ,z o represent the coordinates corresponding to the three-dimensional space position vector of the obstacle on the x, y, and z axes.

[0082] Due to the large calculation complexity of the three-dimensional EDT, a three-dimensional two-dimensional EDT is usually used for calculation, and the following expressions are solved respectively:

[0083]

[0084] wherein x, y, and z respectively represent the coordinates corresponding to the three-dimensional space position vector on the x, y, and z axes, and x o ,y o ,z o represent the coordinates corresponding to the three-dimensional space position vector of the obstacle on the x, y, and z axes.

[0085] The last three results are superimposed and divided by 2 to obtain the Euclidean distance field of the three-dimensional local map. The Euclidean distance field centered on the unmanned aerial vehicle is updated in real time whenever the local environment map changes; by obtaining the value corresponding to any point in the local environment in the Euclidean distance field, it can be judged that the distance from the point to the nearest obstacle, which provides a constraint condition for subsequent space-time trajectory planning of the unmanned aerial vehicle; the effect diagram of the Euclidean distance field is shown in Figure 1 .

[0086] In the embodiment, in S4, the cluster allocation scheme in the construction set allocates a corresponding target to each rotor unmanned aerial vehicle cluster individual, including: after obtaining the position information of all observed targets, considering the value of the observed target and the distance factor between the observed target and the cluster unmanned aerial vehicle, target allocation is performed on each cluster unmanned aerial vehicle; the target allocation needs to meet the requirement that each target needs to be tracked by at least one unmanned aerial vehicle and each unmanned aerial vehicle can and only can track one target; the target allocation model is optimized on the premise of meeting these constraints; the target allocation model is as follows:

[0087]

[0088] and

[0089]

[0090] wherein, the upper right corner T represents the transposition operation of a vector / matrix, n swarm represents the number of rotor unmanned aerial vehicles, n target represents the number of observed targets; x represents a 0-1 integer optimization vector and represents whether the i t th rotor unmanned aerial vehicle tracks the j t th target, represents tracking, otherwise not tracking; and represents that each rotor unmanned aerial vehicle has and only can assign one target and each target needs to assign at least one rotor unmanned aerial vehicle; f val represents a value vector, f dist represents a value vector, i t is an integer and 0≤i t <n swarm , j t is an integer and 0≤j t <n target ; and represents the value of the j t th target, represents the distance between the i t th rotor unmanned aerial vehicle and the j t th target and is easily obtained v max represents the maximum value in the observed target, d max represents the farthest distance between the rotor unmanned aerial vehicle and the target, represents a real number, represents a non-negative integer.

[0091] The solution of the target assignment is used as the tracking target of each cluster UAV.

[0092] In the embodiment, in S4, the target motion prediction model of the rotor UAV is constructed based on the motion information of a single specific target, including: predicting the motion in a future fixed time period by using the motion state of the target in a past fixed time period; predicting the future target motion by curve fitting using the historical motion information of the target as known parameters; establishing a motion prediction optimization model based on a Bezier curve, the calculation result of the motion fitting optimization model being used as the target motion prediction model, and the motion prediction optimization model based on the Bezier curve being:

[0093]

[0094] wherein, J pred represents a motion prediction optimization function, M pred represents the number of saved historical motion information, n bezier represents the order of the Bezier curve, i p is an integer and 0≤i≤M pred , represents the time corresponding to the target position in the i p th historical motion information, represents the current time, B(t) represents a Bezier curve, represents a target prediction trajectory represented by the Bezier curve, represents the second derivative of the target prediction trajectory represented by the Bezier curve with respect to time, is an n bezier -dimensional Bernstein basis, is a control point of the Bezier curve, respectively represent the i-2, i-1, i Bezier curve control points in the dimension μ and μ∈{x, y, z}, s bezier,t represents a time scaling coefficient, v max,pred represents the maximum speed of the target prediction, a max,pred represents the maximum acceleration of the target prediction, represents the confidence weight of the historical target observation information at different times, f pred,t represents a confidence weight function of the historical target observation information related to time, tanh represents a hyperbolic tangent function, k pred,t represents an adjustable parameter related to the confidence weight of the historical target observation information at different times,

[0095]

[0096] the historical target observation information at different times, represents The historical target observation position information corresponding to the time; In order to prevent overfitting, a second-order regulator structure is added, wherein ω p represents the weight related to the second-order regulator structure.

[0097] The fitting weight of different observation information is adjusted by the confidence weight of historical observation target information , and the closer the time of the observation information to the current time, the greater the corresponding weight; The increase of the second-order regulator in the optimization target is to prevent overfitting; The inequality constraint in the optimization problem is obtained by considering that the target needs to be subject to its own dynamics constraint and the convex hull characteristics and speed characteristics of the Bezier curve; The solution result of the target motion prediction is the Bezier curve control point with the highest fitting degree with the historical motion trajectory of the target, and the estimation of the future motion of the target within a fixed time can be obtained by external interpolation.

[0098] In this embodiment, in S5, the space-time trajectory planning part structure can refer to Figure 7 .

[0099] In this embodiment, in S5, the method for planning a path that does not collide with obstacles includes: the method considers the dynamics characteristics of the unmanned aerial vehicle itself and obtains a front-end path by extending the Motion Primitive based on the discrete control input space, and the principle is similar to the hybrid A* algorithm, which searches for a safe and dynamically feasible path with the minimum control amount and time in the voxel network map; Compared with using a straight line, the Motion Primitive uses a curve based on the dynamics of the rotor unmanned aerial vehicle as an edge of the graph structure, and usually uses a node to record a primitive, a grid at the end of the primitive, a geometric cost loss value g c , and a total cost loss value f c . By continuously expanding in the voxel grid map, the voxel grid map is iteratively expanded, and then by a pruning method, only the primitive with the minimum total cost loss value in the same voxel is retained, and then the safety and dynamic feasibility of the remaining primitives are judged by a feasibility check. The cycle continues until any primitive reaches the target or meets the analytical expansion condition without considering the obstacle avoidance constraint. The schematic diagram of the front-end path search part can refer to Figure 5 .

[0100] The motion primitive in this embodiment is a three-dimensional polynomial curve that only changes with respect to time, which is as follows:

[0101] Wherein, t represents time, μ∈{x,y,z}, p search (t) represents a spatial position curve related to time t, p search,μ (t) represents a spatial position curve in the μ dimension, nsearch an order of a polynomial front-end search path related to time, an i-th coefficient of a polynomial front-end search path, s The quadcopter can be considered as a linear time-invariant system, whose state vector is represented as follows:

[0102]

[0103] wherein, represent a front-end search polynomial path and its 1st, (n search -1)th derivative with respect to time, respectively, denotes a set consisting of all state vectors, denotes a 3n search dimensional real vector; a control vector is represented as follows:

[0104]

[0105] wherein, denotes an n search th derivative of a front-end search polynomial path with respect to time, u max denotes a maximum control input amplitude, denotes a 3-dimensional real vector, and a state space equation is represented as follows:

[0106]

[0107]

[0108] wherein, x denotes a state vector of a dynamic system, x denotes a derivative of the state vector with respect to time t, u denotes a control input in the dynamic system, A denotes a state matrix, B denotes a control input matrix, and I3 denotes a 3-dimensional unit matrix. An analytical solution of the state space equation is represented as follows:

[0109]

[0110] wherein, x(0) denotes an initial state of the quadcopter, e denotes a natural constant, t denotes time, e At denotes a matrix exponential function related to time t; a control input is uniformly discretized to obtain In the present embodiment, n = 2, and the state transition equation corresponds to a second-order integrator; since the input dimension is 3, wherein, μ ∈ {x, y, z}, u max denotes a maximum control input amplitude, and r denotes a degree of discretization of a control input in a certain dimension, i.e., will be discretized to [-u max , u maxThe control input uniformly distributed on the interval is discretized into (2r+1) parts. Considering that the dimension of the control input is 3, the total number of discrete control inputs is (2r+1) 3 .

[0111] To obtain a front path with minimum cost, the cost loss of the path is defined as follows:

[0112]

[0113] Considering that the control input u(t) is replaced by the uniformly discretized control input u d , the cost loss of each primitive is

[0114] e c = (||u d || 2 + ρ search )τ, where τ represents the time interval, T search is the path duration, and ρ search is an adjustable parameter related to the path duration T search .

[0115] Similar to the A* algorithm, the embodiment uses g c to represent the sum of the cumulative costs corresponding to the optimal path from the starting state to the current state; assuming that the optimal path is composed of M search primitives, then Since the A* algorithm uses the heuristic function cost loss to accelerate path searching, the embodiment also uses a self-defined heuristic function that uses the Pontryagin minimum principle and can minimize the cost loss from the current state to the target state, and the form is as follows:

[0116]

[0117] where μ∈{x,y,z}, represents the optimal polynomial path with time t as the variable that satisfies the Pontryagin principle in the dimension μ, p μc represents the current position, v μc represents the current speed, p μg represents the position corresponding to the target point, and v μg represents the speed corresponding to the target point; in order to facilitate calculation, α μ , β μ are constants related to the third-order polynomial trajectory coefficients of time t in the dimension μ;

[0118] To solve the optimal time , α μ , β μSubstitute J * (T search Then find the equation that satisfies the equation. And the root T that makes the path feasible h , J(T h The heuristic cost loss value h is defined as follows: c The final total cost and loss f c =g c +h c By defining the cost loss, a safe and dynamically feasible front-end path composed of multiple primitives can be obtained.

[0119] Based on the planned path that avoids collisions with obstacles and other swarm drones, key waypoints on the flight trajectory are selected where the distance between adjacent points is greater than the lower bound of the minimum distance and less than the upper bound of the maximum distance, and the line connecting two adjacent points does not overlap with an obstacle. Given a known Euclidean distance field centered on the drone's position, the drone's time-track planning is divided into flight trajectory optimization and yaw angle trajectory optimization. The flight trajectory and the yaw angle trajectory are represented as follows:

[0120]

[0121] Among them, Ξ MINCO This represents a polynomial trajectory class that satisfies the above definition, where MINCO represents the minimum control quantity trajectory.

[0122] Indicates a dimension of m opt real vectors, This indicates that the number of rows is m opt The number of columns is M opt A real matrix of -1 The dimension is M opt A positive real vector, q(t) represents the position at time t, T opt,∑ The total time of the flight trajectory is represented by m. opt M represents the dimension of the trajectory. opt This indicates the number of segments in the polynomial flight trajectory. p represents the coefficients of the polynomial locus. opt =(p opt,1 ,,p opt,M-1 T represents the midpoint between the flight path and the yaw angle trajectory. opt =(T opt,1 ,,T opt,M ) T This represents the time vector composed of the times of each segment of the polynomial trajectory in the flight trajectory and yaw angle trajectory, (p opt ,T opt) denotes a parametric mapping with linear complexity, the MINCO trajectory can be obtained from the time parameter and the space parameter opt and the time vector T opt parameterization representation;

[0123] For the spatial-temporal parameterization of continuous flight trajectory, the MINCO trajectory can be obtained from the time parameter and the space parameter with linear time complexity, and the guarantee of the uniqueness of the solution in the multi-stage unconstrained minimum control quantity trajectory of the integral chain system in the flat space makes the sensitivity and smoothness of the minimum functional with respect to the spatial-temporal parameter. Due to the good properties of the optimality condition, the solution of the unconstrained optimal control trajectory of the integral chain system itself can be used as a forward generation process to determine a unique smooth trajectory from the spatial-temporal parameter; the MINCO trajectory refers to a trajectory set parameterized by using the optimal condition as the basis; the advantage of the MINCO trajectory is that a long but single motion mode trajectory can be represented by only a few parameters, and the parameter scale is only a constant multiple of the number of trajectory segments; the MINCO trajectory is different from the traditional geometric spline curve represented by B-spline and Bézier curve. Its sparse parameters can directly control the spatial deformation and temporal deformation of a trajectory, and the two degrees of freedom are equally important for the dynamic characteristics of the multi-rotor unmanned aerial vehicle. In addition, the MINCO trajectory can be spatial-temporally deformed under a user-defined cost function or constraint function and solved by gradient with linear complexity.

[0124] In the spatial-temporal trajectory planning, the time between the waypoint position and the waypoint is selected as the two types of parameters of the unconstrained trajectory planning sub-problem, and the form of the unconstrained sub-problem is as follows:

[0125]

[0126] Where, t opt,0 represents the starting time of the flight trajectory, represents the ending time of the flight trajectory, and z(t) represents the flat output trajectory, and z (s) (t) represents the s-th derivative of the flat output trajectory z(t) with respect to time t, The time interval is divided into M opt +1 fixed time points into M opt stages respectively represent the initial and terminal boundary conditions of the whole trajectory, and the intermediate conditions must be satisfied The condition specifies that the trajectory up to the s-th derivative The specific value, Indicates a dimension of m opt The control input, This represents the adjustable positive definite diagonal weight matrix associated with the control input v(t), i.e.

[0127] In this embodiment, the starting point constraint for the flight trajectory optimization part is second-order (s = 3, m opt =3), meaning the initial and final positions, velocities, and accelerations must be known, and the intermediate point constraint is of order zero. The intermediate point position is known; the starting point constraint for the yaw angle optimization part is first-order (s=2,m). opt =1), meaning the initial and final positions and velocities must be known, and the intermediate point constraint is of order zero ( ). m opt =1), the position of the midpoint is known; therefore, the flight path and yaw angle path are represented as follows:

[0128]

[0129] Where t represents time and β opt (x)=(1,x,,x 5 ) T The natural basis for the polynomial flight trajectory, β ψ,opt (x)=(1,x,,x 3 ) T Let represent the natural basis associated with the polynomial yaw angle trajectory; therefore,

[0130] The representation of is as follows:

[0131]

[0132] Indicates the i-th opt The coefficient matrix corresponding to the polynomial trajectory of the yaw angle segment. Indicates the i-th opt The coefficient matrix corresponding to the segmented polynomial trajectory. For ease of calculation, relative time is used in the polynomial trajectory, i.e., t is defined as... opt,0 =0. A trajectory can be represented by the coefficient matrix of multiple polynomial trajectories. and time vector The only decision is:

[0133]

[0134] in, Indicates the i-th opt The duration of a segment of the polynomial trajectory. The i-th segment...opt The global time and total trajectory time corresponding to each time point can be respectively... and T opt,∑ =||T opt ||1. An M opt Segmented polynomial spline locus Defined as:

[0135]

[0136] Considering that the trajectory must satisfy the boundary conditions of the starting point and intermediate track points, and that adjacent trajectory segments must satisfy the continuity condition, it is represented as follows:

[0137]

[0138] M minco c minco =b minco

[0139]

[0140] in, Let D1 and D2 be the real number matrices corresponding to the values ​​of the boundary conditions and continuity conditions equations, respectively, and D1 be the known boundary condition matrix corresponding to the first intermediate waypoint. Indicates the Mth opt -1 known boundary condition matrix corresponding to intermediate waypoints. F0 and These represent the initial and final boundary condition coefficient matrices, respectively. Let represent a real matrix containing boundary and continuity conditions, with 2M rows. opt s and the number of columns is m opt ; The coefficients of the MINCO polynomial locus are represented; 0 represents a matrix of all zeros with 2s rows and 2s columns, and similarly, Indicates the number of rows. The number of columns is m opt The zero matrix represents the value of the continuity condition equation between two adjacent trajectories at the first intermediate waypoint; Indicates the number of rows. The number of columns is m opt A matrix consisting entirely of zeros, which represents the Mth zero matrix. opt -1 The value of the continuity condition equation between two adjacent tracks at intermediate track points; Indicates the i-th opt Boundary condition coefficient matrix at each track point Indicates the i-th opt The coefficient matrix of the continuity condition between two adjacent trajectories at each waypoint. Representing the i-th opt and iopt +1 coefficients of the MINCO polynomial trajectory. Correspondingly, E1 denotes the coefficient matrix of the boundary condition at the 1st waypoint, and F1 denotes the coefficient matrix of the continuity condition of the adjacent two trajectories at the 1st waypoint; E2 denotes the coefficient matrix of the boundary condition at the 2nd waypoint, and F2 denotes the coefficient matrix of the continuity condition of the adjacent two trajectories at the 2nd waypoint; denotes the coefficient matrix of the continuity condition of the adjacent two trajectories at the M opt -1 waypoint.

[0141] Since the uniqueness of the optimality condition guarantees that M minco is always a non-singular matrix under any positive time allocation vector T opt 0 condition. Therefore, the optimal solution c minco of the linear equation system can be obtained by directly solving the equation system, and considering that M minco is a banded matrix and non-singular, the optimal solution c minco can be obtained by banded PLU decomposition with linear time complexity. Therefore, the trajectory can be solved with linear complexity without other constraints.

[0142] Since the optimality condition guarantees the existence and uniqueness of the solution of the unconstrained trajectory planning subproblem, the sensitivity of the optimal cost functional of the unconstrained trajectory planning subproblem with respect to the two types of parameters, i.e., the intermediate waypoint positions and the flight duration between the waypoints, is well defined. Moreover, the analytical gradients of any demand function or constraint function with respect to their solutions can also be obtained. Under this guarantee of completeness, an iterative method can be used to deform the motion trajectory in time and space by changing the two types of parameters, i.e., the intermediate waypoint positions and the flight duration between the waypoints, and each iteration also has linear computational complexity.

[0143] Since the cost functional or the constraint functional of the polynomial spline curve trajectory is a function of its coefficient matrix and the time vector, the self-defined loss cost function or the constraint cost function can be expressed as where, must satisfy the second-order continuous differentiability condition, and the gradient must be obtainable. The function with respect to the trajectory in the minimum control amount trajectory class Ξ MINCO can be calculated in the following manner:

[0144]

[0145] To use the gradient to iteratively optimize the space-time deformation of the MINCO trajectory, it is necessary to obtain and its gradient with respect to the time and space parameters, i.e., The linear equation system derived from the optimality condition is written in the parameter-dependent form:

[0146] M minco (T opt )c minco (p opt ,T opt )=b minco (p opt )

[0147] set up express The j-th vector opt One element, regarding custom functions. For p opt To obtain the gradient, first consider the gradient of both sides of the linear equation system with respect to... Differentiate, that is:

[0148]

[0149] Therefore, a user-defined function can be obtained. right The gradient, i.e.:

[0150]

[0151] Where Tr(·) represents the trace of the matrix. From b minco (p opt The structure shows that Only in the (2i) opt -1)s+1 row j opt The column has one non-zero element, 1. Therefore, The expression is as follows:

[0152]

[0153] in, Represents the identity matrix The jth opt Each column vector corresponds to... Represents the identity matrix Earth (2i) opt -1)s+j opt Column vector. Considering M minco Matrices possess the property of being PLU decomposable; this property is utilized here to avoid directly inverting the matrix. Let... And it satisfies the following equation:

[0154]

[0155] Let M minco =P mat L mat U mat , where L matU represents a lower triangular matrix with all diagonal elements equal to 1. mat Let P represent an upper triangular matrix. mat Describe a row permutation matrix that satisfies therefore The LUP decomposition is represented as follows:

[0156]

[0157] in, Let G represent the Hadamard product. Matrix G can be solved using LUP decomposition in linear time and space complexity. Matrix G can be partitioned as follows:

[0158]

[0159] The sizes of the submatrices are respectively as well as therefore, Regarding p opt The gradient can be represented as follows:

[0160]

[0161] Where e1 represents the identity matrix I with dimension 2s. 2s The first column vector. Further calculations... Regarding T opt The gradient is obtained by first applying the gradient to both sides of the linear equation. Find the differential, that is:

[0162]

[0163] therefore, Regarding T opt The gradient can be expressed as follows:

[0164]

[0165] Considering the banded matrix M minco It satisfies the following properties:

[0166]

[0167] At this point, you can obtain Regarding T opt The gradient of is represented as follows:

[0168]

[0169] in, The gradient can be obtained analytically; therefore, 1≤i can be calculated. opt ≤M optall of further obtainable Thus, the user-defined function is obtained with respect to p opt ,T opt all gradients. Since all calculations conform to linear time complexity, all gradient calculations satisfy linear time complexity. Subsequently, user-defined functions can be designed according to various task requirements, and MINCO trajectory is used to realize linear time complexity of space-time deformation, and MINCO trajectory guarantees the local smoothness of the optimization trajectory.

[0170] In practical applications, each unmanned aerial vehicle has a distributed asynchronous planner, which can independently plan its own flight trajectory to ensure that the unmanned aerial vehicle cluster can cooperatively track the target. After completing the space-time trajectory planning, the unmanned aerial vehicle broadcasts the result through the cluster communication network, and the planning results are exchanged between the unmanned aerial vehicles. Each unmanned aerial vehicle performs collision detection on its own flight trajectory and the flight trajectories of other unmanned aerial vehicles, that is, whether a collision will occur in the future period of time is detected. If there is a collision risk, the flight trajectory is adjusted.

[0171] Specifically in S5, the safety constraint based on the Euclidean distance field, the self-dynamics constraint, the target tracking constraint, the formation constraint, and the trajectory smoothness constraint are converted into optimization objectives, and the constrained optimization problem is converted into an unconstrained optimization problem, and then a gradient optimization method based on unconstrained optimization is used for solving. The space-time trajectory planning can be divided into a space-time trajectory optimization model of a flight trajectory and a trajectory optimization model of a yaw angle, wherein the flight trajectory optimization model is:

[0172]

[0173] wherein J spatial represents an optimization function of the flight trajectory, J control represents a target function for minimizing the control amount and the total time of the flight trajectory, λ control represents a weight of the target function for minimizing the control amount and the total time of the flight trajectory, p opt (t) represents a position of the flight trajectory at time t, represents a third-order derivative of the position with respect to time at t, represents a penalty function of a single rotor unmanned aerial vehicle navigation constraint, represents a penalty function of a rotor unmanned aerial vehicle cluster navigation constraint, q opt represents an intermediate waypoint of the flight trajectory, T opt,∑ represents the total time of the flight trajectory, ρ opt represents an adjustable parameter related to the total time of the flight trajectory.

[0174] wherein the yaw angle optimization model is:

[0175]

[0176] where J yaw represents an optimization function of the track angle trajectory, J yaw,control represents a target function of the control amount of the track angle to be minimized, λ yaw,control represents a weight corresponding to the target function of the track angle control amount to be minimized, J yaw,vel represents a penalty function corresponding to the yaw angle velocity feasibility constraint of the rotorcraft UAV, λ yaw,vel represents a weight corresponding to the yaw angle velocity feasibility penalty function of the rotorcraft UAV, J yaw,acc represents a penalty function corresponding to the yaw angle acceleration feasibility constraint of the rotorcraft UAV, λ yaw,acc represents a weight corresponding to the yaw angle acceleration feasibility penalty function of the rotorcraft UAV, J yaw,track represents a penalty function corresponding to the yaw angle constraint in the target tracking process of the rotorcraft UAV, λ yaw,track represents a weight corresponding to the yaw angle penalty function in the target tracking process of the rotorcraft UAV, ψ represents an intermediate state point of the yaw angle trajectory, M opt represents the number of polynomial flight trajectory segments, represents the sampling number of each yaw angle trajectory segment, represents the duration of the i opt th flight trajectory segment, i opt is an integer and 0≤i opt ≤M opt , j opt is an integer and represents the j opt th integral coefficient and ψ(t) represents a polynomial yaw angle trajectory related to time t, represents a polynomial yaw angle velocity trajectory related to time t, represents a polynomial yaw angle acceleration trajectory related to time t, ψ * (t) represents an expected polynomial yaw angle trajectory related to time t, represents the square of the maximum yaw angle velocity amplitude, represents the square of the maximum yaw angle acceleration amplitude.

[0177] wherein the penalty function of the single-rotorcraft UAV navigation constraint is:

[0178]

[0179] where J obs represents a penalty function corresponding to the obstacle avoidance related constraint of the rotorcraft UAV, λ obsJ represents the weight corresponding to the penalty function related to obstacle avoidance in rotary-wing UAVs. vel Let λ represent the penalty function corresponding to the speed feasibility constraint of the rotary-wing UAV. vel J represents the weight corresponding to the speed feasibility penalty function for rotary-wing UAVs. acc Let λ represent the penalty function corresponding to the acceleration feasibility constraint of the rotary-wing UAV. acc J represents the weight corresponding to the acceleration feasibility penalty function for rotary-wing UAVs. track Let λ represent the penalty function corresponding to the target tracking constraints of the rotary-wing UAV. track J represents the weight corresponding to the target tracking penalty function of the rotary-wing UAV. var Let λ represent the penalty function corresponding to the smoothness constraint of the flight trajectory of the rotary-wing UAV. var M represents the weight corresponding to the smoothness penalty function of the flight trajectory described by the rotary-wing UAV. opt Indicates the number of flight trajectory segments, i opt The integer is 0 ≤ i opt ≤M opt , This indicates the number of samples for each segment of the flight trajectory. Indicates the i-th opt The duration of the flight path Denotes the integral coefficient and d obs The spatial position p at time t is obtained from the Euclidean distance field. opt (t) Distance to the nearest obstacle, d th v represents the shortest possible distance from a rotary-wing drone to an obstacle. max a represents the maximum permissible flight speed amplitude of a rotary-wing unmanned aerial vehicle. max This indicates the maximum permissible flight acceleration amplitude for a rotary-wing drone. This represents the total number of predicted position points in the target motion prediction. The flight trajectory indicates that... Time and location The vertical component of the distance from the predicted target point. The flight trajectory is represented by Time and location The horizontal component of the distance from the predicted target point. The flight trajectory is represented by Time and location The expected vertical distance between the target predicted point and the target point, where ε represents a small constant, d l d represents the expected distance from the lower bound. u This indicates the expected distance from the upper bound. a penalty function representing horizontal distance, an integral coefficient of a penalty function representing rotorcraft target tracking,

[0180] wherein the penalty function of the navigation constraint of the rotorcraft swarm is:

[0181]

[0182] L graph = D graph - A graph

[0183]

[0184] wherein J rep represents a penalty function corresponding to the rotorcraft swarm internal collision avoidance related constraint, J form represents a penalty function corresponding to the rotorcraft swarm formation related constraint, λ rep represents a weight corresponding to the rotorcraft swarm internal collision avoidance related penalty function, λ form represents a weight corresponding to the rotorcraft swarm formation related penalty function, N swarm represents the number of individuals in the rotorcraft swarm, r s represents the minimum distance acceptable between swarm individuals, τ represents the time interval between the current time and the starting time of the flight trajectory, A graph represents an adjacency matrix, A graph,des represents an adjacency matrix of the desired formation, D graph represents a diagonal matrix, I represents an identity matrix, L graph represents a Laplacian matrix, M opt represents the number of flight trajectory segments, i opt is an integer and 0≤i opt ≤M opt , represents the sampling number of each flight trajectory segment, represents the corresponding duration of each flight trajectory segment, represents an integral coefficient and represents the i opt th polynomial trajectory at time t related to time t, represents the spatial position of the k swarm th rotorcraft at time t swarm , T l represents the duration of the l th trajectory segment planned by other rotorcraft in the swarm, This represents the Laplace matrix of the expected formation after normalization. Indicates the i-th node within the cluster s The drone and the jth s The distance between the drones Represents a diagonal matrix D graph The i-th s Line j s The value corresponding to the column, express The Frobenius norm of a matrix, i.e. Where Tr(·) represents the trace of the matrix.

[0185] After defining the optimization objective and constrained optimization objective according to the requirements of the collaborative target tracking task scenario, it is necessary to perform gradient-based iterative optimization of the optimization objective using the properties of the MINCO trajectory. Most existing optimization methods are for functions defined in Euclidean space. However, the MINCO trajectory uses the time vector T... opt Restricted to simple manifolds To avoid frequent contraction operations in the manifold, it is necessary to adjust the time vector T. opt The simple manifolds of the domain under different regularization terms are given their explicit differential homeomorphisms in Euclidean space. Furthermore, the unconstrained surrogate variables in Euclidean space are directly optimized, which is more conducive to improving the efficiency of the spatiotemporal deformation of trajectories.

[0186] Since the spatiotemporal trajectory planning problem based on polynomial spline trajectories can be uniformly represented as follows:

[0187] J(p opt ,T opt ) = J popt (p opt ,T opt )+ρ opt (||T opt ||1)

[0188] in, Indicates costs and losses other than time. Furthermore, it can be calculated in linear time complexity using the MINCO trajectory characteristics. ||T opt ||1 represents T opt The 1-norm, ρ opt This represents an adjustable parameter related to the total time of the flight trajectory. Due to M... opt The segment trajectory is strictly fixed in total time. The domain of the time cost function J is a (M opt -1)- Simplex in The relative interior of the above, i.e., the domain of the entire trajectory optimization solution, is further restricted to the area within the range of the above. superior.

[0189] Therefore, differential homeomorphism is used to avoid this M. opt -1-dimensional simple manifold constraint. Considering the domain of the time vector in the spatiotemporal trajectory planning problem as follows:

[0190]

[0191] in, That is, in When relative to the interior, T represents opt Every element of the vector is greater than 0; for any T opt J(p) opt ,T opt For nontrivial p opt In other words, it is bounded.

[0192] The following gives the case of (1≤i) opt <M opt Satisfying C ∞ The mapping of an infinitely continuous differential homeomorphic transformation is specifically represented as follows:

[0193]

[0194] Through a set of proxy variables It can be in the domain of τ The optimized cost function J allows the simple manifold constraints on the time vector to be implicitly satisfied.

[0195] At this point, the proxy variable τ and the position of the midpoint p of the trajectory are... opt All defined in European-style space Within this framework, gradient-based optimization methods can be used to iteratively optimize a custom trajectory optimization function, and the resulting trajectory is the optimal solution to the trajectory optimization problem.

[0196] The diagram illustrating the effect of a single rotary-wing UAV in tracking a single target is shown below. Figure 8 .

[0197] The diagram illustrating the effect of a rotorcraft drone swarm achieving cooperative target tracking of a single target is shown below. Figure 9 .

[0198] The diagram illustrating the effect of a rotorcraft drone swarm in achieving collaborative target tracking of multiple targets is shown below. Figure 10 , Figure 11 .

[0199] In this embodiment, in S6, the transmission of the optimized flight trajectory results of cluster members and the detected target position and speed information within the cluster via a broadcast communication mechanism includes: For messages such as flight trajectory optimization results and target position information with small data volumes, the communication protocol of the UAV cluster broadcast communication is based on the User Datagram Protocol (UDP); this is a connectionless, lightweight transport layer protocol, mainly used in scenarios with high requirements for transmission speed and real-time performance but low requirements for reliability; the characteristics of the UDP protocol are consistent with the characteristics of target tracking scenarios, so it is chosen as the main communication protocol; while for messages such as depth images with large data volumes and high reliability requirements, the communication protocol of the UAV cluster broadcast communication is based on the Transmission Control Protocol (TCP), which is a connection-based communication protocol that transmits data through data streams rather than entire data packets, thus enabling reliable transmission of messages with large data volumes, but sacrificing message transmission speed.

[0200] Designing communication mechanisms with different communication protocols for different data types is beneficial for meeting the needs of broadcast communication of various messages between clusters.

[0201] The decision-making algorithm in the autonomous collaborative target tracking rotorcraft UAV swarm system employs a finite state machine (FSM) algorithm. It sets trigger conditions between different states and switches between states through state changes, thereby controlling the rotorcraft UAVs to perform different actions. The UAV states include: waiting, continuous start, generating new trajectory, replanning, executing trajectory, and emergency stop. Different states correspond to different task modules, and different trigger mechanisms are used to switch states and execute different actions. The logic for state switching is as follows: Figure 12 Figure 13 As shown, the cluster of UAVs first initializes and receives relevant sensor data; based on the obtained image information, it uses target detection to obtain target location information and uses the communication mechanism between the clusters to send the obtained target location information. When the UAV receives the assigned target information from the task allocation center node, it enters the program control state and waits for the start trigger signal.

[0202] After receiving the start trigger signal, the distributed asynchronous planner receives the flight trajectory planned by other unmanned aerial vehicles. After receiving all necessary signals, the system will be continuously started and begin to plan the global trajectory to determine the optimal trajectory from the current position to the target position. If the global trajectory planning is successful, the system will enter the execution state according to the planned trajectory; if the planning fails, the system will return to the initial state to try again. During execution, the system continuously checks whether the target position has been reached. If not, the system continues to execute the planned trajectory; if the target position has been reached, the system determines whether re-planning is needed. If re-planning is not needed, the system directly enters the local planning state to optimize the current trajectory segment; if re-planning is needed, the system adjusts the trajectory to adapt to the new situation and then enters the local planning state. During this period, the system always performs safety detection on the planned trajectory. If any unsafe factors are found, the system will stop immediately to ensure safety. After successful emergency stop, the system generates a new trajectory plan and tries global trajectory planning again to ensure the safety and effectiveness of the trajectory. Through this cycle of repeated processes, the system can flexibly and safely complete various tasks. If problems are encountered at any stage, the system will take appropriate measures, such as returning to the initial state or the emergency stop state, to ensure the smooth progress of the overall process.

[0203] The embodiment relates to the fields of environment perception, mapping, target detection and target motion prediction, task planning, real-time space-time trajectory planning of a multi-rotor unmanned aerial vehicle cluster. In actual application, the multi-rotor unmanned aerial vehicle cluster can perceive information such as obstacles and targets in the environment when performing a target tracking task in an environment without prior knowledge, and can perform task allocation, decision-making and planning on this basis to achieve highly autonomous cooperative target tracking. The application comprises: when the system is in an unknown environment, a real-time updated sliding scale map is updated by unmanned aerial vehicle positioning and depth map construction, and a local environment around the unmanned aerial vehicle is constructed in real time; target detection is realized by picture information obtained by the unmanned aerial vehicle carrying a camera to obtain target position information, and further target predicted motion trajectories are obtained by target motion prediction; communication is performed between cluster members to exchange target position information observed by each cluster member; task allocation of the multi-rotor unmanned aerial vehicle cluster is realized on the basis of the observed position information of all known targets; space-time trajectory planning of the multi-rotor unmanned aerial vehicle cluster is performed in the constructed environment and on the premise that the target position information is known, front-end feasible path search and rear-end space-time trajectory optimization are performed; communication is performed between cluster members to exchange the planned paths of each cluster member, and finally the flight trajectories of each cluster member are obtained after planning for possible collisions.

Claims

1. A cooperative multi-target tracking method for rotorcraft unmanned aerial vehicle (UAV) swarms, characterized in that, The steps are as follows: S1. Using an airborne depth camera, acquire point cloud data of the environment in which the rotorcraft drone is located, and construct a local environment map around the rotorcraft drone. The local environment map is a scale map representing the environment around the rotorcraft drone. The local environment map adopts the form of a probabilistic grid map, which divides the environment into small grids. Each grid has two states: occupied and idle. The occupancy probability is used to represent the possibility that an obstacle is occupied. S2. By using an airborne camera, image data of the environment in which the rotary-wing UAV is located is acquired, and a target detection method with image data as input is constructed to obtain the position and velocity information of the target. The target detection method is a method for extracting information related to the target from the perceived information. S3. Based on the local environment map around the rotorcraft drone, construct the Euclidean distance field around the rotorcraft drone. The Euclidean distance field is a method that records the Euclidean distance from each point in space to the nearest object surface in scalar form and distinguishes the inside and outside relationships with signs. S4. Based on the target motion information obtained from target detection and internal cluster communication, a centralized cluster allocation scheme is constructed to assign corresponding targets to each individual rotorcraft UAV in the cluster. For the motion information of a single specific target, a target motion prediction model for rotorcraft UAVs is constructed. The target prediction method is a method that predicts the motion information of a dynamic target in the future period based on the historical motion information of the dynamic target. S5. Perform spatiotemporal trajectory planning for the flight path. Spatiotemporal trajectory planning means planning a path that will not collide with obstacles based on the constructed local environment map. Optimize the flight path based on the planned path and the constructed Euclidean distance field, so that the optimized flight path spatially avoids collisions with obstacles and other individual rotorcraft in the swarm, and temporally minimizes the total time of the planned flight path within the dynamic constraints of the rotorcraft. Then, send the trajectory to the rotorcraft's flight controller to complete the drone's trajectory tracking. S6. The optimized flight trajectories of cluster members, as well as the detected target positions and speeds, are transmitted within the cluster via a broadcast communication mechanism.

2. The cooperative multi-target tracking method for rotorcraft UAV swarms according to claim 1, characterized in that: In step S1, after constructing the local environment map of the rotary-wing UAV, the probability of each occupied grid in the local environment map is updated in real time, where the occupation probability at time t is... Where, n i t represents the probability that grid number i in the local environment map is occupied, and t represents time.

3. The cooperative multi-target tracking method for rotorcraft UAV swarms according to claim 1, characterized in that: In step S2, the target detection method is the YOLOv5 algorithm. The overall structure of the YOLOv5 algorithm includes an input end, a backbone network end, a neck network end, and a prediction end. The input end consists of three parts: mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling. Mosaic enhancement stitches together four images that are randomly scaled, cropped, and arranged to enrich the background and small target information of detected objects; adaptive anchor boxes automatically calculate the optimal anchor box parameters of the input image through learning, without the need for manual setting. Adaptive image scaling maintains the aspect ratio of the image and scales it to an appropriate size, preserving the original information in the image and avoiding information loss; The adaptive anchor box calculation involves calculating anchor box parameters on feature maps at different scales. The specific steps include: S2-1. Feature map extraction: In the backbone network, feature maps of multiple scales are extracted at different stages of the CSPNet structure. Let the feature maps of three scales be extracted and denoted as F1, F2, and F3, which correspond to different resolutions of the original image. Anchor frame parameters are calculated independently for each scale of the feature map; the anchor frame parameters include the width and height of the anchor frame. For the feature map F at the i-th scale i Calculate anchor frame parameter a i The calculation formula for the process is: a i =Clustering(F i ,K), where Clustering is a K-means clustering algorithm used to find the optimal anchor box size, and K is the number of cluster centers, i.e., the number of anchor boxes; S2-3. Multi-scale anchor frame parameter fusion: Design a fusion strategy to weight and fuse anchor frame parameters at different scales; the fusion formula is: a = w1·a1 + w2·a2 + w3·a3 Where w1, w2, w3 are the weights of the three scales, a1, a2, a3 are the corresponding anchor box parameters, and a is the fused anchor box parameter; weight w i The formula is dynamically adjusted based on the target size and feature map resolution: Among them, s i s is the average target size of the i-th feature map. target σ is the average size of the target, and σ is a parameter that controls the smoothness of the weights; S2-4, Bounding box regression during training: Bounding box regression training is performed using the fused anchor box parameters; the bounding box regression loss function formula is: Where N is the number of anchor frames, b i For the true bounding box, CIoU is the complete intersection-union ratio (IoU) loss function; S2-5. Target detection at the prediction end: Target detection is performed using the fused anchor box parameters at the prediction end. The backbone network is used to extract image features. First, a Focus structure is introduced to periodically extract pixels from the input image. The four sub-images are then concatenated to reconstruct a low-resolution image with a small number of pixels, thereby achieving downsampling and feature compression of the input feature map. Then, a CSP structure is introduced to split the input feature map into two parts, which are then concatenated and combined after cross-processing as the input for the next layer. The neck network further improves the feature extraction capability. The neck network adopts the structure of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN). The FPN structure is used to pass strong semantic features from top to bottom, and the PAN module is used to construct a feature pyramid structure to pass strong localization features from bottom to top, so as to realize feature fusion between different layers. The prediction information in the prediction end consists of bounding box location information, category, and confidence score; among them, the bounding box loss function adopts the complete intersection-union ratio (CIoU) loss function to measure the difference between the predicted bounding box and the true bounding box; and non-maximum suppression (NMS) is used to filter the predicted boxes. Where A and B represent different boxes, These represent the predicted boxes. The width and height of the actual bounding box b are given by w and h, respectively, and ρ represents the Euclidean distance between the center points of the two anchor boxes. b This represents the diagonal distance of the smallest closure region that simultaneously contains both the true anchor frame and the predicted anchor frame.

4. The cooperative multi-target tracking method for rotorcraft UAV swarms according to claim 1, characterized in that: In step S3, based on the local environment map around the rotorcraft UAV, an Euclidean distance field is constructed around the rotorcraft UAV. This includes: constructing the Euclidean distance field in real time based on the obstacle information in the known local environment map; the Euclidean distance field represents the distance from any point in space to the nearest obstacle surface and includes positional relationships; the calculation of the Euclidean distance field is divided into the Euclidean distance field of the free region relative to the obstacle and the Euclidean distance field of the obstacle region relative to the free region; the Euclidean distance field of the obstacle region relative to the free region is obtained by inverting the sign and superimposing it with the Euclidean distance field of the free region relative to the obstacle; the value of the Euclidean distance field at each point in space is obtained through Euclidean distance transformation (EDT), and the specific representations of the two-dimensional and three-dimensional EDTs are shown below: Where x, y, and z represent the coordinates of the three-dimensional spatial position vector on the x, y, and z axes, respectively, and x o ,y o ,z o These represent the coordinates of the obstacle's three-dimensional spatial position vector on the x, y, and z axes, respectively; the value of EDT is always greater than or equal to 0, and the value of EDT is 0 if and only if the spatial position is located on the surface of the obstacle; The following expressions are solved using cubic two-dimensional EDT: Where x, y, and z represent the coordinates of the three-dimensional spatial position vector on the x, y, and z axes, respectively. o ,y o ,z o These represent the coordinates of the obstacle's three-dimensional spatial position vector on the x, y, and z axes, respectively. The sum of the last three results and division by 2 yields the Euclidean distance field of the 3D local map. The Euclidean distance field centered on the UAV's position is updated in real time whenever the local environment map changes. By obtaining the value of any point in the local environment corresponding to the Euclidean distance field, the distance from that point to the nearest obstacle is determined, providing constraints for the subsequent spatiotemporal trajectory planning of the UAV.

5. A cooperative multi-target tracking method for a rotorcraft unmanned aerial vehicle (UAV) swarm according to claim 1, characterized in that: In step S4, a centralized cluster allocation scheme is constructed to assign corresponding targets to each individual rotorcraft drone in the cluster. This includes: after acquiring the position information of all observed targets, considering the value of the observed targets and the distance between the observed targets and the cluster drones, target allocation is performed for each cluster drone. Target allocation requires that each target must be tracked by at least one drone and each drone can track one and only one target. The target allocation model is optimized under these constraints. The target allocation model is shown below: and Where, the upper right corner T represents the transpose operation of a vector / matrix, and n swarm n represents the number of rotary-wing drones. target Represents the number of observed targets; x represents the 0-1 integer optimization vector and Indicates the i-th t Does the rotorcraft drone track the j-th...? t One goal, Indicates tracking, otherwise does not track; and This means that each rotary-wing UAV can be assigned to one and only one target, and each target requires at least one rotary-wing UAV; f val f represents the value vector. dist Represents the value vector, i t The integer is 0 ≤ i t <n swarm j t The integer is 0 ≤ j t <n target ; and Indicates the j-th t The value of a goal Indicates the i-th t The rotary-wing drone and the jth t The distance between the targets is easily obtained. v max d represents the maximum value among the observed targets. max This indicates the farthest distance between the rotary-wing drone and the target. Represent real numbers, Represents a non-negative integer.

6. A cooperative multi-target tracking method for a rotorcraft unmanned aerial vehicle swarm according to claim 1 or 5, characterized in that: In step S4, for a single specific target motion information, a target motion prediction model for the rotary-wing UAV is constructed, including: predicting the motion within a future fixed time period using the target's motion state over a past fixed time period; predicting the future target motion by curve fitting using the target's historical motion information as known parameters; establishing a motion prediction optimization model based on Bézier curves, with the calculation result of the motion fitting optimization model serving as the target motion prediction model. The motion prediction optimization model based on Bézier curves is as follows: Among them, J pred Let M represent the motion prediction optimization function. pred n represents the amount of historical movement information preserved. bezier Indicates the order of the Bezier curve, i p The integers are 0 ≤ i ≤ M pred , Indicates the i-th p The time corresponding to the target position in each historical motion information. Indicates the current moment. B(t) represents the Bézier curve. This represents the predicted trajectory of the target, characterized by a Bézier curve. This represents the second derivative of the predicted trajectory of the target, represented by a Bézier curve, with respect to time. is n bezier Bernsteinki, These are the control points of the Bézier curve. Let s represent the (i-2), (i-1), and (i-1)th control points of the Bezier curve in dimension μ, where μ ∈ {x, y, z}. bezier,t v represents the time scaling factor. max,pred a represents the target's predicted maximum speed. max,pred Indicates the target's predicted maximum acceleration. f represents the confidence weight of historical target observation information at different times. pred,t The confidence weighting function represents the historical target observation information related to time, tanh represents the hyperbolic tangent function, and k represents the k-value. pred,t An adjustable parameter representing the confidence weights associated with historical target observation information at different times. express Historical target observation location information corresponding to the given time; ω p This represents the weights associated with the second-order regulator structure.

7. A cooperative multi-target tracking method for a rotorcraft unmanned aerial vehicle (UAV) swarm according to claim 1, characterized in that: In step S5, based on planning a path that avoids collisions with obstacles and other swarm drones, key waypoints on the flight trajectory are selected where the distance between adjacent points is greater than the lower bound of the minimum distance and less than the upper bound of the maximum distance, and the line connecting two adjacent points does not overlap with an obstacle. Given the Euclidean distance field centered on the drone's position, the drone's time-track planning is divided into flight trajectory optimization and yaw angle trajectory optimization. The flight trajectory and yaw angle trajectory are represented as follows: Among them, Ξ MINCO This represents a polynomial trajectory class that satisfies the definition, where MINCO represents the minimum control quantity trajectory. Indicates a dimension of m opt real vectors, This indicates that the number of rows is m opt The number of columns is M opt A real matrix of -1 Indicates a dimension of M opt A positive real vector, q(t) represents the position at time t, T opt,∑ The total time of the flight trajectory is represented by m. opt M represents the dimension of the trajectory. opt This indicates the number of segments in the polynomial flight trajectory. p represents the coefficients of the polynomial locus. opt =(p opt,1 ,…,p opt,M-1 T represents the midpoint between the flight path and the yaw angle path. opt =(T opt,1 ,…,T opt,M ) T This represents the time vector composed of the times of each segment of the polynomial trajectory in the flight path and yaw angle trajectory. This represents a parameter mapping with linear complexity, where the minimum control quantity trajectory is determined by the midpoint p of the trajectory. opt and time vector T opt Parametric representation.

8. A cooperative multi-target tracking method for a rotorcraft unmanned aerial vehicle swarm according to claim 1 or 7, characterized in that: Step S5 includes: transforming the safety constraints, self-dynamic constraints, target tracking constraints, formation constraints, and trajectory smoothness constraints based on the Euclidean distance field into optimization objectives, thus transforming the constrained optimization problem into an unconstrained optimization problem, and then solving it using an unconstrained gradient optimization method; the spatiotemporal trajectory planning is divided into a spatiotemporal trajectory optimization model for flight trajectory and a trajectory optimization model for yaw angle, wherein the flight trajectory optimization model is as follows: Among them, J spatial The optimization function representing the flight trajectory, J control The objective function λ represents minimizing the control input and total time of the flight trajectory. control p represents the weights of the objective function that minimizes the control inputs and total time of the flight trajectory. opt (t) represents the position of the flight trajectory at time t. Let represent the third derivative of the position at time t with respect to time. Denotes the penalty function for navigation constraints of a single-rotor unmanned aerial vehicle. The penalty function q represents the navigation constraints of a rotorcraft UAV swarm. opt T represents the intermediate waypoint of the flight path. opt,∑ ρ represents the total time of the flight trajectory. opt An adjustable parameter related to the total time of the flight trajectory; The optimization model for the track angle is as follows: Among them, J yaw J represents the optimization function for the trajectory angle. yaw,control The objective function is λ, which represents the control variable that minimizes the track angle. yaw,control J represents the weights corresponding to the objective function that minimizes the path angle control quantity. yaw,vel Let λ represent the penalty function corresponding to the feasibility constraint on the yaw rate of the rotary-wing UAV. yaw,vel J represents the weight corresponding to the feasibility penalty function for the yaw rate of a rotary-wing UAV. yaw,acc Let λ represent the penalty function corresponding to the feasibility constraint of the yaw acceleration of the rotary-wing UAV. yaw,acc J represents the weight corresponding to the feasibility penalty function for the yaw angle acceleration of a rotary-wing UAV. yaw,track Let λ represent the penalty function corresponding to the yaw angle constraint during target tracking of a rotary-wing UAV. yaw,track M represents the weight corresponding to the yaw angle penalty function during target tracking of a rotary-wing UAV, ψ represents the intermediate state point of the yaw angle trajectory, and M represents the weight corresponding to the yaw angle penalty function. opt This indicates the number of segments in the polynomial flight trajectory. This indicates the number of samples for each yaw angle trajectory segment. Indicates the i-th opt The duration of the flight path i opt The integer is 0 ≤ i opt ≤M opt ,j opt Integer and Indicates the j-th opt Each integral coefficient and ψ(t) represents the polynomial yaw angle trajectory related to time t. This represents the polynomial yaw rate trajectory related to time t. ψ represents the polynomial yaw acceleration trajectory related to time t. * (t) represents the expected polynomial yaw angle trajectory related to time t. This represents the square of the maximum yaw rate amplitude. It represents the square of the maximum yaw angle acceleration magnitude.

9. A cooperative multi-target tracking method for a rotorcraft unmanned aerial vehicle (UAV) swarm according to claim 8, characterized in that: In step S5, the penalty function for the navigation constraints of a single rotary-wing UAV is: Among them, J obs Let λ represent the penalty function corresponding to the obstacle avoidance constraints of a rotary-wing UAV. obs J represents the weight corresponding to the penalty function related to obstacle avoidance in rotary-wing UAVs. vel Let λ represent the penalty function corresponding to the speed feasibility constraint of the rotary-wing UAV. vel J represents the weight corresponding to the speed feasibility penalty function for rotary-wing UAVs. acc Let λ represent the penalty function corresponding to the acceleration feasibility constraint of the rotary-wing UAV. acc J represents the weight corresponding to the acceleration feasibility penalty function for rotary-wing UAVs. track Let λ represent the penalty function corresponding to the target tracking constraints of the rotary-wing UAV. track J represents the weight corresponding to the target tracking penalty function of the rotary-wing UAV. var Let λ represent the penalty function corresponding to the smoothness constraint of the rotorcraft UAV's flight trajectory. var M represents the weights corresponding to the smoothness penalty function of the rotorcraft UAV's flight trajectory. opt Indicates the number of flight trajectory segments, i opt The integer is 0 ≤ i opt ≤M opt , This indicates the number of samples for each flight path segment. Indicates the i-th opt The duration of the flight path Denotes the integral coefficient and d obs Represents the spatial position p at time t obtained from the Euclidean distance field. opt (t) Distance to the nearest obstacle, d th v represents the shortest possible distance from a rotary-wing drone to an obstacle. max a represents the maximum permissible flight speed amplitude of a rotary-wing unmanned aerial vehicle. max This indicates the maximum permissible flight acceleration amplitude for a rotary-wing drone. This represents the total number of predicted position points in the target motion prediction. Indicates the flight path in Time and location The vertical component of the distance from the target predicted point location. Indicating flight trajectory Time and location The horizontal component of the distance from the target prediction point. Indicating flight trajectory Time and location The expected vertical distance between the target predicted point and the target point, where ε represents a small constant, d l d represents the expected distance from the lower bound. u This indicates the expected distance from the upper bound. The penalty function represents the horizontal distance. The integral coefficients of the target tracking penalty function for rotary-wing UAVs are represented. The penalty function for navigation constraints in a rotorcraft drone swarm is: L graph =D graph -A graph Among them, J rep J represents the penalty function corresponding to the collision avoidance constraints within a rotorcraft UAV swarm. form Let λ represent the penalty function corresponding to the constraints related to the formation of a rotorcraft drone swarm. rep λ represents the weight corresponding to the collision avoidance penalty function within a rotorcraft drone swarm. form N represents the weight corresponding to the penalty function related to the formation of the rotorcraft drone swarm. swarm r represents the number of individuals in a swarm of rotary-wing drones. s Let A represent the minimum acceptable distance between individuals in the cluster, τ represent the time interval between the current moment and the start moment of the flight trajectory, and A represent the minimum acceptable distance between individuals in the cluster. graph Let A represent the adjacency matrix. graph,des Let D be the adjacency matrix of the desired formation. graph Let I denote a diagonal matrix, and L denote the identity matrix. graph M represents the Laplace matrix. opt Indicates the number of flight trajectory segments, i opt The integer is 0 ≤ i opt ≤M opt , This indicates the number of samples for each flight path segment. This indicates the duration of each flight path segment. Denotes the integral coefficient and Represents the i-th time related to time t opt Segment polynomial locus, Indicates the kth swarm A drone at time t swarm The spatial position corresponding to time T l This indicates the duration of the l-th segment of the planned trajectories of other drones within the cluster. This represents the normalized Laplace matrix of the current formation. This represents the Laplace matrix of the expected formation after normalization. Indicates the i-th node within the cluster s The drone and the jth s The distance between the drones Represents a diagonal matrix D graph The i-th s Line j s The value corresponding to the column, express The Frobenius norm of a matrix, i.e. Where Tr(·) represents the trace of the matrix.

10. A cooperative multi-target tracking method for a rotorcraft unmanned aerial vehicle (UAV) swarm according to claim 1, characterized in that: In step S6, the optimized flight trajectory results of cluster members, the detected target position and velocity information are transmitted within the cluster through a broadcast communication mechanism. This includes: for flight trajectory optimization results and target position information with small data volumes, the communication protocol of the UAV cluster broadcast communication is based on User Datagram Protocol (UDP); for depth images, the communication protocol of the UAV cluster broadcast communication is based on Transmission Control Protocol (TCP). The decision-making algorithm in the autonomous collaborative target tracking rotorcraft swarm system adopts a finite state machine algorithm. Trigger conditions are set for different states, and state transitions are achieved through state changes, thereby controlling the rotorcraft to perform different actions. The swarm's states include: waiting, continuous start, generating new trajectory, replanning, executing trajectory, and emergency stop. Different states correspond to different task modules, and different trigger mechanisms are used to switch states and execute different actions. First, the swarm's collaborative planner initializes and receives relevant sensor data. Based on the acquired image information, target location information is obtained using target detection, and the acquired target location information is transmitted using the swarm's communication mechanism. When the swarm receives the assigned target information from the task allocation center node, it enters the program control state and waits for the start trigger signal. Upon receiving the start trigger signal, the distributed asynchronous planner receives flight trajectories planned by other drones. Once all necessary signals are received, the system continuously starts and begins global trajectory planning to determine the optimal trajectory from the current position to the target position. If global trajectory planning is successful, the system enters the execution state according to the planned trajectory; if planning fails, it returns to the initial state and retryes. During execution, the system continuously checks whether it has reached the target position. If not, it continues to execute the predetermined trajectory; if it has reached the target position, the system determines whether replanning is needed. If replanning is not needed, the system directly enters the local planning state to optimize the current trajectory segment; if replanning is needed, the system adjusts the trajectory to adapt to the new situation before entering local planning again. During this period, the system continuously performs safety checks on the planned trajectory; if any unsafe factors are detected, the system will stop urgently to ensure safety. After a successful emergency stop, the system generates a new trajectory plan and attempts global trajectory planning again to ensure the safety and effectiveness of the trajectory. Through this iterative process, the system can flexibly and safely complete various tasks. If a problem is encountered at any stage, the system will take corresponding measures, such as returning to the initial state or emergency stop, to ensure the smooth progress of the overall process.

11. A cooperative multi-target tracking system for rotorcraft unmanned aerial vehicle (UAV) swarms, characterized in that: The system includes: a data transmission module, a perception module, a mapping module, an allocation module, a planning module, and a control module. The data transmission module is used for receiving and sending observation target position information and planned trajectory information between various rotorcraft UAVs. The perception module acquires depth images or laser point cloud data of the surrounding environment through depth cameras or LiDAR and converts them into environmental point cloud data and target observation information. The environmental point cloud data is used to obtain its own positioning information through vision-based or LiDAR-based odometry. The mapping module converts the environmental point cloud data in the perception module into a probabilistic grid map. The allocation module is used to allocate existing target observation information to individual rotorcraft UAVs. The planning module generates a safe, dynamically feasible, and smooth flight trajectory that meets the tracking task constraints based on the current state of the rotorcraft UAV obtained by the perception module, the probabilistic grid information obtained by the mapping module, and the allocated target observation position information obtained by the allocation module. The control module converts the flight trajectory obtained by the planning module into motor control commands based on the rotorcraft UAV's own dynamic characteristics to control the UAV.

Images