Dynamic multi-target tracking method, system and device and storage medium

Abstract

The invention relates to the technical field of multi-target tracking, in particular to a dynamic multi-target tracking method, system and device and a storage medium. According to the dynamic multi-target tracking method, the tracking target image is obtained by using the large view angle of the aircraft in the air, and tracking task allocation is performed on the ground tracking mechanisms based on a pigeon flock simulation and auction mechanism, so that one aircraft can control a plurality of ground tracking mechanisms; compared with the prior art that only one ground tracking mechanism can be controlled by one aircraft, the resource utilization rate is improved; meanwhile, the ground tracking mechanism performs tracking path planning on the tracking target according to conditions by judging whether the corresponding tracking target is located at the first visual angle of the ground tracking mechanism, and executes the planned tracking path to track the ground tracking target; a multi-target tracking task is efficiently executed through an air-ground cooperation method.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-target tracking, and in particular to a dynamic multi-target tracking method, system, device and storage medium. Background Art

[0002] Multi-target tracking has broad application prospects in the fields of transportation, search, tracking, etc. Multi-target tracking tasks are usually carried out in complex and unknown environments. Unknown global maps and unclear target locations lead to inefficient allocation of multi-target tracking tasks, and ground tracking agencies are unable to quickly locate targets. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a dynamic multi-target tracking method, system, device and storage medium.

[0004] To achieve the above object, the present invention provides a dynamic multi-target tracking method, comprising:

[0005] The aircraft acquires a tracking target image, detects the tracking target and acquires a pixel position of the tracking target image, converts the tracking target image into an initial global map, and converts the pixel position of the tracking target image into a global position of the tracking target;

[0006] Acquire the global position of the ground tracking mechanism, and based on the pigeon flock algorithm and the auction mechanism, assign the tracking target to the ground tracking mechanism according to the initial global map, the global position of the tracking target, and the global position of the ground tracking mechanism, to complete the tracking target task assignment, wherein the number of the ground tracking mechanisms is greater than or equal to the number of the tracking targets;

[0007] Outputting the corresponding tracking target image and tracking target global position to the ground tracking mechanism based on the tracking target task allocation;

[0008] Acquire the first-perspective image of the current ground tracking mechanism, and determine whether the tracking target is located in the first perspective of the ground tracking mechanism based on the tracking target image, the global position of the tracking target, and the first-perspective image of the current ground tracking mechanism; if the tracking target is not located in the first perspective of the ground tracking mechanism, acquire the current global position of the tracking target and set it as the end point of the first tracking trajectory of the ground tracking mechanism, and plan the first tracking trajectory for the ground tracking mechanism; if the tracking target is located in the first perspective of the ground tracking mechanism, acquire the pixel position of the tracking target image in the current first-perspective image and convert it into the world coordinate system position of the tracking target, set the world coordinate system position of the tracking target as the end point of the second tracking trajectory of the ground tracking mechanism, and plan the second tracking trajectory for the ground tracking mechanism;

[0009] The planning of the first tracking trajectory and the planning of the second tracking trajectory are optimized and converted into control instructions, and output to a ground tracking mechanism to execute a target tracking task.

[0010] Optionally, the tracking target is in motion.

[0011] Optionally, the steps of assigning tracking targets to the ground tracking mechanism based on the pigeon flock algorithm and the auction mechanism to complete the tracking target task assignment include:

[0012] Step S201, initializing the pigeon-like flock algorithm;

[0013] Step S202, setting the position and speed of each pigeon, wherein each pigeon in the pigeon-like swarm algorithm is a tracking target task allocation scheme, the position vector of the pigeon is a set of bid prices of each ground tracking mechanism relative to each tracking target, and the speed of the pigeon is the update direction of the bid price, wherein the initial position and initial speed of the pigeon are randomly set;

[0014] Step S203, based on the bidding price, the ground tracking mechanisms are sorted from low to high, and the ground tracking mechanisms with low bidding prices are preferentially assigned the tracking targets corresponding to the bidding prices. When N≤p, the ground tracking mechanisms and the tracking targets are not repeatedly assigned. When N>p, the tracking targets are assigned to the p+1th ground tracking mechanisms to the Nth ground tracking mechanisms in the order of bidding prices from low to high, so as to realize the task assignment between the ground tracking mechanisms and the tracking targets, wherein N is the number of ground tracking mechanisms, p is the code of the tracking target, N and p are both natural numbers, and N>p;

[0015] Step S204, judging whether T+1 is greater than or equal to m, if T+1 is greater than or equal to m, outputting the tracking target task assignment, if T+1 is less than m, executing step S206, wherein T is the number of iterations of the pigeon group, and m is the set pigeon group iteration threshold;

[0016] Step S205, obtaining a tracking distance between the ground tracking mechanism and the assigned tracking target based on the tracking target task assignment, the initial global map, the tracking target global position, and the ground tracking mechanism global position;

[0017] Step S206, obtaining a fitness value of each pigeon according to the fitness function and the tracking distance between the ground tracking mechanism and the assigned tracking target;

[0018] Step S207, determine whether T+1 is greater than or equal to n, if T+1 is less than n, then execute step S208, if T+1 is greater than or equal to n, then execute step S209, where T is the number of iterations of the pigeon group, n is the set iteration threshold of the map and compass operator, n<m;

[0019] Step S208, updating the position of the pigeon flock using a map and a compass operator according to the fitness value of each pigeon;

[0020] Step S209, based on the fitness value of each pigeon, the location of the pigeon flock is updated using a landmark operator.

[0021] Optionally, the fitness function is:

[0022] minF=λ1f1+λ2f2+λ3f3,

[0023] Among them, λ1, λ2 and λ3 are weight parameters, f1 is the total tracking distance, f2 is the maximum tracking distance and f3 is the tracking number distribution;

[0024] The total tracking distance f1 is the sum of the tracking distances of all ground tracking mechanisms and the assigned tracking targets, which is:

[0025]

[0026] Where N is the number of ground tracking mechanisms, C pi is the path distance from the jth ground tracking mechanism to the tracking target, j is the code of the ground tracking mechanism, and j is a natural number;

[0027] The maximum tracking distance f2 is the maximum tracking distance between the ground tracking mechanism and the assigned tracking target, which is:

[0028] minf2=max c pj ,

[0029] Among them, C pj is the path distance from the jth ground tracking mechanism to the tracking target;

[0030] The tracking quantity distribution f3 is the average value of the number of ground tracking mechanisms allocated to each tracking target, which is:

[0031]

[0032] Where M is the number of tracking targets, n p The number of ground tracking units assigned to the pth tracking target, The average number of ground tracking agencies assigned to the same tracking target.

[0033] Optionally, the formula for updating the position of the pigeon flock by the map and compass operator is:

[0034]

[0035] Among them, v i is the pigeon speed, x i is the pigeon position, R is a constant, n is the number of iterations of the map and compass operators, rd is a random number between 0 and 1, x best is the optimal pigeon position, e -R·n is the attenuation factor, i is the pigeon code, and i is a natural number;

[0036] The formula for updating the position of the pigeon flock by the landmark operator is:

[0037]

[0038] x i+1 =x i +rd·(X c -x i ),

[0039] Where X c is the landmark position, x i is the position of the pigeon, P is the total number of pigeons in the flock, and F is the fitness value of each pigeon.

[0040] Optionally, the ground tracking mechanism acquires a corresponding tracking target, and the step of determining whether the tracking target is located in a first viewing angle of the ground tracking mechanism includes:

[0041] Step S401, the ground tracking mechanism acquires the tracking target image;

[0042] Step S402, copying the acquired tracking target image, one copy as the tracking target initial image, and one copy as the tracking target dynamic image, and splicing the tracking target initial image and the tracking target dynamic image to obtain a tracking target comparison image;

[0043] Step S403, the ground tracking mechanism obtains the first-viewing angle image of the current ground tracking mechanism, obtains the tracking target image in the current first-viewing angle image based on the first-viewing angle image of the current ground tracking mechanism, and compares the tracking target image in the current first-viewing angle image with the tracking target comparison image until the tracking target is located at the first viewing angle of the ground tracking mechanism, and then confirms that the tracking target is located at the first viewing angle of the ground tracking mechanism;

[0044] Step S404: the ground tracking mechanism updates the tracking target dynamic image to the tracking target image in the current first-view image at regular intervals.

[0045] Optionally, the aircraft includes one of a fisheye lens and a wide-angle lens.

[0046] The present invention also provides a multi-target tracking system, comprising:

[0047] Aircraft module, ground tracking module;

[0048] The aircraft module is used for the aircraft to obtain a tracking target image, detect the tracking target and obtain the pixel position of the tracking target image, and convert the tracking target image into an initial global map, and convert the pixel position of the tracking target image into a global position of the tracking target; obtain the global position of the ground tracking mechanism, and based on the pigeon flock algorithm and the auction mechanism, assign the tracking target to the ground tracking mechanism according to the initial global map, the global position of the tracking target, and the global position of the ground tracking mechanism, and complete the tracking target task assignment, wherein the number of the ground tracking mechanisms is greater than or equal to the number of the tracking targets; output the corresponding tracking target image and tracking target global position to the ground tracking mechanism based on the tracking target task assignment;

[0049] The ground tracking module is used to obtain the first-perspective image of the current ground tracking mechanism, and judge whether the tracking target is located in the first perspective of the ground tracking mechanism based on the tracking target image, the global position of the tracking target, and the first-perspective image of the current ground tracking mechanism. If the tracking target is not located in the first perspective of the ground tracking mechanism, the global position of the current tracking target is obtained and set as the end point of the first tracking trajectory of the ground tracking mechanism, and the first tracking trajectory is planned for the ground tracking mechanism; if the tracking target is located in the first perspective of the ground tracking mechanism, the pixel position of the tracking target image in the current first-perspective image is obtained and converted into the world coordinate system position of the tracking target, and the world coordinate system position of the tracking target is set as the end point of the second tracking trajectory of the ground tracking mechanism, and the second tracking trajectory is planned for the ground tracking mechanism; the planning of the first tracking trajectory and the planning of the second tracking trajectory are optimized and converted into control instructions, which are output to the ground tracking mechanism to perform the target tracking task.

[0050] The present invention also provides a multi-target tracking device, comprising: a memory, a processor, and a multi-target tracking program stored in the memory and executable on the processor, wherein the multi-target tracking program is configured to implement the steps of the above-mentioned dynamic multi-target tracking method.

[0051] The present invention also provides a storage medium, comprising: a multi-target tracking program is stored on the storage medium, and the multi-target tracking program implements the above-mentioned dynamic multi-target tracking method when executed by a processor.

[0052] In summary, the advantages and beneficial effects of the present invention are:

[0053] The present invention provides a dynamic multi-target tracking method, system, device and storage medium. The dynamic multi-target tracking method uses the large viewing angle of the aircraft in the air to obtain the tracking target image, and based on the pigeon flock and auction mechanism, the tracking task is assigned to the ground tracking mechanism, so that one aircraft controls multiple ground tracking mechanisms, which improves the resource utilization rate compared with the existing one aircraft can only control one ground tracking mechanism; at the same time, the ground tracking mechanism determines whether the corresponding tracking target is located in the first viewing angle of the ground tracking mechanism, plans the tracking path for the tracking target according to the situation, executes the planned tracking path to track the tracking target on the ground, and realizes the efficient execution of the multi-target tracking task through the air-ground collaboration method. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of a process of a dynamic multi-target tracking method according to an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of a process of assigning tracking target tasks in a dynamic multi-target tracking method according to an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of decoupling bid price and position of a dynamic multi-target tracking method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the specific implementation methods of the present invention and the corresponding drawings. Obviously, the described implementation methods are only part of the implementation methods of the present invention, not all implementation methods. Based on the implementation methods of the present invention, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0058] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0059] The present invention provides a dynamic multi-target tracking method, such as Figure 1 As shown, including:

[0060] Step S10, the aircraft acquires a tracking target image, detects the tracking target and acquires the pixel position of the tracking target image, converts the tracking target image into an initial global map, and converts the pixel position of the tracking target image into a global position of the tracking target;

[0061] Step S20, obtaining the global position of the ground tracking mechanism, and assigning the tracking target to the ground tracking mechanism based on the pigeon flock algorithm and the auction mechanism according to the initial global map, the global position of the tracking target, and the global position of the ground tracking mechanism, to complete the tracking target task assignment, wherein the number of the ground tracking mechanisms is greater than or equal to the number of the tracking targets;

[0062] Step S30, outputting the corresponding tracking target image and tracking target global position to the ground tracking mechanism based on the tracking target task allocation;

[0063] Step S40, obtaining the first-viewing angle image of the current ground tracking mechanism, and judging whether the tracking target is located in the first-viewing angle of the ground tracking mechanism based on the tracking target image, the global position of the tracking target, and the first-viewing angle image of the current ground tracking mechanism; if the tracking target is not located in the first-viewing angle of the ground tracking mechanism, executing step S50; if the tracking target is located in the first-viewing angle of the ground tracking mechanism, executing step S60;

[0064] Step S50, obtaining the global position of the current tracking target and setting it as the end point of the first tracking track of the ground tracking mechanism, and planning the first tracking track for the ground tracking mechanism;

[0065] Step S60, obtaining the pixel position of the tracking target image in the current first-perspective image and converting it into the world coordinate system position of the tracking target, setting the world coordinate system position of the tracking target as the end point of the second tracking trajectory of the ground tracking mechanism, and planning the second tracking trajectory for the ground tracking mechanism;

[0066] Step S70: Optimize the planning of the first tracking trajectory and the planning of the second tracking trajectory and convert them into control instructions, which are output to a ground tracking mechanism to execute a target tracking task.

[0067] Specifically, in an embodiment of the present invention, the aircraft is a drone, and the ground tracking mechanism is an unmanned vehicle.

[0068] In other embodiments, the aircraft is a manned aircraft, a helicopter, or other suitable flying equipment, and the ground tracking mechanism is a robot dog, an autonomous vehicle, or other suitable tracking equipment.

[0069] Step S10 is executed, the aircraft acquires a tracking target image, detects the tracking target and the pixel position of the tracking target image, converts the tracking target image into an initial global map, and converts the pixel position of the tracking target image into a global position of the tracking target.

[0070] The aircraft in the air has a larger field of view, can cover a larger tracking area, and provide a larger monitoring range for multiple tracking targets.

[0071] In an embodiment of the present invention, the aircraft includes a fisheye lens, and a fisheye perspective image is acquired based on the fisheye lens.

[0072] In an embodiment of the present invention, the step of obtaining the tracking target image based on the fisheye lens and acquiring the tracking target and the pixel position of the tracking target image includes:

[0073] Step S101, using the fisheye lens to obtain a fisheye perspective image.

[0074] Step S102, performing image correction on the fisheye perspective image to obtain the tracking target image.

[0075] In an embodiment of the present invention, a Scaramuzza fisheye camera model is used to correct the fisheye perspective image acquired by the fisheye lens, and finally the tracking target image is obtained, thereby preventing the distortion of the fisheye perspective image from affecting the tracking target image.

[0076] Step S103, cropping and rotating the tracking target image, and making the tracking target in the tracking target image stand upright.

[0077] In the embodiment of the present invention, the acquired tracking target image is cropped and rotated to obtain the cropped and rotated tracking target image, so that the tracking target in the tracking target image is upright.

[0078] Step S104: performing tracking target detection on the cropped and rotated tracking target image to obtain the tracking target and the pixel position of the tracking target image.

[0079] In an embodiment of the present invention, a YOLOv8 model is used to perform tracking target detection based on the cropped and rotated tracking target image. Since the tracking target in the cropped and rotated tracking target image is upright, the problem of misdetection of inverted and tilted tracking targets by the YOLOv8 model is avoided, and the success rate of tracking target detection is improved. Specifically, the acquired tracking target image is cropped into four pieces along -45 degrees, 45 degrees, 135 degrees, and 225 degrees sectors, respectively, and the cropped images are rotated 0 degrees, -90 degrees, 180 degrees, and 90 degrees, respectively, so that the tracking target in the tracking target image is upright.

[0080] In other embodiments, the aircraft includes a wide-angle lens, through which the tracking target image is acquired.

[0081] In an embodiment of the present invention, the MobileSAM method is used to convert the tracking target image into an initial global map. Specifically, the tracking target image is subjected to image semantic segmentation to construct an initial global map. By performing image semantic segmentation on the tracking target image, the passage area of ​​the ground tracking mechanism is determined as the largest single connected area in the top-down view.

[0082] In the embodiment of the present invention, the pixel position of the tracking target image is converted into the global position of the tracking target.

[0083] Execute step S20 to obtain the global position of the ground tracking mechanism, and based on the pigeon flock algorithm and the auction mechanism, assign the tracking target to the ground tracking mechanism according to the initial global map, the global position of the tracking target, and the global position of the ground tracking mechanism to complete the tracking target task assignment.

[0084] In the embodiment of the present invention, the ground tracking mechanism communicates with the aircraft via a local area network, thereby avoiding the problem of communication failure or communication delay caused by navigation system failure or delay.

[0085] In other embodiments, the ground tracking mechanism communicates with the aircraft via a 5G network or other suitable means.

[0086] In the embodiment of the present invention, Figure 2 As shown, based on the pigeon flock algorithm and the auction mechanism, the ground tracking mechanism is assigned a tracking target, and the steps of completing the tracking target task assignment include:

[0087] Step S201, initializing the pigeon-like colony algorithm.

[0088] Step S202, setting the position and speed of each pigeon, wherein each pigeon in the pigeon-like swarm algorithm is a tracking target task allocation scheme, the position vector of the pigeon is a set of bid prices of each ground tracking mechanism relative to each tracking target, and the speed of the pigeon is the update direction of the bid price, wherein the initial position and initial speed of the pigeon are randomly set.

[0089] In the embodiment of the present invention, the speed of the pigeons is used for subsequent map and compass operators to update the position of the pigeon flock, thereby increasing or decreasing the bid price of each ground tracking mechanism relative to each tracking target, thereby optimizing the bid price.

[0090] Step S203, based on the bid price, the ground tracking mechanisms are sorted from low to high, and the ground tracking mechanisms with low bid prices are preferentially assigned the tracking targets corresponding to the bid prices. When N≤p, the ground tracking mechanisms and the tracking targets are not assigned repeatedly. When N>p, the tracking targets are assigned to the p+1th ground tracking mechanisms to the Nth ground tracking mechanisms according to the order of bid prices from low to high, so as to realize task assignment between the ground tracking mechanisms and the tracking targets, wherein N is the number of ground tracking mechanisms, p is the code of the tracking target, N and p are both natural numbers, and N>p.

[0091] In the embodiment of the present invention, Figure 3 As shown in FIG. 1 , it is a set of bidding prices of ground tracking mechanisms A1, A2, ... Aj relative to tracking targets B1, B2 ... Bp, such as Figure 3 It can be seen that the single bidding price of the ground tracking mechanism A2 relative to all tracking targets is the lowest, the single bidding price of the ground tracking mechanism A1 relative to all tracking targets is the second lowest, and the single bidding price of the ground tracking mechanism A3 relative to all tracking targets is the third lowest. The ground tracking mechanisms are sorted based on the bidding prices, namely: A2, A1, A3...Aj..., and the corresponding tracking targets are assigned to the ground tracking mechanisms in sequence according to the sorting, namely: first, the tracking target Bp is assigned to the ground tracking mechanism A2, and then the tracking target Bp is excluded, and the tracking target B1 is assigned to the ground tracking mechanism A1, and the tracking target Bp and the tracking target B1 are excluded again to assign the ground tracking mechanism A3 to the tracking mechanism B2... until all the tracking targets are assigned with the corresponding ground tracking mechanisms. If there are any ground tracking mechanisms left, the corresponding tracking targets are continued to be assigned to the remaining ground tracking mechanisms in the order of low to high bidding prices, and finally the task allocation of all ground tracking mechanisms and all tracking targets is realized.

[0092] Step S204, determine whether T+1 is greater than or equal to m. If T+1 is greater than or equal to m, output the tracking target task assignment. If T+1 is less than m, execute step S206, where T is the number of iterations of the pigeon group, and m is the set pigeon group iteration threshold.

[0093] Step S205 , obtaining a tracking distance between the ground tracking mechanism and the assigned tracking target based on the tracking target task assignment, the initial global map, the tracking target global position, and the ground tracking mechanism global position.

[0094] In the embodiment of the present invention, the tracking distance between the ground tracking mechanism and the assigned tracking target is globally obtained by an A* algorithm through the global map, the global position of the tracking target, and the ground tracking mechanism.

[0095] Step S206, obtaining the fitness value of each pigeon according to the fitness function and the tracking distance between the ground tracking mechanism and the assigned tracking target.

[0096] In this embodiment of the present invention, the fitness function is:

[0097] minF=λ1f1+λ2f2+λ3f3,

[0098] Among them, λ1, λ2 and λ3 are weight parameters, f1 is the total tracking distance, f2 is the maximum tracking distance and f3 is the tracking number distribution;

[0099] The total tracking distance f1 is the sum of the tracking distances of all ground tracking mechanisms and the assigned tracking targets, which is:

[0100]

[0101] Where N is the number of ground tracking mechanisms, C pi is the path distance from the jth ground tracking mechanism to the tracking target, j is the code of the ground tracking mechanism, and j is a natural number;

[0102] The maximum tracking distance f2 is the maximum tracking distance between the ground tracking mechanism and the assigned tracking target, which is:

[0103] minf2=maxc pj ,

[0104] Among them, C pj is the path distance from the jth ground tracking mechanism to the tracking target;

[0105] The tracking quantity distribution f3 is the average value of the number of ground tracking mechanisms allocated to each tracking target, which is:

[0106]

[0107] Where M is the number of tracking targets, n p The number of ground tracking units assigned to the pth tracking target, The average number of ground tracking agencies assigned to the same tracking target.

[0108] In the embodiment of the present invention, the tracking number distribution f3 is used for uniform distribution of the ground tracking mechanisms, that is, when the number of the ground tracking mechanisms is greater than the number of the tracking targets, the number of the ground tracking mechanisms allocated to each tracking target is as uniform as possible.

[0109] Step S207, determine whether T+1 is greater than or equal to n. If T+1 is less than n, execute step S208. If T+1 is greater than or equal to n, execute step S209, where T is the number of iterations of the pigeon flock, n is the set iteration threshold of the map and compass operator, and n<m.

[0110] Step S208, based on the fitness value of each pigeon, the position of the pigeon flock is updated using the map and compass operator, and the formula for updating the position of the pigeon flock using the map and compass operator is:

[0111]

[0112] Among them, v i is the pigeon speed, x i is the pigeon position, R is a constant, n is the number of iterations of the map and compass operators, rd is a random number between 0 and 1, x best is the optimal pigeon position, e -R·n is the attenuation factor, i is the pigeon code, and i is a natural number.

[0113] In the embodiment of the present invention, the pigeon flock is the sum of all pigeons.

[0114] In the embodiment of the present invention, the attenuation factor is such that as the number of iterations increases, the speed gradually decreases.

[0115] Step S209, based on the fitness value of each pigeon, the position of the pigeon group is updated using the landmark operator, and the formula for updating the position of the pigeon group by the landmark operator is:

[0116]

[0117] x i+1 =x i +rd·(X c -x i ),

[0118] Where X c is the landmark position, x i is the position of the pigeon, P is the total number of pigeons in the flock, and F is the fitness value of each pigeon.

[0119] Step S30 is executed to output the corresponding tracking target image and tracking target global position to the ground tracking mechanism based on the tracking target task allocation.

[0120] Execute step S40 to obtain the first-perspective image of the current ground tracking mechanism, and determine whether the tracking target is located in the first perspective of the ground tracking mechanism based on the tracking target image, the global position of the tracking target, and the first-perspective image of the current ground tracking mechanism. If the tracking target is not located in the first perspective of the ground tracking mechanism, execute step S50; if the tracking target is located in the first perspective of the ground tracking mechanism, execute step S60.

[0121] In an embodiment of the present invention, the ground tracking mechanism acquires a corresponding tracking target, and the step of determining whether the tracking target is located at a first viewing angle of the ground tracking mechanism includes:

[0122] Step S401, the ground tracking mechanism acquires the tracking target image;

[0123] Step S402, copying the acquired tracking target image, one copy as the tracking target initial image, and one copy as the tracking target dynamic image, and splicing the tracking target initial image and the tracking target dynamic image to obtain a tracking target comparison image;

[0124] Step S403, the ground tracking mechanism obtains the first-viewing angle image of the current ground tracking mechanism, obtains the tracking target image in the current first-viewing angle image based on the first-viewing angle image of the current ground tracking mechanism, and compares the tracking target image in the current first-viewing angle image with the tracking target comparison image until the tracking target is located at the first viewing angle of the ground tracking mechanism, and then confirms that the tracking target is located at the first viewing angle of the ground tracking mechanism;

[0125] Step S404: the ground tracking mechanism updates the tracking target dynamic image to the tracking target image in the current first-view image at regular intervals.

[0126] In an embodiment of the present invention, the tracking target image in the current first-perspective image is compared with the tracking target initial image. When the similarity between the tracking target initial image and the tracking target image in the current first-perspective image reaches a set range, it is determined that the tracking target is located in the first perspective of the ground tracking mechanism.

[0127] Step S50 is executed to obtain the global position of the current tracking target and set it as the end point of the first tracking trajectory of the ground tracking mechanism, and plan the first tracking trajectory for the ground tracking mechanism.

[0128] In an embodiment of the present invention, the step of planning a first tracking trajectory for the ground tracking mechanism includes:

[0129] Step S501, constructing an occupancy grid map according to the laser radar point cloud carried by the ground tracking mechanism.

[0130] Step S502: in the constructed occupancy grid map, an A* algorithm is used to obtain an initial first tracking trajectory planning path of the ground tracking mechanism based on the first tracking trajectory endpoint and the global position of the current ground tracking mechanism.

[0131] Step S503, parameterizing the initial first tracking trajectory planning path using a B-spline curve to obtain the first tracking trajectory planning path, wherein the B-spline curve parameter is expressed as:

[0132]

[0133] Among them, V a is the speed of the ground tracking mechanism at time k, A a is the acceleration of the ground tracking mechanism at time k, J a is the acceleration derivative of the ground tracking mechanism at time k, Q a is the ath control point of the B-spline curve, Δk is the time interval, a is the control point of the B-spline curve, and a is a natural number.

[0134] In an embodiment of the present invention, the initial tracking path obtained by the A* algorithm search is a plurality of line segments directly connected, and the plurality of line segments obtained by the A* algorithm using B-spline curve fitting are used to obtain a smooth, dynamically feasible, and collision-free first tracking trajectory.

[0135] Execute step S60, obtain the pixel position of the tracking target image in the current first-perspective image and convert it into the world coordinate system position of the tracking target, set the world coordinate system position of the tracking target as the end point of the second tracking trajectory of the ground tracking mechanism, and plan the second tracking trajectory for the ground tracking mechanism.

[0136] In an embodiment of the present invention, the step of planning a second tracking trajectory for the ground tracking mechanism includes:

[0137] Step S601, constructing an occupancy grid map according to the laser radar point cloud carried by the ground tracking mechanism.

[0138] Step S602: In the constructed occupancy grid map, an A* algorithm is used to obtain an initial second tracking trajectory planning path of the ground tracking mechanism based on the second tracking trajectory endpoint and the global position of the current ground tracking mechanism.

[0139] Step S603, parameterizing the initial second tracking trajectory planning path using a B-spline curve to obtain the second tracking trajectory planning path, wherein the B-spline curve parameter is expressed as:

[0140]

[0141] Among them, V a is the speed of the ground tracking mechanism at time k, A a is the acceleration of the ground tracking mechanism at time k, J a is the acceleration derivative of the ground tracking mechanism at time k, Q a is the ath control point of the B-spline curve, Δk is the time interval, a is the control point number of the B-spline curve, and a is a natural number.

[0142] In an embodiment of the present invention, the initial tracking path obtained by the A* algorithm search is a plurality of line segments directly connected, and the plurality of line segments are obtained by the A* algorithm using B-spline curve fitting, and an optimization method is used to optimize the control points of the B-spline to obtain a smooth, dynamically feasible, and collision-free tracking path.

[0143] In the embodiment of the present invention, when the ground tracking mechanism plans the second tracking trajectory, a tracking distance constraint and a tracking viewing angle constraint are set for the planning of the second tracking trajectory.

[0144] Step S70: Optimize the planning of the first tracking trajectory and the planning of the second tracking trajectory and convert them into control instructions, which are output to a ground tracking mechanism to execute a target tracking task.

[0145] In an embodiment of the present invention, an MPC (Model Predict Control) controller is used, and the forward speed and angular speed of the ground tracking mechanism are used as control inputs of the MPC controller. The MPC controller optimizes the planning of the first tracking trajectory and the planning of the second tracking trajectory to minimize the tracking trajectory error. Specifically, the cost function J of the control input for optimizing the tracking trajectory error is:

[0146]

[0147] Among them, Q is the state error weight matrix, R is the control input weight matrix, Z k is the actual position of the ground tracking mechanism at time k, g k is the expected position of the ground tracking mechanism at time k, u k is the control input at time k, Z N is the position of the end point N of the total number of positions in the expected trajectory, g is the global position of the current tracking target or the expected position of the world coordinate system position of the tracking target, (·) T is the matrix transpose, T is the time interval between time k and time k+1, and the expected position is the planned position of the first tracking trajectory or the second tracking trajectory.

[0148] In an embodiment of the present invention, the maximum speed and acceleration of the ground tracking mechanism are optimized by using the MPC controller, and the maximum speed and acceleration of the ground tracking mechanism are limited to a certain range to ensure the safety and dynamic feasibility of the ground tracking mechanism, specifically:

[0149]

[0150] Z k+1 =Z k +T·f(Z k ,u k )

[0151] Z0=Z init

[0152] -v max ≤v k ≤v max

[0153] -ω max ≤ωk≤ω max

[0154]

[0155] Among them, T is the time interval between time k and time k+1, Z init is the first position of the first tracking trajectory or the first position of the second tracking trajectory, Z k is the current global position of the ground tracking mechanism at time k, v k is the velocity of the ground tracking mechanism at time k, ω k is the angular velocity of the ground tracking mechanism at time k, v max is the maximum speed of the ground tracking mechanism, ω max is the maximum angular velocity of the ground tracking mechanism, u k is the control input at time k, f(Z,u)=[vcos(θ),vsin(θ),ω] T is the dynamic equation of the ground tracking mechanism.

[0156] In an embodiment of the present invention, the tracking trajectory error optimization and the maximum speed and acceleration optimization of the ground tracking mechanism are solved by adopting the CasADi solving program.

[0157] The image of the tracking target is acquired by utilizing the large viewing angle of the aircraft in the air, and tracking tasks are allocated to the ground tracking mechanism based on a pigeon flock and auction mechanism, thereby realizing one aircraft controlling multiple ground tracking mechanisms. Compared with the existing situation where one aircraft can only control one ground tracking mechanism, resource utilization is improved; at the same time, the ground tracking mechanism determines whether the corresponding tracking target is located in the first viewing angle of the ground tracking mechanism, plans the tracking path for the tracking target according to different situations, executes the planned tracking path to track the tracking target on the ground, and realizes efficient execution of multi-target tracking tasks through air-ground collaboration.

[0158] The embodiment of the present invention further provides a multi-target tracking system, comprising:

[0159] Aircraft module, ground tracking module;

[0160] The aircraft module is used for the aircraft to obtain a tracking target image, detect the tracking target and obtain the pixel position of the tracking target image, and convert the tracking target image into an initial global map, and convert the pixel position of the tracking target image into a global position of the tracking target; obtain the global position of the ground tracking mechanism, and based on the pigeon flock algorithm and the auction mechanism, assign the tracking target to the ground tracking mechanism according to the initial global map, the global position of the tracking target, and the global position of the ground tracking mechanism, and complete the tracking target task assignment, wherein the number of the ground tracking mechanisms is greater than or equal to the number of the tracking targets; output the corresponding tracking target image and tracking target global position to the ground tracking mechanism based on the tracking target task assignment;

[0161] The ground tracking module is used to obtain the first-perspective image of the current ground tracking mechanism, and judge whether the tracking target is located in the first perspective of the ground tracking mechanism based on the tracking target image, the global position of the tracking target, and the first-perspective image of the current ground tracking mechanism. If the tracking target is not located in the first perspective of the ground tracking mechanism, the global position of the current tracking target is obtained and set as the end point of the first tracking trajectory of the ground tracking mechanism, and the first tracking trajectory is planned for the ground tracking mechanism; if the tracking target is located in the first perspective of the ground tracking mechanism, the pixel position of the tracking target image in the current first-perspective image is obtained and converted into the world coordinate system position of the tracking target, and the world coordinate system position of the tracking target is set as the end point of the second tracking trajectory of the ground tracking mechanism, and the second tracking trajectory is planned for the ground tracking mechanism; the planning of the first tracking trajectory and the planning of the second tracking trajectory are optimized and converted into control instructions, which are output to the ground tracking mechanism to perform the target tracking task.

[0162] An embodiment of the present invention also provides a multi-target tracking device, comprising: a memory, a processor, and a multi-target tracking program stored in the memory and executable on the processor, wherein the multi-target tracking program is configured to implement the steps of the above-mentioned dynamic multi-target tracking method.

[0163] An embodiment of the present invention further provides a storage medium, comprising: a multi-target tracking program is stored on the storage medium, and the multi-target tracking program implements the above-mentioned dynamic multi-target tracking method when executed by a processor.

[0164] Finally, it is to be noted that any modification or equivalent replacement of part or all of the technical features based on the device structure of the present invention and the technical solution of the described embodiment, which does not deviate from the essence of the corresponding technical solution of the present invention, belongs to the patent scope of the device structure of the present invention and the described implementation scheme.

Claims

1. A dynamic multi-target tracking method, characterized in that: include: The aircraft acquires a tracking target image, detects the tracking target and acquires a pixel position of the tracking target image, converts the tracking target image into an initial global map, and converts the pixel position of the tracking target image into a global position of the tracking target; Acquire the global position of the ground tracking mechanism, and based on the pigeon flock algorithm and the auction mechanism, assign the tracking target to the ground tracking mechanism according to the initial global map, the global position of the tracking target, and the global position of the ground tracking mechanism, to complete the tracking target task assignment, wherein the number of the ground tracking mechanisms is greater than or equal to the number of the tracking targets; Outputting the corresponding tracking target image and tracking target global position to the ground tracking mechanism based on the tracking target task allocation; Acquire the first-perspective image of the current ground tracking mechanism, and determine whether the tracking target is located in the first perspective of the ground tracking mechanism based on the tracking target image, the global position of the tracking target, and the first-perspective image of the current ground tracking mechanism; if the tracking target is not located in the first perspective of the ground tracking mechanism, acquire the current global position of the tracking target and set it as the end point of the first tracking trajectory of the ground tracking mechanism, and plan the first tracking trajectory for the ground tracking mechanism; if the tracking target is located in the first perspective of the ground tracking mechanism, acquire the pixel position of the tracking target image in the current first-perspective image and convert it into the world coordinate system position of the tracking target, set the world coordinate system position of the tracking target as the end point of the second tracking trajectory of the ground tracking mechanism, and plan the second tracking trajectory for the ground tracking mechanism; The planning of the first tracking trajectory and the planning of the second tracking trajectory are optimized and converted into control instructions, which are output to a ground tracking mechanism to execute a target tracking task.

2. A dynamic multi-target tracking method as claimed in claim 1, characterized in that: The tracking target is in a moving state.

3. A dynamic multi-target tracking method as claimed in claim 1, characterized in that: Based on the pigeon flock algorithm and the auction mechanism, the ground tracking mechanism is assigned a tracking target, and the steps of completing the tracking target task assignment include: Step S201, initializing the pigeon-like flock algorithm; Step S202, setting the position and speed of each pigeon, wherein each pigeon in the pigeon-like swarm algorithm is a tracking target task allocation scheme, the position of the pigeon is a set of bid prices of each ground tracking mechanism relative to each tracking target, and the speed of the pigeon is an update direction of the bid price, wherein the initial position and initial speed of the pigeon are randomly set; Step S203, based on the bidding price, the ground tracking mechanisms are sorted from low to high, and the ground tracking mechanisms with low bidding prices are preferentially assigned the tracking targets corresponding to the bidding prices. When N≤p, the ground tracking mechanisms and the tracking targets are not repeatedly assigned. When N>p, the tracking targets are assigned to the p+1th ground tracking mechanisms to the Nth ground tracking mechanisms in the order of bidding prices from low to high, so as to realize the task assignment between the ground tracking mechanisms and the tracking targets, wherein N is the number of ground tracking mechanisms, p is the code of the tracking target, N and p are both natural numbers, and N>p; Step S204, judging whether T+1 is greater than or equal to m, if T+1 is greater than or equal to m, outputting the tracking target task assignment, if T+1 is less than m, executing step S206, wherein T is the number of iterations of the pigeon group, and m is the set pigeon group iteration threshold; Step S205, obtaining a tracking distance between the ground tracking mechanism and the assigned tracking target based on the tracking target task assignment, the initial global map, the tracking target global position, and the ground tracking mechanism global position; Step S206, obtaining a fitness value of each pigeon according to the fitness function and the tracking distance between the ground tracking mechanism and the assigned tracking target; Step S207, determine whether T+1 is greater than or equal to n, if T+1 is less than n, then execute step S208, if T+1 is greater than or equal to n, then execute step S209, where T is the number of iterations of the pigeon group, n is the set iteration threshold of the map and compass operator, n<m; Step S208, updating the position of the pigeon flock using a map and a compass operator according to the fitness value of each pigeon; Step S209, based on the fitness value of each pigeon, the location of the pigeon flock is updated using a landmark operator.

4. A dynamic multi-target tracking method as claimed in claim 3, characterized in that: The fitness function is: minF=λ1f1+λ2f2+λ3f3, Among them, λ1, λ2 and λ3 are weight parameters, f1 is the total tracking distance, f2 is the maximum tracking distance and f3 is the tracking number distribution; The total tracking distance f1 is the sum of the tracking distances of all ground tracking mechanisms and the assigned tracking targets, which is: Wherein, N is the number of ground tracking mechanisms, Cpi is the path distance from the jth ground tracking mechanism to the tracking target, j is the code of the ground tracking mechanism, and j is a natural number; The maximum tracking distance f2 is the maximum tracking distance between the ground tracking mechanism and the assigned tracking target, which is: minf2=max c pj , Among them, C pj is the path distance from the jth ground tracking mechanism to the tracking target; The tracking quantity distribution f3 is the average value of the number of ground tracking mechanisms allocated to each tracking target, which is: Where M is the number of tracked targets, np is the number of ground tracking mechanisms assigned to the pth tracked target, The average number of ground tracking agencies assigned to the same tracking target.

5. A dynamic multi-target tracking method as claimed in claim 3, characterized in that: The formula for updating the position of the pigeon flock by the map and compass operator is: v i+1 =v i ·e -R·n +rd·(x best -x i ) x i+1 =x i +v i+1 Among them, v i is the pigeon speed, x i is the pigeon position, R is a constant, n is the number of iterations of the map and compass operators, rd is a random number between 0 and 1, x best is the optimal pigeon position, e -R·n is the attenuation factor, i is the pigeon code, and i is a natural number; The formula for updating the position of the pigeon flock by the landmark operator is: x i+1 =x i +rd·(X c -x i ) Where X c is the landmark position, x i is the position of the pigeon, P is the total number of pigeons in the flock, and F is the fitness value of each pigeon.

6. A dynamic multi-target tracking method as claimed in claim 1, characterized in that: The ground tracking mechanism acquires a corresponding tracking target, and the step of determining whether the tracking target is located at a first viewing angle of the ground tracking mechanism comprises: Step S401, the ground tracking mechanism acquires the tracking target image; Step S402, copying the acquired tracking target image, one copy as the tracking target initial image, and one copy as the tracking target dynamic image, and splicing the tracking target initial image and the tracking target dynamic image to obtain a tracking target comparison image; Step S403, the ground tracking mechanism obtains the first-viewing angle image of the current ground tracking mechanism, obtains the tracking target image in the current first-viewing angle image based on the first-viewing angle image of the current ground tracking mechanism, and compares the tracking target image in the current first-viewing angle image with the tracking target comparison image until the tracking target is located at the first viewing angle of the ground tracking mechanism, and then confirms that the tracking target is located at the first viewing angle of the ground tracking mechanism; Step S404: the ground tracking mechanism updates the tracking target dynamic image to the tracking target image in the current first-view image at regular intervals.

7. A dynamic multi-target tracking method as claimed in claim 1, characterized in that: The aircraft includes one of a fisheye lens and a wide-angle lens.

8. A multi-target tracking system, characterized in that: include: Aircraft module, ground tracking module; The aircraft module is used for the aircraft to obtain a tracking target image, detect the tracking target and obtain the pixel position of the tracking target image, and convert the tracking target image into an initial global map, and convert the pixel position of the tracking target image into a global position of the tracking target; obtain the global position of the ground tracking mechanism, and based on the pigeon flock algorithm and the auction mechanism, assign the tracking target to the ground tracking mechanism according to the initial global map, the global position of the tracking target, and the global position of the ground tracking mechanism, and complete the tracking target task assignment, wherein the number of the ground tracking mechanisms is greater than or equal to the number of the tracking targets; output the corresponding tracking target image and tracking target global position to the ground tracking mechanism based on the tracking target task assignment; The ground tracking module is used to obtain the first-perspective image of the current ground tracking mechanism, and judge whether the tracking target is located in the first perspective of the ground tracking mechanism based on the tracking target image, the global position of the tracking target, and the first-perspective image of the current ground tracking mechanism. If the tracking target is not located in the first perspective of the ground tracking mechanism, the global position of the current tracking target is obtained and set as the end point of the first tracking trajectory of the ground tracking mechanism, and the first tracking trajectory is planned for the ground tracking mechanism; if the tracking target is located in the first perspective of the ground tracking mechanism, the pixel position of the tracking target image in the current first-perspective image is obtained and converted into the world coordinate system position of the tracking target, and the world coordinate system position of the tracking target is set as the end point of the second tracking trajectory of the ground tracking mechanism, and the second tracking trajectory is planned for the ground tracking mechanism; the planning of the first tracking trajectory and the planning of the second tracking trajectory are optimized and converted into control instructions, which are output to the ground tracking mechanism to perform the target tracking task.

9. A multi-target tracking device, characterized in that: include: A memory, a processor, and a multi-target tracking program stored in the memory and executable on the processor, wherein the multi-target tracking program is configured to implement the steps of the dynamic multi-target tracking method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: include: The storage medium stores a multi-target tracking program, which, when executed by a processor, implements the dynamic multi-target tracking method according to any one of claims 1 to 7.

Images