Unmanned aerial vehicle vision multi-target tracking method based on trend change

Abstract

The application belongs to the technical field of target tracking, and specifically relates to a UAV visual multi-target tracking method based on trend change. First, a target detection model is trained, and the trained target detection model is used for target detection on a UAV visual image; then, it is judged whether compensation of a track is needed; if needed, a global offset is calculated to compensate the track, and after compensation, track and target matching is performed; if not needed, matching is directly performed; then, the track and the target are matched, and a matching result is mapped to a target detection frame to obtain a multi-target tracking result; finally, a trend change adaptive Kalman filter is used to update each track, and the updated track is used for multi-target tracking of a next frame of UAV visual image. The method proposes a trend change factor capable of reflecting a target state change trend, proposes a trend change adaptive Kalman filter, a trend change grouping matching strategy and a fast state compensation mechanism based on the trend change factor, effectively solves the problems of mutual occlusion, view angle change and camera jitter, and further improves tracking effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-target tracking using computer vision, and in particular is a method for multi-target tracking using unmanned aerial vehicle vision based on trend changes. Background Art

[0002] Visual object tracking, a key research area in computer vision, has been widely applied in fields such as autonomous driving, video surveillance, and drone aerial tracking. Existing multi-object tracking methods mostly follow a detection-based tracking paradigm. Due to the lack of clues such as trend change information, they often require the addition of additional neural network models and computational modules to the data association step, or the improvement of relatively independent existing modules. These approaches often incur additional training costs or provide limited performance improvements.

[0003] Furthermore, current multi-target tracking methods primarily focus on conventional multi-target tracking scenarios. However, target tracking in drone scenarios primarily focuses on directly improving tracking performance by optimizing the general performance of multi-target tracking. Compared to conventional multi-target tracking scenarios, multi-target tracking in drone vision has its own particularities. For example, in most drone scenarios, drones often remain at high altitudes and are constantly moving, subject to mutual occlusion, camera shake, and changes in perspective. Furthermore, drone devices themselves have limitations, resulting in existing multi-target tracking methods often failing to achieve a good balance between detection and tracking, and thus failing to achieve excellent tracking performance.

[0004] For camera motion compensation, traditional methods include ORB, SIFT, etc., and there are also deep learning methods that apply optical flow detection models for compensation. These methods are all direct responses to surface problems, and traditional methods are not fast, while deep learning methods often require additional training costs.

[0005] Based on the above problems, the present invention conducts an in-depth analysis of the characteristics of multi-target tracking under UAV vision and proposes a UAV vision multi-target tracking method based on trend changes. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the technical problem to be solved by the present invention is to provide a UAV visual multi-target tracking method based on trend changes.

[0007] The present invention solves the technical problem by adopting the following technical solutions:

[0008] A method for tracking multiple targets using UAV visual system based on trend change, comprising the following steps:

[0009] The first step is to train a target detection model, and use the trained target detection model to detect targets in the UAV visual image; according to the confidence level, the targets are divided into high confidence targets and low confidence targets, and a high confidence target set and a low confidence target set are obtained;

[0010] The second step is to determine whether the trajectory needs to be compensated; if so, the global offset is calculated to compensate the trajectory, and after the compensation is completed, the third step is entered; if not, the third step is directly entered;

[0011] The global offset The calculation formula is as follows:

[0012] (1)

[0013] (2)

[0014] (3)

[0015] (4)

[0016] Among them, is the reliability weight of the i th matched successful trajectory, , is the matching cost and confidence of the i th matched successful trajectory, and is a hyperparameter, , is the confidence change value and matching cost change value of the i th matched successful trajectory, , is the coordinate offset and size offset of the i th matched successful trajectory, and n is the number of matched successful trajectories, is the confidence of the i th trajectory at time t-1, t-2, , is the matching cost of the i th trajectory at time t-1, t-2;

[0017] The third step is to match the trajectory with the target, and map the matching result to the target detection frame to obtain the multi-target tracking result;

[0018] The fourth step is to update each trajectory respectively using the trend change adaptive Kalman filter, and the updated trajectory is used for multi-target tracking of the next frame of UAV visual image;

[0019] The trend change adaptive Kalman filter updates the trajectory according to the tracking result of the current frame of UAV visual image, and the noise covariance update formula is:

[0020] (6)

[0021] (7)

[0022] in, is the noise covariance of the adaptive Kalman filter for the i-th trend change at time t, is the influence factor of the i-th trajectory at time t, is a hyperparameter; 、 、 and are the noise covariance of the abscissa, ordinate, width, and height of the i-th track at time t. If the track matches the target successfully, the abscissa, ordinate, width, and height of the track are the abscissa, ordinate, width, and height of the target; if the track does not match successfully, the abscissa, ordinate, width, and height of the track remain at the current values.

[0023] Furthermore, for the matching of trajectories and targets, the priority weight of the trajectory is calculated according to formula (5). The smaller the priority weight, the higher the priority of the trajectory;

[0024] (5)

[0025] in, It is The priority weight of each trajectory;

[0026] Set the number of matches to C, the priority range to Z, initialize the set of matching trajectories to empty, and the set of non-matching trajectories to the set of all trajectories;

[0027] Match the trajectories according to their priorities. For the kth match, add the trajectories with priority weights less than k×Z to the set of participating matching trajectories and remove them from the set of unmatched trajectories. Match each trajectory in the set of participating matching trajectories with each high-confidence target in the set of high-confidence targets. If the intersection-over-union ratio of the trajectory and the high-confidence target is greater than the set threshold, the match is successful. The successfully matched trajectory is removed from the set of participating matching trajectories, and the unmatched trajectory is retained in the set of participating matching trajectories to continue to participate in the next match. This cycle of matching is repeated C-1 times.

[0028] For the last match, the current set of unmatched trajectories is merged with the set of matched trajectories to obtain a merged set of unmatched trajectories; the unmatched high-confidence targets in the high-confidence target set are merged with the low-confidence target set to obtain a mixed set of high and low-confidence targets; each target in the mixed set of high and low-confidence targets is combined with each trajectory in the merged set of unmatched trajectories to obtain multiple candidate matching pairs; the intersection-over-union ratio of each candidate matching pair is calculated, and the intersection-over-union ratio is used as a matrix element to obtain a matching cost matrix; for low-confidence targets, if the intersection-over-union ratio of the candidate matching pair where the low-confidence target is located is less than the filtering threshold, it is an unreliable candidate matching pair, and the intersection-over-union ratio of the unreliable candidate matching pair is set to zero, and the matching cost matrix is ​​updated. All candidate matching pairs where the low-confidence targets are located are traversed to obtain an updated matching cost matrix;

[0029] According to the updated matching cost matrix, the Hungarian algorithm is used to match the trajectory with the target, and then the matching of all trajectories is completed.

[0030] Furthermore, for the setting of the filtering threshold, all low-confidence targets are grouped according to the confidence interval of the low-confidence target and the set confidence interval interval, and a filtering threshold is set for each group according to the value range of the confidence interval; the higher the confidence group, the lower the filtering threshold.

[0031] Furthermore, the process of determining whether a trajectory needs to be compensated is as follows: 1) screening the trajectories to select stable ones; 2) matching the screened trajectories with each high-confidence target in the high-confidence target set. If the intersection-over-union ratio of the trajectory with the high-confidence target is greater than a set threshold, the match is successful; the average matching cost of all successfully matched trajectories is calculated; if the ratio of the average matching cost of the successfully matched trajectories at the previous moment to the current moment is less than or equal to the set threshold, no compensation is required; otherwise, compensation is required.

[0032] Furthermore, the screening condition of the trajectory is at least one of confidence, matching cost and change amplitude.

[0033] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0034] 1. The present invention conducts an in-depth analysis of the drone scene, observes the characteristics of the target movement in the scene, and adds the matching cost as a new clue to the attributes of the trajectory. The matching cost is the motion similarity between the trajectory and the target when it was most recently successfully matched, and uses it and the confidence of the trajectory to construct a trend change factor. The trend change factor includes a confidence change value and a matching cost change value. Because the change in confidence can intuitively reflect the change in the clarity of the target for the target detection model, and the change in matching cost can reflect the stability of the target movement, the trend change factor can reflect the trend of target state changes. The trend change factor can not only simply and intuitively reflect the change in the target state, but can also adapt to different situations after expansion, more effectively estimate the target state, and further determine whether the target is about to be occluded by a large area or about to emerge from the occlusion. This method can extract and utilize the potential information of the target movement without the need for complex deep learning.

[0035] 2. The present invention improves and constructs a Kalman filter and matching strategy by introducing a trend change factor, and constructs a fast and effective fast compensation mechanism, thereby combining the entire data association step into an organic whole, thereby achieving multi-target tracking with excellent speed and superior performance. The trend change adaptive Kalman filter of the present invention combines the trend change factor and the analysis of the characteristics of the drone scene to improve its motion perception ability, thereby enhancing the motion prediction performance. The trend change group matching strategy uses the trend change factor to enhance the distinguishability of the trajectory, and groups the trajectories precisely and effectively, thereby enhancing the matching accuracy and reducing the tracking error caused by the mutual occlusion of dense targets. The fast compensation mechanism adopts the idea of ​​the optical flow method, but does not use numerous pixels with low average information validity as the analysis target, but uses a small number of information-rich trajectories as the analysis target, and adjusts the compensation intensity in conjunction with the trend change factor to achieve fast and effective camera motion compensation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the overall multi-target tracking framework of the present invention;

[0037] Figure 2 is a schematic diagram of the rapid compensation mechanism of the present invention; DETAILED DESCRIPTION

[0038] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific implementations, but this does not limit the scope of protection of the present application.

[0039] The present invention is a method for tracking multiple targets of UAV vision based on trend change (hereinafter referred to as method, see Figures 1 and 2 ), including the following steps:

[0040] Step 1: Train the target detection model and use the trained target detection model to detect targets in the UAV visual image to obtain multiple targets; divide the targets into high-confidence targets and low-confidence targets according to the confidence level to obtain a high-confidence target set. and low confidence target set .

[0041] Step 2: Based on the Fast State Compensation (FSC) mechanism, determine whether the trajectory needs to be compensated. If so, calculate the global offset and compensate the trajectory based on the global offset. After compensation is completed, proceed to step 3. If not, proceed directly to step 3.

[0042] The fast state compensation mechanism of the present invention is used to compensate for the errors caused by camera shake due to the characteristics of the drone itself, and has the characteristics of fast operation and no need for additional hardware equipment. FSC has a good effect in suppressing camera shake caused by the special motion trajectory of the drone. The main theoretical basis of FSC is similar to the optical flow method, which uses the changes in a large number of pixels in adjacent frames to reflect the movement of the object. FSC only quantifies camera shake by calculating the offset between the trajectory and detection, thereby avoiding a large number of pixel calculations, and compared with pixels, the trajectory in the multi-target tracking scene has richer and easier to understand states, such as confidence, size, position and survival time. Therefore, FSC can achieve camera motion compensation in a simple and efficient manner. The FSC mechanism is roughly divided into four steps: screening, matching, calculation and compensation.

[0043] 1) Screen the trajectories to select stable ones. FSC uses high confidence, low matching cost, and low variation as screening conditions. The conditions can be in an OR or AND relationship. For example, if the confidence of a trajectory is greater than 0.7, the matching cost of the trajectory is less than 0.37, or the number of consecutive successful matches of the trajectory (also called trajectory length) is greater than 7, then the trajectory is considered stable.

[0044] 2) Combine the filtered trajectories with the high confidence target set Match each high-confidence target in the trajectory. If the intersection-over-union ratio of the trajectory and the high-confidence target is greater than the set threshold, the match is successful; calculate the average matching cost of all successfully matched trajectories The smaller the value, the smaller the camera shake and the more stable the drone state. If the ratio of the average matching cost of the successful trajectories at adjacent moments is / If the average matching cost ratio is less than or equal to the set threshold, it means that the camera shake is small and the trajectory does not need to be compensated. If the average matching cost ratio is greater than the set threshold, it means that the camera shake is greater and the trajectory needs to be compensated, then go to step 3);

[0045] 3) Calculate the global offset , compensate the trajectory according to the global offset; calculate the reliability weight of the successfully matched trajectory based on the trend change factor The larger the value, the more obvious the impact of the successfully matched trajectory on the global offset. The global offset is calculated according to the reliability weight, and the calculation formula is as follows:

[0046] (1)

[0047] (2)

[0048] in, is the reliability weight of the i-th successfully matched trajectory, 、 is the matching cost and confidence of the i-th successfully matched trajectory, and is a hyperparameter, 、 is the confidence change value and matching cost change value of the i-th successfully matched trajectory, 、 is the coordinate offset and size offset of the i-th successfully matched trajectory, and n is the number of successfully matched trajectories;

[0049] The confidence change value of the i-th trajectory and matching cost change Satisfy the following formulas respectively:

[0050] (3)

[0051] (4)

[0052] in, is the confidence of the i-th trajectory at time t-1 and t-2, 、 is the matching cost of the i-th trajectory at time t-1 and t-2; in the initial stage, both values ​​are initialized to 0.

[0053] Add a global offset to the coordinates and dimensions of each track , completing trajectory compensation, thereby suppressing the impact of camera shake.

[0054] Step 3: Match the trajectory with the target according to the trend change group matching strategy to achieve multi-target tracking;

[0055] The application proposes a Trend Grouping Matching Strategy (TGM) based on trend change, which improves matching accuracy and alleviates errors caused by mutual occlusion in target dense scenes. The strategy evaluates the priority of trajectories based on the trend change factor, and then groups and matches the trajectories according to the priority. Compared with other conventional grouping matching methods, the grouping based on the trend change factor is more effective, not limited to the distinction between targets in a single frame image, but more concerned about the potential trend of motion change between frames, so that the targets have better distinguishability. High-priority trajectories will be matched earlier, which can effectively ensure that the trajectories are matched to the correct targets, and the unmatched trajectories in the high-priority trajectory group will participate in the matching process of the subsequent low-priority trajectory group. In summary, TGM relies on the trend change factor to accurately assign priority according to the stability of the trajectory state, ensuring the accuracy of relatively stable object matching and alleviating the mutual occlusion problem in unmanned aerial scenes.

[0056] Calculate priority weight based on trend change factor , used to evaluate the priority of the trajectory, the smaller the priority weight, the higher the priority of the trajectory; the priority weight calculation formula is:

[0057] (5)

[0058] wherein, is the priority weight of the i-th trajectory;

[0059] It can be seen from formula (5) that the matching cost of the trajectory is is the main parameter, the matching cost change value is the main correction parameter, and the confidence change value is the fluctuation factor of ; compared with a single matching cost , the priority weight combines the trend change factor between frames, which can more effectively measure the stability of the unmanned aerial vehicle state and improve the matching accuracy of trajectory grouping. For the priority of the trajectory, each priority is assigned a value range, and when the priority weight of the trajectory is within the priority value range, the trajectory will be assigned to the same priority.

[0060] Set the matching number as C, the priority value range size as Z, the trajectory set participating in matching as initialized as empty, and the trajectory set not participating in matching as initialized as the set formed by all trajectories; match the trajectories according to the priority, and for the k-th matching, add the trajectories with a priority weight less than k×Z to the trajectory set participating in matching ​and never participated in the matching trajectory set Eliminate these trajectories from the matching trajectory set Each trajectory in the set is matched with each high-confidence target in the set of high-confidence targets. If the intersection-over-union ratio of the trajectory and the high-confidence target is greater than the set threshold, the trajectory is considered to be successfully matched with the high-confidence target and is removed from the set of participating matching trajectories. The successfully matched trajectories are removed from the set, while the unmatched trajectories are retained in the set of participating matching trajectories. Continue to participate in the next matching, and cycle matching C-1 times. For the last matching, merge the current unmatched trajectory set with the matched trajectory set to obtain the merged unmatched trajectory set; at the same time, merge the unmatched high-confidence targets in the high-confidence target set with the low-confidence target set. Merge to obtain a mixed set of high and low confidence targets; perform a final match on the trajectories in the merged set of unmatched trajectories and the targets in the mixed set of high and low confidence targets; first, combine each target in the mixed set of high and low confidence targets with each trajectory in the merged set of unmatched trajectories to obtain multiple candidate matching pairs; calculate the intersection-over-union ratio of each candidate matching pair, and use the intersection-over-union ratio as a matrix element to obtain a matching cost matrix; for low confidence targets, filter unreliable candidate matching pairs according to the low confidence matching screening strategy, set the intersection-over-union ratio of unreliable candidate matching pairs to zero, and update the matching cost matrix to obtain an updated matching cost matrix;

[0061] The low-confidence matching screening strategy is to traverse the intersection-over-union ratios of all candidate matching pairs where low-confidence targets are located. If the intersection-over-union ratio is less than the filtering threshold, it is considered an unreliable candidate matching pair and the matching of the corresponding low-confidence target and the trajectory is not considered.

[0062] For low confidence targets, the conventional method usually matches them directly with limited active tracks, which is neither sufficient nor rigorous enough, and cannot fully explore the correlation between low confidence targets and tracks. According to the characteristics of low confidence targets, the matching rule is analyzed, and combined with the special performance of multi-target tracking in the unmanned aerial vehicle scene, a low confidence matching filtering strategy is proposed. The filtering threshold is according to the confidence interval of the low confidence target, and all low confidence targets are grouped according to the set confidence interval interval, and each group is set a filtering threshold according to the value range of the confidence interval; the higher the confidence of the group, the lower the filtering threshold; for example, if the confidence of the low confidence target is less than 0.6 and greater than 0.4, it is divided into two groups according to the confidence interval interval of 0.1, and the value range of the confidence interval is [0.4, 0.5) and [0.5, 0.6], and the filtering threshold is set to 0.6 and 0.5 respectively, so that the truly reliable low confidence target can be matched successfully, and the unreliable one can be filtered out, increasing the matching accuracy, reducing the error, and effectively improving the correlation between the low confidence target and the track.

[0063] According to the updated matching cost matrix, the Hungarian algorithm is used to match the tracks and targets, and then the track matching is completed, and the matching result is obtained; the matching result is mapped to the target detection frame to obtain the multi-target tracking result; thus, the multi-target tracking of the current frame of unmanned aerial vehicle vision image is completed.

[0064] Fourth step: respectively using trend change adaptive Kalman filter (Trend-NSA Kalman Filter, T-NSA) to update each track, and the updated track is used for multi-target tracking of the next frame of unmanned aerial vehicle vision image;

[0065] The trend change adaptive Kalman filter is an improvement of the adaptive Kalman filter. The adaptive Kalman filter uses the confidence of the detected object to adaptively adjust the noise covariance of the Kalman filter. However, this form is relatively simple and does not fully consider other potential influencing factors. Only relying on the confidence of the track as the influencing parameter, it is difficult to deal with complex and variable situations. In order to improve the prediction accuracy and robustness, the present application proposes a track influence factor based on trend change factor, which uses the track influence factor to affect the noise covariance of the Kalman filter, and then obtains the trend change adaptive Kalman filter, to better adapt to more complex situations in the unmanned aerial vehicle scene. The trend change adaptive Kalman filter updates the track according to the tracking result of the current frame of unmanned aerial vehicle vision image, and its noise covariance update formula is:

[0066] (6)

[0067] (7)

[0068] wherein, is the noise covariance of the t-th trend change adaptive Kalman filter of the i-th trajectory, is the influence factor of the t-th trajectory, is a hyperparameter; , , and are the noise covariances of the horizontal coordinate, the vertical coordinate, the width and the height of the t-th trajectory, respectively, if the trajectory matches the target successfully, the horizontal coordinate, the vertical coordinate, the width and the height of the trajectory are the horizontal coordinate, the vertical coordinate, the width and the height of the target, respectively; if the trajectory does not match successfully, the horizontal coordinate, the vertical coordinate, the width and the height of the trajectory remain the current values.

[0069] As can be seen from equations (6-7), the matching cost change value is used as the cost change coefficient of the trajectory, and the confidence change value is used as the confidence change coefficient of the trajectory, so that the Kalman filter can perceive the trend change of the trajectory under the control of the noise covariance, and further optimize the motion prediction accuracy; in addition, the trajectory influence factor only affects the noise covariance of the horizontal coordinate and the noise covariance of the vertical coordinate, because under the perspective of the unmanned aerial vehicle, the object usually does not deform significantly in a short time, and therefore, excessive influence on the noise covariance of the width and the noise covariance of the height will lead to performance degradation.

[0070] Embodiment

[0071] In this embodiment, the variant edgeyolo of the yolox model is used as the target detection model, experiments are performed on two datasets, VisDrone2019 and UAVDT, respectively, and the method of the present application is compared with the current advanced multi-target tracking method, and the experimental results are shown in Table 1. Among them, the average number ratio takes into account the number of correctly matched targets and the number of unmatched targets, and is used to evaluate the tracking performance at the target level, and the larger the value, the longer the time for accurate tracking of a target. The multi-target tracking accuracy refers to the distance between the predicted trajectory and the true trajectory, reflecting the accuracy of the tracking result, and the larger the value, the higher the tracking accuracy. The multi-target tracking accuracy comprehensively considers factors such as missed detection, false detection and trajectory error, and is one of the main indicators for evaluating the performance of multi-target tracking, and the larger the value, the better the tracking performance.

[0072]

[0073] Experimental results show that compared to current state-of-the-art multi-target tracking methods, the proposed method demonstrates superior performance on both datasets while maintaining good speed. This is due to an in-depth analysis of target tracking characteristics in drone scenarios, which leads to the proposal of a trend change factor that reflects the target state change trend. Furthermore, based on this trend change factor, a trend change adaptive Kalman filter, a trend change group matching strategy, and a fast state compensation mechanism are proposed, effectively addressing issues such as mutual occlusion, viewpoint changes, and camera shake. This method not only improves the accuracy of target motion state prediction but also achieves overall improvements in speed and performance. Specifically, the trend change adaptive Kalman filter enhances trajectory stability by introducing a trend change factor, thereby improving motion prediction accuracy. The trend change group matching strategy constructs priority weights based on trend changes, thereby expanding a new dimension to further differentiate trajectories and effectively reducing matching errors in the case of mutual occlusion. Finally, the fast state compensation mechanism addresses the error amplification caused by camera shake through simple and efficient camera motion compensation. Furthermore, the proposed method only requires training of the target detection model, eliminating the need for additional training costs, further highlighting its superiority.

[0074] The sources of the above methods are as follows:

[0075] [1]Liu S, Li X, Lu H, et al. Multi-object tracking meets moving UAV[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and PatternRecognition. 2022: 8876-8885.

[0076] [2]Cao J, Pang J, Weng X, et al. Observation-centric sort: Rethinkingsort for robust multi-object tracking[C] / / Proceedings of the IEEE / CVFconference on computer vision and pattern recognition. 2023: 9686-9696.

[0077] [3]Yao M, Wang J, Peng J, et al. FOLT: Fast Multiple Object Trackingfrom UAV-captured Videos Based on Optical Flow[C] / / Proceedings of the 31stACM International Conference on Multimedia. 2023: 3375-3383.

[0078] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A UAV visual multi-target tracking method based on trend change, characterized in that: The steps include: The first step is to train the target detection model and use it to detect targets in the UAV visual image. The targets are divided into high-confidence targets and low-confidence targets according to their confidence levels, thus obtaining a high-confidence target set and a low-confidence target set. Step 2: Determine whether the trajectory needs to be compensated; if necessary, calculate the global offset to compensate the trajectory, and then proceed to step 3 after compensation is completed; If not required, proceed directly to step 3; Global offset The calculation formula is as follows: (1) (2) (3) (4) in, is the reliability weight of the i-th successfully matched trajectory, 、 is the matching cost and confidence of the i-th successfully matched trajectory, and is a hyperparameter, 、 is the confidence change value and matching cost change value of the i-th successfully matched trajectory, 、 is the coordinate offset and size offset of the i-th successfully matched trajectory, n is the number of successfully matched trajectories, is the confidence of the i-th trajectory at time t-1 and t-2, 、 is the matching cost of the i-th trajectory at time t-1 and t-2; Step 3: Match the trajectory with the target, map the matching result to the target detection frame, and obtain the multi-target tracking result; Step 4: Use the trend change adaptive Kalman filter to update each trajectory. The updated trajectory is used for multi-target tracking in the next frame of UAV visual image. The trend change adaptive Kalman filter updates the trajectory according to the tracking result of the current frame UAV visual image, and its noise covariance update formula is: (6) (7) in, is the noise covariance of the adaptive Kalman filter for the i-th trend change at time t, is the influence factor of the i-th trajectory at time t, is a hyperparameter; 、 、 and are the noise covariance of the abscissa, ordinate, width, and height of the i-th track at time t. If the track matches the target successfully, the abscissa, ordinate, width, and height of the track are the abscissa, ordinate, width, and height of the target; if the track does not match successfully, the abscissa, ordinate, width, and height of the track remain at the current values.

2. The UAV visual multi-target tracking method based on trend change according to claim 1 is characterized in that: For the matching of trajectory and target, the priority weight of the trajectory is calculated according to formula (5). The smaller the priority weight, the higher the priority of the trajectory; (5) in, It is The priority weight of each trajectory; Set the number of matches to C, the priority range to Z, initialize the set of matching trajectories to empty, and the set of non-matching trajectories to the set of all trajectories; Match the trajectories according to their priorities. For the kth match, add the trajectories with priority weights less than k×Z to the set of participating matching trajectories and remove these trajectories from the set of unmatched trajectories. Match each trajectory in the set of participating matching trajectories with each high-confidence target in the set of high-confidence targets. If the intersection-over-union ratio of the trajectory and the high-confidence target is greater than the set threshold, the match is successful. The successfully matched trajectory is removed from the set of participating matching trajectories, and the unmatched trajectory is retained in the set of participating matching trajectories to continue to participate in the next match. This cycle of matching is repeated C-1 times. For the last match, the current set of unmatched trajectories is merged with the set of matched trajectories to obtain a merged set of unmatched trajectories; the unmatched high-confidence targets in the high-confidence target set are merged with the low-confidence target set to obtain a mixed set of high and low-confidence targets; each target in the mixed set of high and low-confidence targets is combined with each trajectory in the merged set of unmatched trajectories to obtain multiple candidate matching pairs; the intersection-over-union ratio of each candidate matching pair is calculated, and the intersection-over-union ratio is used as a matrix element to obtain a matching cost matrix; for low-confidence targets, if the intersection-over-union ratio of the candidate matching pair where the low-confidence target is located is less than the filtering threshold, it is an unreliable candidate matching pair, and the intersection-over-union ratio of the unreliable candidate matching pair is set to zero, and the matching cost matrix is ​​updated. All candidate matching pairs where the low-confidence targets are located are traversed to obtain an updated matching cost matrix; According to the updated matching cost matrix, the Hungarian algorithm is used to match the trajectory with the target, and then the matching of all trajectories is completed.

3. The UAV visual multi-target tracking method based on trend change according to claim 2 is characterized in that: For the setting of filtering threshold, all low-confidence targets are grouped according to the confidence interval of the low-confidence targets and the set confidence interval interval. A filtering threshold is set for each group according to the value range of the confidence interval; the higher the confidence group, the lower the filtering threshold.

4. The method for tracking multiple targets using UAV vision based on trend change according to any one of claims 1 to 3, characterized in that: The process of determining whether a trajectory needs compensation is as follows: 1) screening the trajectories to select stable ones; 2) matching the selected trajectories with each high-confidence target in the high-confidence target set. If the intersection-over-union ratio of the trajectory with the high-confidence target is greater than a set threshold, the match is successful; the average matching cost of all successfully matched trajectories is calculated; if the ratio of the average matching cost of the successfully matched trajectories at the previous moment to the current moment is less than or equal to the set threshold, no compensation is required; otherwise, compensation is required.

5. The UAV visual multi-target tracking method based on trend change according to claim 4 is characterized in that: The screening condition of the trajectory is at least one of confidence, matching cost and change range.

Images