Multi-target tracking method, device, equipment and medium

Abstract

The invention discloses a multi-target tracking method, device and equipment and a medium, and relates to the technical field of target tracking, and the method reduces the number of times of appearance feature extraction as much as possible through a provided WPFE selection mechanism, and improves the real-time performance of a multi-target tracking algorithm on a quadruped robot with limited resources. Meanwhile, the appearance feature library is updated through a provided feature attenuation strategy, so that the importance of the appearance features is exponentially decreased along with the increase of the frame number. And when the appearance characteristics change under the shielding condition, matching is carried out by depending on new appearance characteristics, and the accuracy of the multi-target tracking algorithm under complicated conditions such as shielding is not reduced.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a multi-target tracking method, apparatus, device, and medium for quadruped robots. Background Technology

[0002] Quadruped robots possess biomimetic leg locomotion structures, enabling them to adapt to various terrains with remarkable stability, navigating rugged mountains, rubble, and narrow streets. When performing tasks, they can penetrate dangerous areas to complete reconnaissance, material transport, casualty search and rescue, and explosive ordnance disposal, playing a crucial role in industrial inspection and disaster relief. Real-time tracking of multiple targets in complex environments is a vital component of a quadruped robot's reconnaissance and perception capabilities, and a fundamental ability required for tasks such as material transport and casualty search and rescue. However, quadruped robots have limited computing resources and memory, placing high demands on the real-time performance of multi-target tracking algorithms deployed on them.

[0003] In recent years, with the continuous development of deep learning and target detection technology, the research on multi-target tracking technology has achieved good results, and detection-based multi-target tracking methods have been widely used. Detection-based multi-target tracking methods divide tracking into two steps: (1) detecting targets frame by frame using a detector to generate detection boxes; (2) data matching: matching the detection boxes of the same target between frames to form the motion trajectories of various targets.

[0004] However, multi-target tracking algorithms face many challenges in complex environments. For example, when tracking small targets, the limited information about the target in the image and interference from the complex environment can easily lead to target loss or incorrect target ID matching. Simultaneously, overlapping occlusion between targets, similar appearances, changes in ambient lighting, and camera shake can also cause target ID matching errors and frequent target ID switching. Furthermore, quadruped robots have limited computational resources, placing high demands on the real-time performance of multi-target tracking algorithms; poor real-time performance can also lead to target loss and target ID matching errors.

[0005] In 2017, N. Wojke et al. added an appearance feature extraction network to the SORT multi-object tracking algorithm, proposing the Deepsort algorithm, which improved the accuracy of the algorithm in complex scenes. The specific workflow of the Deepsort algorithm is as follows: Figure 3 As shown, it is mainly divided into the following three stages: prediction stage, observation stage, and update stage.

[0006] The program is divided into three phases: the prediction phase, which uses Kalman filtering to predict the target's trajectory based on motion information; the observation phase, which uses a target detector to confirm the target's actual position; and the update phase, which is responsible for updating the target's real-time status. The program begins in the prediction phase, using Kalman filtering to predict the target's position in the next video frame and providing two different predicted bounding boxes (confirmed and unconfirmed). When the predicted bounding box is in the confirmed state, it is cascaded with the target detection bounding box generated in the observation phase. The cascaded matching result is categorized into three types: successful match, predicted bounding box mismatch, and detection bounding box mismatch.

[0007] When a match is successful, the system proceeds to the update phase to confirm the latest position of the target in the image. When a match fails, the unmatched predicted bounding boxes, unmatched detection bounding boxes, and unconfirmed predicted bounding boxes undergo an Interchange of Units (IOU) match. If the match is successful, the system proceeds to the update phase. If the match fails, it is determined whether the unmatched predicted bounding boxes are due to occlusion or the detection bounding boxes are due to the presence of a new target.

[0008] To address the issue of tracking failures caused by the emergence of new targets, Deepsort employs a continuous confirmation mechanism. First, a new predicted bounding box is created for the new target and set to an unconfirmed state. If, after processing three consecutive video frames, the predicted bounding box successfully matches the detected bounding box, then the predicted bounding box is updated to a confirmed state.

[0009] To address the issue of mismatched predicted bounding boxes caused by occlusion by other objects, the Deepsort algorithm first determines the state of the predicted bounding box. If it is in an unconfirmed state, it is directly deleted; if it is in a confirmed state, the maximum frame drop time is determined. If the maximum frame drop time is exceeded, it means the object has left the frame, and the trajectory is directly deleted; otherwise, it is re-tracked.

[0010] The Strongsort algorithm is currently the most commonly used multi-object tracking algorithm in industry. It adopts the tracking framework of the Deepsort algorithm and makes many improvements on it. The improvements of the Strongsort algorithm over the Deepsort algorithm are mainly reflected in the appearance branch, motion branch, and matching strategy. The specific workflow of the Strongsort algorithm is as follows: Figure 4As shown, in the appearance branch, the Strongsort algorithm uses a BoT feature extractor built from a ResNeSt50 network to replace the simple convolutional neural network structure in the Deepsort algorithm; simultaneously, after extracting appearance features, it uses an EMA feature update strategy to replace the feature information database in the Deepsort algorithm. In the motion branch, the Strongsort algorithm first uses the ECC algorithm for camera motion compensation, and then uses the NSA Kalman filter algorithm to replace the traditional Kalman filter algorithm in the Deepsort algorithm. Regarding the matching strategy, the Strongsort algorithm uses a global linear matching strategy instead of the cascaded matching strategy in the Deepsort algorithm, thereby improving the accuracy of model matching.

[0011] Multi-target tracking algorithms such as Deepsort and Strongsort extract the appearance features of targets using a deep neural network, and then combine these appearance features with motion information for matching. This improves the accuracy of tracking algorithms in complex environments and reduces the number of target ID switching. However, the appearance feature extraction step relies on computationally expensive convolutional neural networks. When executed on a quadruped robot embedded development board with limited computing resources, the FPS (frames per second) is low, making it difficult to meet the requirements of real-time tracking.

[0012] It is evident that both Deepsort and Strongsort improve the tracking accuracy of the algorithm in complex scenarios such as occlusion by extracting appearance features. However, the appearance feature extraction network has high requirements for computing resources, and its real-time performance is poor when applied to quadruped robots with limited computing resources.

[0013] Some speed-intensive methods, such as OC-Sort, neglect the appearance feature extraction step and rely solely on motion information for matching. However, these methods exhibit poor tracking accuracy in complex scenes with occlusion.

[0014] Therefore, how to provide a multi-target tracking algorithm that balances tracking accuracy and real-time performance in complex scenarios is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0015] In view of the above problems, the present invention provides a multi-target tracking method, apparatus, device and medium for overcoming the above problems or at least partially solving the above problems.

[0016] This invention provides the following solution:

[0017] A multi-target tracking method, comprising:

[0018] Obtain the detection bounding box output by the detector and all trajectory points that have been successfully matched at least once before;

[0019] Calculate the aspect ratio similarity between the detection box and all the trajectory points;

[0020] Determine the relationship between the aspect ratio similarity and the similarity threshold;

[0021] After determining that the aspect ratio similarity is greater than the similarity threshold, the detection box is determined to be a non-risk detection. Based on the Strongsort multi-target tracking algorithm, the target corresponding to the detection box is tracked by matching only motion information.

[0022] After determining that the length-width similarity ratio is less than the similarity threshold, the detection box is determined to be a risk detection. Based on the Strongsort multi-target tracking algorithm, an appearance feature extraction step is performed to obtain appearance features. The appearance features and motion information are then combined to match and track the target corresponding to the detection box.

[0023] Preferably: the detection box is determined to be a non-risk detection, and the appearance feature library of the trajectory remains unchanged;

[0024] Once the detection box is identified as a risk detection, the appearance feature library of the trajectory is updated.

[0025] Preferably, the appearance feature library for updating the trajectory includes:

[0026] The importance of appearance features when not obscured in the early stages of attenuation.

[0027] Preferably, the importance of appearance features when attenuation is not obscured in the early stages includes:

[0028] Assign the current feature vector e t-1 A high weight α is assigned to the newly emerging feature f t A low weight 1-α is used to make the importance of the feature in the mean decrease exponentially with increasing frame number, as shown in the following equation:

[0029] e t =α.e t-1 +(1-α).f t .

[0030] Preferably, the exponent of the weight α is determined as the number of frames since the last appearance feature extraction;

[0031] When the appearance feature library of the trajectory remains unchanged, the currently updated weight α ， The exponent increases with the number of interval frames;

[0032] When updating the appearance feature library of the trajectory, the current weight α is used. ， Update the current appearance feature vector e tThe feature update formula is as follows:

[0033]

[0034] Preferably, the formula for calculating the aspect ratio similarity is as follows:

[0035]

[0036] In the formula: ABS represents aspect ratio similarity, w1 represents the length of detection box 1, h1 represents the width of detection box 1, w2 represents the length of detection box 2, and h2 represents the width of detection box 2.

[0037] A multi-target tracking device for performing the above-described multi-target tracking method, the device comprising:

[0038] The detection box acquisition unit is used to acquire the detection boxes output by the detector and all trajectory points that have been successfully matched at least once before;

[0039] A similarity calculation unit is used to calculate the aspect ratio similarity between the detection box and all the trajectory points;

[0040] The judgment unit is used to determine the relationship between the aspect ratio similarity and the similarity threshold;

[0041] The non-risk detection tracking unit is used to determine that the detection box is a non-risk detection after the similarity ratio of the length and width is greater than the similarity threshold. Based on the Strongsort multi-target tracking algorithm, the target corresponding to the detection box is tracked by matching only the motion information.

[0042] The risk detection and tracking unit is used to determine that the detection box is a risk detection after determining that the similarity ratio of the length and width is less than the similarity threshold. It performs an appearance feature extraction step based on the Strongsort multi-target tracking algorithm to obtain appearance features, and combines the appearance features and the motion information to match and track the target corresponding to the detection box.

[0043] A multi-target tracking device, the device including a processor and a memory:

[0044] The memory is used to store program code and transmit the program code to the processor;

[0045] The processor is used to execute the multi-target tracking method described above according to the instructions in the program code.

[0046] A computer-readable storage medium for storing program code for performing the multi-target tracking method described above.

[0047] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] This application provides a multi-target tracking method, apparatus, device, and medium. The method minimizes the number of appearance feature extractions through a provided WPFE selection mechanism, improving the real-time performance of the multi-target tracking algorithm on resource-constrained quadruped robots. Simultaneously, it updates the appearance feature library through a provided feature attenuation strategy, causing the importance of appearance features to decrease exponentially with increasing frame count. When appearance features change due to occlusion, matching is performed using the new appearance features, without reducing the accuracy of the multi-target tracking algorithm under complex conditions such as occlusion.

[0049] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0051] Figure 1 This is a flowchart of a multi-target tracking method provided in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart of the improved multi-target tracking algorithm provided in the embodiments of the present invention;

[0053] Figure 3 This is a flowchart of the Deepsort multi-target tracking algorithm provided in an embodiment of the present invention;

[0054] Figure 4 This is a flowchart of the Strongsort multi-target tracking algorithm provided in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of a multi-target tracking device provided in an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of a multi-target tracking device provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0058] See Figure 1 This invention provides a multi-target tracking method, such as... Figure 1 As shown, the method may include:

[0059] S101: Obtain the detection box output by the detector and all trajectory points that have been successfully matched at least once before;

[0060] S102: Calculate the aspect ratio similarity between the detection box and all the trajectory points;

[0061] S103: Determine the relationship between the aspect ratio similarity and the similarity threshold;

[0062] S104: After determining that the length-width similarity ratio is greater than the similarity threshold, the detection box is determined to be a non-risk detection. Based on the Strongsort multi-target tracking algorithm, the target corresponding to the detection box is tracked by matching only motion information.

[0063] S105: After determining that the length-width similarity ratio is less than the similarity threshold, the detection box is determined to be a risk detection. Based on the Strongsort multi-target tracking algorithm, the appearance feature extraction step is performed to obtain appearance features. The appearance features and the motion information are then combined to match and track the target corresponding to the detection box.

[0064] The multi-target tracking method provided in this application addresses the issue that the appearance feature extraction step in existing multi-target tracking algorithms affects the real-time performance of multi-target tracking in quadruped robots. Based on the Strongsort algorithm framework, it improves upon this method by proposing a selection mechanism for whether or not to perform feature extraction (WPFE selection mechanism). This mechanism determines whether or not to perform the appearance feature extraction step based on a length-width similarity ratio threshold. In the absence of risk detection, the appearance feature extraction step is skipped, thereby improving the real-time performance of the multi-target tracking algorithm on quadruped robots with limited computing resources.

[0065] To address the issue that the WPFE selection mechanism reduces the tracking accuracy of the algorithm under complex conditions such as occlusion, a feature attenuation strategy is designed to update the appearance feature library. This ensures that the accuracy of the multi-target tracking algorithm is not affected by the WPFE selection mechanism and maintains high accuracy even under complex conditions such as occlusion.

[0066] In specific implementation, the embodiments of this application can provide the ability to determine that the detection box is a non-risk detection while keeping the appearance feature library of the trajectory unchanged;

[0067] Once the detection box is identified as a risk detection, the appearance feature library of the trajectory is updated.

[0068] Furthermore, the appearance feature library for updating the trajectory includes:

[0069] The importance of appearance features when not obscured in the early stages of attenuation.

[0070] The importance of appearance features in the early stages of attenuation when the area is not occluded includes:

[0071] Assign the current feature vector e t-1 A high weight α is assigned to the newly emerging feature f t A low weight 1-α is used to make the importance of the feature in the mean decrease exponentially with increasing frame number, as shown in the following equation:

[0072] e t =α.e t-1 +(1-α).f t .

[0073] The exponent of the weight α is determined as the number of frames since the last appearance feature extraction.

[0074] When the appearance feature library of the trajectory remains unchanged, the currently updated weight α ， The exponent increases with the number of interval frames;

[0075] When updating the appearance feature library of the trajectory, the current weight α is used. ， Update the current appearance feature vector e t The feature update formula is as follows:

[0076]

[0077] The formula for calculating the aspect ratio similarity is as follows:

[0078]

[0079] In the formula: ABS represents aspect ratio similarity, w1 represents the length of detection box 1, h1 represents the width of detection box 1, w2 represents the length of detection box 2, and h2 represents the width of detection box 2.

[0080] The multi-target tracking method provided in this application adopts a WPFE selection mechanism. When the aspect ratio (ARS) between the detection box and a trajectory that has been successfully matched at least once before is greater than a set threshold, it indicates that the current scene is a non-risk detection, and high-accuracy matching can be achieved without performing an appearance feature extraction step. When the aspect ratio (ARS) is less than the set threshold, it indicates that the current scene is a risk detection, and an appearance feature extraction step is required, combining appearance features and motion information for matching.

[0081] By employing the WPFE selection mechanism, the computationally intensive appearance feature extraction step is skipped in non-risk detection scenarios, improving the real-time performance of multi-target tracking in quadruped robots. In risk detection scenarios, the tracking algorithm follows the same steps as Strongsort, still requiring computational resources for appearance feature extraction. Overall, the WPFE selection mechanism minimizes the number of appearance feature extractions, thereby reducing computational costs and improving the real-time performance of multi-target tracking in quadruped robots without sacrificing excessive accuracy.

[0082] The appearance feature library is updated using a feature attenuation strategy, causing the importance of appearance features to decrease exponentially with increasing frame rate. When appearance features change, matching is performed using new appearance features from later periods, without reducing tracking accuracy in complex scenes such as occlusion.

[0083] like Figure 2 As shown, improvements are made to the WPFE selection mechanism and feature update based on the Strongsort multi-target tracking algorithm framework. Figure 2 The improved part is shown inside the dashed box. By using the WPFE selection mechanism, the algorithm skips the number of appearance feature extractions that consume a lot of computing resources when there is no risk detection, thus improving the real-time performance of the algorithm on quadruped robots with limited computing resources. By using a feature attenuation strategy, the importance of early appearance features is reduced, without reducing the accuracy of the algorithm in complex situations such as occlusion.

[0084] (1) WPFE selection mechanism.

[0085] To determine whether an appearance feature extraction step is necessary, we introduce Aspect Ratio Similarity (ARS) as the criterion for the WPFE selection mechanism. The detector outputs the detected bounding boxes. For each bounding box, the selection mechanism calculates the Aspect Ratio Similarity (ARS) between that bounding box and all confirmed trajectory points. A confirmed trajectory is one that has successfully matched the bounding box at least once in a previous frame. If the ARS is greater than a set threshold, the detection is marked as a non-risk detection, meaning the current detection scene is relatively simple and high-accuracy tracking can be achieved without appearance feature extraction. In this case, the computationally intensive appearance feature extraction step is skipped, and matching is performed solely based on motion information. If the ARS is less than the set threshold, the detection is marked as a risk detection, meaning the current detection scene is relatively complex and appearance feature extraction is required for matching. In this case, the appearance feature extraction step is performed, combining appearance features and motion information for matching; the tracking steps are the same as Strongsort.

[0086] Overall, the WPFE selection mechanism can minimize the steps of appearance feature extraction that consume a lot of computing resources, thereby improving the real-time tracking performance of quadruped robots with limited computing resources.

[0087] As in the complete IoU loss, the aspect ratio similarity (ARS) of two bounding boxes is calculated as shown in Equation (1):

[0088]

[0089] In the formula: ABS represents aspect ratio similarity, w1 represents the length of detection box 1, h1 represents the width of detection box 1, w2 represents the length of detection box 2, and h2 represents the width of detection box 2.

[0090] (2) Feature decay strategy.

[0091] According to the selection mechanism, if the detection is classified as a risk detection and an appearance feature extraction step is performed, the appearance feature database of the trajectory will be updated; otherwise, the appearance feature database remains unchanged. In cases of occlusion, the appearance features of the target will change, therefore it is necessary to attenuate the importance of the appearance features when the target was not occluded in the early stages, relying on the new appearance features when the target is occluded in later stages for matching.

[0092] The Strongsort algorithm addresses this by maintaining appearance features as the frame-by-frame exponentially moving average (EMA) of previous features, causing the importance of appearance features to decrease exponentially with increasing frame count. Specifically, it assigns a value to the current feature vector e. t-1 A high weight α is assigned to the newly emerging feature f t A low weight 1-α is used, as shown in Equation (2). In this way, the importance of the feature in the average value decreases exponentially with the increase of the number of frames.

[0093] e t =α.e t-1 +(1-α).f t (2)

[0094] In the selection mechanism provided in this application embodiment, when appearance feature extraction is not performed in non-risk detection situations, the appearance feature library is not updated. Therefore, the importance of early features does not decay, leading to a decrease in tracking accuracy under occlusion conditions. To avoid this situation, mimicking the decay effect of EMA, the exponent of the weight α is set to the number of frames since the last appearance feature extraction.

[0095] When non-risk detection does not perform feature updates, the currently updated weight α ， The exponent increases with the number of interval frames.

[0096] When risk detection updates features, it uses the current weight α. ， Update the current appearance feature vector e t The larger the frame interval since the last appearance feature extraction, the higher the weight α of the earlier the feature was. ， The smaller the value, the less important the early features become in the average value. Therefore, as the number of frames increases, the importance of early features in the average value decreases exponentially, ensuring that the tracking accuracy of the quadruped robot does not decrease in complex situations such as occlusion.

[0097]

[0098] In summary, the multi-object tracking method provided in this application minimizes the number of appearance feature extractions through the provided WPFE selection mechanism, thereby improving the real-time performance of the multi-object tracking algorithm on resource-constrained quadruped robots. Simultaneously, the provided feature decay strategy updates the appearance feature library, causing the importance of appearance features to decrease exponentially with increasing frame count. When appearance features change due to occlusion, matching is performed using the new appearance features, without compromising the accuracy of the multi-object tracking algorithm under complex conditions such as occlusion.

[0099] See Figure 5 This application embodiment can also provide a multi-target tracking device, such as... Figure 5 As shown, the apparatus for performing the above-described multi-target tracking method may include:

[0100] The detection box acquisition unit 501 is used to acquire the detection boxes output by the detector and all trajectory points that have been successfully matched at least once before;

[0101] The similarity calculation unit 502 is used to calculate the aspect ratio similarity between the detection box and all the trajectory points;

[0102] The judgment unit 503 is used to judge the relationship between the aspect ratio similarity and the similarity threshold;

[0103] The non-risk detection tracking unit 504 is used to determine that the detection box is a non-risk detection after determining that the similarity ratio of the length and width is greater than the similarity threshold, and to track the target corresponding to the detection box by matching only the motion information based on the Strongsort multi-target tracking algorithm.

[0104] The risk detection and tracking unit 505 is used to determine that the detection box is a risk detection after determining that the similarity ratio of the length and width is less than the similarity threshold. It performs an appearance feature extraction step based on the Strongsort multi-target tracking algorithm to obtain appearance features, and combines the appearance features and the motion information to match and track the target corresponding to the detection box.

[0105] This application embodiment can also provide a multi-target tracking device, the device including a processor and a memory:

[0106] The memory is used to store program code and transmit the program code to the processor;

[0107] The processor is used to execute the steps of the multi-target tracking method described above according to the instructions in the program code.

[0108] like Figure 6 As shown in the figure, a multi-target tracking device provided in this application embodiment may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0109] In this embodiment, the processor 10 may be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0110] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can execute operations in the embodiments of the multi-target tracking method.

[0111] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0112] Obtain the detection bounding box output by the detector and all trajectory points that have been successfully matched at least once before;

[0113] Calculate the aspect ratio similarity between the detection box and all the trajectory points;

[0114] Determine the relationship between the aspect ratio similarity and the similarity threshold;

[0115] After determining that the aspect ratio similarity is greater than the similarity threshold, the detection box is determined to be a non-risk detection. Based on the Strongsort multi-target tracking algorithm, the target corresponding to the detection box is tracked by matching only motion information.

[0116] After determining that the length-width similarity ratio is less than the similarity threshold, the detection box is determined to be a risk detection. Based on the Strongsort multi-target tracking algorithm, an appearance feature extraction step is performed to obtain appearance features. The appearance features and motion information are then combined to match and track the target corresponding to the detection box.

[0117] In one possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (such as file creation or data read / write). The data storage area may store data created during use, such as initialization data.

[0118] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0119] Communication interface 12 can be an interface for a communication model, used to connect with other devices or systems.

[0120] Of course, it should be noted that, Figure 6 The structure shown does not constitute a limitation on the multi-target tracking device in the embodiments of this application. In practical applications, the multi-target tracking device may include more than Figure 6 More or fewer components as shown, or combinations of certain components.

[0121] This application embodiment may also provide a computer-readable storage medium for storing program code for executing the steps of the multi-target tracking method described above.

[0122] The abbreviations and key terms are defined as follows:

[0123] Deepsort, Deep Simple Online and Realtime Tracking.

[0124] FPS, Frames Per Second, is the number of frames transmitted per second.

[0125] EMA, Exponential moving average.

[0126] ECC stands for Enhanced correlation coefficient maximization.

[0127] NSA, Noise-Sensitive Adaptive.

[0128] WPFE, Whether to perform feature extraction.

[0129] IOU stands for Intersection over Union.

[0130] ARS stands for Aspect Ratio Similarity.

[0131] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0132] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0133] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A multi-target tracking method, characterized in that, include: Obtain the detection bounding box output by the detector and all trajectory points that have been successfully matched at least once before; Calculate the aspect ratio similarity between the detection box and all the trajectory points; Determine the relationship between the aspect ratio similarity and the similarity threshold; After determining that the aspect ratio similarity is greater than the similarity threshold, the detection box is determined to be a non-risk detection. Based on the Strongsort multi-target tracking algorithm, the target corresponding to the detection box is tracked by matching only motion information. After determining that the length-width similarity ratio is less than the similarity threshold, the detection box is determined to be a risk detection. Based on the Strongsort multi-target tracking algorithm, an appearance feature extraction step is performed to obtain appearance features. The appearance features and motion information are then combined to match and track the target corresponding to the detection box.

2. The multi-target tracking method according to claim 1, characterized in that, The detection box is determined to be a non-risk detection, and the appearance feature library of the trajectory remains unchanged; Once the detection box is identified as a risk detection, the appearance feature library of the trajectory is updated.

3. The multi-target tracking method according to claim 2, characterized in that, The appearance feature library for updating the trajectory includes: The importance of appearance features when not obscured in the early stages of attenuation.

4. The multi-target tracking method according to claim 2, characterized in that, The importance of appearance features in the early stages of attenuation when the area is not occluded includes: Assign the current feature vector e t-1 A high weight α is assigned to the newly emerging feature f t A low weight 1-α is used to make the importance of the feature in the mean decrease exponentially with increasing frame number, as shown in the following equation: e t =a.e t-1 +(1-a).f t 。 5. The multi-target tracking method according to claim 2, characterized in that, The exponent of the weight α is determined as the number of frames since the last appearance feature extraction. When the appearance feature library of the trajectory remains unchanged, the currently updated weight α ， The exponent increases with the number of interval frames; When updating the appearance feature library of the trajectory, the current weight α is used. ， Update the current appearance feature vector e t The feature update formula is as follows:

6. The multi-target tracking method according to claim 1, characterized in that, The formula for calculating the aspect ratio similarity is as follows: In the formula: ABS represents aspect ratio similarity, w1 represents the length of detection box 1, h1 represents the width of detection box 1, w2 represents the length of detection box 2, and h2 represents the width of detection box 2.

7. A multi-target tracking device, characterized in that, The apparatus for performing the multi-target tracking method according to any one of claims 1-6, the apparatus comprising: The detection box acquisition unit is used to acquire the detection boxes output by the detector and all trajectory points that have been successfully matched at least once before; A similarity calculation unit is used to calculate the aspect ratio similarity between the detection box and all the trajectory points; The judgment unit is used to determine the relationship between the aspect ratio similarity and the similarity threshold; The non-risk detection tracking unit is used to determine that the detection box is a non-risk detection after the similarity ratio of the length and width is greater than the similarity threshold. Based on the Strongsort multi-target tracking algorithm, the target corresponding to the detection box is tracked by matching only the motion information. The risk detection and tracking unit is used to determine that the detection box is a risk detection after determining that the similarity ratio of the length and width is less than the similarity threshold. It performs an appearance feature extraction step based on the Strongsort multi-target tracking algorithm to obtain appearance features, and combines the appearance features and the motion information to match and track the target corresponding to the detection box.

8. A multi-target tracking device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the multi-target tracking method according to any one of claims 1-6 according to the instructions in the program code.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the multi-target tracking method according to any one of claims 1-6.

Images