An automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model
By combining the self-attention module of re-identification and motion model with adaptive appearance features, the problem of tracking loss caused by target occlusion in dense traffic scenes is solved, and higher accuracy multi-target tracking effect is achieved.
Patent Information
- Application Number
- CN202411967794.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2044-12-30
AI Technical Summary
In complex traffic scenarios, especially dense traffic scenarios, pedestrians and vehicles may block each other, making it impossible for vehicle cameras to directly observe the target. Trackers that rely solely on motion models cannot accurately predict the movement trajectory and state of the occluded target, which can easily lead to the loss of tracking trajectory or identity switching, affecting tracking accuracy.
An autonomous vehicle multi-target tracking algorithm based on re-identification and motion models is adopted. It combines a similarity learning self-attention module and adaptive appearance features, predicts target motion through Kalman filtering, and combines GIOU intersection-union ratio and appearance cosine distance in the data association stage. The Hungarian algorithm is used for trajectory matching and appearance recall to improve tracking accuracy.
In dense traffic scenarios, by enhancing appearance features and adaptive weight adjustment, the accuracy and stability of target tracking are improved, trajectory loss and identity switching caused by occlusion are reduced, and the overall performance of multi-target tracking is enhanced.
Smart Images

Figure CN119888267B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of intelligent vehicles, and particularly relates to a multi-target tracking algorithm for an automatic driving vehicle based on re-identification and a motion model. BACKGROUND
[0002] With the rapid development of intelligent transportation systems, intelligent vehicle environment perception has become a key technology to ensure driving safety and improve driving experience. Pedestrian multi-target tracking is an important task in environment perception research, which mainly refers to data association and matching of multiple pedestrian targets observed by a vehicle-mounted camera during driving, to obtain the state information such as trajectory, speed, direction, etc. of each detection target, serving the planning, prediction, decision, etc. modules of the automatic driving system.
[0003] The multi-target tracking task is often completed through a motion model, which refers to the prediction and estimation of the motion trajectory of the target in the video sequence. The most typical method is the Kalman filter, but in complex traffic scenes (such as occlusion, similar motion features), once the tracking trajectory is lost, it is difficult to find it back only by relying on the motion model, thereby affecting the tracking performance. In the traffic conditions in China, there are many complex traffic conditions such as crossroads and dense crowds during school hours, and at this time the accuracy of multi-target tracking will greatly affect the driving reliability of automatic driving. In dense traffic scenes, pedestrians and vehicles are mutually occluded, making it impossible for the vehicle-mounted camera to directly observe the target, and the tracker relying only on the motion model is difficult to accurately predict the motion trajectory and state of the occluded target, which easily leads to tracking trajectory loss or identity switching, causing the tracking trajectory to be fragmented. Therefore, many researchers have begun to introduce appearance model assisted pedestrian multi-target tracking algorithms to complete the target tracking task.
[0004] The appearance model refers to the pedestrian re-identification technology (Re-ID), which extracts the visual features of the detected target in the video sequence, such as color, texture, shape, etc., to identify and distinguish the target. Common re-identification technologies include deep learning based feature extraction, appearance model based matching, pose estimation, etc. The appearance model assisted method combines the appearance features and motion information of the target, and realizes more accurate and robust target tracking by considering the appearance similarity, motion model prediction, etc. In the multi-target tracking task, the pedestrian re-identification technology can help solve the identity switching problem caused by occlusion between targets, and improve the tracking performance. SUMMARY
[0005] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification of the present application to avoid obscuring the purpose of this section, the abstract and the title of the specification, and such simplifications or omissions cannot be used to limit the scope of the present application.
[0006] The present application is realized by the following technical scheme: a multi-target tracking algorithm for an automatic driving vehicle based on re-identification and a motion model, comprising the following steps:
[0007] 1) using a target detection module to obtain detection box information through a detection network;
[0008] 2) using a re-identification module, including slicing the image before inputting it into the network, inputting it into a Resnest50 backbone network to obtain four slice image features, embedding the four segmented positions into linear projection Q, K, V, and inputting them into a Q, K, V attention mechanism module, calculating the self-attention and cross-attention between the four slices, and finally integrating the four slice features to obtain the global appearance feature of the detected pedestrian;
[0009] 3) using a tracker initialization module to read the detection file generated by the detector, obtaining the detection box information of each target in the current frame and dividing the detection boxes into high-score detection boxes and low-score detection boxes according to the set threshold, and storing their appearance features and sending them to the subsequent trajectory matching and association module;
[0010] 4) using a motion prediction module to predict the next frame target boundary box through Kalman filtering;
[0011] 5) using a trajectory association matching module, relying on the detection box obtained in step 1, the predicted boundary box obtained in step 2, and the appearance feature obtained in step 3, using cascade matching, obtaining the cost matrix of the detection boundary box and the trajectory by calculating the GIOU intersection ratio and the appearance cosine distance in the first two data associations, and adding an appearance adaptive weight coefficient, and if it is greater than the threshold, it is put into the Hungarian algorithm for matching, and in the last data association, the lost trajectory is recalled through the appearance, and if the threshold condition is met, the Hungarian algorithm is used for data association.
[0012] In order to further better realize the multi-target tracking algorithm for an automatic driving vehicle based on re-identification and a motion model, the following setting method is particularly used: the step 1) comprises the following specific steps:
[0013] 1.1) converting the obtained video sequence into the form of image frames;
[0014] 1.2) acquiring detection box information of each target in all image frames by the detector, wherein the information comprises coordinates and confidence of the detection box;
[0015] 1.3) storing the detection box information in a detection file.
[0016] In order to further better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 2) comprises the following specific steps:
[0017] 2.1) taking out the detection box image of each target in the image frame acquired by the detector;
[0018] 2.2) performing slicing operation on the detection box image;
[0019] 2.3) inputting the sliced image into the Resnest50 backbone network to obtain four sliced image features F i , and inputting the sliced image into the Resnest50 backbone network to obtain four sliced image feature matrices;
[0020] 2.4) embedding the four segmented positions into linear projections Q, K and V, and inputting into the Q, K and V attention mechanism modules to calculate self-attention and cross-attention among the four slices;
[0021] 2.5) integrating the four slice features to obtain the global appearance feature of the detected pedestrian.
[0022] In order to further better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 2.2) comprises the following steps:
[0023] 2.2.1) dividing the input image into upper left corner part, lower left corner part, upper right corner part and lower right corner part, and the process is as follows:
[0024] {P i} i=1,...,4 (1)
[0025] P i =W P ×H P (2)
[0026] Wherein, P i represents each image after slicing, W p , H p respectively represent the width and height of each slice;
[0027] 2.2.2) representing each input target image feature vector as four feature vectors, and the process is as follows:
[0028]
[0029] where F V represents the feature vector of the target image, D P represents W p ×H p .
[0030] 2.2.3) Normalize the pixels of S p P i to [0,1], while adding position embedding attributes to maintain the relative position relationship between slices, and obtain the ability to process position information, the process is as follows:
[0031] O i = P i +E i (4)
[0032]
[0033] where O represents the target image input into the network, P i represents each image block after slicing, E i represents the position embedding vector, D P represents W p ×H p .
[0034] In order to better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted:
[0035] 2.4.1) Map F i to Q i , K i , V i , the process is as follows:
[0036] Q i = F i ×T Q (6)
[0037] K i = F i ×T K (7)
[0038] V i = F i ×T V (8)
[0039] where F i represents the feature matrix, T Q , T K, T V respectively represent three linear transformations;
[0040] 2.4.2) Input to Q, K, V attention mechanism module, calculate the self-attention and cross-attention between the four slices, the process is as follows:
[0041]
[0042] Where, alpha i,j represent each O i The correlation coefficient of O j , Hadamard product, Sum represents element-wise sum.
[0043] In order to better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 2.5) comprises the following steps:
[0044] 2.5.1) Integrate the four slice features to obtain the global appearance feature of the detected pedestrian, the process is as follows:
[0045] F attention =(V1,V2...,V4) (11)
[0046] Where, F attention represent the appearance feature vector based on attention.
[0047] In order to better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 3) comprises the following specific steps:
[0048] 3.1) Obtain the detection frame information;
[0049] 3.2) According to the set confidence threshold, the detection frame is divided into high score detection frame and low score detection frame;
[0050] 3.3) Store the appearance feature of the target detection frame found, and send it into the trajectory association matching module.
[0051] In order to better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 4) motion prediction module comprises the following specific steps:
[0052] 4.1) Take out the trajectory stored in the last frame tracking list;
[0053] 4.2) Get the predicted bounding box of the trajectory in the last frame in the current frame through Kalman filter.
[0054] To further better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 5) track association matching module comprises the following specific steps:
[0055] 5.1) the cost matrix of the detection bounding box and the track is obtained by calculating the GIOU intersection over union and the appearance cosine distance in the previous two times of data association, and an appearance adaptive weight coefficient is added, and greater than a threshold value is put into the Hungarian algorithm for matching again;
[0056] 5.2) the last time of data association finds back the lost track through the appearance, and continues to use the Hungarian algorithm for data association after meeting the threshold condition.
[0057] To further better realize the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 5.1) comprises the following steps:
[0058] 5.1.1) calculate the appearance weighting factor α p ;
[0059] χ d =min(max A dh -max A dl ) (12)
[0060] χ t =min(max A th -max A tl ) (13)
[0061] α p =(χ t +χ d ) / 2 (14)
[0062] Wherein, χ d represents the similarity difference value in the detection box, χ t represents the similarity difference value in the track, A dh represents the highest value in the detection box cost matrix, A dl represents the second highest value in the detection box cost matrix, A th represents the highest value in the track cost matrix, A tl represents the second highest value in the track cost matrix, and α p represents the appearance adaptive weighting factor;
[0063] 5.1.2) calculate the GIOU intersection over union and the appearance cosine distance to obtain the cost matrix of the detection bounding box and the track, and greater than a threshold value is put into the Hungarian algorithm for matching again;
[0064] C = GIOU + (l w + a p )A c (15)
[0065] where C represents a cost matrix, GIOU represents an intersection over union, l w represents a hyperparameter, a p represents an appearance adaptive weighting factor, and A c represents an appearance cosine distance.
[0066] In order to better achieve the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, the following setting mode is particularly adopted: the step 5.2) comprises the following steps:
[0067] 5.2.1) using Kalman filtering to predict the center point position of the unmatched tracking target trajectory;
[0068] 5.2.2) in order to compensate for the error of Kalman filtering, the appearance embedding vectors within a 3*3 range around the predicted center point are selected;
[0069] 5.2.3) calculating the minimum cosine distance between the vector and the appearance embedding vectors stored in the unmatched tracking trajectory. If the cosine distance is less than a threshold, it is matched, and the lost trajectory is recovered.
[0070] Compared with the prior art, the present application has the following beneficial effects:
[0071] (1) In the dense traffic scene, pedestrians and vehicles are mutually occluded, which causes the vehicle-mounted camera to be unable to directly observe the target, and only relying on the tracker of the motion model is difficult to accurately predict the motion trajectory and state of the occluded target, which easily leads to loss of tracking trajectory or identity switching, and causes fragmentation of the tracking trajectory. Therefore, the similarity learning self-attention mechanism module is adopted to strengthen the appearance features of the target in the occlusion scene and improve the tracking accuracy in the dense traffic scene.
[0072] (2) In different dense traffic scenes, the degree of occlusion of pedestrians is different, which causes differences in the obtained appearance features. Therefore, in the data association stage, the GIOU intersection over union and the adaptive appearance are combined to change the weight of the appearance features in the cost matrix, which not only retains the advantages of the motion model in the ordinary traffic scene, but also enhances the tracking performance in different complex congested traffic scenes.
[0073] (3) For the loss of track due to occlusion, the cost matrix is calculated by the last observation value of the lost target and the unmatched track (lost track). Due to the large number of image frames during track loss, using motion prediction will cause a large deviation. The lost track is found back through enhanced appearance features, so as to obtain better tracking effect. BRIEF DESCRIPTION OF DRAWINGS
[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0075] Among them:
[0076] Figure 1 The flowchart of the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model described in the embodiments of the present application is shown.
[0077] Figure 2 The re-identification module diagram of the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model described in the embodiments of the present application is shown.
[0078] Figure 3 The track matching and association module of the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model described in the embodiments of the present application is shown. DETAILED DESCRIPTION
[0079] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0080] The purpose of the present application is to overcome the shortcomings of the prior art, and to propose an automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model. The method of similarity learning self-attention module and adaptive combination of appearance and motion features is adopted to improve the tracking effect of pedestrian multi-target tracking algorithm on automatic driving vehicles.
[0081] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.
[0082] EMBODIMENT
[0083] For an autonomous vehicle, in a dense traffic scene, pedestrians and vehicles occlude each other, so that the vehicle-mounted camera cannot directly observe the target, and it is difficult for a tracker relying only on a motion model to accurately predict the motion trajectory and state of the occluded target, which easily leads to loss of tracking trajectory or identity switching, thereby affecting tracking accuracy. Therefore, the present application improves the tracking effect of the pedestrian multi-target tracking algorithm on the autonomous vehicle through the similarity learning self-attention module and the method of adaptively combining appearance and motion features.
[0084] Reference Figure 1 , Figure 2 and Figure 3 For the embodiments of the present application, an autonomous vehicle multi-target tracking algorithm based on re-identification and motion model includes the following steps:
[0085] Step 1: Obtain the detection box information through the detection network.
[0086] Step 2: The target image is sliced before being input into the network, and four slice image features are obtained by inputting into the Resnest50 backbone network. Four segmented positions are embedded into linear projection Q, K, V, and input into the Q, K, V attention mechanism module. The Q, K, V attention mechanism module calculates the self-attention and cross-attention between the four slices, and the obtained four slice features are finally integrated to obtain the global appearance feature of the detected pedestrian.
[0087] The slicing operation in step 2 is:
[0088] The input image is divided into the upper left corner part, the lower left corner part, the upper right corner part and the lower right corner part, and the process is as follows:
[0089] {P i} i=1,...,4 (1)
[0090] P i =W P ×H P (2)
[0091] Wherein, P i represents each slice of the image, W p , H p respectively represent the width and height of each slice;
[0092] Each input target image feature vector is represented as a sequence of four feature vectors, and the process is as follows:
[0093]
[0094] Wherein, F V represents the feature vector of the target image, D PW p ×H p ;
[0095] The four P i pixels are normalized to [0, 1], while the position embedding attribute is added to maintain the relative position relationship between the slices, obtain the processing position information ability, and the process is as follows:
[0096] O i = P i +E i (4)
[0097]
[0098] Wherein, O represents the target image input into the network, P i represents each image after slicing, E i represents the position embedding vector, D P represents W p ×H p .
[0099] The four slice image feature matrices obtained by inputting the sliced images into the Resnest50 backbone network in step 2 are:
[0100]
[0101] Wherein, F i represents the feature matrix, L f represents the number of network layers of the last layer of Resnest50, and Ch represents the feature channel number.
[0102] In step 2, the four divided position embeddings are embedded into linear projection Q, K, V, and input into the Q, K, V attention mechanism module, and the self-attention and cross-attention between the four slices are calculated as:
[0103] F i is mapped to Q i , K i , V i , and the process is as follows:
[0104] Q i = F i ×T Q (7)
[0105] K i = F i ×T K (8)
[0106] V i = F i ×T V (9)
[0107] where F i denotes the feature matrix, T Q , T K , T V denote three linear transformations respectively;
[0108] The input is fed into the Q, K, V attention mechanism module to calculate the self-attention and cross-attention between the four slices, and the process is as follows:
[0109]
[0110] where α i,j denotes the correlation coefficient of each O i ; j denotes the Hadamard product, and Sum denotes the element-wise summation.
[0111] The four slice features obtained in step 2 are finally integrated to obtain the global appearance feature of the detected pedestrian:
[0112] A attention = (V1, V2,..., V4) (12)
[0113] where A attention denotes the appearance feature vector based on attention.
[0114] Step 3: Read the detection file generated by the detector, obtain the detection box information of each target in the current frame, and according to the set threshold, divide the detection box into high-score detection box and low-score detection box, and store their appearance features, and send them into the subsequent trajectory matching association module.
[0115] Step 4: Take out the trajectory stored in the tracking list in the previous frame, and obtain the predicted bounding box of the trajectory in the current frame through the Kalman filter.
[0116] Step 5: In the previous two data associations, the cost matrix of the detection bounding box and the trajectory is obtained by calculating the GIOU intersection-over-union and the appearance cosine distance, and the appearance adaptive weight coefficient is added. Greater than the threshold value, put it into the Hungarian algorithm for matching, and the last data association loses the trajectory through the appearance recall, and continues to use the Hungarian algorithm for data association after meeting the threshold condition.
[0117] The appearance weighting factor α p is calculated in step 5, and the cost matrix of the detection bounding box and the trajectory is obtained by calculating the GIOU intersection-over-union and the appearance cosine distance, and the appearance adaptive weight coefficient is added. Greater than the threshold value, put it into the Hungarian algorithm for matching:
[0118] The appearance weighting factor α p is calculated, and the process is as follows:
[0119] χ d = min(max A dh - max A dl ) (13)
[0120] χ t = min(max A th - max A tl ) (14)
[0121] α p = (χ t + χ d ) / 2 (15)
[0122] wherein χ d represents the similarity difference value in the detection box, χ t represents the similarity difference value in the track, A dh represents the highest value in the detection box cost matrix, A dl represents the second highest value in the detection box cost matrix, A th represents the highest value in the track cost matrix, A tl represents the second highest value in the track cost matrix, α p represents the appearance adaptive weighting factor;
[0123] The GIOU intersection over union and the appearance cosine distance are calculated to obtain the cost matrix of the detection bounding box and the track, and greater than the threshold value is put into the Hungarian algorithm for matching again, the process is as follows:
[0124] C = GIOU + (λ w + α p ) A c (16)
[0125] wherein C represents the cost matrix, GIOU represents the intersection over union, λ w represents the hyperparameter, α p represents the appearance adaptive weighting factor, and A c represents the appearance cosine distance.
[0126] The last data association in step 5 loses the track by appearance recall, and continues to use the Hungarian algorithm for data association when the threshold condition is met as follows:
[0127] Kalman filtering is used to predict the center point position of the unmatched tracking target track. In order to compensate for the error of Kalman filtering, the appearance embedding vectors within a 3x3 range around the predicted center point are selected. The minimum cosine distance between the vector and the appearance embedding vectors stored in the unmatched tracking track is calculated. If the cosine distance is less than the threshold value, it is matched, and the lost track is recovered.
[0128] Preliminary experimental results
[0129] To verify the feasibility and effectiveness of the present application, the present application is verified on the validation set and test set of MOT17, and the preliminary results are as follows:
[0130]
[0131] The above table is the performance comparison of the present application on the MOT17 dataset and the other two benchmark algorithms, the MOT17 dataset includes seven indoor and outdoor public scene scenes with pedestrians, the video of each scene is divided into two segments for training and testing respectively, among which MOT17-13 and MOT17-14 are collected in the process of vehicle driving in traffic scene, therefore the dataset is used to represent the pedestrian multi-target tracking in traffic scene. In terms of evaluation indicators, MOTA represents the accuracy of multi-target tracking, the higher the value, the better the tracking effect of the algorithm, IDS represents the number of times of identity switching of pedestrian targets caused by occlusion in multi-target tracking, the fewer the number of switching, the better the tracking effect of the algorithm.
[0132] In summary, the automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model improves the pedestrian target tracking effect.
[0133] The above describes the examples of the present application in combination with the drawings, although the present application has been disclosed as above with the preferred embodiments, but it is not intended to limit the present application, anyone skilled in the art can make various modifications and modifications without departing from the spirit and scope of the present application, therefore the protection scope of the present application should be limited by the claims.
Claims
1. An automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model, characterized in that: The algorithm adopts a re-identification system of similarity self-attention learning and a tracking strategy combining adaptive appearance features and motion prediction, and the specific method comprises the following steps: Step 1: The target detection module obtains the detection frame information through the detection network; Step 2: re-identification module, including image slicing operation before inputting into the network, inputting into the re-identification backbone network to obtain four sliced image features, four segmented positions being embedded into linear projection Q , K , V Q , K , V Attention mechanism module, Q , K , V The attention module calculates self-attention and cross-attention between the four slices, and the obtained four slice features are finally integrated to obtain the global appearance feature of the detected pedestrian; Step 3: The tracker initialization module is used for reading the detection file generated by the detector and the appearance feature generated by the re-identification network, obtaining the detection frame information of each target in the current frame, and dividing the detection frame into high-score detection frame and low-score detection frame according to the set threshold, and storing their appearance features and sending them into the subsequent track matching association module; Step 4: The motion prediction module predicts the next frame boundary frame through Kalman filtering; Step 5: The track association matching module relies on the detection frame obtained in step 1, the appearance feature obtained in step 2 and the predicted boundary frame obtained in step 4, adopts cascade matching, obtains the cost matrix of the detection boundary frame and the track through calculating the intersection-over-union GIOU and the appearance cosine distance in the first two data associations, and adds an appearance adaptive weight coefficient, and greater than the threshold value is put into the Hungarian algorithm for matching, and the last data association loses the track through the appearance recall, and after meeting the threshold condition, the Hungarian algorithm is used for data association again; Step 51: In the first two data associations, the cost matrix of the detection boundary frame and the track is obtained through calculating the GIOU intersection-over-union and the appearance cosine distance, and an appearance adaptive weight coefficient is added, and greater than the threshold value is put into the Hungarian algorithm for matching; The step 51 specifically comprises the following operations: Step 511 : Calculate appearance weighting factor α p ; (12); (13); (14); wherein, χ d represents the similarity difference in the detection frame, χ t represents the similarity difference in the track, A dh represents the highest value in the detection frame cost matrix, A dl represents the second highest value in the detection frame cost matrix, A th represents the highest value in the track cost matrix, A tl represents the second highest value in the track cost matrix, α p represents the appearance-adaptive weighting factor. 2.The algorithm according to claim 1, wherein: The step 1 specifically comprises the following sub-steps: Step 11: convert the obtained video sequence into the form of image frames; Step 12: obtain the detection frame information of each target in all image frames by using the detector, wherein the information includes the coordinates and confidence of the detection frame; Step 13: store the detection frame information in the detection file. 3.The algorithm of automatic driving vehicle multi-target tracking based on re-identification and motion model according to claim 1, characterized in that: The step 2 specifically comprises the following sub-steps: Step 21: take out the detection frame image of each target in the image frame obtained by the detector; Step 22: perform slicing operation on the detection frame image; Step 23: input the slice image into the Resnest50 backbone network to obtain four slice image features F i input the slice image into the Resnest50 backbone network to obtain four slice image feature matrices; Step 24: Embedding the four split positions into linear projections Q , K , V and input to Q , K , V Attention mechanism module, calculate the self-attention and cross-attention between the four slices; Step 25: integrate the four slice features to obtain the global appearance feature of the detected pedestrian. 4.The algorithm of automatic driving vehicle multi-target tracking based on re-identification and motion model according to claim 3, characterized in that: The step 22 specifically comprises the following operations: Step 221: divide the input image into the upper left corner part, the lower left corner part, the upper right corner part and the lower right corner part, and the process is as follows: (1); (2); wherein, P i denotes each piece of image after slicing, W p , H p denotes the width and height of each slice, respectively; Step 222: express each input target image feature vector as four feature vectors, and the process is as follows: (3); wherein F V a feature vector representing the target image, D p represents W p x H p ; Step 223: Place the 4 P i The pixels are normalized to [0,1], and a position embedding attribute is added to maintain the relative positional relationship between slices, thereby gaining the ability to process positional information. The process is as follows: (4); (5); wherein, O represents a target image inputted into a network, P i represents each piece of image after slicing, E i represents a position embedding vector, D p represents W p x H p .
5. The automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model according to claim 3, characterized in that: The step 24 specifically comprises the following operations: Step 241 : mapping F i to Q i , K i , V i The procedure is as follows: (6); (7); (8); wherein F i denotes a feature matrix, T Q , T K , T V denote three linear transformations, respectively; Step 242: input to Q , K , V The attention mechanism module calculates the self-attention and cross-attention between the four slices and obtains the appearance feature vector based on attention, as follows: (9); (10); wherein α i,j denotes each O i about O j the correlation coefficient of denotes the Hadamard product, Sum denotes the element-wise sum. 6.The algorithm of automatic driving vehicle multi-target tracking based on re-identification and motion model according to claim 3, characterized in that: The step 25 specifically comprises the following operations: Step 251: integrate the four slice features to obtain the global appearance feature of the detected pedestrian, and the process is as follows: (11); wherein, F attention represents an appearance feature vector based on attention.
7. The automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model according to claim 1, characterized in that: The step 3 specifically comprises the following sub-steps: Step 31: obtain the detection frame information; Step 32: divide the detection frame into high-score detection frame and low-score detection frame according to the set confidence threshold; Step 33: store the appearance feature of the above target detection frame and send it into the track association matching module. 8.The algorithm of automatic driving vehicle multi-target tracking based on re-identification and motion model according to claim 1, characterized in that: The step 4 specifically comprises the following sub-steps: Step 41: take out the track stored in the tracking list of the previous frame; Step 42: Obtain the predicted bounding box of the track in the previous frame in the current frame through the Kalman filter. 9.The algorithm of automatic driving vehicle multi-target tracking based on re-identification and motion model according to claim 1, characterized in that: The step 5 specifically comprises the following sub-steps: Step 51: In the previous two times of data association, the cost matrix of the detection bounding box and the track is obtained by calculating the GIOU intersection over union and the appearance cosine distance, and an appearance adaptive weight coefficient is added, and greater than a threshold value is put into the Hungarian algorithm for matching again; Step 52: In the last time of data association, the lost track is recalled through the appearance, and after meeting the threshold condition, the Hungarian algorithm is used for data association; The step 51 further comprises the following operations: Step 512: Calculate the GIOU intersection over union and the appearance cosine distance to obtain the cost matrix of the detection bounding box and the track, and greater than a threshold value is put into the Hungarian algorithm for matching again; (15); wherein, C denotes a cost matrix, GIOU denotes Generalized Intersection over Union, Lambda w denotes a hyperparameter, α p denotes an appearance-adaptive weighting factor, A c denotes an appearance cosine distance. 10.The algorithm of automatic driving vehicle multi-target tracking based on re-identification and motion model according to claim 9, characterized in that: The step 52 specifically comprises the following operations: Step 521: The Kalman filter is used to predict the center point position of the unmatched tracking target track; Step 522: In order to compensate for the error of the Kalman filter, the appearance embedding vector within the range of 3*3 around the predicted center point is selected; Step 523: Calculate the minimum cosine distance between the vector and the appearance embedding vector stored in the unmatched tracking track; if the cosine distance is less than a threshold value, match and recover the lost track.
Citation Information
Patent Citations
Multi-target tracking method for intelligent driving
CN116402850A
Road vehicle tracking method based on lightweight re-recognition network
CN118154642A