Automatic driving vehicle multi-target tracking algorithm based on re-identification and motion model

Abstract

The invention discloses an automatic driving vehicle multi-target tracking algorithm based on re-identification and a motion model. A target detection module obtains information such as a target position and confidence through a detection network. And the re-identification module performs slicing operation on the detection image through the image slicing module, slices are conveyed to the backbone network to obtain image features of each slice, and appearance feature enhancement is performed through the similarity learning self-attention module. The tracker initialization module obtains detection frame information obtained by a target detection system, divides a high-score detection frame and a low-score detection frame into a high-score detection frame and a low-score detection frame according to a set threshold value, and sends the high-score detection frame and the low-score detection frame into the follow-up track matching association module. The motion prediction module uses a Kalman filter to predict a next frame of target bounding box. The trajectory matching association module calculates a cost matrix of a target detection frame and a trajectory by combining an intersection-to-union ratio and an adaptive weighted appearance cosine distance. According to the invention, the tracking effect of the pedestrian multi-target tracking algorithm on the automatic driving vehicle is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent vehicles, and in particular relates to a multi-target tracking algorithm for an autonomous driving vehicle based on re-identification and motion model. Background Art

[0002] With the rapid development of intelligent transportation systems, intelligent vehicle environmental perception has become a key technology to ensure driving safety and improve driving experience. Pedestrian multi-target tracking is an important task in environmental perception research. It mainly refers to the data association and matching of multiple pedestrian targets observed by the vehicle-mounted camera during driving to obtain the trajectory, speed, direction and other status information of each detected target, which serves the planning, prediction, decision-making and other modules of the autonomous driving system.

[0003] The multi-target tracking task is often completed through motion models. Motion models refer to the prediction and estimation of the motion trajectory of the target in the video sequence. The most typical method is the Kalman filter. However, in complex traffic scenes (such as occlusion and similar motion features), once the tracking trajectory is lost, it is difficult to retrieve it only by relying on the motion model, which affects the tracking performance. In my country's traffic conditions, there are many complex traffic conditions such as crossroads and dense crowds at school. At this time, the accuracy of multi-target tracking will greatly affect the driving reliability of autonomous driving. In dense traffic scenes, pedestrians and vehicles block each other, resulting in the inability of the on-board camera to directly observe the target. The tracker that relies solely on the motion model is difficult to accurately predict the motion trajectory and state of the occluded target, which easily leads to the loss of tracking trajectory or identity switching, making the tracking trajectory fragmented. Therefore, many researchers have begun to introduce appearance model-assisted pedestrian multi-target tracking algorithms to complete target tracking tasks.

[0004] Appearance model refers to person re-identification technology (Re-ID), which extracts visual features of detected targets in video sequences, such as color, texture, shape, etc., to identify and distinguish targets. Common re-identification technologies include feature extraction based on deep learning, matching based on appearance models, and pose estimation. The appearance model-assisted method combines the appearance features and motion information of the target, and achieves more accurate and robust target tracking by comprehensively considering multiple aspects of information such as appearance similarity and motion model prediction. In multi-target tracking tasks, pedestrian re-identification technology can help solve problems such as identity switching caused by occlusion between targets and improve tracking performance. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] The present invention is implemented by the following technical solution: an algorithm for multi-target tracking of an autonomous driving vehicle based on re-identification and motion model, comprising the following steps:

[0007] 1) Use the target detection module to obtain the detection box information through the detection network;

[0008] 2) Using the re-identification module, including slicing the image before entering the network, inputting it into the Resnest50 backbone network to obtain four slice image features, the four segmented positions are embedded into the linear projection Q, K, V, and input into the Q, K, V attention mechanism module, the Q, K, V attention module calculates the self-attention and cross-attention between the four slices, and the obtained four slice features are finally integrated to obtain the global appearance features of the detected pedestrian;

[0009] 3) Using the tracker initialization module, it is used to read the detection file generated by the detector, obtain the detection frame information of each target in the current frame, and divide the detection frame into high-score detection frame and low-score detection frame according to the set threshold, and store their appearance features and send them to the subsequent trajectory matching association module;

[0010] 4) Using the motion prediction module, the target bounding box of the next frame is predicted through Kalman filtering;

[0011] 5) Using the trajectory association matching module, relying on the detection box obtained in step 1, the predicted bounding box obtained in step 2, and the appearance features obtained in step 3, cascade matching is adopted. In the first two data associations, the cost matrix of the detection bounding box and the trajectory is obtained by calculating the GIOU intersection-union ratio and the appearance cosine distance, and the appearance adaptive weight coefficient is added. If it is greater than the threshold, it is put into the Hungarian algorithm for matching. The last data association recalls the lost trajectory through appearance. After the threshold condition is met, the Hungarian algorithm is continued to be used for data association.

[0012] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 1) includes the following specific steps:

[0013] 1.1) Convert the obtained video sequence into the form of image frames;

[0014] 1.2) Using the detector to obtain the detection box information of each target in all image frames, where the information includes the coordinates and confidence of the detection box;

[0015] 1.3) Store the detection box information in the detection file.

[0016] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 2) includes the following specific steps:

[0017] 2.1) Take out the detection frame image of each target in the image frame acquired by the detector;

[0018] 2.2) Slicing the detection frame image;

[0019] 2.3) Input the slice image into the Resnest50 backbone network to obtain four slice image features F i , input the slice image into the Resnest50 backbone network to obtain four slice image feature matrices;

[0020] 2.4) The positions of the four segments are embedded into the linear projection Q, K, V and input into the Q, K, V attention mechanism module to calculate the self-attention and cross-attention between the four slices;

[0021] 2.5) Integrate the four slice features to obtain the global appearance features of the detected pedestrian.

[0022] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 2.2) includes the following steps:

[0023] 2.2.1) Divide the input image into the upper left corner, lower left corner, upper right corner, and lower right corner. The process is as follows:

[0024] {P i} i＝1，...，4 (1)

[0025] P i =W P ×H P (2)

[0026] Among them, P i Represents each image after slicing, W p , H p Respectively represent the width and height of each slice;

[0027] 2.2.2) Represent each input target image feature vector as 4 feature vectors. The process is as follows:

[0028]

[0029] Among them, F V Represents the feature vector of the target image, D P W p ×H p ;

[0030] 2.2.3) S p P i The pixels are normalized to [0,1], and the position embedding attribute is added to maintain the relative position relationship between slices and obtain the ability to process position information. The process is as follows:

[0031] O i =P i +E i (4)

[0032]

[0033] Among them, O represents the target image of the input network, P i Represents each image after slicing, E i represents the position embedding vector, D P W p ×H p .

[0034] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 2.4) includes the following steps:

[0035] 2.4.1) F i Mapping to Q i , K i 、V i , the process is as follows:

[0036] Q i =F i ×T Q (6)

[0037] K i =F i ×T K (7)

[0038] V i =F i ×T V (8)

[0039] Among them, F i represents the feature matrix, T Q 、T K、T V Represent three linear transformations respectively;

[0040] 2.4.2) Input to the Q, K, V attention mechanism module to calculate the self-attention and cross-attention between the four slices. The process is as follows:

[0041]

[0042] Among them, α i,j Indicates that each O i About O j The correlation coefficient of represents the Hadamard product and Sum represents the element-by-element summation.

[0043] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 2.5) includes the following steps:

[0044] 2.5.1) Integrate the four slice features to obtain the global appearance features of the detected pedestrian. The process is as follows:

[0045] F attention =(V1,V2...,V4) (11)

[0046] Among them, F attention represents the attention-based appearance feature vector.

[0047] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 3) includes the following specific steps:

[0048] 3.1) Obtain detection box information;

[0049] 3.2) According to the set confidence threshold, the detection frame is divided into a high-score detection frame and a low-score detection frame;

[0050] 3.3) Store the appearance features of the target detection box found above and send them to the trajectory association matching module.

[0051] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the motion prediction module in step 4) includes the following specific steps:

[0052] 4.1) Take out the track stored in the tracking list of the previous frame;

[0053] 4.2) Get the predicted bounding box of the trajectory in the previous frame in the current frame through the Kalman filter.

[0054] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the trajectory association matching module in step 5) includes the following specific steps:

[0055] 5.1) In the first two data associations, the cost matrix of the detection bounding box and trajectory is obtained by calculating the GIOU intersection-union ratio and the appearance cosine distance, and the appearance adaptive weight coefficient is added. If it is greater than the threshold, it is put into the Hungarian algorithm for matching;

[0056] 5.2) The last data association is to retrieve the lost tracks through appearance, and continue to use the Hungarian algorithm for data association after the threshold condition is met.

[0057] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 5.1) includes the following steps:

[0058] 5.1.1) Calculate the appearance weighting factor α p ;

[0059] χ d =min(max A dh -max A dl ) (12)

[0060] χ t =min(max A th -max A tl ) (13)

[0061] α p =(χ t +χ d ) / 2 (14)

[0062] Among them, χ d Represents the similarity difference in the detection box, χ t Represents the similarity difference in the trajectory, A dh Represents the highest value in the detection box cost matrix, A dl Represents the second highest value in the detection box cost matrix, A th Represents the highest value in the cost matrix in the trajectory, A tl represents the second highest value in the cost matrix in the trajectory, α p represents the appearance adaptive weighting factor;

[0063] 5.1.2) Calculate the GIOU intersection-over-union ratio and the appearance cosine distance to obtain the cost matrix of the detection bounding box and trajectory. If it is greater than the threshold, it will be put into the Hungarian algorithm for matching;

[0064] C=GIOU+(λ w +α p )A c (15)

[0065] Among them, C represents the cost matrix, GIOU represents the intersection-over-union ratio, and λ w represents the hyperparameter, α p represents the appearance adaptive weighting factor, A c Represents the appearance cosine distance.

[0066] In order to further better implement the multi-target tracking algorithm of an autonomous driving vehicle based on re-identification and motion model described in the present invention, the following setting is particularly adopted: the step 5.2) includes the following steps:

[0067] 5.2.1) Use Kalman filtering to predict the center point position of the unmatched tracking target trajectory;

[0068] 5.2.2) In order to compensate for the error of Kalman filtering, an appearance embedding vector within a 3×3 range around the prediction center point is selected;

[0069] 5.2.3) Calculate the minimum cosine distance between this vector and the appearance embedding vector stored in the unmatched tracking trajectory. If the cosine distance is less than a threshold, then it matches and the lost trajectory is recovered.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] (1) In dense traffic scenes, pedestrians and vehicles block each other, making it impossible for the on-board camera to directly observe the target. Trackers that rely solely on motion models are difficult to accurately predict the motion trajectory and state of the occluded target, which can easily lead to tracking trajectory loss or identity switching, resulting in tracking trajectory fragmentation. Therefore, a similarity learning self-attention mechanism module is used to enhance the appearance features of the target in occluded scenes and improve the tracking accuracy in dense traffic scenes.

[0072] (2) In different dense traffic scenes, the degree of occlusion of pedestrians is different, resulting in differences in the acquired appearance features. Therefore, in the data association stage, the weight of the appearance features in the cost matrix is ​​changed by combining the GIOU intersection-over-union method with the adaptive appearance. This not only retains the advantages of the motion model in ordinary traffic scenes, but also enhances the tracking performance in different complex congested traffic scenes.

[0073] (3) For trajectory loss due to occlusion, the cost matrix is ​​calculated by losing the last observation value of the target and the unmatched trajectory (lost trajectory). Since the number of image frames is large during the trajectory loss process, the use of motion prediction will cause a large deviation. The lost trajectory is retrieved through enhanced appearance features, thereby obtaining a better tracking effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0075] in:

[0076] Figure 1 A schematic diagram of a flow chart of a multi-target tracking algorithm for an autonomous driving vehicle based on re-identification and motion model according to an embodiment of the present invention;

[0077] Figure 2 A schematic diagram of a re-identification module of a multi-target tracking algorithm for an autonomous driving vehicle based on re-identification and motion model according to an embodiment of the present invention;

[0078] Figure 3 This is a trajectory matching association module of the multi-target tracking algorithm for an autonomous driving vehicle based on re-identification and motion model as described in an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a multi-target tracking algorithm for autonomous driving vehicles based on re-identification and motion model. The method adopts a self-attention module through similarity learning and adaptively combines appearance and motion features to improve the tracking effect of the pedestrian multi-target tracking algorithm on autonomous driving vehicles.

[0081] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0082] Example

[0083] For autonomous vehicles. In dense traffic scenes, pedestrians and vehicles block each other, resulting in the inability of the on-board camera to directly observe the target. Trackers that rely solely on motion models are difficult to accurately predict the motion trajectory and state of the blocked target, which can easily lead to tracking trajectory loss or identity switching, thereby affecting tracking accuracy. Therefore, the present invention improves the tracking effect of the pedestrian multi-target tracking algorithm on autonomous vehicles through a similarity learning self-attention module and a method of adaptively combining appearance and motion features.

[0084] Reference Figure 1 , Figure 2 and Figure 3 , is an embodiment of the present invention, a multi-target tracking algorithm for an autonomous driving vehicle based on re-identification and motion model, comprising the following steps:

[0085] Step 1: Obtain detection box information through the detection network.

[0086] Step 2: The target image is sliced ​​before being input into the network and input into the Resnest50 backbone network to obtain four slice image features. The four segmented positions are embedded into the linear projection Q, K, V and input into the Q, K, V attention mechanism module. The Q, K, V attention module calculates the self-attention and cross-attention between the four slices. The obtained four slice features are finally integrated to obtain the global appearance features of the detected pedestrian.

[0087] The slicing operation in step 2 is:

[0088] The input image is divided into the upper left corner, the lower left corner, the upper right corner, and the lower right corner. The process is as follows:

[0089] {P i} i＝1，...，4 (1)

[0090] P i =W P ×H P (2)

[0091] Among them, P i Represents each image after slicing, W p , H p Respectively represent the width and height of each slice;

[0092] Each input target image feature vector is represented as a sequence of 4 feature vectors. The process is as follows:

[0093]

[0094] Among them, F V Represents the feature vector of the target image, D PW p ×H p ;

[0095] The 4 P i The pixels are normalized to [0,1], and the position embedding attribute is added to maintain the relative position relationship between slices and obtain the ability to process position information. The process is as follows:

[0096] O i =P i +E i (4)

[0097]

[0098] Among them, O represents the target image of the input network, P i Represents each image after slicing, E i represents the position embedding vector, D P W p ×H p .

[0099] In step 2, the slice image is input into the Resnest50 backbone network to obtain four slice image feature matrices:

[0100]

[0101] Among them, F i represents the feature matrix, L f Resnest50 represents the number of the last network layer, and Ch represents the number of feature channels.

[0102] In step 2, the positions of the four segments are embedded into the linear projection Q, K, V and input into the Q, K, V attention mechanism module to calculate the self-attention and cross-attention between the four slices as:

[0103] F i Mapping to Q i , K i 、V i , the process is as follows:

[0104] Q i =F i ×T Q (7)

[0105] K i =F i ×T K (8)

[0106] V i =F i ×T V (9)

[0107] Among them, F i represents the feature matrix, T Q 、T K 、T V Represent three linear transformations respectively;

[0108] Input to the Q, K, V attention mechanism module to calculate the self-attention and cross-attention between the four slices. The process is as follows:

[0109]

[0110] Among them, α i,j Indicates that each O i About O j The correlation coefficient of represents the Hadamard product and Sum represents the element-by-element summation.

[0111] The four slice features obtained in step 2 are finally integrated to obtain the global appearance features of the detected pedestrian:

[0112] A attention =(V1,V2...,V4) (12)

[0113] Among them, A attention represents the attention-based appearance feature vector.

[0114] Step 3: Read the detection file generated by the detector, obtain the detection frame information of each target in the current frame, and divide the detection frame into high-score detection frame and low-score detection frame according to the set threshold, store their appearance features, and send them to the subsequent trajectory matching association module.

[0115] Step 4: Take out the trajectory stored in the tracking list of the previous frame, and use the Kalman filter to obtain the predicted bounding box of the trajectory in the previous frame in the current frame.

[0116] Step 5: In the first two data associations, the cost matrix of the detection bounding box and trajectory is obtained by calculating the GIOU intersection-union ratio and the appearance cosine distance, and the appearance adaptive weight coefficient is added. If it is greater than the threshold, it is put into the Hungarian algorithm for matching. The last data association is performed by recalling the lost trajectory through appearance. After the threshold condition is met, the Hungarian algorithm is used to continue data association.

[0117] In step 5, the appearance weighting factor α is calculated p , calculate the GIOU intersection-and-union ratio and the appearance cosine distance to obtain the cost matrix of the detection bounding box and trajectory. If it is greater than the threshold, it will be put into the Hungarian algorithm for matching:

[0118] Calculate the appearance weighting factor α p , the process is as follows:

[0119] χ d =min(max A dh -max A dl ) (13)

[0120] χ t =min(max A th -max A tl ) (14)

[0121] α p =(χ t +χ d ) / 2 (15)

[0122] Among them, χ d Represents the similarity difference in the detection box, χ t Represents the similarity difference in the trajectory, A dh Represents the highest value in the detection box cost matrix, A dl Represents the second highest value in the detection box cost matrix, A th Represents the highest value in the cost matrix in the trajectory, A tl represents the second highest value in the cost matrix in the trajectory, α p represents the appearance adaptive weighting factor;

[0123] Calculate the GIOU intersection-over-union ratio and the appearance cosine distance to obtain the cost matrix of the detection bounding box and trajectory. If it is greater than the threshold, put it into the Hungarian algorithm for matching. The process is as follows:

[0124] C=GIOU+(λ w +α p )A c (16)

[0125] Among them, C represents the cost matrix, GIOU represents the intersection-over-union ratio, and λ w represents the hyperparameter, α p represents the appearance adaptive weighting factor, A c Represents the appearance cosine distance.

[0126] The last data association in step 5 is to recall the lost track through appearance, and then continue to use the Hungarian algorithm to associate data after the threshold condition is met:

[0127] Use Kalman filtering to predict the center point position of the unmatched tracking target trajectory. To compensate for the error of Kalman filtering, select the appearance embedding vector within a 3×3 range around the predicted center point. Calculate the minimum cosine distance between this vector and the appearance embedding vector stored in the unmatched tracking trajectory. If the cosine distance is less than the threshold, it is matched and the lost trajectory is recovered.

[0128] Preliminary experimental results

[0129] In order to verify the feasibility and effectiveness of the present invention, the present invention is verified on the validation set and test set of MOT17, and the preliminary results are as follows:

[0130]

[0131] The above table is a performance comparison of the present invention on the MOT17 dataset and the other two benchmark algorithms. The MOT17 dataset includes seven indoor and outdoor public places with pedestrians. The video of each scene is divided into two clips, which are used for training and testing respectively. Among them, MOT17-13 and MOT17-14 are vehicle driving processes collected in traffic scenes. Therefore, this dataset is used to represent pedestrian multi-target tracking in traffic scenes. In terms of evaluation indicators, MOTA represents the accuracy of multi-target tracking. The higher the value, the better the algorithm tracking effect. IDS represents the number of pedestrian target identity switches due to occlusion in multi-target tracking. The fewer the number of switches, the better the algorithm tracking effect.

[0132] In summary, a multi-target tracking algorithm for autonomous driving vehicles based on re-identification and motion model improves the pedestrian target tracking effect.

[0133] The examples of the present invention are described above in conjunction with the accompanying drawings. Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the definition of the claims.

Claims

1. A multi-target tracking algorithm for autonomous driving vehicles based on re-identification and motion model, characterized by: The algorithm adopts a similarity self-attention learning re-identification system and a tracking strategy combining adaptive appearance features and motion prediction. The specific method includes the following steps: Step 1: The target detection module obtains the detection box information through the detection network; Step 2: Re-ID module, including slicing the image before inputting into the network, inputting into the re-ID backbone network to obtain four slice image features, the four segmented positions are embedded into the linear projection Q, K, V, and input into the Q, K, V attention mechanism module, the Q, K, V attention module calculates the self-attention and cross-attention between the four slices, and the obtained four slice features are finally integrated to obtain the global appearance features of the detected pedestrian; Step 3: The tracker initialization module is used to read the detection file generated by the detector and the appearance features generated by the re-identification network, obtain the detection frame information of each target in the current frame, and divide the detection frame into high-score detection frames and low-score detection frames according to the set threshold, and store their appearance features and send them to the subsequent track matching association module; Step 4: Motion prediction module predicts the next frame bounding box through Kalman filtering; Step 5: The trajectory association matching module uses cascade matching based on the detection box obtained in step 1, the appearance features obtained in step 2, and the predicted bounding box obtained in step 4. In the first two data associations, the cost matrix of the detection bounding box and the trajectory is obtained by calculating the intersection-over-union ratio (GIOU) and the appearance cosine distance, and the appearance adaptive weight coefficient is added. If it is greater than the threshold, it is put into the Hungarian algorithm for matching. The last data association recalls the lost trajectory through appearance, and continues to use the Hungarian algorithm for data association after the threshold condition is met.

2. An algorithm according to claim 1, characterized in that: The step 1 specifically includes the following sub-steps: Step 11: Convert the obtained video sequence into the form of image frames; Step 12: Use the detector to obtain the detection box information of each target in all image frames, where the information includes the coordinates and confidence of the detection box; Step 13: Store the detection box information in the detection file.

3. An algorithm according to claim 1, characterized in that: The step 2 specifically includes the following sub-steps: Step 21: Take out the detection frame image of each target in the image frame acquired by the detector; Step 22: Slice the detection frame image; Step 23: Input the slice image into the Resnest50 backbone network to obtain four slice image features F i , input the slice image into the Resnest50 backbone network to obtain four slice image feature matrices; Step 24: Embed the positions of the four segments into the linear projection Q, K, V and input them into the Q, K, V attention mechanism module to calculate the self-attention and cross-attention between the four slices; Step 25: Integrate the four slice features to obtain the global appearance features of the detected pedestrian.

4. An algorithm according to claim 3, characterized in that: The step 22 specifically includes the following operations: Step 221: Divide the input image into an upper left corner portion, a lower left corner portion, an upper right corner portion, and a lower right corner portion, and the process is as follows: {P i} i＝1，...，4 (1) P i =W P ×H P (2) Among them, P i Represents each image after slicing, W p , H p Respectively represent the width and height of each slice; Step 222: Represent each input target image feature vector as 4 feature vectors, the process is as follows: Among them, F V Represents the feature vector of the target image, D P W p ×H p ; Step 223: Put the 4 P i The pixels are normalized to [0,1], and the position embedding attribute is added to maintain the relative position relationship between slices and obtain the ability to process position information. The process is as follows: O i =P i +E i (4) Among them, O represents the target image of the input network, P i Represents each image after slicing, E i represents the position embedding vector, D P W p ×H p .

5. An algorithm according to claim 3, characterized in that: The step 24 specifically includes the following operations: Step 241: F i Mapping to Q i , K i 、V i The process is as follows: Q i =F i ×T Q (6) K i =F i ×T K (7) V i =F i ×T V (8) Among them, F i represents the feature matrix, T Q 、T K 、T V Represent three linear transformations respectively; Step 242: Input to the Q, K, V attention mechanism module, calculate the self-attention and cross-attention between the four slices, and obtain the attention-based appearance feature vector. The process is as follows: Among them, α i,j Indicates that each O i About O j The correlation coefficient of represents the Hadamard product and Sum represents the element-by-element summation.

6. An algorithm according to claim 3, characterized in that: The step 25 specifically includes the following operations: Step 251: Integrate the four slice features to obtain the global appearance features of the detected pedestrian. The process is as follows: F attention =(V1,V2...,V4) (11) Among them, F attention represents the attention-based appearance feature vector.

7. An algorithm according to claim 1, characterized in that: The step 3 specifically includes the following sub-steps: Step 31: Get detection frame information; Step 32: Divide the detection frame into a high-score detection frame and a low-score detection frame according to the set confidence threshold; Step 33: Store the appearance features of the target detection box found above and send them to the trajectory association matching module.

8. An algorithm according to claim 1, characterized in that: The step 4 specifically includes the following sub-steps: Step 41: Retrieve the track stored in the tracking list of the previous frame; Step 42: Obtain the predicted bounding box of the trajectory in the previous frame in the current frame through the Kalman filter.

9. An algorithm according to claim 1, characterized in that: The step 5 specifically includes the following sub-steps: Step 51: In the first two data associations, the cost matrix of the detection bounding box and the trajectory is obtained by calculating the GIOU intersection-union ratio and the appearance cosine distance, and the appearance adaptive weight coefficient is added. If it is greater than the threshold, it is put into the Hungarian algorithm for matching; Step 52: The last data association is to retrieve the lost track through appearance, and continue to use the Hungarian algorithm for data association after the threshold condition is met; The step 51 specifically includes the following operations: Step 511: Calculate appearance weighting factor α p ; x d =min(max A dh -max A dl ) (12) x t =min(max A th -max A tl ) (13) a p =(x t +x d ) / 2 (14) Among them, χ d Represents the similarity difference in the detection box, χ t Represents the similarity difference in the trajectory, A dh Represents the highest value in the detection box cost matrix, A dl Represents the second highest value in the detection box cost matrix, A th Represents the highest value in the cost matrix in the trajectory, A tl represents the second highest value in the cost matrix in the trajectory, α p represents the appearance adaptive weighting factor; Step 512: Calculate the GIOU intersection-over-union ratio and the appearance cosine distance to obtain the cost matrix of the detection bounding box and the trajectory. If it is greater than the threshold, it is put into the Hungarian algorithm for matching; C=GIOU+(λ w +a p )A c (15) Among them, C represents the cost matrix, GIOU represents the intersection-over-union ratio, and λ w represents the hyperparameter, α p represents the appearance adaptive weighting factor, A c Represents the appearance cosine distance.

10. An algorithm according to claim 9, characterized in that: The step 52 specifically includes the following operations: Step 521: Use Kalman filtering to predict the center point position of the unmatched tracking target trajectory; Step 522: In order to compensate for the error of the Kalman filter, an appearance embedding vector within a 3×3 range around the prediction center point is selected; Step 523: Calculate the minimum cosine distance between the vector and the appearance embedding vector stored in the unmatched tracking trajectory; if the cosine distance is less than a threshold, it is matched and the lost trajectory is restored.

Images