Target detection method based on human head and human body combination, multi-target pedestrian tracking method and application

Abstract

The invention discloses a target detection method based on human head and human body combination, a multi-target pedestrian tracking method and application, relates to the technical field of computer vision, and can effectively relieve the problem of pedestrian occlusion in dense or complex scenes. Firstly, based on a single-stage target detection network, a target detector for joint detection of a human head and a human body is designed, a human head and human body target matching mechanism based on position correlation is introduced into the detector, and human head and human body detection frames output by the target detector can be accurately associated and matched; secondly, on the basis of a human head and human body joint detection frame output by a detector, an efficient target association and track updating strategy is provided, and the problem of target loss or identity switching caused by pedestrian shielding in dense crowds can be effectively relieved; finally, the multi-target pedestrian tracking method based on joint detection and tracking of the head and the body can be efficiently deployed in various computing-limited terminal devices in real time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a target detection method based on head and body joint, a multi-target pedestrian tracking method and application. BACKGROUND

[0002] In recent years, with the rapid development of deep learning technology, multi-target pedestrian tracking technology has made significant progress. The traditional tracking-by-detection multi-target tracking framework first performs target detection on each frame of image, and then establishes the continuous trajectory of the target between different frames through the association algorithm. However, in dense or complex scenes of pedestrians, this algorithm often encounters some challenges, such as target occlusion. When pedestrians are occluded, the tracking algorithm may mistakenly lose the target or incorrectly switch the identity of the target, which seriously affects the accuracy of tracking. Especially in crowded crowds, this problem is particularly prominent, because occlusion and close pedestrians easily lead to errors and discontinuity of target detection.

[0003] In order to improve the tracking effect in complex environments, many studies have proposed various improvement methods, including using more complex detection networks, increasing feature and context information, and optimizing association strategies. In order to pursue high detection accuracy, existing algorithms usually adopt complex network structures, which have large computational complexity and are difficult to realize real-time deployment on computing limited devices such as embedded devices and edge computing terminals, limiting their application range in actual scenes.

[0004] Therefore, how to design a tracking method that can guarantee multi-target pedestrian tracking accuracy in dense or complex scenes and can be efficiently deployed on computing limited terminal devices has become a technical problem to be solved in the field. SUMMARY

[0005] The purpose of the present application is to provide a target detection method based on head and body joint, which can effectively alleviate the problem of pedestrian occlusion in dense or complex scenes.

[0006] In order to achieve the above purpose, the solution of the present application is: A target detection method based on head and body joint, comprising the following steps: A1: constructing a target detector for head and body joint detection, the target detector taking a single-stage target detection network YOLOX as a basic framework; A2: collecting and labeling a data set, the data set containing multiple images and multiple head annotation boxes and multiple pedestrian annotation boxes corresponding to the multiple images, and the head annotation boxes and the pedestrian annotation boxes one-to-one corresponding; A3: defining a loss function of the target detector, and training the target detector by a back propagation algorithm of stochastic gradient descent; A4: detecting each image of the input video by using the trained target detector to obtain a pedestrian prediction box set and a head prediction box set; A5: matching the pedestrian prediction box and the head prediction box by a head and human target matching score mechanism based on position correlation, combining the Hungarian algorithm and overlap checking, and outputting a head-pedestrian joint detection box.

[0007] Further, in step A1, the output set of the target detector is wherein, represents the prediction box output of the target detector, represents the pedestrian box output set, represents the head box output set; the prediction box output of the target detector is wherein, represents the horizontal coordinate of the top-left corner of the prediction box, represents the vertical coordinate of the top-left corner of the prediction box, represents the width of the prediction box, represents the height of the prediction box, represents the horizontal coordinate of the relative detection displacement of the prediction box, represents the vertical coordinate of the relative detection displacement of the prediction box, and the prediction label of the prediction box of the target detector is .

[0008] Further, in step A2, the annotation box set of the data set is wherein, represents the data set annotation box, represents the pedestrian annotation box, represents the head annotation box; the data set annotation box wherein represents the horizontal coordinate of the top-left corner of the annotation box, represents the vertical coordinate of the top-left corner of the annotation box, represents the width of the annotation box, represents the height of the annotation box; the relative displacement of the annotation box is the horizontal coordinate and the vertical coordinate .

[0009] Further, in step A3, the loss function includes a CIoU loss between the prediction box and the annotation box , a binary cross-entropy loss between the prediction label and the true label , and an Euclidean distance loss between the prediction relative displacement and the annotation relative displacement The target detector is trained using the backpropagation algorithm of stochastic gradient descent. Pedestrian marking boxes and head label box The relative displacement of the annotation box is the x-axis. with the vertical axis Define the pedestrian annotation box. , and This indicates the center coordinates of the header box corresponding to the pedestrian, i.e. , , This represents the x-coordinate of the top-left vertex of the header label box. This indicates the width of the header label box. This represents the ordinate of the top-left vertex of the header label box. This indicates the height of the head label box; for the head label box , and This indicates the center coordinates of the pedestrian marker box corresponding to the person's head, i.e. , , This represents the x-coordinate of the top-left vertex of the pedestrian marker box. This indicates the width of the pedestrian marker box. This represents the ordinate of the top-left corner of the pedestrian marker box. Indicates the height of the pedestrian marker box; the actual label of the marker box. , This indicates that the annotation box is a head frame. This indicates that the bounding box is a pedestrian box; therefore, the loss function of the single-stage object detector is:

[0010]

[0011]

[0012] in Indicates the number of prediction boxes. Represents the model prediction box With the target annotation box The distance between CIoU, This represents the binary cross-entropy loss function. The predicted label for the predicted bounding box. The loss function is trained using the backpropagation algorithm based on stochastic gradient descent to obtain the true label of the bounding box, thus obtaining a single-stage object detector.

[0013] Furthermore, in step A4, the pedestrian prediction box set , Head prediction box set ,in ; The specific method for step A5 is as follows: First, calculate the pedestrian prediction box. With head prediction box The matching score is calculated using the following formula:

[0014] in Indicates the first Individual pedestrian prediction box With the Individual head prediction box The matching score; The x-coordinate represents the relative displacement of the i-th pedestrian prediction box to the detection location; This represents the x-coordinate of the top-left corner of the predicted head frame for the j-th person. This represents the width of the prediction box for the j-th person's head. Let represent the ordinate of the displacement of the i-th pedestrian prediction box relative to the detection location. The y-coordinate of the top-left vertex of the j-th person's head prediction box. This represents the height of the predicted head bounding box for the j-th person; Secondly, this invention uses the Hungarian algorithm to obtain the scores. By performing a bipartite graph maximum matching operation, the pedestrian prediction box and the head prediction box can be matched. Finally, for each successfully paired pedestrian prediction box and head prediction box, an overlap check is performed. If the overlap check is greater than 0, that is, the pedestrian prediction box and the head prediction box intersect, and the joint detection box of pedestrian and head is output. If the overlap check is less than or equal to 0, that is, the pedestrian prediction box and the head prediction box do not intersect, and the matching result is discarded.

[0015] Another objective of this invention is to provide a multi-target pedestrian tracking method, which performs multi-target tracking based on the aforementioned head and body joint target detection method, and can effectively alleviate the pedestrian occlusion problem in dense or complex scenes.

[0016] To achieve the above objectives, the solution of the present invention is: A multi-target pedestrian tracking method employs the aforementioned head and body joint target detection method to perform target association and trajectory update strategies. It achieves initial association between the target pedestrian and the trajectory of the current frame through head bounding box intersection-union comparison, and then performs secondary association between unmatched targets and unmatched trajectories through pedestrian bounding box intersection-union comparison. Unmatched trajectories are deleted, and new trajectories are initialized for unmatched targets to complete the trajectory update.

[0017] Further, the multi-target pedestrian tracking method specifically comprises the following steps: Step B1, using the trained single-stage target detector, for the tth frame image of any video, obtaining a target pedestrian image set , wherein t represents the tth frame image of the video, represents the i th target in the frame image, represents the pedestrian prediction frame of the frame image, represents the head prediction frame of the frame image; the track set at the tth frame time is , represents the i th pedestrian track in the frame track, represents the pedestrian frame of the frame image, represents the head frame of the frame image; Step B2, for the target pedestrian image set to be matched and the track set of the current frame , target association is performed, and the target association similarity calculation formula is:

[0018] That is, the head frame coordinates are used for the associated matching of the target pedestrian and the track of the current frame, represents the head prediction frame of the target pedestrian , and the head frame of the pedestrian track to be matched ; Step B3, according to the similarity between the pedestrian image set and the track set to be matched , the existing track is matched and updated using the Hungarian algorithm to generate ; after association, the set of pedestrian targets that are not successfully matched in is defined as , and the track that is not matched in is defined as ; Step B4, for the i th target in the set of not successfully matched pedestrian targets and the target to be associated in the set of not matched tracks , , , , target association is performed, and the target association similarity calculation formula is:

[0019] That is, the pedestrian frame coordinates are used for the associated matching of the target pedestrian and the track of the current frame, represents the pedestrian prediction frame of the target pedestrian ;​​ pedestrian frame of the pedestrian track to be matched intersection over union of the pedestrian frame of the pedestrian track to be matched Step B5, according to the unsuccessful matching target set the similarity between the track set to be matched and the existing track set , the existing track is matched and updated using the Hungarian algorithm, and the result is merged into ; after association, the set of pedestrian targets in that are not successfully matched is defined as , the pedestrian targets in that are not matched to the track are defined as ; the unmatched is deleted, the new track is initialized for the pedestrian image in and updated and merged into , and the track updating is completed.

[0020] Another purpose of the present application is to provide a kind of based on the application of head and body joint target detection method and multi-target pedestrian tracking method in various computing limited terminal equipment, effectively alleviate the problem of pedestrian occlusion in dense or complex scene.

[0021] In order to achieve the above purpose, the solution of the present application is: The application of a target detection method based on head and body joint, the target detection method based on head and body joint is applied to camera or terminal equipment needing to process video or image.

[0022] The present application can also adopt the following solution: The application of a multi-target pedestrian tracking method, the multi-target pedestrian tracking method is applied to camera or terminal equipment needing to process video or image.

[0023] After adopting the above scheme, the target detection method based on head and body joint of the present application is based on single-stage target detection network, and a head and body joint detection target detector is designed, which introduces a head and body target matching mechanism based on position correlation, can accurately associate and match the head and body detection frame output by the target detector, and effectively alleviate the problem of pedestrian occlusion in dense or complex scene.

[0024] The multi-target pedestrian tracking method of the present application proposes an efficient target association and track updating strategy, based on the joint detection frame of head and body, which can effectively alleviate the problem of target loss or identity switching caused by pedestrian occlusion in dense crowd.

[0025] ​The multi-target pedestrian tracking method based on head and body joint detection and tracking can be efficiently and real-timely deployed on various computing limited terminal devices, and can effectively alleviate the pedestrian occlusion problem in dense or complex scenes. DETAILED DESCRIPTION

[0026] In order to further explain the technical solutions of the present application, the present application will be described in detail below through specific embodiments.

[0027] The present application discloses a target detection method based on head and body joint, comprising the following steps: A1: constructing a target detector for head and body joint detection, the target detector taking a single-stage target detection network YOLOX [1] as a basic framework; The output set of the target detector is , wherein represents the prediction box output of the target detector, represents the pedestrian box output set, represents the head box output set; the prediction box output of the target detector is , wherein represents the horizontal coordinate of the top-left corner of the prediction box, represents the vertical coordinate of the top-left corner of the prediction box, represents the width of the prediction box, represents the height of the prediction box, represents the horizontal coordinate of the relative detection displacement of the prediction box, represents the vertical coordinate of the relative detection displacement of the prediction box, and the prediction label of the prediction box of the target detector is .

[0028] A2: collecting and labeling the data set, the present application uses a private data set, which contains 79115 images and 701257 head annotation boxes and pedestrian annotation boxes corresponding to the 79115 images, and the head annotation boxes and the pedestrian annotation boxes are one-to-one corresponding; The annotation box set of the data set is , wherein represents the data set annotation box, represents the pedestrian annotation box, represents the head annotation box; the data set annotation box , wherein represents the horizontal coordinate of the top-left corner of the annotation box, represents the vertical coordinate of the top-left corner of the annotation box, represents the width of the annotation box, represents the height of the annotation box; the relative displacement of the annotation box is the horizontal coordinate and the vertical coordinate .

[0029] A3: Define the loss function for the object detector, which includes the CIoU loss between the predicted bounding box and the labeled bounding box. Binary cross-entropy loss between predicted labels and true labels And the Euclidean distance loss between the predicted relative displacement and the labeled relative displacement. The target detector is trained using the backpropagation algorithm of stochastic gradient descent. Pedestrian marking boxes and head label box The relative displacement of the annotation box is the x-axis. with the vertical axis Define the pedestrian annotation box. , and This indicates the center coordinates of the header box corresponding to the pedestrian, i.e. , , This represents the x-coordinate of the top-left vertex of the header label box. This indicates the width of the header label box. This represents the ordinate of the top-left vertex of the header label box. This indicates the height of the head label box; for the head label box , and This indicates the center coordinates of the pedestrian marker box corresponding to the person's head, i.e. , , This represents the x-coordinate of the top-left vertex of the pedestrian marker box. This indicates the width of the pedestrian marker box. This represents the ordinate of the top-left corner of the pedestrian marker box. Indicates the height of the pedestrian marker box. The actual label of the marker box. , This indicates that the annotation box is a head frame. This indicates that the bounding box is a pedestrian box. Therefore, the loss function of the single-stage object detector is...

[0030]

[0031]

[0032] in Indicates the number of prediction boxes. Represents the model's prediction box. With the target annotation box The CIoU distance between them[2], This represents the binary cross-entropy loss function. The predicted label for the predicted bounding box. The values ​​represent the true labels of the bounding boxes. A single-stage object detector can be obtained by training the loss function using a backpropagation algorithm based on stochastic gradient descent.

[0033] A4: Use the trained object detector to detect each frame of the input video to obtain a set of pedestrian prediction boxes. , , and the set of head prediction boxes ,in .

[0034] A5: To make the pedestrian prediction box set With head prediction box set To perform matching, this invention proposes a head and body target matching score mechanism based on location correlation.

[0035] First, calculate the pedestrian prediction box. With head prediction box The matching score is calculated using the following formula:

[0036] in Indicates the first Individual pedestrian prediction box With the Individual head prediction box The matching score. The x-coordinate represents the relative displacement of the i-th pedestrian prediction box to the detection location; This represents the x-coordinate of the top-left corner of the predicted head frame for the j-th person. This represents the width of the prediction box for the j-th person's head. Let represent the ordinate of the displacement of the i-th pedestrian prediction box relative to the detection location. The y-coordinate of the top-left vertex of the j-th person's head prediction box. This represents the height of the predicted head bounding box for the j-th person.

[0037] Secondly, this invention uses the Hungarian algorithm [3] to obtain the scores. By performing a bipartite graph maximum matching operation, the pedestrian prediction box and the head prediction box can be matched.

[0038] Finally, the intersection-over-union (IoU) check is performed on each successfully matched pair of pedestrian and head prediction boxes. If the IoU is greater than 0, the pedestrian and head prediction boxes intersect, and a joint pedestrian-head detection box is output; if the IoU is less than or equal to 0, the pedestrian and head prediction boxes do not intersect, and the matching result is discarded.

[0039] According to the head and body target matching mechanism based on the position correlation, the head and body detection frames output by the target detector can be quickly and accurately associated and matched.

[0040] The application also discloses a multi-target pedestrian tracking method, which uses the target detection method based on the head and body combination to perform a target association and trajectory updating strategy, preliminarily associates the target pedestrian with the trajectory of the current frame through the intersection-over-union of the head frame, secondarily associates the unmatched target with the unmatched trajectory through the intersection-over-union of the pedestrian frame, deletes the unmatched trajectory, initializes a new trajectory for the unmatched target, and completes the trajectory updating. Specifically, the method comprises the following steps. Step B1, using the trained single-stage target detector, obtaining a target pedestrian image set for the t-th frame image of any video , wherein t represents the t-th frame image of the video, represents the i-th target in the frame image, represents the pedestrian prediction frame of the frame image, represents the head prediction frame of the frame image. The trajectory set at the t-th frame time is , represents the i-th pedestrian trajectory in the frame trajectory, represents the pedestrian frame of the frame image, represents the head frame of the frame image. Step B2, performing target association on the target pedestrian image set and the trajectory set

[0041] of the current frame, and the target association similarity calculation formula is:

[0042] That is, the head frame coordinates are used to associate and match the target pedestrian with the trajectory of the current frame, represents the intersection-over-union of the head prediction frame of the target pedestrian and the head frame of the pedestrian trajectory to be matched.

[0043] Step B3, according to the similarity between the pedestrian image set and the to-be-matched trajectory set , using the Hungarian algorithm [3] to match and update the existing trajectory to generate . After the association, the set of unmatched pedestrian targets in is defined as , and the set of unmatched trajectories in ​​The unmatched trajectory is defined as .

[0044] Step B4: For the above-mentioned set of pedestrian targets that did not match successfully In With the set of unmatched trajectories The target to be associated The formula for calculating the similarity of target associations is as follows:

[0045] Even if the pedestrian bounding box coordinates are used to match the target pedestrian with the trajectory of the current frame, Indicates the target pedestrian pedestrian prediction box Pedestrian trajectories to be matched pedestrian frame The intersection and union ratio.

[0046] Step B5: Based on the set of unsuccessfully matched targets The set of trajectories to be matched is The similarity between them is used to match and update existing trajectories using the Hungarian algorithm, and the results are then merged into... After association, it will The set of pedestrian targets that were not successfully matched is defined as follows: ,Will The unmatched trajectory is defined as Delete unmatched pairs. ,against The pedestrian image in the image is initialized with a new trajectory and updated and incorporated. In the middle, the trajectory update is completed.

[0047] This invention utilizes a joint detection bounding box of human heads and bodies, and based on the above-mentioned efficient target association and trajectory update strategy, to effectively alleviate the problem of target loss or identity switching caused by pedestrian occlusion in dense crowds.

[0048] This invention uses a private video set to test the multi-object tracking method for cedar trees, achieving a multi-object tracking accuracy (MOTA) of 73.4%.

[0049] Furthermore, the multi-target tracking method proposed in this invention can be efficiently and in real-time deployed on computationally limited embedded terminal devices, achieving a computing performance of 30 frames per second (30fps) on the HiSilicon Hi3516CV500 chip.

[0050] The above embodiments are not limited to the product form and style of the present application, and any appropriate changes or modifications made by those skilled in the art to the above embodiments shall be deemed not to depart from the scope of the patent of the present application.

[0051] References: [1] Ge, Zheng, et al. YOLOX: A Better Performance of Object Detection than YOLO Series in 2021 [EB / OL]. arXiv preprint, arXiv:2107.08430, 2021.

[0052] [2] Zheng, Zhaohui, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren. Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression [C]. AAAI Conference on Artificial Intelligence, 34 (07): 12993-13000, 2020.

[0053] [3] Harold W. Kuhn. Hungarian Methods for the Assignment Problem [J]. Naval Research Logistics Quarterly, 2 (1-2): 83-97, 1955. W Harold W. Kuhn. Hungarian Methods for the Assignment Problem [J]. Naval Research Logistics Quarterly, 2 (1-2): 83-97, 1955.

Claims

1. A target detection method based on head and body joint, comprising the following steps: A1: constructing a target detector for head and body joint detection, which takes a single-stage target detection network YOLOX as the basic framework; A2: collecting and labeling a data set containing multiple images and multiple head annotation boxes and multiple pedestrian annotation boxes corresponding to the multiple images, and the head annotation boxes and the pedestrian annotation boxes correspond one by one; A3: defining the loss function of the target detector, and training the target detector by the back propagation algorithm of stochastic gradient descent; A4: using the trained target detector to detect each frame of image of the input video to obtain a set of pedestrian prediction boxes and a set of head prediction boxes; A5: realizing the matching of pedestrian prediction boxes and head prediction boxes by a head and body target matching score mechanism based on position correlation, combining the Hungarian algorithm and overlap checking, and outputting the head-pedestrian joint detection box.

2. The method of claim 1, wherein the method is based on a combination of human head and human body detection. In step A1, the output set of the target detector is wherein, denotes the predicted bounding box output of the target detector, denotes the pedestrian bounding box output set, denotes the head bounding box output set; the predicted bounding box output of the target detector is wherein, denotes the horizontal coordinate of the top-left corner of the predicted bounding box, denotes the vertical coordinate of the top-left corner of the predicted bounding box, denotes the width of the predicted bounding box, denotes the height of the predicted bounding box, denotes the horizontal coordinate of the relative detection displacement of the predicted bounding box, denotes the vertical coordinate of the relative detection displacement of the predicted bounding box, and the predicted label of the predicted bounding box of the target detector is .

3. The method of claim 1, wherein the method is based on a combination of human head and human body detection. In step A2, the set of bounding boxes of the data set is wherein, denotes a bounding box of the data set, denotes a pedestrian bounding box, denotes a head bounding box; the set of bounding boxes of the data set wherein denotes the horizontal coordinate of the top-left vertex of the bounding box, denotes the vertical coordinate of the top-left vertex of the bounding box, denotes the width of the bounding box, denotes the height of the bounding box; the relative displacement of the bounding box is the horizontal coordinate and the vertical coordinate .

4. The method of claim 1, wherein the method is based on a combination of human head and human body detection. In step A3, the loss function comprises a CIoU loss between the predicted box and the labeled box , a binary cross-entropy loss between the predicted label and the true label , and an Euclidean distance loss between the predicted relative displacement and the labeled relative displacement , and the target detector is trained by a back propagation algorithm of stochastic gradient descent. For pedestrian bounding box and head bounding box The relative displacement of the bounding box is the horizontal coordinate and the vertical coordinate For pedestrian bounding box , and represents the center coordinates of the head bounding box corresponding to the pedestrian, that is , , represents the horizontal coordinate of the top-left vertex of the head bounding box, represents the width of the head bounding box, represents the vertical coordinate of the top-left vertex of the head bounding box, represents the height of the head bounding box; for head bounding box , and represents the center coordinates of the pedestrian bounding box corresponding to the head, that is , , represents the horizontal coordinate of the top-left vertex of the pedestrian bounding box, represents the width of the pedestrian bounding box, represents the vertical coordinate of the top-left vertex of the pedestrian bounding box, represents the height of the pedestrian bounding box; the true label of the bounding box , represents that the bounding box is a head box, represents that the bounding box is a pedestrian box; Therefore, the loss function of the single-stage target detector is: wherein denotes the number of predicted boxes, denotes the predicted box of the model and the CIoU distance between the target labeled box, denotes the binary cross-entropy loss function, is the predicted label of the predicted box, is the true label of the labeled box, using the back propagation algorithm based on stochastic gradient descent to train the above loss function, a single-stage target detector can be obtained.​ 5. The method of claim 1, wherein the method is based on a combination of human head and human body detection. In step A4, the pedestrian prediction box set , , the human head prediction box set wherein ; The specific method of step A5 is: First, the pedestrian prediction box is calculated The matching score with the human head prediction box is calculated using the following formula: wherein represents the matching score of the i-th pedestrian prediction box with the j-th person head prediction box represents the horizontal coordinate of the i-th pedestrian prediction box relative to the detection displacement; represents the horizontal coordinate of the top-left corner vertex of the j-th person head prediction box, represents the width of the j-th person head prediction box, represents the vertical coordinate of the i-th pedestrian prediction box relative to the detection displacement, the vertical coordinate of the top-left corner vertex of the j-th person head prediction box, represents the height of the j-th person head prediction box;​​​ Secondly, the application uses the Hungarian algorithm to the obtained scores Carries out the bipartite graph maximum matching operation, namely, the matching of the pedestrian prediction frame and the human head prediction frame. Finally, overlap checking is performed on each pair of successfully matched pedestrian prediction box and head prediction box, if the overlap checking is greater than 0, that is, the pedestrian prediction box intersects with the head prediction box, that is, the head-pedestrian joint detection box is output; if the overlap checking is less than or equal to 0, that is, the pedestrian prediction box does not intersect with the head prediction box, that is, the matching result is discarded.

6. A multi-target pedestrian tracking method, characterized by, The target association and trajectory updating strategy is executed by the target detection method based on head and body joint of any one of claims 1-5, the target pedestrian is preliminarily associated with the current frame trajectory through the head box intersection-over-union, and the secondary association is performed on the unmatched target and the unmatched trajectory through the pedestrian box intersection-over-union, the unmatched trajectory is deleted, the unmatched target is initialized as a new trajectory, and the trajectory updating is completed.

7. The multi-target pedestrian tracking method of claim 6, wherein, Specifically, the following steps are included: Step B1: Using the single-stage object detector trained above, obtain a set of target pedestrian images for the t-th frame of any video. Where t represents the t-th frame of the video. This indicates the first [image element] in this frame. One goal, This represents the predicted pedestrian bounding box for this frame of the image. The bounding box for the head in this frame is represented; the set of trajectories at time t is... , Indicates the first [frame] in the trajectory of this frame. A pedestrian's trajectory This represents the pedestrian bounding box in this frame of the image. This represents the bounding box of the human head in the image frame; Step B2, for the target pedestrian image set to be matched Trajectory set of the current frame Target association is performed, and a target association similarity calculation formula is: Even if the head bounding box coordinates are used for the association matching of the target pedestrian and the current frame trajectory, a head prediction bounding box representing the target pedestrian a head bounding box of the pedestrian trajectory to be matched the intersection over union of the head prediction bounding box and the head bounding box of the pedestrian trajectory to be matched​​ Step B3: Based on the pedestrian image set The set of trajectories to be matched is The similarity between them is used to match and update existing trajectories using the Hungarian algorithm. After association, The set of pedestrian targets that were not successfully matched is defined as follows: ,Will The unmatched trajectory is defined as ; Step B4, target association for the above unmatched pedestrian target set with the to-be-associated targets in the unmatched trajectory set Target association similarity calculation formula:​​ Even if the pedestrian box coordinates are used for the target pedestrian and the current frame trajectory association matching, representing the target pedestrian pedestrian prediction box pedestrian box of the pedestrian trajectory to be matched pedestrian box intersection over union; Step B5: Based on the set of unsuccessfully matched targets The set of trajectories to be matched is The similarity between them is used to match and update existing trajectories using the Hungarian algorithm, and the results are then merged into... After association, The set of pedestrian targets that were not successfully matched is defined as follows: ,Will The unmatched trajectory is defined as ; Delete unmatched ,against The pedestrian image in the image is initialized with a new trajectory and updated and incorporated. In the middle, the trajectory update is completed.

8. An application of a target detection method based on a combination of human head and human body, characterized in that: The target detection method based on head and body joint of any one of claims 1-5 is applied to a camera or a terminal device that needs to process video or image.

9. Use of a multi-target pedestrian tracking method, characterized in that: The multi-target pedestrian tracking method of claim 6 or 7 is applied to a camera or a terminal device that needs to process video or image.

Images