Multi-target tracking anti-occlusion processing method integrating directional scores and geometric features

Abstract

The present invention discloses a multi-target tracking anti-occlusion processing method that integrates directional scores and geometric features, comprising: tracking a target in a video to obtain N tracks, N prediction frames of a current frame, information of the prediction frames, M detection frames of the current frame, and information of the detection frames; the N tracks are obtained after tracking from a first frame to a frame previous to the current frame; determining motion features and geometric features of the detection frames and the prediction frames; matching each detection frame with the N tracks based on the above information, motion features, geometric features, and the N prediction frames, and when there is a detection frame that does not match any of the N tracks, determining a directional score between the detection frame and each prediction frame; matching the detection frame with the N tracks again based on the directional scores, and when there is a track that matches the detection frame, updating the N tracks to obtain an updated track; and tracking the target of the next frame based on the updated track until the last frame of the video is tracked.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a multi-target tracking anti-occlusion processing method that integrates directional scores and geometric features. Background Art

[0002] In the field of drone aerial photography, target detection, recognition, and tracking are crucial research areas, encompassing multiple fields, including computer vision, image processing, pattern recognition, and artificial intelligence. Target recognition and tracking for drone aerial photography involves identifying and marking targets in the ground scene captured by a drone during flight. For moving targets, tracking is achieved by determining their position, size, and trajectory across successive frames of video. Currently, drone target recognition and tracking technology is widely used in civilian applications and has proven to be highly valuable.

[0003] Because target tracking often fails due to occlusion, efficient and robust target tracking in occluded scenarios has become a hot topic of research. Deep learning-based target tracking algorithms can better extract target features and more accurately represent target information, demonstrating significant advantages in tracking in occluded scenarios. However, existing multi-target tracking algorithms often fail in occluded scenarios due to reduced or even absent target surface information. Summary of the Invention

[0004] In order to solve the above problems existing in the related art, the present invention provides a multi-target tracking anti-occlusion processing method that integrates directional scores and geometric features. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] The present invention provides a multi-target tracking anti-occlusion processing method that integrates directional scores and geometric features, comprising:

[0006] Obtain a video sequence to be tracked;

[0007] Track the target in the video sequence to be tracked, and obtain N tracks, N prediction frames of the current frame and information about each prediction frame, and M detection frames of the current frame and information about each detection frame; the N tracks are tracks obtained from the first frame to the frame before the current frame; M and N are both integers greater than 0;

[0008] Determine motion features and geometric features of each detection frame and each prediction frame based on the information;

[0009] Matching each detection frame with the N trajectories based on the information, the motion features, the geometric features, and the N prediction frames, and determining a directional score between the mismatched detection frame and each prediction frame when there is a mismatched detection frame that does not match any of the N trajectories; the directional score is a fusion of an intersection-over-union (IoU) and a directional similarity between the mismatched detection frame and the prediction frame;

[0010] Matching the mismatch detection frame with the N tracks again according to the directionality score, and when there is a track matching the mismatch detection frame, updating the N tracks to obtain an updated track;

[0011] The target of the next frame is tracked according to the updated trajectory until the last frame of the video sequence to be tracked is tracked.

[0012] In some embodiments, the information includes: the width and height of the detection box, and the coordinates of the center point; and determining the motion features and geometric features of each detection box and each prediction box based on the information includes:

[0013] For each detection frame or each prediction frame, extracting a motion feature of the frame from the current frame according to the information of the frame;

[0014] According to the first preset weight, and the width, height, and center point coordinates of each detection box and each prediction box, the geometric features between each detection box and each prediction box are calculated.

[0015] In some embodiments, the geometric features between the j-th detection box and the l-th prediction box in the current frame are expressed as follows:

[0016]

[0017] Wherein, j is an integer greater than 0 and less than or equal to M, l is an integer greater than 0 and less than or equal to N, d ge (l,j) is the geometric feature between the jth detection frame and the lth prediction frame, β1 and β2 are the first preset weights, (x j ,y j ) is the center point coordinate of the j-th detection frame, w j is the width of the j-th detection box, h j is the height of the j-th detection box, (x l ,y l ) is the center point coordinate of the l-th prediction box, w l is the width of the l-th prediction box, h l is the height of the l-th prediction box.

[0018] In some embodiments, matching each detection frame with the N tracks based on the information, the motion features, the geometric features, and the N prediction frames, and when there is a mismatched detection frame that does not match any of the N tracks, determining a directionality score between the mismatched detection frame and each detection frame includes:

[0019] Calculate the similarity between each detection frame and each prediction frame based on the motion features of each prediction frame in the N prediction frames, the motion features of each detection frame, and the geometric features between each prediction frame and each detection frame;

[0020] For each detection frame, when the similarity between the detection frame and N prediction frames is less than a preset threshold, the detection frame is determined to be the mismatched detection frame, and the directionality score between the mismatched detection frame and each prediction frame is determined.

[0021] In some embodiments, the mismatch detection box is the xth detection box, each of the prediction boxes is the yth prediction box, and the yth prediction box corresponds to the yth track of the N tracks, x is an integer greater than 0 and less than or equal to M, and y is an integer greater than 0 and less than or equal to N; determining the directionality score between the mismatch detection box and each prediction box includes:

[0022] Obtain the angle between the prediction box and the detection box corresponding to the y-th track in the previous frame;

[0023] Determine the angle between the xth detection frame and the yth prediction frame based on the angle between the prediction frame and the detection frame corresponding to the yth trajectory in the previous frame, and the coordinates of the center point of the xth detection frame and the coordinates of the center point of the yth prediction frame;

[0024] Calculate the directional similarity between the prediction box and the detection box in the current frame and the previous frame based on the angle between the prediction box and the detection box corresponding to the y-th trajectory in the previous frame, and the angle between the x-th detection box and the y-th prediction box;

[0025] Calculate the intersection-over-union (IoU) ratio between the x-th detection frame and the y-th prediction frame;

[0026] Determine a directionality score between the x-th detection frame and the y-th prediction frame according to a second preset weight, the directional similarity, and the intersection-over-union ratio.

[0027] In some embodiments, determining the angle between the xth detection frame and the yth prediction frame based on the angle between the prediction frame and the detection frame corresponding to the yth trajectory in the previous frame, and the center point coordinates of the xth detection frame and the center point coordinates of the yth prediction frame includes:

[0028] According to the center coordinates of the xth detection frame and the center coordinates of the yth prediction frame, calculate the initial angle θ' corresponding to the xth detection frame and the yth prediction frame (n) ;

[0029] According to the initial angle θ' (n) , the angle θ between the prediction box and the detection box corresponding to the y-th track in the previous frame (n-1) , and a preset coefficient to determine the angle between the x-th detection frame and the y-th prediction frame.

[0030] In some embodiments, the expression of the directionality score between the x-th detection box and the y-th prediction box is:

[0031] IoU θ =β·IoU+(1-β)·θ closeness ;

[0032]

[0033] Among them, IoU θ is the directionality score, β is the second preset weight, IoU is the intersection over union ratio, θ closeness is the direction similarity, θ2 is the angle between the x-th detection frame and the y-th prediction frame, and θ1 is the angle between the prediction frame and the detection frame corresponding to the y-th track in the previous frame.

[0034] In some embodiments, matching the mismatch detection frame with the N tracks again according to the directionality score, and updating the N tracks when there is a track matching the mismatch detection frame to obtain an updated track, includes:

[0035] Comparing the directionality score between the mismatch detection frame and each prediction frame with a preset threshold to obtain a comparison result;

[0036] When the comparison result shows that the directionality score between the mismatching detection frame and the nth prediction frame among the N prediction frames is greater than or equal to the preset threshold, it indicates that there is a track matching the mismatching detection frame, the mismatching detection frame is associated with the nth track corresponding to the nth prediction frame, and information of the mismatching detection frame is used as updated track information of the nth track to obtain an updated nth track;

[0037] The N trajectories are updated according to the updated n-th trajectory to obtain an updated trajectory.

[0038] In some embodiments, the method further comprises:

[0039] When it is determined according to the directionality score that the mismatched detection frame still does not match the N trajectories, obtaining an updated trajectory based on the matching conditions between the detection frames other than the mismatched detection frame among the M detection frames and the N trajectories, and using the mismatched detection frame as a candidate trajectory;

[0040] performing frame-by-frame tracking of video frames after the current frame according to the candidate trajectory and the updated trajectory to obtain tracking results of multiple consecutive frames after the current frame;

[0041] When the tracking results of the plurality of consecutive frames all indicate that there is no detection frame matching the candidate track, all relevant information of the candidate track is deleted, and target tracking of subsequent video frames is continued;

[0042] When the tracking results of the plurality of consecutive frames all indicate that there is a detection frame matching the candidate trajectory, the candidate trajectory is used as a new trajectory, and target tracking is continued for subsequent video frames based on all currently obtained trajectories.

[0043] In some embodiments, performing target tracking on the video sequence to be tracked to obtain N tracks, N prediction frames of the current frame and information about each prediction frame, and M detection frames of the current frame and information about each detection frame includes:

[0044] Performing target tracking on the video sequence to be tracked to obtain the N trajectories, as well as the M detection frames of the current frame and the information of each detection frame;

[0045] According to the N trajectories, N prediction frames in the current frame are predicted by a Kalman filtering method; the N prediction frames correspond one-to-one to the N trajectories.

[0046] The present invention has the following beneficial technical effects:

[0047] The present invention improves the accuracy of the association between trajectories and targets between adjacent video frames by introducing geometric features as the metric for the first matching. At the same time, since the direction of the tracking trajectory will not change suddenly during the target tracking process, the present invention introduces a directional score that combines the IoU score and directional similarity as the metric for the second matching, thereby greatly reducing the situation where tracking failures are caused by the reduction or even disappearance of target appearance information in occlusion scenarios, enhancing the tracking effect of occluded targets, and improving the robustness of target tracking.

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 An optional flowchart of a multi-target tracking and anti-occlusion processing method that integrates directional scores and geometric features provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0051] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0052] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0053] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0054] Figure 1 FIG. 1 is a flow chart of a multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0055] S101: Acquire a video sequence to be tracked.

[0056] S102. Track the target in the video sequence to be tracked, and obtain N trajectories, N prediction frames of the current frame and information about each prediction frame, and M detection frames of the current frame and information about each detection frame; the N trajectories are trajectories obtained after tracking from the first frame to the frame before the current frame; M and N are both integers greater than 0.

[0057] Here, the target tracker starts tracking the target from the first frame of the video sequence to be tracked until it tracks the previous frame of the current frame, and obtains N tracks. Then, the target tracker performs target detection on the current frame, obtains M detection frames of the current frame and information of each detection frame, and predicts N prediction frames in the current frame based on these N tracks through the Kalman filtering method, and these N prediction frames correspond one-to-one to these N tracks.

[0058] Here, the information of each detection frame includes: the width, height, center point coordinates of the detection frame, and the category of the object contained in the detection frame.

[0059] Here, each of the N trajectories has a trajectory number (trajectory ID), and different trajectories have different IDs. Each trajectory has trajectory information, and the trajectory information of each trajectory is the information of the last detection box associated with the trajectory in the video frame and the trajectory ID of the trajectory.

[0060] S103: Determine motion features and geometric features of each detection frame and each prediction frame based on the information.

[0061] Here, the motion features of each detection frame or each predicted frame are the Mahalanobis distance and cosine distance of the frame. Specifically, based on the center point coordinates, width, and height of the frame, the image of the target corresponding to the frame is extracted from the current frame, and the Mahalanobis distance and cosine distance of the frame are extracted from the image.

[0062] Here, the geometric feature between each detection box and each prediction box is the distance between the detection box and the prediction box in terms of geometric information. For example, the geometric feature between the jth detection box and the lth prediction box in the current frame is expressed as follows: Wherein, j is an integer greater than 0 and less than or equal to M, l is an integer greater than 0 and less than or equal to N, d ge (l,j) is the geometric feature between the jth detection box and the lth prediction box, β1 and β2 are the first preset weights, (x j ,y j ) is the center point coordinate of the jth detection box, w j is the width of the j-th detection box, h j is the height of the j-th detection box, (x l ,y l ) is the coordinate of the center point of the l-th prediction box, w l is the width of the l-th prediction box, h l is the height of the l-th prediction box. Specifically, It represents the offset distance. It represents the target scale, and β1 and β2 are used as weights to adjust the parameters.

[0063] S104. Match each detection frame with N tracks based on the information, motion features, geometric features, and N prediction frames. When there is a mismatched detection frame that does not match any of the N tracks, determine the directional score between the mismatched detection frame and each prediction frame. The directional score is a fusion of the intersection-over-union ratio and the directional similarity between the mismatched detection frame and the prediction frame.

[0064] Here, step S104 includes:

[0065] S1041. Calculate the similarity between each detection frame and each prediction frame based on the motion features of each prediction frame in the N prediction frames, the motion features of each detection frame, and the geometric features between each prediction frame and each detection frame.

[0066] Specifically, the data association matrix is generated by combining Mahalanobis distance and geometric features Among them, C i,j is the (i, j)th element of the similarity matrix C, is the motion Mahalanobis distance between the predicted box of the i-th track and the j-th detection box, indicating the similarity of motion features; d ge(i, j) is the geometric feature between the prediction box of the i-th track and the j-th detection box, indicating the similarity of geometric information; data association matrix That is, the similarity between the predicted box of the i-th track and the j-th detection box.

[0067] In some embodiments, a screening threshold can be set, for example, 0.5. When the motion Mahalanobis distance or geometric feature between the prediction box of the i-th trajectory and the j-th detection box is less than 0.5, no matching processing is performed between the prediction box of the i-th trajectory and the j-th detection box.

[0068] S1042. For each detection frame, when the similarity between the detection frame and the N prediction frames is less than a preset threshold, determine the detection frame as a mismatch detection frame, and determine the directionality score between the mismatch detection frame and each prediction frame.

[0069] Specifically, for each detection frame, when the similarity between the detection frame and the N prediction frames is less than a preset threshold, it means that the detection frame fails to match the N tracks, so that the detection frame can be used as a mismatched detection frame, and the directional score between the mismatched detection frame and each of the N prediction frames is calculated. In addition, when the similarity between the detection frame and the n-th prediction frame among the N prediction frames is greater than or equal to the preset threshold, it means that the target corresponding to the detection frame and the target corresponding to the n-th track corresponding to the n-th prediction frame belong to the same target. Therefore, the detection frame is associated with the n-th track corresponding to the n-th prediction frame (that is, the Hungarian algorithm is used to assign the n-th tracker to the detection frame), and the information of the mismatched detection frame is used as the updated track information of the n-th track to obtain the updated n-th track; n is an integer greater than 0 and less than or equal to N.

[0070] In some embodiments, when the mismatched detection frame is the xth detection frame among M detection frames, it is necessary to calculate the directionality score between the xth detection frame and the yth prediction frame among N prediction frames, and the y prediction frames correspond to the yth track among N tracks, the directionality score can be calculated by the following steps:

[0071] S201, obtain the angle θ between the prediction box and the detection box corresponding to the y-th track in the previous frame (n-1) .

[0072] Here, n represents the current frame in the video to be tracked, and n-1 represents the previous frame of the current frame.

[0073] S202: Calculate the initial angle θ' between the xth detection frame and the yth prediction frame based on the center point coordinates of the xth detection frame and the center point coordinates of the yth prediction frame. (n) .

[0074] Specifically, Among them, (x1, y1) is the coordinate of the center point of the y-th prediction box, and (x2, y2) is the coordinate of the center point of the x-th detection box.

[0075] S203, according to the initial angle θ' (n) , the angle θ between the prediction box and the detection box corresponding to the y-th track in the previous frame (n-1) , and the preset coefficient α, determine the angle θ between the xth detection frame and the yth prediction frame (n) .

[0076] Specifically, θ (n) =α·θ' (n) +(1-α)·θ (n-1) .

[0077] Here, θ simply calculates the angle between two points and updates the direction of the target tracker at each time step. The present invention handles extreme fluctuations in the direction measurement by adding a decay variable α to the direction measurement. The setting of α helps to smooth the direction readings over a period of time, so that subsequent θ-based tracking can be used to update the direction of the target tracker. (n) The calculated directionality score is more stable.

[0078] S204: Angle θ between the prediction frame and the detection frame corresponding to the y-th track in the previous frame (n-1) , and the angle θ between the x-th detection box and the y-th prediction box (n) , calculate the directional similarity θ between the prediction box and the detection box in the current frame and the previous frame closeness .

[0079] Specifically, Where abs(.) is the absolute value function, θ1 is the above θ (n-1) , θ2 is the above θ (n) θ closeness That is, it characterizes the directionality of the target tracker and how close the possible detections are.

[0080] S205. Calculate the intersection over union (IoU) between the x-th detection box and the y-th prediction box.

[0081] Here, the area intersection-over-union ratio between the xth detection box and the yth prediction box can be calculated based on the width, height, and center point coordinates of the xth detection box and the yth prediction box.

[0082] S206, according to the second preset weight β, the direction similarity θ closeness And the intersection over union (IoU) determines the directional score IoU between the x-th detection box and the y-th prediction box θ .

[0083] Specifically, IoU θ =β·IoU+(1-β)·θ closeness , where β controls the weight given to IoU and a 1-β weight is assigned to θ closeness ,Using β to control the corresponding weights helps to parameterize the contribution of the IOU metric, and at the same time can achieve the directionality of the expected results under different conditions. β values will be higher when the objects move with dynamic trajectories. At this time, the directionality metric may not achieve the best results due to the constant change of direction. Similarly, when the directionality of the objects is more uniform, the value of β will be lower, which can better handle occlusion when objects pass each other in different directions.

[0084] S105 , matching the mismatched detection frame with the N tracks again according to the directionality score. When there is a track that matches the mismatched detection frame, updating the N tracks to obtain an updated track.

[0085] Specifically, the directional score between the mismatch detection frame and each prediction frame is compared with a preset threshold to obtain a comparison result; when the comparison result shows that the directional score between the mismatch detection frame and the nth prediction frame among the N prediction frames is greater than or equal to the preset threshold, it indicates that there is a track matching the mismatch detection frame, the mismatch detection frame is associated with the nth track corresponding to the nth prediction frame, and the information of the mismatch detection frame is used as the updated track information of the nth track to obtain the updated nth track; the nth track among the N tracks is updated according to the updated nth track to obtain the updated track (i.e., the updated N tracks).

[0086] S106 , tracking the target of the next frame according to the updated trajectory until the last frame of the video sequence to be tracked is tracked.

[0087] Specifically, the updated trajectory obtained in S105 is all the trajectories obtained after the tracking of the current frame is completed. When tracking the target in the next frame of the current frame, all the trajectories obtained after the tracking of the current frame is completed can be used to continue tracking the target in the next frame, and the tracking principle is the same as that of the current frame.

[0088] In some embodiments, the above method further includes steps S107 to S110, and step S107 is a case other than the case in step S105 where N trajectories are updated to obtain updated trajectories when there is a trajectory matching the mismatched detection frame. Steps S107 to S110 are as follows:

[0089] S107. When it is determined according to the directionality score that the mismatched detection frame still does not match the N trajectories, an updated trajectory is obtained according to the matching conditions between the detection frames other than the mismatched detection frame in the M detection frames and the N trajectories, and the mismatched detection frame is used as a candidate trajectory.

[0090] For example, when the first M-1 detection frames among M detection frames are matched with one of the N trajectories, and the Mth detection frame among the M detection frames still does not match the N trajectories during the first matching and the second matching, the information of the trajectory matching the detection frame can be updated according to the information of each detection frame among the first M-1 detection frames of the M detection frames, and after the first M-1 detection frames among the M detection frames are all updated, the updated trajectory can be obtained, and the updated trajectory is all the trajectories obtained after the tracking of the current frame is completed.

[0091] S108 , tracking the video frames after the current frame frame by frame according to the candidate trajectory and the updated trajectory, and obtaining tracking results of multiple consecutive frames after the current frame.

[0092] Continuing with the above example, after all trajectories are obtained after tracking the current frame, the Mth detection frame is used as a candidate trajectory to continue target tracking for the next frame of the current frame. Specifically, all detection frames in the next frame are detected, and based on all trajectory predictions obtained after tracking the current frame and the candidate trajectory, all prediction frames in the next frame are predicted by the Kalman filtering method. Based on all detection frames, all prediction frames, and all trajectories obtained after tracking the current frame and the candidate trajectory, specific target tracking is performed on the next frame (the specific tracking principle is the same as the tracking principle for the current frame) to obtain the target tracking result of the next frame. Afterwards, target tracking is continued for the next frame based on the target tracking result of the next frame. Tracking is continued in this way until tracking results for multiple consecutive frames after the current frame are obtained, for example, until tracking results for three consecutive frames after the current frame are obtained.

[0093] S109 : When the tracking results of multiple consecutive frames all indicate that there is no detection frame matching the candidate track, all relevant information of the candidate track is deleted, and target tracking of subsequent video frames is continued.

[0094] For example, when the tracking results of three consecutive frames after the current frame are obtained, and the tracking results of these three consecutive frames all indicate that there is no detection box matching the candidate track, all relevant information of the candidate track is deleted from the track list used to store track information, and the Kalman filter used for prediction box prediction is updated to continue target tracking for subsequent video frames.

[0095] S110 : When the tracking results of multiple consecutive frames all indicate that there is a detection frame matching the candidate trajectory, the candidate trajectory is used as a new trajectory, and target tracking is continued for subsequent video frames based on all currently obtained trajectories.

[0096] For example, when the tracking results of three consecutive frames after the current frame are obtained, and the tracking results of these three consecutive frames all indicate that there is a detection box matching the candidate track, the candidate track is taken as a new track and a track ID is assigned to it. After that, the target tracking of subsequent video frames is continued based on all the currently obtained tracks.

[0097] The present invention first introduces geometric features as the metric for the first data matching, which improves the accuracy of trajectory and target association between adjacent image frames. At the same time, by fusing the Intersection over Union (IoU) score and directionality metric, it solves the problem of target ID jumps in long-term tracking and dense target scenes. Finally, combined with cascade matching to perform trajectory matching in stages, it enhances the tracking effect of occluded targets and improves the robustness of target tracking.

[0098] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A multi-target tracking anti-occlusion processing method that integrates directional scores and geometric features, characterized in that: include: Obtain a video sequence to be tracked; Track the target in the video sequence to be tracked, and obtain N tracks, N prediction frames of the current frame and information about each prediction frame, and M detection frames of the current frame and information about each detection frame; the N tracks are tracks obtained from the first frame to the frame before the current frame; M and N are both integers greater than 0; Determine motion features and geometric features of each detection frame and each prediction frame based on the information; Matching each detection box with the N tracks based on the information, the motion features, the geometric features, and the N prediction boxes, and when there is a mismatched detection box that does not match any of the N tracks, determining a directionality score between the mismatched detection box and each prediction box; The directionality score is a fusion of the intersection-over-union ratio and the direction similarity between the mismatch detection frame and the prediction frame; Matching the mismatch detection frame with the N tracks again according to the directionality score, and when there is a track matching the mismatch detection frame, updating the N tracks to obtain an updated track; The target of the next frame is tracked according to the updated trajectory until the last frame of the video sequence to be tracked is tracked.

2. The multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features according to claim 1 is characterized in that: The information includes: the width and height of the detection frame, and the coordinates of the center point; and determining the motion features and geometric features of each detection frame and each prediction frame based on the information includes: For each detection frame or each prediction frame, extracting a motion feature of the frame from the current frame according to the information of the frame; According to the first preset weight, and the width, height, and center point coordinates of each detection box and each prediction box, the geometric features between each detection box and each prediction box are calculated.

3. The multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features according to claim 2, characterized in that: The geometric features between the jth detection frame and the lth prediction frame in the current frame are expressed as follows: Wherein, j is an integer greater than 0 and less than or equal to M, l is an integer greater than 0 and less than or equal to N, d ge (l,j) is the geometric feature between the jth detection frame and the lth prediction frame, β1 and β2 are the first preset weights, (x j ,y j ) is the coordinate of the center point of the j-th detection frame, w j is the width of the j-th detection box, h j is the height of the j-th detection box, (x l ,y l ) is the center point coordinate of the l-th prediction box, w l is the width of the l-th prediction box, h l is the height of the l-th prediction box.

4. The multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features according to claim 1, characterized in that: Matching each detection frame with the N trajectories according to the information, the motion features, the geometric features, and the N prediction frames, and when there is a mismatched detection frame that does not match any of the N trajectories, determining a directionality score between the mismatched detection frame and each detection frame, including: Calculate the similarity between each detection frame and each prediction frame based on the motion features of each prediction frame in the N prediction frames, the motion features of each detection frame, and the geometric features between each prediction frame and each detection frame; For each detection frame, when the similarity between the detection frame and N prediction frames is less than a preset threshold, the detection frame is determined to be the mismatched detection frame, and the directionality score between the mismatched detection frame and each prediction frame is determined.

5. The multi-target tracking anti-occlusion processing method according to claim 1 or 4, characterized in that: The mismatch detection box is the xth detection box, each of the prediction boxes is the yth prediction box, and the yth prediction box corresponds to the yth track in the N tracks, x is an integer greater than 0 and less than or equal to M, and y is an integer greater than 0 and less than or equal to N; determining the directionality score between the mismatch detection box and each prediction box includes: Obtain the angle between the prediction box and the detection box corresponding to the y-th track in the previous frame; Determine the angle between the xth detection frame and the yth prediction frame based on the angle between the prediction frame and the detection frame corresponding to the yth trajectory in the previous frame, and the coordinates of the center point of the xth detection frame and the coordinates of the center point of the yth prediction frame; Calculate the directional similarity between the prediction box and the detection box in the current frame and the previous frame based on the angle between the prediction box and the detection box corresponding to the y-th trajectory in the previous frame, and the angle between the x-th detection box and the y-th prediction box; Calculate the intersection-over-union (IoU) ratio between the x-th detection frame and the y-th prediction frame; Determine a directionality score between the x-th detection frame and the y-th prediction frame according to a second preset weight, the directional similarity, and the intersection-over-union ratio.

6. The multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features according to claim 5, characterized in that: The determining, based on the angle between the prediction frame and the detection frame corresponding to the y-th trajectory in the previous frame, and the center point coordinates of the x-th detection frame and the center point coordinates of the y-th prediction frame, the angle between the x-th detection frame and the y-th prediction frame includes: According to the center coordinates of the xth detection frame and the center coordinates of the yth prediction frame, calculate the initial angle θ' corresponding to the xth detection frame and the yth prediction frame (n) ; According to the initial angle θ' (n) , the angle θ between the prediction box and the detection box corresponding to the y-th track in the previous frame (n-1) , and a preset coefficient to determine the angle between the x-th detection frame and the y-th prediction frame.

7. The multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features according to claim 5, characterized in that: The expression of the directionality score between the x-th detection box and the y-th prediction box is: IoU θ =β·IoU+(1-β)·θ closeness ; Among them, IoU θ is the directionality score, β is the second preset weight, IoU is the intersection over union ratio, θ closeness is the direction similarity, θ2 is the angle between the x-th detection frame and the y-th prediction frame, and θ1 is the angle between the prediction frame and the detection frame corresponding to the y-th track in the previous frame.

8. The multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features according to claim 1, characterized in that: The step of re-matching the mismatch detection frame with the N tracks according to the directionality score, and updating the N tracks when a track matching the mismatch detection frame exists to obtain an updated track, includes: Comparing the directionality score between the mismatch detection frame and each prediction frame with a preset threshold to obtain a comparison result; When the comparison result shows that the directionality score between the mismatching detection frame and the nth prediction frame among the N prediction frames is greater than or equal to the preset threshold, it indicates that there is a track matching the mismatching detection frame, the mismatching detection frame is associated with the nth track corresponding to the nth prediction frame, and information of the mismatching detection frame is used as updated track information of the nth track to obtain an updated nth track; The N trajectories are updated according to the updated n-th trajectory to obtain an updated trajectory.

9. The multi-target tracking anti-occlusion processing method for fusing directional scores and geometric features according to claim 1, characterized in that: The method further comprises: When it is determined according to the directionality score that the mismatched detection frame still does not match the N trajectories, obtaining an updated trajectory based on the matching conditions between the detection frames other than the mismatched detection frame among the M detection frames and the N trajectories, and using the mismatched detection frame as a candidate trajectory; performing frame-by-frame tracking of video frames after the current frame according to the candidate trajectory and the updated trajectory to obtain tracking results of multiple consecutive frames after the current frame; When the tracking results of the plurality of consecutive frames all indicate that there is no detection frame matching the candidate track, all relevant information of the candidate track is deleted, and target tracking of subsequent video frames is continued; When the tracking results of the plurality of consecutive frames all indicate that there is a detection frame matching the candidate trajectory, the candidate trajectory is used as a new trajectory, and target tracking is continued for subsequent video frames based on all currently obtained trajectories.

10. The multi-target tracking anti-occlusion processing method integrating directional scores and geometric features according to claim 1, characterized in that: The target tracking is performed on the video sequence to be tracked to obtain N tracks, N prediction frames of the current frame and information about each prediction frame, and M detection frames of the current frame and information about each detection frame, including: Performing target tracking on the video sequence to be tracked to obtain the N trajectories, as well as the M detection frames of the current frame and the information of each detection frame; According to the N trajectories, N prediction frames in the current frame are predicted by a Kalman filtering method; the N prediction frames correspond one-to-one to the N trajectories.

Images