Multi-target tracking method based on shielding state and related device thereof
By identifying the occlusion status of the target and selecting the appropriate matching method, the error problem of multi-object tracking in occlusion and camera disturbance scenarios is solved, and the tracking accuracy and robustness are improved.
Patent Information
- Application Number
- CN202411812412.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-03-04
AI Technical Summary
The existing multi-objective tracking method has the problem of large detection errors when target occlusion, especially in camera disturbance or high-speed motion scenarios, the credibility of IoU matching is reduced, and ReID matching fails under occlusion.
By identifying the target's occlusion status, select different matching methods: use ReID matching when occluding lightly; use IoU matching when moderate and severely; use geometric feature similarity matching when camera disturbance is high.
It improves the accuracy and robustness of target tracking in occlusion state, reduces errors, and is suitable for target matching and tracking in sports scenarios.
Smart Images
Figure CN119559212A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of target tracking technology, and in particular to a multi-target tracking method based on occlusion status and related devices. Background Art
[0002] There are two classic multi-target tracking and matching methods: (i) ReID vector matching: The ReID (Re-identification) algorithm is used to calculate the feature vector of the target, and the similarity between targets is measured by calculating the similarity of the ReID vector. In the absence of occlusion, the ReID credibility of the target is relatively high. In the case of occlusion, the credibility of ReID is not high because the corresponding area may have features of other objects. (ii) IoU matching: The overlap between target detection frames (Intersection over Union, IoU) is used to measure the similarity between targets. When the target moves slowly or moderately, IoU shows a high credibility even in the presence of occlusion. However, when the target moves faster, the credibility of IoU may be reduced. If the camera itself is also moving, the credibility of IoU will be further reduced. If there is loss in the middle frames, the credibility of IoU will be further affected.
[0003] Traditional classical multi-target tracking methods assume that lost tracks can only be found through ReID vector matching, that is, when the target reappears after being lost, the ReID vector is still valid. However, this assumption does not take into account the performance of the detector: when the detector performance is good enough, pedestrians may still be in an occluded state when they reappear after being lost. At this time, the ReID vector is invalid, resulting in large errors and low precision in tracking results. At present, the more novel multi-target tracking methods only use IoU-based methods for matching. This method ignores the natural advantage of ReID vectors in target matching based on long time differences. At the same time, although its detector performance is extremely excellent, the IoU-based matching strategy is not robust enough for scenes with camera perturbations or target occlusion under high-speed motion. Summary of the invention
[0004] The present application provides a multi-target tracking method based on occlusion status and a related device thereof, which are used to improve the technical problem of large detection errors in existing multi-target tracking methods when targets are occluded.
[0005] In view of this, the first aspect of the present application provides a multi-target tracking method based on occlusion state, comprising:
[0006] Performing target detection on the image to be detected captured by the camera, and identifying the occlusion state of each target according to the position of the detection frame of each target; the occlusion state includes light occlusion, moderate occlusion and heavy occlusion;
[0007] Match the detection box of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity;
[0008] Match the detection boxes of moderately occluded and heavily occluded objects with the unmatched confirmed tracks based on IoU;
[0009] Match the unconfirmed tracks with all unmatched detection boxes based on IoU;
[0010] After all matches are completed, the trajectory is updated.
[0011] Optionally, the identifying the occlusion state of each target according to the position of the detection frame of each target includes:
[0012] Determine the front and back relationship of each target by comparing the bottom values of the detection boxes of all targets;
[0013] Calculate the sum of the overlap between each target and the detection frame of the target in front to get the occlusion amount of each target;
[0014] The occlusion state of each target is determined according to the occlusion amount of each target.
[0015] Optionally, after matching the detection frames of the moderately occluded and heavily occluded targets with the unmatched confirmed tracks based on IoU, the method further includes:
[0016] When it is detected that the camera disturbance level is greater than a preset disturbance level threshold, all unmatched tracks are matched with all unmatched detection frames based on geometric feature similarity.
[0017] Optionally, matching all unmatched trajectories with all unmatched detection boxes based on geometric feature similarity includes:
[0018] Calculate the scale distance between each unmatched track and each unmatched detection frame according to the width and height of each unmatched track and each unmatched detection frame;
[0019] Calculating the spatial distance between each unmatched trajectory and each unmatched detection frame according to the corner point coordinates of each unmatched trajectory and each unmatched detection frame;
[0020] Match the trajectories whose scale distance is less than or equal to the maximum scale distance and whose spatial distance is less than or equal to the maximum spatial distance with the detection box.
[0021] Optionally, the calculation formula of the scale distance is:
[0022]
[0023] In the formula, For the trajectory With detection box The scale distance; The trajectory Width and height of The detection frame Width and height of .
[0024] Optionally, the calculation formula of the spatial distance is:
[0025]
[0026] In the formula, For the trajectory With detection box spatial distance; The trajectory The horizontal coordinate and vertical coordinate of the corner point; The detection frame The horizontal coordinate and vertical coordinate of the corner point.
[0027] Optionally, a scale distance constraint of geometric features is added to the matching process based on ReID similarity.
[0028] A second aspect of the present application provides a multi-target tracking device based on occlusion state, comprising:
[0029] A detection unit, used to perform target detection on the image to be detected acquired by the camera, and identify the occlusion state of each target according to the position of the detection frame of each target; the occlusion state includes light occlusion, moderate occlusion and heavy occlusion;
[0030] A first matching unit, configured to match the detection frame of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity;
[0031] The second matching unit is used to match the detection boxes of the moderately occluded and heavily occluded targets with the unmatched confirmed tracks based on IoU;
[0032] The third matching unit is used to match the unconfirmed track with all unmatched detection boxes based on IoU;
[0033] The trajectory updating unit is used to update the trajectory after all matching is completed.
[0034] A third aspect of the present application provides an electronic device, the device comprising a processor and a memory;
[0035] The memory is used to store program code and transmit the program code to the processor;
[0036] The processor is used to execute any one of the multi-target tracking methods based on occlusion status described in the first aspect according to the instructions in the program code.
[0037] In a fourth aspect, the present application provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, it implements any one of the multi-target tracking methods based on occlusion status described in the first aspect.
[0038] It can be seen from the above technical solutions that this application has the following advantages:
[0039] The multi-target tracking method based on occlusion state provided by the present application identifies the occlusion state of the target. When the target is lightly occluded, the ReID vector of the target is determined to be still valid, and the target is tracked by using the ReID matching method; when the target is moderately or severely occluded, the ReID vector of the target is determined to be invalid, and the target is tracked by using the IoU matching method. The present application selects different matching methods by identifying the occlusion state of the target, thereby achieving robust tracking of the target under occlusion state, helping to improve the accuracy of target tracking under occlusion state, and improving the existing technology that only uses the ReID matching method or only uses the IoU matching method to achieve target matching and tracking in motion scenes, and there is a technical problem of large detection error when the target is occluded;
[0040] Furthermore, when the camera disturbance level is detected to be large, the present application determines that both the ReID matching method and the IoU matching are invalid, and adopts geometric feature similarity matching to solve the situation where both ReID and IoU matching are invalid due to camera motion interference, thereby further improving the target tracking accuracy and reducing errors;
[0041] Furthermore, the present application also adds scale constraints of geometric features when performing ReID matching, thereby solving the problem that the ReID matching method is insensitive to scale. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 A flowchart of a multi-target tracking method based on occlusion state provided in an embodiment of the present application;
[0044] Figure 2 Another flowchart of a multi-target tracking method based on occlusion state provided in an embodiment of the present application;
[0045] Figure 3 A comparison chart of the matching strategy proposed in this application and the results of DeepSort and ByteTrack provided in the embodiments of this application;
[0046] Figure 4 A comparison chart of the results before and after adding geometric feature similarity matching to the tracker provided in the embodiment of the present application;
[0047] Figure 5 A schematic diagram of the structure of a multi-target tracking device based on occlusion state provided in an embodiment of the present application;
[0048] Figure 6 A structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0049] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0050] For easier understanding, please refer to Figure 1 , the embodiment of the present application provides a multi-target tracking method based on occlusion state, including:
[0051] Step 110: perform target detection on the image to be detected captured by the camera, and identify the occlusion state of each target according to the position of the detection frame of each target.
[0052] The existing target detection model can be used to perform target detection on the image to be detected collected by the camera, obtain the detection frame of each target, and then identify the occlusion state of each target according to the position of the detection frame of each target. The judgment of the occlusion state directly determines what matching strategy should be used for the target. In the embodiment of the present application, the occlusion state includes three types: light occlusion, moderate occlusion and heavy occlusion; when the target is not occluded or lightly occluded, the ReID vector of the target is considered to be reliable, and it is more suitable to use the ReID matching method; when the target is in moderate occlusion or heavy occlusion, the ReID vector is no longer reliable, and the IoU matching method is more reliable. It should be noted that when the pedestrian image is collected, the pedestrian image comes from a legally authorized one, or with the consent of the party concerned, and the collected pedestrian image is used for pedestrian tracking, which is mainly used for public safety.
[0053] Furthermore, the specific process of identifying the occlusion state of each target according to the position of the detection frame of each target includes:
[0054] Step 210, determining the front-back relationship of each target by comparing the bottom values of the detection boxes of all targets;
[0055] The front and back relationship of each target is determined by comparing the bottom values of the detection frames of all targets. The larger the bottom value of the target's detection frame, the closer the target is to the camera.
[0056] Step 220: Calculate the sum of the overlap between each target and the detection frame of the target in front to obtain the occlusion amount of each target; determine the occlusion state of each target according to the occlusion amount of each target;
[0057] For any target detection box D, after obtaining the front-back relationship of each target, calculate the sum of the overlap (IoU) of all detection boxes in front of the detection box D and the detection box D to get the occlusion of the detection box D. , that is, the occlusion amount of the target corresponding to the detection box D. Then, the occlusion state of each target is determined according to the occlusion amount of each target. Specifically, if the occlusion amount of the target < the first threshold, the occlusion state of the target is determined to be light occlusion; if the first threshold ≤ the occlusion amount of the target < the second threshold, the target is judged to be moderately blocked; if the target's occlusion amount is ≥ the second threshold, the occlusion state of the target is determined to be severe occlusion. If the bottom value of the detection frame D is greater than the bottom value of all other detection frames that overlap with the detection frame D, then the occlusion state of the target corresponding to the detection frame D is determined to be light occlusion. Among them, the first threshold is less than the second threshold, for example, the first threshold is 0.1, and the second threshold is 0.3. The first threshold and the second threshold can be set according to actual conditions and are not specifically limited here.
[0058] Step 120: Match the detection box of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity.
[0059] This application divides the detection frame of each target into and Two categories, the detection frame with slight occlusion is divided into the first category ; Classify the detection frames with moderate occlusion and severe occlusion into the second category . Indicates that ReID matching is preferred. Indicates that IoU matching is preferred. With confirmed track The historical ReID vector set is matched based on ReID similarity. The matching method based on ReID similarity belongs to the prior art, and its specific process will not be described here.
[0060] Furthermore, the embodiments of the present application take into account that the current ReID matching method has a fatal flaw, namely, the ReID matching method does not consider the geometric characteristics and the position correlation of the object. When their physical positions are far apart and the geometric features are quite different, the ReIDs of the two targets may still have a relatively high similarity. In order to solve the problem that the ReID matching method is insensitive to scale, the present application adds additional scale constraints of geometric features when matching the detection box of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity, that is, further calculating the scale distance between the detection box of the lightly occluded target and the confirmed trajectory. On the basis of satisfying the ReID vector matching, the scale distance also needs to satisfy the distance constraint to determine whether the detection box and the confirmed trajectory match. Please refer to step 150 for the calculation process of the scale distance.
[0061] Step 130: Match the detection boxes of the moderately occluded and heavily occluded targets with the unmatched confirmed tracks based on IoU.
[0062] The detection boxes of moderately occluded and heavily occluded objects Confirmed tracks that are not matched in step 120 The IoU-based matching is performed on all the tracks (including the lost tracks), which to some extent solves the problem that the ReID feature is invalid for long-distance queries when the track is lost and reappears and is still severely occluded. The IoU-based matching method belongs to the prior art and its specific process will not be described here.
[0063] Step 140: Match the unconfirmed track with all unmatched detection boxes based on IoU.
[0064] Unconfirmed tracks and all unmatched detection boxes in step 120 and step 130 ( ) for IoU-based matching.
[0065] Step 160: After all matches are completed, the trajectory is updated.
[0066] After all matches are completed, the successfully matched trajectory is sent to the Kalman filter for trajectory update. It should be noted that special constraints are added when updating the history vector of trajectory ReID. The ReID vectors with matching detection box occlusion status of light occlusion or moderate occlusion are retained and updated, and the ReID vectors with matching detection box occlusion status of heavy occlusion are discarded.
[0067] Furthermore, after step 140 and before step 160, the following steps are further included:
[0068] Step 150: When it is detected that the camera disturbance level is greater than a preset disturbance level threshold, all unmatched tracks are matched with all unmatched detection frames based on geometric feature similarity.
[0069] When it is detected that the camera disturbance level is greater than the preset disturbance level threshold, that is, when there is a large camera disturbance, all unmatched trajectories in step 130 and step 140 are matched with all unmatched detection frames in step 140 based on geometric feature similarity. Please refer to Figure 2 Specifically, the scale distance between each unmatched trajectory and each unmatched detection frame is calculated according to the width and height of each unmatched trajectory and each unmatched detection frame; the spatial distance between each unmatched trajectory and each unmatched detection frame is calculated according to the corner coordinates of each unmatched trajectory and each unmatched detection frame; the trajectory and detection frame corresponding to the scale distance less than or equal to the maximum scale distance and the spatial distance less than or equal to the maximum spatial distance are matched. Among them, the detection process of the camera disturbance level belongs to the prior art and will not be described in detail here.
[0070] The expression of geometric features must consider both the similarity in scale and the spatial distance relationship. When performing similarity matching based on geometric features, the search space of trajectories of different scales is subject to the constraints of their scales: the larger the scale, the larger the spatial range; the smaller the scale, the smaller the spatial range. For any trajectory , due to the imaging characteristics of the camera, the near is large and the far is small, this application believes that (w, h) represents the scale constraint and expresses the vertical search space in the image space; it is believed represents the distance constraint and expresses the lateral search space in the image space.
[0071] For any trajectory information to be matched , search for the matching i-th detection box , the calculation formula of scale distance is:
[0072]
[0073] The calculation formula for spatial distance is:
[0074]
[0075] In the formula, For the trajectory With detection box The scale distance; For the trajectory With detection box spatial distance; The trajectory The horizontal coordinate of the corner point, the vertical coordinate of the corner point, the width and the height; The detection frame The corner point abscissa, corner point ordinate, width, and height. The corner point coordinates are preferably the upper left corner point coordinates.
[0076] When the scale distance and the spatial distance are mutually constrained to be less than a certain threshold, a valid geometric feature is formed. Regarding the threshold information involved in the representation of scale distance and spatial distance, the following information is obtained by statistically analyzing the target label information in the data set:
[0077] 1) The maximum scale distance between objects in adjacent frames is TH0 and the maximum spatial distance is D0;
[0078] 2) When the interval is k frames, the maximum scale distance between targets is TH k =k*TH0 (k≥1), maximum spatial distance D k =k*D0.
[0079] The multi-target tracking method based on occlusion state provided by the present application identifies the occlusion state of the target. When the target is lightly occluded, the ReID vector of the target is determined to be still valid, and the target is tracked by using the ReID matching method; when the target is moderately or severely occluded, the ReID vector of the target is determined to be invalid, and the target is tracked by using the IoU matching method. The present application selects different matching methods by identifying the occlusion state of the target, thereby achieving robust tracking of the target under occlusion state, helping to improve the accuracy of target tracking under occlusion state, and improving the existing technology that only uses the ReID matching method or only uses the IoU matching method to achieve target matching and tracking in motion scenes, and there is a technical problem of large detection error when the target is occluded;
[0080] Furthermore, when the camera disturbance level is detected to be large, the present application determines that both the ReID matching method and the IoU matching are invalid, and adopts geometric feature similarity matching to solve the situation where both ReID and IoU matching are invalid due to camera motion interference, thereby further improving the target tracking accuracy and reducing errors;
[0081] Furthermore, the present application also adds scale constraints of geometric features when performing ReID matching, thereby solving the problem that the ReID matching method is insensitive to scale.
[0082] The embodiment of the present application conducted an experimental quantitative analysis of the proposed matching strategy, and compared the test results of DeepSort and ByteTrack on the MOT17 training set; in addition, in order to verify the effectiveness of the geometric features proposed in this application, a specific analysis of the geometric features was conducted. The embodiment of the present application used the target detection box results provided by ByteTrack, and the detection box confidence threshold was 0.5. At the same time, in order to ensure the fairness of the experimental comparison, the fuse_score strategy in ByteTrack was not used, because this strategy is optimized for a specific data set and is not universal.
[0083] The comparison results of the matching strategies (OM and OMGeo) proposed in this application with DeepSort and ByteTrack are shown in the figure. Figure 3 As shown. OM represents the matching strategy without adding geometric feature similarity matching ( Figure 2 Remove process ④), OMGeo represents the matching strategy of adding geometric similarity feature matching.
[0084] Depend on Figure 3 It can be seen from the results that the tracking strategy OM proposed in the embodiment of the present application has an IDF1 index of 5.7% and 2.6% respectively compared with DeepSort and ByteTrack. The tracking strategy OMGeo after adding geometric feature similarity matching has an IDF1 index of 6.5% and 3.4% compared with DeepSort and ByteTrack, respectively, and an improvement of 0.8% compared with OM, which improves the accuracy of trajectory ID prediction. The MOTA index is not much different overall. This is because our goal of optimizing tracking defects based on DeepSort and ByteTrack is to improve the accuracy of trajectory ID prediction, which is reflected by the indicator IDF1; MOTA is more of a reflection of the performance of the detector. Since the same detector is used, the MOTA index is not much different.
[0085] This application compares the results of the OM strategy and the OMGeo strategy on each subset of the MOT17 training set to observe which data sets have undergone more obvious changes after adding geometric feature similarity matching, so as to verify the idea that the geometric feature similarity can be used to solve the camera motion interference in the embodiment of this application. The results are as follows: Figure 4As shown in the figure, by comparing the results of OM and OMGeo, it can be seen that there is a certain improvement on the MOT17-10, MOT17-11, and MOT17-13 datasets. There are camera perturbations in these three datasets, especially in MOT17-10, which has a large camera perturbation. Therefore, it can be seen that the geometric feature similarity matching has a better effect when facing large camera perturbations; and compared with ByteTrack, it can be found that the multi-target tracking method proposed in this application is better than ByteTrack in terms of indicators when facing dense small-scale crowds (MOT17-02), special camera postures (MOT17-09), and large camera perturbations (MOT17-10).
[0086] It can be seen from the results that it is not feasible to directly and blindly apply geometric similarity matching to all data sets. When the camera perturbation is not strong (MOT17-05, MOT17-09), geometric feature similarity matching may bring more erroneous matching results, resulting in a decrease in the IDF1 index. Geometric similarity matching shows better performance in the case of larger camera perturbations.
[0087] The above is an embodiment of a multi-target tracking method based on occlusion state provided by the present application, and the following is an embodiment of a multi-target tracking device based on occlusion state provided by the present application.
[0088] Please refer to Figure 5 , an embodiment of the present application provides a multi-target tracking device based on occlusion state, comprising:
[0089] The detection unit 310 is used to perform target detection on the image to be detected collected by the camera, and identify the occlusion state of each target according to the position of the detection frame of each target; the occlusion state includes light occlusion, medium occlusion and heavy occlusion;
[0090] A first matching unit 320, configured to match the detection frame of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity;
[0091] A second matching unit 330 is used to match the detection frames of the moderately occluded and heavily occluded targets with the unmatched confirmed tracks based on IoU;
[0092] A third matching unit 340 is used to match the unconfirmed track with all unmatched detection boxes based on IoU;
[0093] The trajectory updating unit 350 is used to update the trajectory after all the matching is completed.
[0094] Furthermore, the embodiments of the present application take into account that the current ReID matching method has a fatal flaw, namely, the ReID matching method does not consider the geometric characteristics and the position correlation of the object. When their physical positions are far apart and the geometric features are quite different, the ReIDs of the two targets may still have a relatively high similarity. In order to solve the problem that the ReID matching method is insensitive to scale, the present application adds additional scale constraints of geometric features when matching the detection box of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity, that is, further calculating the scale distance between the detection box of the lightly occluded target and the confirmed trajectory. On the basis of satisfying the ReID vector matching, the scale distance also needs to satisfy the distance constraint to determine whether the detection box and the confirmed trajectory match.
[0095] As a further improvement, the detection unit 310 is specifically used for:
[0096] Perform target detection on the image to be detected captured by the camera to obtain the detection frame of each target;
[0097] Determine the front and back relationship of each target by comparing the bottom values of the detection boxes of all targets;
[0098] Calculate the sum of the overlap between each target and the detection frame of the target in front to obtain the occlusion amount of each target;
[0099] The occlusion state of each target is determined according to the occlusion amount of each target.
[0100] As a further improvement, the device further comprises:
[0101] The fourth matching unit 360 is configured to match all unmatched trajectories with all unmatched detection frames based on geometric feature similarity when it is detected that the camera disturbance level is greater than a preset disturbance level threshold.
[0102] As a further improvement, the fourth matching unit 360 is specifically configured to:
[0103] Calculate the scale distance between each unmatched track and each unmatched detection frame according to the width and height of each unmatched track and each unmatched detection frame;
[0104] Calculating the spatial distance between each unmatched trajectory and each unmatched detection frame according to the corner point coordinates of each unmatched trajectory and each unmatched detection frame;
[0105] Match the trajectories whose scale distance is less than or equal to the maximum scale distance and whose spatial distance is less than or equal to the maximum spatial distance with the detection box.
[0106] The multi-target tracking method based on occlusion state provided by the present application identifies the occlusion state of the target. When the target is lightly occluded, the ReID vector of the target is determined to be still valid, and the target is tracked by using the ReID matching method; when the target is moderately or severely occluded, the ReID vector of the target is determined to be invalid, and the target is tracked by using the IoU matching method. The present application selects different matching methods by identifying the occlusion state of the target, thereby achieving robust tracking of the target under occlusion state, helping to improve the accuracy of target tracking under occlusion state, and improving the existing technology that only uses the ReID matching method or only uses the IoU matching method to achieve target matching and tracking in motion scenes, and there is a technical problem of large detection error when the target is occluded;
[0107] Furthermore, when the camera disturbance level is detected to be large, the present application determines that both the ReID matching method and the IoU matching are invalid, and adopts geometric feature similarity matching to solve the situation where both ReID and IoU matching are invalid due to camera motion interference, thereby further improving the target tracking accuracy and reducing errors;
[0108] Furthermore, the present application also adds scale constraints of geometric features when performing ReID matching, thereby solving the problem that the ReID matching method is insensitive to scale.
[0109] Please refer to Figure 6 , the embodiment of the present application further provides an electronic device, the device includes a processor 410 and a memory 420;
[0110] The memory 420 is used to store program codes and transmit the program codes to the processor 410;
[0111] The processor 410 is used to execute the multi-target tracking method based on occlusion status in the aforementioned method embodiment according to the instructions in the program code.
[0112] An embodiment of the present application also provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, the multi-target tracking method based on occlusion state in the aforementioned method embodiment is implemented.
[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0114] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0115] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0116] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0118] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.
[0120] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A multi-target tracking method based on occlusion state, characterized in that: include: Perform target detection on the image to be detected captured by the camera, and identify the occlusion status of each target according to the position of the detection frame of each target; The occlusion state includes light occlusion, moderate occlusion and heavy occlusion; Match the detection box of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity; Match the detection boxes of moderately occluded and heavily occluded objects with the unmatched confirmed tracks based on IoU; Match the unconfirmed tracks with all unmatched detection boxes based on IoU; After all matches are completed, the trajectory is updated.
2. The multi-target tracking method based on occlusion state according to claim 1, characterized in that: The identifying the occlusion state of each target according to the position of the detection frame of each target includes: Determine the front and back relationship of each target by comparing the bottom values of the detection boxes of all targets; Calculate the sum of the overlap between each target and the detection frame of the target in front to obtain the occlusion amount of each target; The occlusion state of each target is determined according to the occlusion amount of each target.
3. The multi-target tracking method based on occlusion state according to claim 1, characterized in that: After matching the detection frames of the moderately occluded and heavily occluded targets with the unmatched confirmed tracks based on IoU, the method further includes: When it is detected that the camera disturbance level is greater than a preset disturbance level threshold, all unmatched tracks are matched with all unmatched detection frames based on geometric feature similarity.
4. The multi-target tracking method based on occlusion state according to claim 3, characterized in that: The matching of all unmatched trajectories with all unmatched detection frames based on geometric feature similarity includes: Calculate the scale distance between each unmatched track and each unmatched detection frame according to the width and height of each unmatched track and each unmatched detection frame; Calculating the spatial distance between each unmatched trajectory and each unmatched detection frame according to the corner point coordinates of each unmatched trajectory and each unmatched detection frame; Match the trajectories whose scale distance is less than or equal to the maximum scale distance and whose spatial distance is less than or equal to the maximum spatial distance with the detection box.
5. The multi-target tracking method based on occlusion state according to claim 4, characterized in that: The calculation formula of the scale distance is: In the formula, For the trajectory With detection box The scale distance; The trajectory Width and height of The detection frame Width and height of .
6. The multi-target tracking method based on occlusion state according to claim 4, characterized in that: The calculation formula of the spatial distance is: In the formula, For the trajectory With detection box spatial distance; The trajectory The horizontal coordinate and vertical coordinate of the corner point; The detection frame The horizontal coordinate and vertical coordinate of the corner point.
7. The multi-target tracking method based on occlusion state according to claim 4, characterized in that: The scale distance constraint of geometric features is added to the matching process based on ReID similarity.
8. A multi-target tracking device based on occlusion state, characterized in that: include: A detection unit, used to perform target detection on the image to be detected acquired by the camera, and identify the occlusion state of each target according to the position of the detection frame of each target; The occlusion state includes light occlusion, moderate occlusion and heavy occlusion; A first matching unit, configured to match the detection frame of the lightly occluded target with the historical ReID vector set of the confirmed trajectory based on ReID similarity; The second matching unit is used to match the detection boxes of the moderately occluded and heavily occluded targets with the unmatched confirmed tracks based on IoU; The third matching unit is used to match the unconfirmed track with all unmatched detection boxes based on IoU; The trajectory updating unit is used to update the trajectory after all matching is completed.
9. An electronic device, characterized in that: The device comprises a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the multi-target tracking method based on occlusion status according to any one of claims 1-7 according to the instructions in the program code.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and when the program codes are executed by a processor, the multi-target tracking method based on occlusion status according to any one of claims 1 to 7 is implemented.