Multi-camera multi-target personnel tracking method, device, equipment and medium

Abstract

The present application relates to a method, device, equipment and medium for tracking personnel with multiple cameras and multiple targets, wherein the method includes: obtaining the monocular tracking and positioning results of multiple cameras in the same batch to obtain initial monocular tracking information; clustering high-confidence classes from the initial monocular tracking information according to the spatial distance threshold and the human feature similarity threshold; merging high-confidence classes by polling and adjusting the spatial distance threshold and the human feature similarity threshold to obtain a first merged high-confidence class; merging and filtering the free classes to obtain a second merged high-confidence class; initializing the second merged high-confidence class as a multi-target object, and obtaining the tracked target information of multiple cameras in adjacent frames of the initial monocular tracking information; matching the tracked target information with the multi-target object to obtain the target tracking result. The present application realizes multi-camera and multi-target personnel tracking without a large amount of pre-training data.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a multi-camera multi-target personnel tracking method, device, equipment and medium. Background Art

[0002] Multi-camera multi-target tracking is a function that tracks every individual in a multi-camera environment. Tracking each individual can help site managers record the trajectory of personnel and restore and analyze the trajectory of personnel. The basic architecture of the current multi-target multi-camera tracking method is: (1) A single camera performs single-camera tracking of personnel within its control range; (2) The results of single-camera tracking are converted from image coordinates to physical space coordinate systems to obtain the physical space position of each person tracked by a single camera, and based on the appearance characteristics of personnel across cameras and the physical space relationship, personnel unification and tracking under multiple cameras are achieved. However, this type of technology currently faces the following challenges in indoor office scenarios:

[0003] 1. When multiple single cameras track people to achieve unified physical space positions, a skeleton prediction model is needed to estimate the ankle point position, and then convert the image space to the physical space. However, in indoor office scenes, due to the occlusion of the desk, the ankle point position is often not accurately obtained, so the estimated position of the ankle point often has a large error. 2. When multiple cameras track people to achieve cross-camera unification through human features, pedestrian re-identification technology (Person Re-identification) is needed. Pedestrian re-identification technologies that can support high frame rate tracking are mostly based on CNN convolutional neural network architectures, and the generalization ability of scenes is poor. Therefore, a large amount of training data is required for the scene to accurately achieve cross-camera pedestrian re-identification. Therefore, the existing multi-camera multi-target personnel tracking methods have failed to achieve accurate multi-camera multi-target tracking in indoor environments without a lot of pre-training, only through the generalization ability of the model, and the tracking accuracy of the existing personnel tracking methods is low. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a multi-camera multi-target personnel tracking method, device, equipment and medium, so as to realize multi-camera multi-target personnel tracking without a large amount of pre-training data, while improving the accuracy of personnel tracking.

[0005] In order to solve the above technical problems, the present application provides a multi-camera multi-target personnel tracking method, including:

[0006] Obtain the monocular tracking and positioning results of multiple cameras in the same batch to obtain the initial monocular tracking information;

[0007] Clustering a high confidence class from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold;

[0008] Merging the high confidence classes by adjusting the spatial distance threshold and the human feature similarity threshold in a polling manner to obtain a first merged high confidence class;

[0009] Obtaining free classes, merging and filtering the free classes to obtain a second merged high-confidence class;

[0010] Initialize the second merged high confidence class as a multi-target object, and obtain the tracked target information of the multiple cameras in the adjacent frames of the initial monocular tracking information;

[0011] The tracked target information is matched with the multiple target objects to obtain a target tracking result.

[0012] In order to solve the above technical problems, the embodiment of the present application provides a multi-camera multi-target personnel tracking device, comprising:

[0013] An initial monocular tracking information acquisition unit is used to obtain monocular tracking and positioning results of multiple cameras in the same batch to obtain initial monocular tracking information;

[0014] A high confidence class clustering unit, used for clustering high confidence classes from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold;

[0015] A first merging unit, configured to merge the high confidence classes by adjusting the spatial distance threshold and the human feature similarity threshold in a polling manner to obtain a first merged high confidence class;

[0016] A second merging unit is used to obtain a free class, merge and filter the free class to obtain a second merged high confidence class;

[0017] An adjacent frame information acquisition unit, configured to initialize the second merged high confidence class as a multi-target object, and acquire the tracked target information of the plurality of cameras in adjacent frames of the initial monocular tracking information;

[0018] The target tracking result generating unit is used to match the tracked target information with the multiple target objects to obtain a target tracking result.

[0019] In order to solve the above technical problems, a technical solution adopted by the present invention is: to provide a computer device, including one or more processors; a memory for storing one or more programs, so that the one or more processors can implement any one of the multi-camera multi-target personnel tracking methods described above.

[0020] In order to solve the above technical problems, a technical solution adopted by the present invention is: a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the multi-camera multi-target personnel tracking method described in any one of the above is implemented.

[0021] The embodiment of the present invention provides a multi-camera multi-target personnel tracking method, device, equipment and medium. The method includes: obtaining the monocular tracking and positioning results of multiple cameras in the same batch to obtain initial monocular tracking information; clustering high-confidence classes from the initial monocular tracking information according to the spatial distance threshold and the human feature similarity threshold; merging the high-confidence classes by polling and adjusting the spatial distance threshold and the human feature similarity threshold to obtain a first merged high-confidence class; obtaining a free class, merging and filtering the free class to obtain a second merged high-confidence class; initializing the second merged high-confidence class as a multi-target object, and obtaining the tracked target information of multiple cameras in adjacent frames of the initial monocular tracking information; matching the tracked target information with the multi-target object to obtain a target tracking result. The embodiment of the present invention obtains a high confidence class through the adjustment of the spatial distance threshold, the human feature similarity threshold and the threshold, thereby realizing the target unification of multiple cameras and multiple targets in the same time frame and realizing the target unified tracking of multiple cameras and multiple targets in adjacent frames. It can realize multi-camera multi-target personnel tracking without a large amount of pre-training data, and merge and screen the tracking targets, which is beneficial to reducing the error of multi-camera multi-target tracking and improving the accuracy of personnel tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 It is a flowchart of the implementation process of the multi-camera multi-target personnel tracking method provided in an embodiment of the present application;

[0024] Figure 2 It is a flowchart for implementing the first sub-process in the multi-camera multi-target personnel tracking method provided in an embodiment of the present application;

[0025] Figure 3 It is a flowchart for implementing the second sub-process in the multi-camera multi-target personnel tracking method provided in an embodiment of the present application;

[0026] Figure 4 It is a flowchart for implementing the third sub-process in the multi-camera multi-target personnel tracking method provided in an embodiment of the present application;

[0027] Figure 5 It is a flowchart for implementing the fourth sub-process in the multi-camera multi-target personnel tracking method provided in an embodiment of the present application;

[0028] Figure 6 It is a flowchart for implementing the fifth sub-process in the multi-camera multi-target personnel tracking method provided in an embodiment of the present application;

[0029] Figure 7 It is a flowchart for implementing the sixth sub-process in the multi-camera multi-target personnel tracking method provided in an embodiment of the present application;

[0030] Figure 8 Schematic diagram of a multi-camera multi-target personnel tracking device provided in an embodiment of the present application;

[0031] Fig. 9 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0033] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0035] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the multi-camera multi-target personnel tracking method provided in the embodiment of the present application is generally executed by a server, and accordingly, the multi-camera multi-target personnel tracking device is generally configured in the server.

[0037] The multi-camera multi-target personnel tracking method provided in the embodiment of the present application is applied to personnel tracking in multi-camera multi-target scenes, such as personnel tracking in indoor office scenes, or other multi-camera multi-target indoor spaces. In the embodiment of the present application, the unification of multi-camera same frame targets and the unification of multi-camera adjacent frame targets are described. Among them, the unification of targets in the same frame of multiple cameras is to achieve the unification of personnel identities under multiple cameras at the same time. For example, the same person appears under multiple cameras and needs to be matched into a single person in the physical space. The unification of targets in adjacent frames of multiple cameras is to achieve the unification of personnel identities at continuous moments, such as the same person in the physical space at continuous moments, and its tracking and identification should be maintained. The embodiment of the present application can utilize the human feature re-identification technology based on SwinTransformer with zero samples, and utilize its strong scene generalization ability to realize cross-camera and cross-region personnel tracking. However, the introduction of this architecture will cause the processing frame rate of the system to become lower. Therefore, there is a time difference in the detection results of each single camera between the same frames, and it is necessary to tolerate a larger spatial position deviation from the algorithm to achieve the unification of targets in the same frame of multiple cameras.

[0038] See also Figure 1 , Figure 1 A specific implementation of a multi-camera multi-target personnel tracking method is shown.

[0039] It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 The process sequence shown is limited to the following steps:

[0040] S1: Obtain the monocular tracking and positioning results of multiple cameras in the same batch to obtain the initial monocular tracking information.

[0041] In an embodiment of the present application, the monocular tracking and positioning results (single-camera tracker) of multiple cameras in the same batch are read, and the monocular tracking and positioning results of the same batch are used as the initial monocular tracking information. After obtaining the initial monocular tracking information, in order to achieve the target unification of the same frame of multiple cameras, it is necessary to screen out reliable results with high confidence through the spatial threshold and the human body similarity threshold. Then the two thresholds are alternately relaxed respectively, and the reliable results are merged to tolerate the possible spatial positioning errors and the errors of the re-identification model. Then, in a low-weighted manner, the remaining free results are merged into the high-confidence results to supplement the information. Finally, the free results that cannot be merged are filtered out by rules to remove errors. Among them, the initial monocular tracking information is the result of tracking and positioning multiple cameras and multiple targets in the same batch.

[0042] S2: Clustering high confidence classes from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold.

[0043] Specifically, the spatial distance threshold and the human feature similarity threshold are pre-set, and the high confidence class is clustered from the initial monocular tracking information based on the spatial distance threshold and the human feature similarity threshold through the Hungarian algorithm and the matching algorithm. The spatial distance threshold and the human feature similarity threshold ensure that under the premise of multiple cameras, only targets with very close distances and very high human feature similarity will be clustered. It can be considered that the clustered results have very high credibility and can be considered as the same target under multiple cameras. Among them, the high confidence class refers to the personnel identity category with high credibility obtained after screening, merging and absorbing through a series of algorithms in the process of multi-camera target unification and tracking. These categories provide a reliable foundation for subsequent cross-camera and cross-region personnel tracking.

[0044] It should be noted that the spatial distance threshold and the human feature similarity threshold are set according to the actual situation and are not limited here. In a specific embodiment, a high confidence class is clustered from the initial monocular tracking information by setting a strict spatial distance threshold (highconfidence_spatial_radius) and a strict human feature similarity threshold (highconfidence_feat_radius). A class refers to an identity category of a tracking target.

[0045] See also Figure 2 , Figure 2 A specific implementation of step S2 is shown, which is described in detail as follows:

[0046] S21: extracting human features of each tracking target in the initial monocular tracking information.

[0047] S22: Calculating the similarity of the tracking target according to the human body feature to obtain a human body feature similarity value, and determining a target similarity value based on the human body feature similarity value.

[0048] Specifically, due to the weak cross-domain (camera) capability of human features, the human features under each camera are retained, and there are as many versions of human features as there are cameras. Therefore, in the embodiment of the present application, the human features of each tracking target in the initial monocular tracking information are first extracted, and then the human cosine similarity of the tracking target is calculated based on the human features to obtain the human feature similarity value. The minimum value among the human feature similarity values ​​is used as the target similarity value. For example, class 1 has n human feature values, and class 2 has m human feature values. The minimum value among the n*m ​​matches is taken as the target similarity value.

[0049] The similarity of human features between classes or trackers of a single camera is the minimum value of the cosine similarities of these human features. For example, if class 1 has n human feature values ​​and class 2 has m human feature values, the minimum value among n*m matches is taken.

[0050] S23: Determine the target class from the initial monocular tracking information, and calculate the spatial position distance of the target class to obtain the target spatial distance.

[0051] S24: Clustering the high confidence class from the initial monocular tracking information according to the spatial distance threshold, the human feature similarity threshold, the target similarity value and the target spatial distance.

[0052] Specifically, a class has only one spatial position, and which spatial position is selected as the only spatial position of this class is obtained by the algorithm. The principle is that the advantageous position of each camera is configured in the configuration (assuming that this camera has the best viewing angle within this physical space range and can best avoid occlusion and spatial positioning errors). In a class, only the monocular tracking and positioning results under these cameras are retained, and the monocular tracking and positioning results are used as the target class. At the same time, their average spatial position is calculated and returned as the only spatial position of this class, that is, the only spatial position is used as the spatial position of the target class. When the subsequent classes are merged, they no longer become the spatial position of the class with high confidence. Therefore, the calculation of the spatial distance is that the spatial position of each class is provided by this unique spatial position. Therefore, after determining the spatial position of the target class and the target class in the embodiment of the present application, the spatial position distance between the target classes is calculated to obtain the distance between the targets. Finally, according to the spatial distance threshold, the human feature similarity threshold, the target similarity value and the target spatial distance, a high confidence class is clustered from the initial monocular tracking information. High confidence classes are clustered by adopting the Hungarian algorithm and the matching algorithm. Among them, a class is composed of tracking targets under multiple cameras.

[0053] S3: Merging the high confidence classes by adjusting the spatial distance threshold and the human feature similarity threshold in a polling manner to obtain a first merged high confidence class.

[0054] Specifically, after the above high-confidence classes are generated, it is possible that the same target part of the camera is clustered into multiple classes because the threshold is too strict, so the high-confidence classes are merged by polling and relaxing the two thresholds. It should be noted that high-confidence classes will only be merged if the cameras do not overlap. If two high-confidence classes contain tracking targets under the same camera, they cannot be merged.

[0055] See also Figure 3 , Figure 3 A specific implementation of step S3 is shown, which is described in detail as follows:

[0056] S31: Identify categories of cameras that do not overlap from the high confidence categories to obtain categories to be merged.

[0057] S32: merging the classes to be merged by polling and relaxing any one of the spatial distance threshold and the human feature similarity threshold to obtain an initial merged high-confidence class.

[0058] S33: When it is impossible to continue to merge categories, the initially merged high-confidence category and the high-confidence category that has not been merged are used as the first merged high-confidence category.

[0059] Specifically, the categories of cameras that do not overlap are identified from the high-confidence categories to obtain the categories to be merged. The specific polling process is as follows: Step (1) first keeps the spatial distance threshold unchanged, relaxes the human feature similarity threshold, that is, increases the threshold range of the human feature similarity threshold, and attempts to merge the two high-confidence categories. Step (2) keeps the human feature similarity threshold unchanged, relaxes the spatial distance threshold, and attempts to merge the two high-confidence categories; repeat steps (1) and (2) until no new categories are merged, and obtain the current initial merged high-confidence category. The initial merged high-confidence category and the high-confidence category that has not been merged are taken as the first merged high-confidence category, so that the first merged high-confidence category can be processed later.

[0060] S4: Obtain free classes, merge and filter the free classes to obtain a second merged high-confidence class.

[0061] Specifically, the free class refers to the class with only one camera's monocular tracking and positioning results after the above clustering is completed. Generally, the free class appears because the camera angle is poor or there is a person blocking it, resulting in a large positioning error. Therefore, in the embodiment of the present application, the free class needs to be merged and filtered to obtain the high-confidence class in the free class, thereby obtaining the second merged high-confidence class.

[0062] See also Figure 4 , Figure 4 A specific implementation of step S4 is shown, which is described in detail as follows:

[0063] S41: Acquire the free class, merge the free class according to the human feature similarity threshold and the relaxed spatial distance threshold to obtain a merged free class, and add the merged free class to the first merged high-confidence class to obtain the second merged high-confidence class.

[0064] Specifically, the free classes are merged according to the human feature similarity threshold and the relaxed spatial distance threshold to obtain a merged free class, and the merged free class is added to the first merged high-confidence class to obtain a second merged high-confidence class, thereby achieving the absorption of the free class by the high-confidence class, avoiding the omission of categories, and effectively improving the accuracy of tracking targets.

[0065] S42: Acquire the current remaining free classes, filter the current remaining free classes to obtain filtered free classes, and convert the filtered free classes into the second merged high-confidence classes.

[0066] Specifically, after the free classes are merged, there are still some free classes left, and some of these remaining free classes should be retained, such as: a person is only photographed by a camera with a better viewing angle, although the person may be blocked by other cameras. Therefore, in order to improve the recall rate, it is necessary to obtain the current remaining free classes, filter the current remaining free classes, obtain the filtered free classes, and convert the filtered free classes into the second merged high confidence classes.

[0067] See also Figure 5 , Figure 5 A specific implementation of step S43 is shown, which is described in detail as follows:

[0068] S431: Acquire the current remaining free class, and take the area of ​​the detection box corresponding to the current remaining free class as a first condition being smaller than a preset area.

[0069] S432: Taking the current remaining free class being within a preset target area as a second condition.

[0070] S433: The third condition is that the proportion of the currently remaining free classes being blocked is less than a preset proportion.

[0071] S434: taking the current remaining free classes that satisfy the first condition, the second condition, and the third condition as the filtered free classes, and converting the filtered free classes into the second merged high-confidence classes.

[0072] Specifically, if the free class accounts for a very small proportion in the picture, it may cause an error and need to be filtered. At the same time, if the free class accounts for a relatively large proportion in the picture, it needs to be retained, so the area of ​​the detection box corresponding to the current remaining free class is smaller than the preset area as the first condition. If it is in a remote location of the camera, an error may be caused and it needs to be filtered. However, if the free class is in a key area of ​​a camera, it may need to be retained, and the current remaining free class is in the preset target area as the second condition. If the free class is clearly visible without being blocked to a large extent, otherwise it may cause an error and need to be filtered, so when the above three conditions are met, the current remaining free class will be converted into a high-confidence class and retained as a single person in the physical space of multiple cameras at the same time.

[0073] It should be noted that the preset area, preset target area and preset ratio are set according to actual conditions and are not limited here.

[0074] S5: Initialize the second merged high-confidence class as a multi-target object, and obtain the tracked target information of multiple cameras in adjacent frames of the initial monocular tracking information.

[0075] Specifically, after the construction of the high-confidence class of the same frame of multiple cameras is completed, the second merged high-confidence class will be converted into multiple target objects (detections) in the macroscopic physical space to match the multi-camera tracking and positioning results (multi-camera trackers) in the macroscopic physical space, completing the target unification of the previous and next frames in the multi-camera physical space, thereby continuing the tracking of the multi-camera tracking and positioning results multi-camera trackers. Therefore, in an embodiment of the present application, the second merged high-confidence class is initialized as a multi-target object, and the tracked target information of multiple cameras in adjacent frames of the tracking information is obtained. The tracked target information refers to the multi-camera tracking and positioning results (multi-camera trackers) in adjacent frames of the tracking information.

[0076] It should be noted that the initial single-camera tracking information (single-camera tracker) in the embodiment of the present application is used to describe the tracking information from a single camera at the current processing moment of multiple cameras. The multi-camera tracking and positioning results (multi-camera trackers) are multi-camera tracking information in the logistics space under the coordination of multiple cameras.

[0077] S6: Match the tracked target information with the multiple target objects to obtain a target tracking result.

[0078] The principle of unifying the targets of adjacent frames of multiple cameras proposed in the embodiment of the present application is mainly as follows: first, through spatial distance and human body similarity, try to match the multiple target objects in the physical space with the target information to be tracked. When the human weight recognition fails due to errors such as human occlusion, the tracking is continued through the voting method. Finally, try to merge multiple tracking to avoid occasional tracking failures, so as to obtain the target tracking result.

[0079] See also Figure 6 , Figure 6 A specific implementation of step S6 is shown, which is described in detail as follows:

[0080] S61: Matching the tracked target information with the multiple target objects according to the human feature similarity threshold and the relaxed spatial distance threshold to obtain matched tracking targets and unmatched tracking targets.

[0081] Specifically, the matching between multi-target object detection and multi-camera trackers also depends on spatial similarity and human feature similarity. Since the frame rate of physical space tracking is low, the spatial similarity is selected as a relatively loose spatial distance threshold. Since the human feature model based on swintransformer has very good discrimination between different people under a single camera (unless the clothes are all black), the human feature similarity uses a more stringent human feature similarity threshold. This matching idea is based on the following observation: Generally speaking, a person is less likely to be lost under multiple cameras at the same time, and the monocular tracking and positioning results exist for a long time, so there are human feature values ​​under each camera. Therefore, the human feature matching between multi-target object detection and multi-camera trackers will be very indicative (therefore, the human feature similarity between multi-target object detection and multi-camera trackers is the minimum value, which is generally the matching of human features under the same camera, which is very good for different people, unless the clothes are all black), so the spatial similarity can be relaxed without any worries. Therefore, after this round of matching, there is a greater probability that the multi-target object detections with better visibility will be matched with the multi-camera trackers of the tracked target information that has not disappeared in the short term.

[0082] S62: re-matching the unmatched tracking target by using the relaxed spatial distance threshold and voting method to obtain a new matching tracking target.

[0083] Specifically, although the human feature model based on SwinTransformer performs very well, the problem of human occlusion may cause the multi-target object detections and the tracked target information multi-camera trackers to be unable to be matched. In this case, if the tracking of the multi-target object detections under multiple cameras is not interrupted, it can be restored through the macroscopic monocular tracking and positioning results of the physical space in which the monocular tracking under each camera was matched in the previous frame. Therefore, all detections and tracked target information multi-cameratrackers that are not matched in the matching process of the above multi-target object detections and the tracked target information multi-cameratrackers use a loose spatial distance threshold and complete the matching by voting.

[0084] S63: Obtain the split tracking information, and merge the split tracking information with the matching tracking target to obtain the current matching tracking target. The split tracking information is the newly added multi-target tracking and positioning result.

[0085] See also Figure 7 , Figure 7 A specific implementation of step S63 is shown, which is described in detail as follows:

[0086] S631: Obtain tracking information of the split.

[0087] S632: The tracking target in the split tracking information is judged to be merged with the matching tracking target by using the human feature similarity threshold and the relaxed spatial distance threshold to obtain the current matching tracking target.

[0088] Specifically, the matching of multi-target object detections based on a single frame with the tracked target information multi-cameratrackers may fail to complete the matching due to the poor cross-domain capability of the human feature model. Specifically, the multi-target object detections cannot match the tracked target information multi-cameratrackers due to the strict human feature threshold, so a new tracked target information multi-camera trackers has to be added. After this split multi-camera trackers appears for a short time and is captured by multiple cameras, the cross-domain human features will become the human feature comparison between the same camera, so it can be merged into the multi-cameratrackers that appeared before to eliminate the split phenomenon. Therefore, in the end, try to merge the multi-camera trackers under the relaxed spatial similarity using a strict human feature threshold, so as to eliminate the split phenomenon as much as possible by merging after the split phenomenon occurs. That is, in the embodiment of the present application, the split tracking information is obtained; the tracking target in the split tracking information is judged to be merged with the matching tracking target by the human feature similarity threshold and the relaxed spatial distance threshold to obtain the current matching tracking target.

[0089] S64: Generate the target tracking result based on the current matching tracking target and the new matching tracking target.

[0090] Specifically, all matched tracking targets are taken as the final target tracking results.

[0091] In an embodiment of the present application, monocular tracking and positioning results of multiple cameras in the same batch are obtained to obtain initial monocular tracking information; high confidence classes are clustered from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold; the high confidence classes are merged by polling and adjusting the spatial distance threshold and the human feature similarity threshold to obtain a first merged high confidence class; a free class is obtained, and the free class is merged and filtered to obtain a second merged high confidence class; the second merged high confidence class is initialized as a multi-target object, and tracked target information of multiple cameras in adjacent frames of the initial monocular tracking information is obtained; the tracked target information is matched with the multi-target object to obtain a target tracking result. The embodiment of the present invention obtains a high confidence class through the adjustment of the spatial distance threshold, the human feature similarity threshold and the threshold, thereby realizing the target unification of multiple cameras and multiple targets in the same time frame and realizing the target unified tracking of multiple cameras and multiple targets in adjacent frames. It can realize multi-camera multi-target personnel tracking without a large amount of pre-training data, and merge and screen the tracking targets, which is beneficial to reducing the error of multi-camera multi-target tracking and improving the accuracy of personnel tracking.

[0092] The embodiment of the present application has the characteristics of strong scene generalization ability. The zero-sample person re-identification module does not need to be trained for the scene, has a strong ability to maintain the identity of the person for a long time, and is not prone to person confusion.

[0093] The embodiment of the present application has the characteristic of strong scene generalization ability. The use of pedestrian re-identification technology based on the SwinTransformer model will cause the tracking frame rate to become lower (for example, a single NVIDIA RTX4090 graphics card only supports 3 to 4 frames per second per camera). The algorithm designed for the tracking logic of low frame rate avoids the problem of large spatial error and achieves accurate tracking even at low frame rate.

[0094] Please refer to Figure 8 , as a response to the above Figure 1 The present application provides an embodiment of a multi-camera multi-target personnel tracking device, and the device embodiment is Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0095] like Figure 8 As shown, the multi-camera multi-target personnel tracking device of this embodiment includes: an initial monocular tracking information acquisition unit 71, a high confidence class clustering unit 72, a first merging unit 73, a second merging unit 74, an adjacent frame information acquisition unit 75 and a target tracking result generation unit 76, wherein:

[0096] An initial monocular tracking information acquisition unit 71 is used to acquire monocular tracking and positioning results of multiple cameras in the same batch to obtain initial monocular tracking information;

[0097] A high confidence class clustering unit 72, configured to cluster a high confidence class from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold;

[0098] A first merging unit 73, configured to merge the high confidence classes by adjusting the spatial distance threshold and the human feature similarity threshold in a polling manner to obtain a first merged high confidence class;

[0099] A second merging unit 74 is used to obtain a free class, merge and filter the free class to obtain a second merged high confidence class;

[0100] An adjacent frame information acquisition unit 75, configured to initialize the second merged high confidence class as a multi-target object, and acquire the tracked target information of the plurality of cameras in adjacent frames of the initial monocular tracking information;

[0101] The target tracking result generating unit 76 is used to match the tracked target information with the multiple target objects to obtain a target tracking result.

[0102] Further, the high confidence clustering unit 72 includes:

[0103] A feature extraction unit, used to extract the human features of each tracking target in the initial monocular tracking information;

[0104] A similarity calculation unit, configured to calculate the similarity of the tracking target according to the human body feature, obtain a human body feature similarity value, and determine a target similarity value based on the human body feature similarity value;

[0105] A spatial distance calculation unit, used to determine the target class from the initial monocular tracking information, and calculate the spatial position distance of the target class to obtain the target spatial distance;

[0106] A clustering unit is used to cluster the high confidence class from the initial monocular tracking information according to the spatial distance threshold, the human feature similarity threshold, the target similarity value and the target spatial distance.

[0107] Further, the first merging unit 73 includes:

[0108] A to-be-merged class identification unit, used to identify the categories of cameras that do not overlap from the high-confidence classes to obtain the to-be-merged classes;

[0109] A polling unit, configured to merge the classes to be merged by polling and relaxing any one of the spatial distance threshold and the human feature similarity threshold, to obtain an initial merged high confidence class;

[0110] The merging completion unit is used to use the initial merged high-confidence class and the high-confidence class that has not been merged as the first merged high-confidence class when the class merging cannot be continued.

[0111] Further, the second merging unit 74 includes:

[0112] A free class acquisition unit, used to acquire the free class, merge the free class according to the human feature similarity threshold and the relaxed spatial distance threshold to obtain a merged free class, and add the merged free class to the first merged high-confidence class to obtain the second merged high-confidence class;

[0113] The filtering unit is used to obtain the current remaining free class, filter the current remaining free class to obtain the filtered free class, and convert the filtered free class into the second merged high confidence class.

[0114] Furthermore, the filtering unit comprises:

[0115] A first condition unit, used for obtaining the current remaining free class, and taking the area of ​​the detection box corresponding to the current remaining free class as a first condition being smaller than a preset area;

[0116] A second condition unit, used for taking the current remaining free class being in a preset target area as a second condition;

[0117] A third condition unit, configured to take the current remaining free class occlusion ratio being less than a preset ratio as a third condition;

[0118] A filtering unit is used to use the current remaining free classes that all meet the first condition, the second condition and the third condition as the filtered free classes, and convert the filtered free classes into the second merged high-confidence classes.

[0119] Furthermore, the target tracking result generating unit 76 includes:

[0120] A first matching unit, configured to match the tracked target information with the multiple target objects according to the human feature similarity threshold and the relaxed spatial distance threshold, to obtain matched tracking targets and unmatched tracking targets;

[0121] A second matching unit, configured to re-match the unmatched tracking target by using the relaxed spatial distance threshold and a voting method to obtain a new matching tracking target;

[0122] A current matching tracking target generating unit is used to obtain split tracking information and merge the split tracking information with the matching tracking target to obtain a current matching tracking target, wherein the split tracking information is a newly added multi-target tracking and positioning result;

[0123] A result generating unit is used to generate the target tracking result based on the current matching tracking target and the new matching tracking target.

[0124] Furthermore, the current matching pursuit target generation unit includes:

[0125] A split information acquisition unit, used to acquire tracking information of the split;

[0126] The tracking target merging unit is used to determine the merging of the tracking target in the split tracking information and the matching tracking target through the human feature similarity threshold and the relaxed spatial distance threshold to obtain the current matching tracking target.

[0127] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0128] The computer device 8 includes a memory 81, a processor 82, and a network interface 83 which are interconnected through a system bus. It should be noted that: Fig. 9 Only a computer device 8 having three components, memory 81, processor 82, and network interface 83, is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0129] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0130] The memory 81 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 81 can be an internal storage unit of the computer device 8, such as a hard disk or memory of the computer device 8. In other embodiments, the memory 81 can also be an external storage device of the computer device 8, such as a plug-in hard disk equipped on the computer device 8, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 81 can also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 81 is generally used to store the operating system and various application software installed on the computer device 8, such as the program code of the multi-camera multi-target personnel tracking method, etc. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or are to be output.

[0131] In some embodiments, the processor 82 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 82 is generally used to control the overall operation of the computer device 8. In this embodiment, the processor 82 is used to run the program code stored in the memory 81 or process data, such as running the program code of the above-mentioned multi-camera multi-target personnel tracking method to implement various embodiments of the multi-camera multi-target personnel tracking method.

[0132] The network interface 83 may include a wireless network interface or a wired network interface, and the network interface 83 is generally used to establish a communication connection between the computer device 8 and other electronic devices.

[0133] The present application also provides another implementation, namely, providing a computer-readable storage medium, which stores a computer program, and the computer program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned multi-camera multi-target personnel tracking method.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0135] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A multi-camera multi-target personnel tracking method, characterized in that: include: Obtain the monocular tracking and positioning results of multiple cameras in the same batch to obtain the initial monocular tracking information; Clustering a high confidence class from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold; Merging the high confidence classes by adjusting the spatial distance threshold and the human feature similarity threshold in a polling manner to obtain a first merged high confidence class; Obtaining free classes, merging and filtering the free classes to obtain a second merged high-confidence class; Initialize the second merged high confidence class as a multi-target object, and obtain the tracked target information of the multiple cameras in the adjacent frames of the initial monocular tracking information; The tracked target information is matched with the multiple target objects to obtain a target tracking result.

2. The multi-camera multi-target personnel tracking method according to claim 1, characterized in that: The step of clustering a high confidence class from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold comprises: Extracting human features of each tracking target in the initial monocular tracking information; Calculating the similarity of the tracking target according to the human body feature to obtain a human body feature similarity value, and determining a target similarity value based on the human body feature similarity value; Determine the target class from the initial monocular tracking information, and calculate the spatial position distance of the target class to obtain the target spatial distance; The high confidence class is clustered from the initial monocular tracking information according to the spatial distance threshold, the human feature similarity threshold, the target similarity value and the target spatial distance.

3. The multi-camera multi-target personnel tracking method according to claim 1, characterized in that: The method of adjusting the spatial distance threshold and the human feature similarity threshold by polling to merge the high confidence classes to obtain a first merged high confidence class includes: Identifying categories where cameras do not overlap from the high confidence categories to obtain categories to be merged; Merging the classes to be merged by polling and relaxing any one of the spatial distance threshold and the human feature similarity threshold to obtain an initial merged high confidence class; When the category merging cannot be continued, the initially merged high-confidence category and the high-confidence category that has not been merged are used as the first merged high-confidence category.

4. The multi-camera multi-target personnel tracking method according to claim 1, characterized in that: The obtaining of the free class, merging and filtering the free class to obtain a second merged high confidence class, includes: Acquire the free class, merge the free class according to the human feature similarity threshold and the relaxed spatial distance threshold to obtain a merged free class, and add the merged free class to the first merged high-confidence class to obtain the second merged high-confidence class; The current remaining free classes are obtained, and the current remaining free classes are filtered to obtain filtered free classes, and the filtered free classes are converted into the second merged high-confidence classes.

5. The multi-camera multi-target personnel tracking method according to claim 4, characterized in that: The obtaining of the current remaining free classes, filtering the current remaining free classes to obtain filtered free classes, and converting the filtered free classes into the second merged high confidence classes includes: Acquire the current remaining free class, and take the area of ​​the detection box corresponding to the current remaining free class as a first condition being smaller than a preset area; Taking the current remaining free class as being within a preset target area as a second condition; The third condition is that the proportion of the currently remaining free classes being blocked is less than a preset proportion; The current remaining free classes that satisfy the first condition, the second condition, and the third condition are used as the filtered free classes, and the filtered free classes are converted into the second merged high-confidence classes.

6. The multi-camera multi-target personnel tracking method according to any one of claims 1 to 5, characterized in that: The step of matching the tracked target information with the multiple target objects to obtain a target tracking result includes: Matching the tracked target information with the multiple target objects according to the human feature similarity threshold and the relaxed spatial distance threshold to obtain matched tracking targets and unmatched tracking targets; Re-matching the unmatched tracking target by using the relaxed spatial distance threshold and voting method to obtain a new matching tracking target; Acquire the split tracking information, and merge the split tracking information with the matched tracking target to obtain the current matched tracking target, wherein the split tracking information is the newly added multi-target tracking and positioning result; The target tracking result is generated based on the current matching pursuit target and the new matching pursuit target.

7. The multi-camera multi-target personnel tracking method according to claim 6, characterized in that: The obtaining of the split tracking information and merging the split tracking information with the matching tracking target to obtain the current matching tracking target includes: Obtaining tracking information for the split; The tracking target in the split tracking information is judged to be merged with the matching tracking target through the human feature similarity threshold and the relaxed spatial distance threshold to obtain the current matching tracking target.

8. A multi-camera multi-target personnel tracking device, characterized in that: include: An initial monocular tracking information acquisition unit is used to obtain monocular tracking and positioning results of multiple cameras in the same batch to obtain initial monocular tracking information; A high confidence class clustering unit, used for clustering high confidence classes from the initial monocular tracking information according to a spatial distance threshold and a human feature similarity threshold; A first merging unit, configured to merge the high confidence classes by adjusting the spatial distance threshold and the human feature similarity threshold in a polling manner to obtain a first merged high confidence class; A second merging unit is used to obtain a free class, merge and filter the free class to obtain a second merged high confidence class; An adjacent frame information acquisition unit, configured to initialize the second merged high confidence class as a multi-target object, and acquire the tracked target information of the plurality of cameras in adjacent frames of the initial monocular tracking information; The target tracking result generating unit is used to match the tracked target information with the multiple target objects to obtain a target tracking result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the multi-camera multi-target personnel tracking method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the multi-camera multi-target personnel tracking method according to any one of claims 1 to 7 is implemented.

Images