Target tracking method, device and equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a target tracking method, device and equipment and a computer readable storage medium. The method comprises: obtaining a target video to be detected; performing target detection on video frames in the target video to obtain a detection frame of a target object and identification information in the video frames; extracting the detection frame and the identification information of consecutive video frames in the target video to obtain a first target detection frame, first identification information, a second target detection frame and second identification information; and adjusting the first identification information and the second identification information according to the similarity between the first target detection frame and the second target detection frame to obtain target identification information. Embodiments of the present application extract and compare the similarity between different detection frames in consecutive video frames, and use the similarity to adjust the identification information of the detection frame, thereby effectively avoiding the problem of ID identification error and further improving the effect of target tracking.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of target tracking, and in particular, to a target tracking method, device, equipment and computer readable storage medium. BACKGROUND

[0002] Target tracking refers to a series of continuous detection of a target in a video or continuous multiple frames of images to obtain the moving characteristics of the target.

[0003] In general, objects in different images that are detected as the same target are marked with the same ID in the detection process. However, due to the detection performance of the model itself, the detection of the target cannot be accurately completed, thereby causing problems such as ID identification errors in the actual tracking process. SUMMARY

[0004] Embodiments of the present application provide a target tracking method, device, equipment and computer readable storage medium, aiming to solve the technical problem of ID identification errors existing in the prior art target tracking method.

[0005] In one aspect, the present application provides a target tracking method, comprising:

[0006] obtaining a target video to be detected;

[0007] performing target detection on video frames in the target video to obtain detection boxes corresponding to target objects in the video frames and identification information;

[0008] extracting the detection boxes and the identification information of consecutive video frames in the target video respectively to obtain first target detection boxes and corresponding first identification information, and second target detection boxes and corresponding second identification information;

[0009] adjusting the first identification information and the second identification information according to the similarity between the first target detection boxes and the second target detection boxes to obtain target identification information corresponding to the target objects.

[0010] As an optional embodiment of the present application, the step of adjusting the first identification information and the second identification information according to the similarity between the first target detection boxes and the second target detection boxes to obtain target identification information corresponding to the target objects comprises:

[0011] obtaining the similarity between the first target detection boxes and the second target detection boxes, and the comparison result between the first identification information and the second identification information;

[0012] performing conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result;

[0013] If the conflict detection result is conflict, the step of adjusting the first identification information and the second identification information to obtain the target identification information corresponding to the target object is performed.

[0014] As an optional embodiment of the present application, the step of performing conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result comprises:

[0015] If the similarity is greater than a preset first similarity threshold and the comparison result is different identification information, conflict is set as the conflict detection result of the similarity and the comparison result.

[0016] As an optional embodiment of the present application, the step of performing conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result comprises:

[0017] If the similarity is less than a preset second similarity threshold and the comparison result is the same identification information, conflict is set as the conflict detection result of the similarity and the comparison result.

[0018] As an optional embodiment of the present application, the step of extracting the detection frame and the identification information of the continuous video frames in the target video to obtain the first target detection frame and the corresponding first identification information and the second target detection frame and the corresponding second identification information comprises:

[0019] determining a target video frame from the target video;

[0020] extracting the detection frame and the identification information in the target video frame to obtain the first target detection frame and the corresponding first identification information;

[0021] extracting the detection frame and the identification information in the adjacent video frame of the target video frame to obtain the second target detection frame and the corresponding second identification information.

[0022] As an optional embodiment of the present application, the step of determining a target video frame from the target video comprises:

[0023] setting the video frame with a preset number of detection frames in the target video as a target video frame; and / or

[0024] The video frame with a distance between the bounding boxes smaller than the preset distance threshold in the target video is set as a target video frame.

[0025] As an optional embodiment of the present application, the step of adjusting the first identification information and the second identification information comprises:

[0026] obtaining a reference bounding box corresponding to the first target bounding box and the second target bounding box, and reference identification information corresponding to the reference bounding box;

[0027] if the similarity between the first target bounding box and the reference bounding box is greater than a preset third similarity threshold, adjusting the first identification information corresponding to the first target bounding box to the reference identification information; and / or

[0028] if the similarity between the second target bounding box and the reference bounding box is greater than a preset third similarity threshold, adjusting the second identification information corresponding to the second target bounding box to the reference identification information.

[0029] In another aspect, the embodiments of the present application also provide a target tracking device, comprising:

[0030] an acquisition module configured to acquire a target video to be detected;

[0031] a detection module configured to perform target detection on video frames in the target video to obtain bounding boxes corresponding to target objects in the video frames and identification information;

[0032] an extraction module configured to extract the bounding boxes and the identification information of consecutive video frames in the target video respectively to obtain a first target bounding box and corresponding first identification information, and a second target bounding box and corresponding second identification information;

[0033] an adjustment module configured to adjust the first identification information and the second identification information according to the similarity between the first target bounding box and the second target bounding box to obtain target identification information corresponding to the target objects.

[0034] In another aspect, the embodiments of the present application also provide a target tracking device, comprising a processor, a memory, and a target tracking program stored in the memory and executable on the processor, wherein the processor executes the target tracking program to implement the steps in the above target tracking method.

[0035] In another aspect, the embodiments of the present application also provide a computer readable storage medium having a target tracking program stored thereon, wherein the target tracking program is executed by a processor to implement the steps in the above target tracking method.

[0036] The technical scheme provided by the embodiments of the present application, after obtaining the detection frame and identification information by performing target detection on the video frames in the target video, extracts and compares the similarity between different detection frames in the continuous video frames, and adjusts the identification information of the detection frame by using the similarity, so that the identification information of the adjusted detection frame can match the similarity between the detection frames, thereby effectively eliminating the ID identification error that may occur in the prior art when only a tracking algorithm is used, and further improving the target tracking effect. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical scheme in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 An implementation scenario diagram of a target tracking provided by the embodiments of the present application;

[0039] Figure 2 A step flow diagram of a target tracking method provided by the embodiments of the present application;

[0040] Figure 3 A step flow diagram of adjusting identification based on detection frame similarity provided by the embodiments of the present application;

[0041] Figure 4 A step flow diagram of conflict detection on similarity and comparison result provided by the embodiments of the present application;

[0042] Figure 5 Another step flow diagram of conflict detection on similarity and comparison result provided by the embodiments of the present application;

[0043] Figure 6 A step flow diagram of extracting detection frame and identification information provided by the embodiments of the present application;

[0044] Figure 7 A step flow diagram of determining target video frame provided by the embodiments of the present application;

[0045] Figure 8 A step flow diagram of adjusting identification information provided by the embodiments of the present application;

[0046] FIGS. 9(a), 9(b), 9(c) and 9(d) are specific scene diagrams of ID misassignment provided by the embodiments of the present application;

[0047] Figure 10 A structural schematic diagram of a target tracking device provided in an embodiment of the present application is shown in FIG. 1.

[0048] Figure 11 A structural schematic diagram of a target tracking device provided in an embodiment of the present application is shown in FIG. 1. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, any other embodiments obtained by a person skilled in the art without creative work fall within the scope of the present application.

[0050] In the embodiments of the present application, the word "exemplary" is used to mean "serving as an example, instance, or illustration." Any embodiment described as "exemplary" in the embodiments of the present application is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the present application. In the following description, for the purposes of explanation, numerous details are set forth in order to provide a thorough understanding. It should be apparent to those skilled in the art that the present application can be practiced without the use of these specific details. In other instances, well known structures and processes are not elaborated upon in order to avoid obscuring the description of the present application. Thus, the present application is not intended to be limited by the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0051] The embodiments of the present application provide a target tracking method, device, equipment and computer readable storage medium, which are described in detail as follows.

[0052] As shown in FIG. 1, Figure 1 Figure 1 A schematic diagram of an implementation scenario of target tracking provided in an embodiment of the present application is shown in FIG. 1. Specifically, the implementation scenario includes a collection device 100 and a target tracking device 200. The collection device 100 is mainly used to collect a video to be detected and continuously transmit the video to the target tracking device 200 in the form of a video stream. The target tracking device 200 will implement the target tracking method provided in the embodiments of the present application by executing a target tracking program deployed thereon after obtaining the video, so as to complete tracking of targets in the video. For example, in a security check project, the collection device 100 will continuously collect a video of a security check machine conveyor belt and send the video to the target tracking device 200. At this time, the target tracking device will continuously track and detect packages on the security check machine conveyor belt.

[0053] It should be noted that, Figure 1 ​The implementation scenario diagram of target tracking shown is only an example, and the implementation scenario diagram of target tracking described in the embodiments of the present application is used to more clearly illustrate the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application.

[0054] Based on the implementation scenario diagram of target tracking described above, specific embodiments of a target tracking method are provided.

[0055] As shown in Figure 2 , Figure 2 A step flow diagram of a target tracking method provided by the embodiments of the present application is shown, and the target tracking method includes steps 201-204.

[0056] 201, obtaining a target video to be detected.

[0057] In the embodiments of the present application, the target video to be detected refers to a video that needs to be tracked, which is usually obtained by a video acquisition device. The target video usually contains a plurality of targets, and target tracking refers to target detection on each image frame in the target video, and different targets in the target video are expected to be marked with different labels, and the same target is marked with the same label, so as to facilitate user identification. Specifically, the label usually exists in the form of ID, that is, different IDs are assigned to different targets, for example, digital IDs.

[0058] Specifically, the target video can be obtained in real time, and at this time, the target tracking processing of the target video is also performed in real time. For example, in a common security check scene, a video acquisition device can collect the video on a conveyor belt in real time, and track the package on the conveyor belt in real time. Of course, the target video can also be a complete video that has been collected, for example, in a common monitoring video, a target tracking algorithm provided by the embodiments of the present application is used to track and detect a target person or a target vehicle in the video.

[0059] However, it should be noted that no matter how the specific implementation scenario is, the video frames in the target video will be processed, that is, the video needs to be preprocessed, for example, the video frames collected in real time are sent to a target tracking device for processing according to a certain time period, or a plurality of continuous video frames are obtained by sampling the video according to a certain time period, and then sent to a target tracking device for processing. Specifically, the time period can be selected as 0.25 seconds, that is, the video is sampled at a frequency of 4 frames per second.

[0060] 202, performing target detection on the video frames in the target video to obtain detection boxes corresponding to target objects in the video frames and label information.

[0061] In the embodiments of the present application, the target video in the target video can be detected by using the existing tracking model, and the detection frame and the identification information corresponding to each target object in each video can be obtained. Specifically, for example, the target video in the target video can be detected by using the target detection model such as yolov5, Faster R-CNN, and on the basis of target detection, the detection frame is further marked with corresponding identification information, that is, ID, by using the tracking algorithm such as deepsort, deepsocial, and centertrack.

[0062] However, target tracking expects to mark different targets in the target video with different IDs and the same target with the same ID to facilitate user identification, but in the prior art, due to the detection performance problem of the target detection model itself, the tracking algorithm often misjudges, that is, marks the wrong ID.

[0063] For example, it is common that if multiple objects are too close and are not detected at the same time by the target detection model, ID switch, that is, the serial number exchange problem, will occur. Specifically, taking a conveyor belt moving from left to right as an example, when the A package on the right is identified and the B package on the left of the small package is not identified, the A package is marked with an A ID. However, after a period of time, the target detection model detects the A package and the B package at the same time, and if the A package and the B package are close enough, the tracking algorithm may mark the B package with an A ID, that is, inherit the identification information of the A package, that is, mark different objects with the same ID, and mark the A package with a new ID, thereby causing the ID of the A package to be inconsistent before and after.

[0064] In addition, if the generalization ability of the detection model is insufficient, that is, the target object cannot be accurately identified, Bodymismatch, that is, the problem of misidentification of the object to be detected, will also occur, that is, the same object is marked with different IDs.

[0065] In order to more clearly understand the two ID misidentification problems provided by the embodiments of the present application, as shown in Figures 9(a) to 9(d) , a specific scene diagram of ID misidentification in the prior art is shown.

[0066] 203, the detection frame and the identification information of the continuous video frames in the target video are extracted respectively to obtain the first target detection frame and the corresponding first identification information, and the second target detection frame and the corresponding second identification information.

[0067] In the embodiments of the present application, the first target detection frame and the second target detection frame are respectively a detection frame in a first video frame and a detection frame in a second video frame in consecutive video frames, the first identification information is identification information marked by the first target detection frame, and the second identification information is identification information marked by the second target detection frame.

[0068] Specifically, in order to realize the extraction of the detection frame and the identification information in the video frame, taking the real-time target tracking processing of the video as an example, a cache queue can be used to record the detection frame and the corresponding identification information in the latest two frames of pictures, of course, it is also feasible to extract the detection frame and the identification information of the consecutive video frames in the target video in other manners, and the present application examples will not be repeated here.

[0069] Further, in general, for any two consecutive video frames in the target video, the target tracking device will extract the detection frame and the identification information, so as to subsequently complete the error correction of the identification information by using the similarity between the detection frames, but this will occupy more computing resources, considering that the two ID misassignment problems mentioned above, the serial number exchange problem mainly occurs in the video frames with detection frames close to each other, and the to-be-detected object misassignment problem mainly occurs in the video frames with only a single detection frame, therefore, whether the video frame has a high probability of ID misassignment can be judged by extracting some features of the detection frame in the video frame. Specifically, reference can be made to the subsequent Figure 6 and the content explained and described.

[0070] 204. Adjusting the first identification information and the second identification information according to the similarity between the first target detection frame and the second target detection frame, to obtain the target identification information corresponding to the target object.

[0071] In the embodiments of the present application, after obtaining the detection frame in the first video frame and the detection frame in the second video frame in the consecutive video frames, by comparing the similarity of the images in the two target detection frames, whether the identification information is misjudged can be further judged, so that the first identification information and the second identification information can be adjusted to obtain the accurate identification information corresponding to each target object in the video frame. Specifically, the specific implementation scheme of adjusting the identification information according to the similarity can be referred to the subsequent Figure 3 and the content explained and described.

[0072] Furthermore, it should be noted that improving the model's object detection performance by modifying the model structure or training process can also improve the tracking algorithm's effectiveness to some extent. However, improving the model's object detection performance requires more model training resources, and the improvement is limited. For example, conventional detection models typically have an 8%–10% probability of ID mismatch, while object detection models trained with supplementary sample data can only improve accuracy by about 2%, ultimately still having a 6%–8% ID mismatch rate. The object tracking method provided in this application, however, can effectively improve object tracking performance without consuming more model training resources, ultimately achieving an accuracy of over 95% in tracking target objects.

[0073] The technical solution provided in this application, after obtaining detection boxes and identification information by performing target detection on video frames in the target video, extracts and compares the similarity between different detection boxes in consecutive video frames, and uses this similarity to adjust the identification information of the detection boxes, so that the identification information of the adjusted detection boxes can match the similarity between the detection boxes, thereby effectively eliminating the problem of ID identification errors that may occur when using tracking algorithms alone in the prior art, and further improving the target tracking effect.

[0074] like Figure 3 As shown, Figure 3 This document presents a flowchart illustrating the steps for adjusting identifiers based on bounding box similarity, as provided in an embodiment of this application. Details are as follows.

[0075] In this application embodiment, an implementation scheme for adjusting the label based on the similarity between detection boxes is provided, specifically including steps 301 to 303:

[0076] 301. Obtain the similarity between the first target detection box and the second target detection box, as well as the comparison result between the first identification information and the second identification information.

[0077] In this embodiment of the application, the similarity between the first target detection box and the second target detection box refers to the similarity between the image content in the first target detection box and the image content in the second target detection box. Since the target detection box can be understood as the object detected by the target detection model, the similarity between the first target detection box and the second target detection box can be understood as the similarity between two target objects.

[0078] Specifically, there are many implementation schemes for calculating the similarity between the detection boxes, for example, the simplest one can be obtained by counting the similarity between each pixel point in the image in the detection box. Of course, as another feasible implementation scheme of the present application, the detection box can also be input into the Siamese network with shared weight parameters, that is, the Siamese network, to extract the feature vector of the detection box, so as to complete the calculation of the similarity, so as to obtain a more accurate similarity calculation result.

[0079] In the embodiments of the present application, the comparison result between the first identification information and the second identification information generally refers to the comparison result of whether the first identification information and the second identification information are the same, that is, the comparison result contains two kinds, the same identification information and the different identification information. The first identification information and the second identification information are the same, and the corresponding comparison result is the same identification information. The first identification information and the second identification information are different, and the corresponding comparison result is different identification information.

[0080] 302. According to the preset conflict detection rule, the similarity and the comparison result are detected for conflict, and the conflict detection result of the similarity and the comparison result is obtained.

[0081] In the embodiments of the present application, it can be known from the foregoing description that for the same target object, its identification information should be the same, and the similarity between the corresponding target detection boxes is also high, and vice versa, for different target objects, its identification information should be different, and the similarity between the corresponding target detection boxes is also low. That is, under normal circumstances, the similarity between the target detection boxes matches the comparison result between the identification information. Based on this, the similarity and the comparison result can be detected for conflict according to the pre-set conflict detection rule, so as to judge whether the similarity and the comparison result match. The conflict detection result generally includes two kinds, conflict and no conflict, which correspond to the matching and non-matching relationship between the similarity and the comparison result respectively. Specifically, there are two feasible implementation ways of the preset conflict detection rule, which are described in the following Figures 4 to 5 and the content of the interpretation and description.

[0082] 303. It is judged whether the conflict detection result is conflict. If yes, step 304 is executed; if no, other steps are executed.

[0083] In the embodiment of the present application, if the conflict detection result is that there is a conflict, it indicates that the similarity between the comparison result of the identification information and the detection frame does not match, and the first identification information and / or the second identification information may have a misfit problem, and the first identification information and the second identification information need to be adjusted, that is, step 304 is performed. On the contrary, if the conflict detection result is that there is no conflict, it indicates that the similarity between the comparison result of the identification information and the detection frame matches, and at this time, the first identification information and the second identification information can be directly output as accurate identification information.

[0084] 304. Adjusting the first identification information and the second identification information to obtain target identification information corresponding to the target object.

[0085] In the embodiment of the present application, adjusting the first identification information and the second identification information means adjusting the identification information that may be incorrect identification information in the first identification information and the second identification information. In order to identify the incorrect identification information, the target detection frame and the identification information in other image frames can be used as a reference, and details can be referred to subsequent Figure 8 and the explanatory content thereof.

[0086] As Figure 4 shown, Figure 4 is a step flow diagram for conflict detection of similarity and comparison result provided by the embodiment of the present application. Specifically, steps 401-403 are included.

[0087] 401. If the comparison result is different identification information, it is judged whether the similarity is greater than a preset first similarity threshold. If yes, step 402 is performed; if no, other steps are performed.

[0088] As can be known from the foregoing description, if the comparison result is different identification information, that is, the first identification information and the second identification information are different, that is, the target detection model output considers that the two objects are different, therefore, whether the contents contained in the two detection frames are really different can be determined by judging the size relationship between the similarity and the preset first similarity threshold. Specifically, if the similarity is less than or equal to the preset first similarity threshold, it can be considered that the contents contained in the two detection frames are not similar, which conforms to the comparison result between the first identification information and the second identification information, that is, it is considered that the conflict detection result is that there is no conflict. On the contrary, if the similarity is greater than the preset first similarity threshold, it indicates that the contents contained in the two detection frames are similar, that is, the comparison result between the similarity and the identification information is conflicting.

[0089] Furthermore, similarity is generally described using a numerical value between 0 and 1, where a similarity of 0 indicates that the two are completely dissimilar, and a similarity of 1 indicates that they are completely identical. Considering that ID mismatches are infrequent and have a low probability in practical applications, meaning that the comparison results will be correct in most cases, the first similarity threshold should be set relatively high. Specifically, the first similarity threshold can be set based on the probability of ID mismatches during testing. For example, if statistics show an 8% to 10% probability of ID mismatches during testing, then the first similarity threshold can be set to 0.9 to 0.92.

[0090] 402, the existence of a conflict is set as the conflict detection result of the similarity and the comparison result.

[0091] In this embodiment of the application, when the comparison result is different identification information and the similarity is greater than the preset first similarity threshold, it can be considered that the comparison result between the similarity and the identification information is conflicting. Therefore, the conflict detection result between the similarity and the comparison result can be set as having a conflict.

[0092] like Figure 5 As shown, Figure 5 This is a schematic flowchart illustrating another step for conflict detection of similarity and comparison results provided in an embodiment of this application. Specifically, it includes steps 501 to 502:

[0093] 501. If the comparison result is the same identification information, then determine whether the similarity is less than a preset second similarity threshold. If yes, proceed to step 502; otherwise, proceed to other steps.

[0094] and Figure 4 Unlike the schematic diagram illustrating conflict detection steps for similarity and comparison results, this embodiment focuses on conflict detection when the comparison results show identical identifier information. Specifically, if the comparison results show identical identifier information (i.e., the first identifier information and the second identifier information are the same), the target detection model outputs a target detection result indicating that the two objects are identical. Therefore, the similarity and a preset second similarity threshold can be used to determine whether the content contained in the two detection boxes is truly identical. If the similarity is greater than or equal to the preset second similarity threshold, the content contained in the two detection boxes can be considered similar, conforming to the comparison result between the first and second identifier information, meaning the conflict detection result is considered non-conflicting. Conversely, if the similarity is less than the preset second similarity threshold, it indicates that the content contained in the two detection boxes is dissimilar, meaning the comparison result between the similarity and the identifier information is conflicting.

[0095] Similarly, considering that the ID misjudgment occurs less frequently in the actual application process, the probability is low, that is, in most cases, the comparison result will be correct, therefore, the second similarity threshold should be set relatively low. Specifically, the second similarity threshold can also be set according to the probability of ID misjudgment in the test process, for example, it is found through statistics in the test process that there is an 8% to 10% probability of ID misjudgment, and the second similarity threshold can be set to 0.08 to 0.1.

[0096] 502, set the conflict as the similarity and the conflict detection result of the comparison result.

[0097] In the embodiment of the application, when the comparison result is the same identification information and the similarity is less than the preset second similarity threshold, it can be considered that the comparison result between the similarity and the identification information is in conflict, and therefore, the conflict detection result of the similarity and the comparison result can be set as the existence of the conflict.

[0098] As shown in Figure 6 , a step flow diagram for extracting a detection frame and identification information provided by the embodiment of the application. Details are as follows. Figure 6 Considering that ID misjudgment mainly occurs in specific image frames in the target tracking process, the embodiment of the application provides a technical solution for determining the target image frame in which ID misjudgment is more likely to occur from the video, specifically, including steps 601-603:

[0099] 601, determine a target video frame from the target video.

[0100] In the embodiment of the application, considering that the misjudgment of the to-be-detected object mainly occurs in the video frame with only a single target detection frame, and the serial number exchange mainly occurs in the video frame with too close target detection frames, therefore, the feature information of each video frame can be obtained by extracting the features of each video frame in the video, and then the target video frame in which ID misjudgment is more likely to occur is extracted from the target video frame based on the feature information of the target detection frame in the video frame. Specifically, the feature information can use the number feature and / or the spacing feature, at this time, the specific implementation scheme of extracting the target video frame can refer to the subsequent

[0101] and the explanatory content thereof. Figure 7

[0102] In addition, for the video that performs real-time target tracking processing, the specific method for determining the target video frame according to the feature information is to determine whether the current video frame is the target video frame in which ID misjudgment is more likely to occur by judging whether the feature information of the target detection frame in the current video frame meets the preset condition after obtaining the current video frame in real time.

[0103] ​602, Extract the detection box and identification information from the target video frame to obtain the first target detection box and its corresponding first identification information.

[0104] In this embodiment of the application, after extracting the target video frame that is more likely to have ID mismatch from the video, the detection box and the identification information in the target video frame can be extracted, and the extracted detection box is used as the first target detection box, and the identification information of the detection box is used as the first identification information.

[0105] 603. Extract the detection boxes and identification information from the adjacent video frames of the target video frame to obtain the second target detection box and its corresponding second identification information.

[0106] In this embodiment of the application, after taking the detection box and the identification information in the target video frame as the first target detection box and the first identification information respectively, the adjacent video frames of the target video frame will be further extracted. For example, the detection box in the previous video frame of the target video frame can be taken as the second target detection box, and its corresponding identification information can be taken as the second identification information.

[0107] The technical solution provided in this application extracts the feature information of the target detection box in the video frame to determine the target video frame that is more likely to be mismatched with ID, thereby facilitating subsequent correction of the target video frame. While effectively reducing the amount of computation, it can effectively avoid the occurrence of ID mismatch anomalies.

[0108] like Figure 7 As shown, Figure 7 This is a flowchart illustrating the steps for determining the target video frame provided in an embodiment of this application. Specifically, it may include steps 701 to 702:

[0109] 701, The video frame in the target video with a preset number of detection boxes is set as the target video frame.

[0110] In this embodiment of the application, the target tracking device extracts the number of detection boxes in each video frame of the target video and sets the number of detection boxes to a preset value. Typically, the preset value is set to 1, which means that a video frame containing only one detection box is set as the target video frame.

[0111] In addition, it is also necessary to point out that, for the video frame containing only one detection box, considering that its main appearance will appear detection object misassignment problem, that is, the same object will be marked with different labels, therefore, if the number of detection boxes in the target video is set to the preset value of the video frame, then in the subsequent similarity and comparison result conflict detection process, mainly determine whether the comparison result is different label information, and after determining that the comparison result is different label information, continue to judge whether the similarity is greater than the preset first similarity threshold, that is, mainly execute Figure 4 the steps of conflict detection of similarity and comparison result as shown.

[0112] 702, set the video frame in which the distance between the detection boxes in the target video is less than the preset distance threshold as the target video frame.

[0113] In the embodiment of the application, as another feasible step, the target tracking device extracts the distance between the detection boxes in each video frame in the target video, and sets the video frame in which the distance between the detection boxes is less than the preset distance threshold as the target video frame. Specifically, the embodiment of the application does not limit the implementation scheme of specifically obtaining the distance between the detection boxes, for example, it can be the distance between the two closest pixel points in the two detection boxes, or it can be the distance between the center points of the detection boxes.

[0114] In addition, similar to the foregoing step 701, for the video frame in which the distance between the detection boxes is less than the preset distance threshold, considering that its main appearance will appear the problem of serial number exchange, that is, different objects will be marked with the same label, therefore, if the video frame in which the distance between the detection boxes in the target video is less than the preset distance threshold is set as the target video frame, then in the subsequent similarity and comparison result conflict detection process, mainly determine whether the comparison result is the same label information, and after determining that the comparison result is the same label information, continue to judge whether the similarity is less than the preset second similarity threshold, that is, mainly execute Figure 5 the steps of conflict detection of similarity and comparison result as shown.

[0115] The embodiment of the application provides a specific implementation scheme for determining the target video frame according to the feature information of the target detection box in the video frame. Specifically, the number of detection boxes in the video frame can be extracted to determine whether the video frame is a target video frame prone to detection object misassignment, or the distance between the detection boxes in the video can be extracted to determine whether the video frame is a target video frame prone to serial number exchange. The above steps of determining the target video frame can exist independently, that is, only the number of detection boxes or only the distance between the detection boxes is used to determine the target video frame, of course, more commonly, the number of detection boxes and the distance between the detection boxes are used to determine the target video frame.

[0116] As Figure 8 shown, Figure 8 A step flow diagram for adjusting identification information provided by an embodiment of the present application is provided, as follows.

[0117] In the embodiment of the present application, it specifically includes steps 801-803:

[0118] 801, obtaining the reference detection frame corresponding to the first target detection frame and the second target detection frame, and the reference identification information corresponding to the reference detection frame.

[0119] Considering the continuity of target tracking, that is, under normal circumstances, the same target often appears in multiple consecutive video frames, and the target tracking provided by the present application is continuous, that is, the detection frame in the previous video frame is usually after the correction processing, and the identification result is often accurate. Therefore, the reference detection frame corresponding to the first target detection frame and the second target detection frame refers to the detection frame in the previous video frame adjacent to the video frame where the first target detection frame and the second target detection frame are located. At this time, the reference detection frame and its corresponding identification information can be further used to correct the identification information of the first target detection frame and / or the second target detection frame.

[0120] 802, judging whether the similarity of the first target detection frame and the reference detection frame is greater than a preset third similarity threshold. If yes, step 903 is executed; if no, other steps are executed.

[0121] In the embodiment of the present application, by judging whether the similarity of the first target detection frame and each reference detection frame is greater than a preset similarity threshold in turn, it is determined whether the two targets are the same object. It can be understood that if the similarity of the first target detection frame and a certain reference detection frame is greater than a preset third similarity threshold, it indicates that the targets corresponding to the two detection frames are the same, and their identification information should also be the same.

[0122] Further, as an optional implementation scheme of the present application, the third similarity threshold can be equal to the first similarity threshold mentioned above.

[0123] 803, adjusting the first identification information corresponding to the first target detection frame to the reference identification information.

[0124] In the embodiment of the present application, as can be known from the foregoing related description, if the similarity of the first target detection frame and the reference detection frame is greater than the preset third similarity threshold, the first identification information corresponding to the first target detection frame can be adjusted to the reference identification information.

[0125] It should be noted that in this embodiment of the application, the example of adjusting the identification information corresponding to the first target detection box is only used. In fact, the same steps are performed for the second target detection box, that is, to determine whether the similarity between the second target detection box and the reference detection box is greater than the preset third similarity threshold. If so, the second identification information corresponding to the second target detection box is adjusted to the reference identification information.

[0126] Furthermore, in practical applications, to enable users to clearly understand that the target tracking algorithm provided in this application has corrected the identification information, thus facilitating subsequent manual screening, the normally output identification information and the corrected identification information can be displayed differently. For example, this can be distinguished by the display format. That is, after adjusting the first identification information corresponding to the first target detection box or the second identification information corresponding to the second target detection box, the format of the adjusted identification information is different from that of the normally output identification information. Specifically, the normally assigned identification information can exist in the form of pure numbers, while the corrected identification information can exist in the form of numbers + letters. That is, the normally assigned identification information can be 1, 2, 3, etc., and when the identification information of a certain detection box is adjusted to 3 after being processed by the target tracking algorithm proposed in this application, it can be labeled as 3-A. Subsequently, manual screening of detection boxes with identification information in the form of numbers + letters can be more convenient.

[0127] To better understand the ID mismatch problem provided in the embodiments of this application, such as Figures 9(a) to 9(d) The diagram illustrates a specific scenario of ID mismatch.

[0128] Specifically, Figures 9(a) and 9(b) illustrate ID mismatch diagrams caused by sequence number swapping. In Figure 9(a), package A on the right is marked with an ID identifier of 239, while package B on the left has not yet been detected by the target detection model. After a period of time, both package A and package B, which are being transmitted to the right, are detected by the target detection model simultaneously, as shown in Figure 9(b). At this point, because package A and package B are too close, the tracking algorithm may mark package B with an ID identifier of 239 and update the ID identifier of package A to 244. This is a sequence number swapping anomaly, meaning that two different targets, package A and package B, have been marked with the same ID identifier.

[0129] Correspondingly, FIG. 9(c) and FIG. 9(d) describe the schematic diagram of the misassignment of the to-be-detected object. In FIG. 9(c), the single package C is marked with the ID 1645, however, due to the performance problem of the target detection model itself, the package C can be identified as another object in the subsequent video frames, so that the package C is marked with a new ID, for example, as shown in FIG. 9(d), at this time, the package C is marked with the ID 1646, at this time, the misassignment of the to-be-detected object occurs, that is, the same target is marked with different IDs in different video frames.

[0130] In order to more clearly know the complete implementation scheme of the target tracking algorithm provided by the embodiments of the present application, specifically, the complete implementation process of real-time target tracking in the package security inspection process will be given below in combination with the foregoing Figures 2 to 8 step flowchart, and Figures 9(a) to 9(d) specific scene schematic diagram.

[0131] The camera is installed on the package security inspection equipment, which will collect images of the packages on the conveyor belt at a frequency of 4 frames per second, and transmit the real-time collected video frames to the target tracking device. After obtaining the real-time collected current video frame, the target tracking device will obtain the detection frame and the initial identification information of each detection frame in the current video frame based on the conventional target tracking algorithm, and extract the number information and distance feature of the detection frame in the current video frame. In addition, the target tracking device will also cache the latest obtained several video frames, that is, the several video frames before the current video frame, through a cache queue.

[0132] If the target tracking device determines that the number of detection boxes in the current video frame is 1, it means that the current video frame is a target image frame prone to misassignment of the to-be-detected object, at this time, the previous video frame of the current video frame is extracted from the cache queue, and the identification information of the detection box in the previous video frame is compared with the identification information of the detection box in the current video frame. If it is determined that they are the same, no subsequent steps need to be performed, and the newly acquired video frame can continue to be processed. If it is determined that they are not the same, the similarity of the two detection boxes needs to be further calculated. If the obtained similarity is lower than or equal to 90%, the identification information is normal and can be output. If the obtained similarity is higher than 90%, it is considered that misassignment of the to-be-detected object has occurred, and the identification information in the two video frames needs to be adjusted. Considering that the processing of the video frame is performed in sequence in the real-time target tracking process, the detection box and the identification information in the previous video frame have been corrected and will not be abnormal, so the identification information of the detection box in the current video frame can be directly adjusted to the identification information of the detection box in the previous video frame. Of course, it is also feasible to adjust the identification information of the detection box in the current video frame and / or the previous video frame by using the identification information of the detection box in other video frames in the cache queue and the similarity.

[0133] If the target tracking device determines that the distance between the detection boxes in the current video frame is less than a certain distance threshold, for example, less than 5 cm, it means that the current video frame is a target image frame prone to sequence number exchange abnormality. At this time, the previous video frame of the current video frame is extracted from the cache queue, and the identification information of the detection box in the previous video frame is compared with the identification information of the detection box in the current video frame. If it is determined that they are different, no subsequent steps need to be performed, and the newly acquired video frame can continue to be processed. If it is determined that they are the same, the similarity of the two detection boxes needs to be further calculated. If the obtained similarity is greater than or equal to 10%, the identification information is normal and can be output. If the obtained similarity is less than 10%, it is considered that sequence number exchange abnormality has occurred, and the identification information in the two video frames needs to be adjusted. Similarly, considering that the processing of the video frame is performed in sequence in the real-time target tracking process, the detection box and the identification information in the previous video frame have been corrected and will not be abnormal, so only the identification information of each detection box in the current video frame needs to be adjusted in the adjustment process, and the adjusted identification information of each detection box can be obtained by comparing with the detection box and the identification information of the video frame in the cache queue. Of course, it should be noted that if the similarity of the detection box in the current video frame and the similarity of the detection box in the video frame in the cache queue are both not higher than 80%, the identification information of the detection box can be adjusted to a new identification information.

[0134] In addition, for the current video frame with the number of detection boxes not being 1 and the distance between the detection boxes being greater than 5 cm, the situation of ID mismatching exception occurring for such a video frame is less common, and therefore, it can be considered that no subsequent rectification processing is performed, and after the detection boxes and the initial identification information of each detection box are obtained based on the conventional target tracking algorithm, the initial identification information is directly output. Of course, in order to improve the accuracy, rectification processing can also be performed, but the efficiency of target tracking will be affected. As an optional solution, the rectification processing can be performed with a certain probability, for example, the rectification processing is performed on such a video frame with a probability of 10% to 20%.

[0135] In order to better implement the target tracking method in the embodiments of the present application, on the basis of the target tracking method, a target tracking device is further provided in the embodiments of the present application, as shown in Figure 10 Figure 10 a structure diagram of the target tracking device provided in the embodiments of the present application. Specifically, the target tracking device comprises:

[0136] an acquisition module 1001 configured to acquire a target video to be detected;

[0137] a detection module 1002 configured to perform target detection on a video frame in the target video to obtain a detection box corresponding to a target object in the video frame and identification information;

[0138] an extraction module 1003 configured to extract the detection box and the identification information of consecutive video frames in the target video respectively to obtain a first target detection box and corresponding first identification information and a second target detection box and corresponding second identification information;

[0139] an adjustment module 1004 configured to adjust the first identification information and the second identification information according to the similarity between the first target detection box and the second target detection box to obtain target identification information corresponding to the target object.

[0140] In some embodiments of the present application, the adjustment module comprises:

[0141] a comparison sub-module configured to obtain the similarity between the first target detection box and the second target detection box and a comparison result between the first identification information and the second identification information;

[0142] a conflict detection sub-module configured to perform conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result;

[0143] ​The adjusting sub-module is configured to, if the conflict detection result is that there is a conflict, perform the adjusting of the first identification information and the second identification information to obtain the target identification information corresponding to the target object.

[0144] In some embodiments of the present application, the conflict detection sub-module includes:

[0145] The first conflict detection unit is configured to, if the similarity is greater than a preset first similarity threshold and the comparison result is different identification information, set that there is a conflict as the conflict detection result of the similarity and the comparison result.

[0146] In some embodiments of the present application, the conflict detection sub-module includes:

[0147] The second conflict detection unit is configured to, if the similarity is less than a preset second similarity threshold and the comparison result is the same identification information, set that there is a conflict as the conflict detection result of the similarity and the comparison result.

[0148] In some embodiments of the present application, the extraction module includes:

[0149] The target frame screening sub-module is configured to determine a target video frame from the target video.

[0150] The first extraction sub-module is configured to extract a detection box and identification information in the target video frame to obtain a first target detection box and corresponding first identification information.

[0151] The second extraction sub-module is configured to extract a detection box and identification information in an adjacent video frame of the target video frame to obtain a second target detection box and corresponding second identification information.

[0152] In some embodiments of the present application, the target frame screening sub-module includes:

[0153] The number screening unit is configured to set a video frame in which the number of detection boxes in the target video is a preset value as a target video frame; and / or

[0154] The distance screening unit is configured to set a video frame in which the distance between detection boxes in the target video is less than a preset distance threshold as a target video frame.

[0155] In some embodiments of the present application, the adjusting sub-module includes:

[0156] The reference information acquisition unit is configured to acquire a reference detection box corresponding to the first target detection box and the second target detection box, and reference identification information corresponding to the reference detection box.

[0157] The first adjusting unit is configured to adjust the first identification information corresponding to the first target detection frame to the reference identification information if the similarity between the first target detection frame and the reference detection frame is greater than a preset third similarity threshold.

[0158] The second adjusting unit is configured to adjust the second identification information corresponding to the second target detection frame to the reference identification information if the similarity between the second target detection frame and the reference detection frame is greater than a preset third similarity threshold.

[0159] The embodiment of the present application further provides a target tracking device, as shown in the accompanying drawings. Figure 11 Figure 11 is an embodiment structure schematic diagram of the target tracking device provided in the present application.

[0160] The target tracking device comprises a memory, a processor, and a target tracking program stored in the memory and executable on the processor, and the processor implements the steps in the target tracking method in any embodiment when executing the target tracking program.

[0161] Specifically, the target tracking device can comprise a processor 1101 with one or more processing cores, a memory 1102 with one or more storage media, a power supply 1103, an input unit 1104, and the like. Those skilled in the art can understand that the structure of the target tracking device shown in the accompanying drawings does not constitute a limitation on the target tracking device, and can comprise more or fewer components than shown, or combine certain components, or different component arrangements. Among them: Figure 11

[0162] The processor 1101 is the control center of the target tracking device, connects various parts of the target tracking device through various interfaces and lines, executes software programs and / or modules stored in the memory 1102 and data stored in the memory 1102, processes various functions and data of the target tracking device, and thus monitors the target tracking device as a whole. Optionally, the processor 1101 can comprise one or more processing cores; preferably, the processor 1101 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1101.

[0163] ​​The memory 1102 can be used to store software programs and modules, and the processor 1101 can execute various function applications and data processing by running the software programs and modules stored in the memory 1102. The memory 1102 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the target tracking device, etc. In addition, the memory 1102 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 1102 can also include a memory controller to provide the processor 1101 with access to the memory 1102.

[0164] The target tracking device also includes a power supply 1103 for supplying power to various components. Preferably, the power supply 1103 can be logically connected to the processor 1101 through a power management system, so as to realize functions such as charge management, discharge management, and power consumption management through the power management system. The power supply 1103 can also include one or more direct current or alternating current power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

[0165] The target tracking device can also include an input unit 1104, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0166] Although not shown, the target tracking device can also include a display unit, etc., which will not be described here. Specifically, in the present embodiment, the processor 1101 in the target tracking device will load executable files corresponding to the processes of one or more application programs into the memory 1102 according to the following instructions, and run the application programs stored in the memory 1102 by the processor 1101, so as to realize the steps in any one of the target tracking methods provided by the embodiments of the present application.

[0167] To this end, the embodiments of the present application provide a computer readable storage medium, which can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. The computer readable storage medium has a target tracking program stored thereon, and the target tracking program realizes the steps in any one of the target tracking methods provided by the embodiments of the present application when executed by a processor.

[0168] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the detailed description of other embodiments above, which will not be repeated here.

[0169] In the implementation, the above units or structures can be implemented as independent entities, or combined as the same or several entities, and the specific implementation of the above units or structures can be referred to the method embodiments above, which will not be repeated here.

[0170] The specific implementation of the above operations can be referred to the above embodiments, which will not be repeated here.

[0171] The above describes in detail the target tracking method provided by the embodiments of the present application, and the principle and implementation manner of the present application are described by using specific examples; the above embodiment descriptions are only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, the specific implementation manner and application range can be changed according to the idea of the present application, and the above description should not be understood as the limitation of the present application.

Claims

1. A target tracking method characterized by, The method comprises the following steps: acquiring a target video to be detected; performing target detection on video frames in the target video to obtain detection boxes corresponding to target objects in the video frames and identification information; extracting detection boxes and identification information of consecutive video frames in the target video respectively to obtain first target detection boxes and corresponding first identification information and second target detection boxes and corresponding second identification information; adjusting the first identification information and the second identification information according to the similarity between the first target detection boxes and the second target detection boxes to obtain target identification information corresponding to the target objects; the target identification information corresponding to the target objects is accurate identification information corresponding to the target objects; the step of adjusting the first identification information and the second identification information according to the similarity between the first target detection boxes and the second target detection boxes to obtain target identification information corresponding to the target objects comprises the following steps: acquiring the similarity between the first target detection boxes and the second target detection boxes and the comparison result between the first identification information and the second identification information; performing conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result; if the conflict detection result is a conflict, performing the step of adjusting the first identification information and the second identification information to obtain target identification information corresponding to the target objects.

2. The object tracking method of claim 1, wherein, the step of performing conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result comprises the following steps: if the similarity is greater than a preset first similarity threshold value and the comparison result is different identification information, setting a conflict as the conflict detection result of the similarity and the comparison result.

3. The object tracking method of claim 1, wherein, the step of performing conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result comprises the following steps: if the similarity is less than a preset second similarity threshold value and the comparison result is the same identification information, setting a conflict as the conflict detection result of the similarity and the comparison result.

4. The object tracking method of claim 1, wherein, the step of extracting detection boxes and identification information of consecutive video frames in the target video respectively to obtain first target detection boxes and corresponding first identification information and second target detection boxes and corresponding second identification information comprises the following steps: determining a target video frame from the target video; extracting detection boxes and identification information in the target video frame to obtain first target detection boxes and corresponding first identification information; extracting detection boxes and identification information in adjacent video frames of the target video frame to obtain second target detection boxes and corresponding second identification information.

5. The target tracking method according to claim 4, characterized in that, the step of determining a target video frame from the target video comprises the following steps: setting a video frame with a preset number of detection boxes in the target video as a target video frame; and / or setting a video frame with a detection box spacing less than a preset distance threshold value in the target video as a target video frame.

6. The object tracking method according to any one of claims 1 to 5, characterized in that, The step of adjusting the first identification information and the second identification information comprises: obtaining a reference detection frame corresponding to the first target detection frame and the second target detection frame, and reference identification information corresponding to the reference detection frame; if a similarity between the first target detection frame and the reference detection frame is greater than a preset third similarity threshold, adjusting the first identification information corresponding to the first target detection frame to the reference identification information; and / or if a similarity between the second target detection frame and the reference detection frame is greater than a preset third similarity threshold, adjusting the second identification information corresponding to the second target detection frame to the reference identification information.

7. A target tracking device, characterized by comprise: an acquisition module configured to acquire a target video to be detected; a detection module configured to perform target detection on video frames in the target video to obtain detection frames corresponding to target objects in the video frames and identification information; an extraction module configured to extract detection frames and identification information of consecutive video frames in the target video respectively to obtain a first target detection frame and first identification information corresponding to the first target detection frame, and a second target detection frame and second identification information corresponding to the second target detection frame; an adjustment module configured to adjust the first identification information and the second identification information according to a similarity between the first target detection frame and the second target detection frame to obtain target identification information corresponding to the target objects; the target identification information corresponding to the target objects is accurate identification information corresponding to the target objects; the step of adjusting the first identification information and the second identification information according to the similarity between the first target detection frame and the second target detection frame to obtain target identification information corresponding to the target objects comprises: obtaining a similarity between the first target detection frame and the second target detection frame, and a comparison result between the first identification information and the second identification information; performing conflict detection on the similarity and the comparison result according to a preset conflict detection rule to obtain a conflict detection result of the similarity and the comparison result; if the conflict detection result is a conflict, adjusting the first identification information and the second identification information to obtain target identification information corresponding to the target objects.

8. A target tracking device, characterized by, The target tracking device comprises a processor, a memory, and a target tracking program stored in the memory and executable on the processor, and the processor executes the target tracking program to implement the steps in the target tracking method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a target tracking program, and the target tracking program is executed by a processor to implement the steps in the target tracking method of any one of claims 1 to 6.

Images