Target tracking method, electronic device and storage medium

Abstract

The present application discloses a target tracking method, electronic device, and storage medium, which can improve the reliability of tracking the object to be tracked when the object to be tracked appears discontinuously in a video frame. The target tracking method includes: obtaining a target video frame; obtaining target features of the object to be tracked and a target position of the object to be tracked based on the target video frame; performing similarity calculation on the target features and N tracked object features included in each of M pre-stored cache queues to obtain the similarity between the object to be tracked and each tracked object, where the tracked object features are features of the tracked object, each cache queue in the M cache queues corresponds one-to-one to the identity of the tracked object, and N is a positive integer not less than 1; if the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, binding the target position of the object to be tracked to the identity of the target tracked object.

Description

Technical field

[0001] The embodiments of the present application relate to the field of video surveillance technology, and in particular to a target tracking method, electronic device, and storage medium. [Background Technology]

[0002] With the development of computer vision technology, visual target tracking has been widely used in the field of video surveillance. The purpose is to determine the position of the target to be tracked in a continuous video sequence, so as to obtain the complete motion trajectory of the target.

[0003] Existing technologies often use the correlation between adjacent video frames to track an object. However, in crowded places, the target can be easily obscured. For example, if a target appears in the first video frame, disappears in the second, and reappears in the third, existing target tracking methods cannot exploit the correlation between the second and third frames to track the reappearing target. This means that reliable tracking is impossible when the same target appears repeatedly in non-consecutive video frames. [Summary of the invention]

[0004] Embodiments of the present application provide a target tracking method, an electronic device, and a storage medium, which can improve the reliability of tracking an object to be tracked when the object to be tracked appears discontinuously in a video frame.

[0005] In a first aspect, an embodiment of the present application discloses a method for target tracking, the method comprising:

[0006] Get the target video frame;

[0007] Obtaining target features of the object to be tracked and the target position of the object to be tracked based on the target video frame;

[0008] Calculate similarity between the target feature and N tracked object features included in each of the pre-stored M cache queues to obtain similarity between the target object and each tracked object, where the tracked object feature is the feature of the tracked object, and each cache queue in the M cache queues corresponds to an identity identifier of the tracked object, where N is a positive integer not less than 1;

[0009] If the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, the target position of the object to be tracked is bound to the identity identifier of the target tracked object.

[0010] In the embodiment of the present application, the cache queue can be considered as a cache queue formed by the tracked object features corresponding to the tracked objects. If there is an object to be tracked in the target video frame, the target features and target position of the object to be tracked can be obtained, and then the target features are used to determine whether the object to be tracked is a tracked object in history. If it is a tracked object in history, the identity of the tracked object in history is bound to the target position of the current object to be tracked in the target video frame, that is, the object to be tracked is tracked in the target video frame. Compared with the prior art, even if the object to be tracked in the target video frame does not appear in the previous video frame, as long as it appears in any video frame in history, reliable target tracking can be achieved.

[0011] Optionally, performing similarity calculation between the target feature and N tracked object features included in each of the pre-stored M cache queues to obtain the similarity between the object to be tracked and each tracked object includes:

[0012] Calculate the target feature and the nth tracked object feature T in the i-th cache queue in the M cache queues (i,n) The similarity S (i,n) , i is a positive integer not exceeding M, n is a positive integer not exceeding N;

[0013] Determine the similarity S (i,n) The corresponding weight value Q (i,n) , T (i,n) From the j-th video frame X j , n is positively correlated with j, Q (i,n) Positively correlated with n, j is a positive integer not less than n, and the weight values ​​Q in the same cache queue are (i,n) The sum is 1;

[0014] According to the similarity S (i,n) And the corresponding weight value Q (i,n) , determine the similarity between the object to be tracked and the i-th tracked object.

[0015] In the embodiment of the present application, since target tracking is a continuous process, during this process, the features of the same object to be tracked extracted from video frames corresponding to different frame sequences may undergo some slight changes. The closer the frame sequences between different video frames, the closer the features of the same object to be tracked extracted therefrom can be considered, that is, the higher the similarity between the features of the same object to be tracked. Therefore, when calculating and judging whether the object to be tracked and the object that has been tracked in history are the same object, the similarity between the target features of the object to be tracked and the tracked object features of the tracked object can be calculated respectively, and different weight values ​​can be set for the above similarity according to the degree of proximity between the frame sequence corresponding to the tracked object features and the frame sequence corresponding to the current target features, that is, the closer the frame sequence corresponding to the tracked object features and the target video frame sequence corresponding to the current target features are, the higher the weight corresponding to the similarity between the two; conversely, the farther the interval between the frame sequence corresponding to the tracked object features and the target video frame sequence corresponding to the current target features is, the smaller the weight corresponding to the similarity between the two. In other words, the similarity between the tracked object features extracted from the historical video frame closest to the target video frame sequence and the target features is used as the main basis for judging whether the current object to be tracked is the object that has been tracked in history, thereby improving the accuracy of the judgment.

[0016] Optionally, the similarity between the feature of the object to be tracked and the i-th tracked object is obtained based on the following calculation formula:

[0017] S i =Q (i,n) ×S (i,n)

[0018] S i Indicates the similarity between the object to be tracked and the i-th tracked object;

[0019] Q (i,n) Represents the similarity S between the target feature and the nth tracked object feature in the i-th cache queue (i,n) The corresponding weight value;

[0020] S (i,n) Represents the similarity between the target feature and the feature of the nth tracked object in the i-th cache queue.

[0021] In an embodiment of the present application, based on the obtained similarity between the features of the object to be tracked and the features of the tracked object, and the weight value corresponding to the similarity, the similarity between the features of the object to be tracked and the tracked object can be obtained through the above-mentioned weighted calculation formula, so as to serve as a basis for judging whether the subsequent object to be tracked is a tracked object in history.

[0022] Optionally, each of the M cache queues includes a face feature subqueue and a body feature subqueue; the face feature subqueue includes A face features of the tracked objects, and the body feature subqueue includes B body features of the tracked objects, where A and B are both positive integers not less than 1. The target feature is calculated and the feature T of the nth tracked object in the i-th cache queue in the M cache queues is calculated. (i,n) The similarity S (i,n) include;

[0023] When the target feature is a face feature, the similarity S between the target feature and the face feature of the nth tracked object included in the i-th face feature sub-queue is calculated. (i,n) ;

[0024] When the target feature is a human feature, the similarity S between the target feature and the nth tracked human feature included in the i-th human feature sub-queue is calculated. (i,n) .

[0025] In an embodiment of the present application, each cache queue is configured with a facial feature subqueue and a human feature subqueue. The facial subqueue is used to store facial features, and the human subqueue is used to store human features. Since the target features extracted from the video frame can be either facial features or human features, it is possible to first determine whether the target feature captured in the target video frame is a facial feature or a human feature. After determining the specific type of the target feature, the similarity between the target feature and each facial / human feature in the facial / human feature subqueue included in the target cache queue is calculated, providing a basis for subsequent target tracking. In other words, regardless of the type of the target feature currently extracted, it can serve as the basis for subsequent target tracking.

[0026] Optionally, if the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, after binding the target position of the object to be tracked with the identity identifier of the target tracked object, the method further includes:

[0027] The target feature is used as a tracked object feature of the target tracked object, and the target feature is added to a target cache queue corresponding to the target tracked object.

[0028] In an embodiment of the present application, when the similarity between the target features included in the object to be tracked and the target tracked object exceeds a preset similarity, the target features can be used as tracked object features of the target tracked object and stored in the target cache queue corresponding to the target tracked object, thereby providing a basis for continuous tracking of the target tracked object in subsequent video frames.

[0029] Optionally, after taking the target feature as a tracked object feature of the target tracked object and adding the target feature to a target cache queue corresponding to the target tracked object, the method further includes:

[0030] If the number of updated tracked object features in the target cache queue is greater than a set threshold, the first tracked object feature in the target cache queue is removed from the target cache queue.

[0031] In an embodiment of the present application, after the target feature is added to the target cache queue corresponding to the target tracked object, it is necessary to determine whether the number of updated tracked object features in the target cache queue is greater than a set threshold. If it is greater than the set threshold, the first tracked object feature in the target cache queue is removed from the target cache queue, thereby saving the storage space occupied by the cache queue while ensuring that the cache queue can accurately determine whether the object to be tracked is a historically tracked object.

[0032] Optionally, after taking the target feature as a tracked object feature of the target tracked object and adding the target feature to a target cache queue corresponding to the target tracked object, the method further includes:

[0033] If the difference between the first frame sequence of the target video frame and the second frame sequence of the video frame corresponding to the first appearance of the tracked object feature of the target tracked object exceeds a preset difference, and the number of updated tracked object features in the target cache queue is less than a set threshold, the target cache queue is deleted.

[0034] In an embodiment of the present application, if the difference between the frame sequence in which a tracked object feature first appears and the frame sequence of the target video frame is large, it means that the time interval between the target video frame and the video frame corresponding to the first appearance of the tracked object feature is long. At this time, if there are fewer tracked object features stored in the cache queue corresponding to the tracked object, it can be considered that the tracked object only entered the monitoring area accidentally and there is no value in target tracking it. The cache queue corresponding to the tracked object can be deleted, thereby saving storage space of the electronic device.

[0035] In a second aspect, an embodiment of the present invention provides an electronic device, comprising:

[0036] A first acquisition unit, configured to acquire a target video frame;

[0037] a second acquiring unit, configured to acquire a target feature of an object to be tracked and a target position of the object to be tracked based on the target video frame;

[0038] a calculation unit, configured to calculate a similarity between the target feature and N tracked object features included in each of the pre-stored M cache queues, to obtain a similarity between the object to be tracked and each tracked object, where the tracked object feature is a feature of the tracked object, each cache queue in the M cache queues corresponds to an identity identifier of the tracked object, and M and N are positive integers not less than 1;

[0039] The tracking unit is configured to bind the target position of the object to be tracked with the identity identifier of the target tracked object when the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold.

[0040] Optionally, similarity calculation is performed between the target feature and N tracked object features included in each of the pre-stored M cache queues to obtain similarity between the object to be tracked and each tracked object, and the calculation unit includes:

[0041] The similarity calculation subunit is used to calculate the similarity between the target feature and the nth tracked object feature T in the i-th cache queue in the M cache queues. (i,n) The similarity S (i,n) , i is a positive integer not exceeding M, n is a positive integer not exceeding N;

[0042] The weight determination subunit is used to determine the similarity S (i,n) The corresponding weight value Q (i,n) , the T (i,n) From the j-th video frame X j , n is positively correlated with j, Q (i,n) Positively correlated with n, j is a positive integer not less than n, and the weight values ​​Q in the same cache queue are (i,n) The sum is 1;

[0043] The weighted similarity calculation subunit is configured to determine the similarity between the object to be tracked and the i-th tracked object according to the similarity and the corresponding weight value.

[0044] Optionally, the similarity between the object to be tracked and the i-th tracked object is obtained based on the following calculation formula:

[0045] S i =Q (i,n) ×S (i,n)

[0046] The S i Represents the similarity between the object to be tracked and the i-th tracked object;

[0047] The Q (i,n) Represents the similarity S between the target feature and the nth tracked object feature in the i-th cache queue (i,n) The corresponding weight value;

[0048] The S (i,n) Indicates the similarity between the target feature and the nth tracked object feature in the i-th cache queue.

[0049] Optionally, each cache queue in the M cache queues includes a face feature subqueue and a body feature subqueue; the face feature subqueue includes A face features of the tracked object, and the body feature subqueue includes B body features of the tracked object, where A and B are both positive integers not less than 1. The target feature and the nth tracked object feature T in the i-th cache queue in the M cache queues are calculated. (i,n) The similarity S (i,n) , the similarity calculation subunit is specifically used for:

[0050] When the target feature is a face feature, the similarity S between the target feature and the face feature of the nth tracked object included in the i-th face feature sub-queue is calculated. (i,n) ;

[0051] When the target feature is a human feature, the similarity S between the target feature and the nth human feature of the tracked object included in the i-th human feature sub-queue is calculated. (i,n) .

[0052] Optionally, if the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, after binding the target position of the object to be tracked with the identity identifier of the target tracked object, the tracking unit is further configured to:

[0053] The target feature is used as a tracked object feature of the target tracked object, and the target feature is added to a target cache queue corresponding to the target tracked object.

[0054] Optionally, after taking the target feature as the tracked object feature of the target tracked object and adding the target feature to the target cache queue corresponding to the target tracked object, the tracking unit is further configured to:

[0055] If the number of updated tracked object features in the target cache queue is greater than a set threshold, the first tracked object feature in the target cache queue is removed from the target cache queue.

[0056] Optionally, after taking the target feature as the tracked object feature of the target tracked object and adding the target feature to the target cache queue corresponding to the target tracked object, the tracking unit is further configured to:

[0057] If the difference between the first frame sequence of the target video frame and the second frame sequence of the video frame corresponding to the first appearance of the tracked object feature of the target tracked object exceeds a preset difference, and the number of updated tracked object features in the target cache queue is less than a set threshold, the target cache queue is deleted.

[0058] In a third aspect, an embodiment of the present invention provides an electronic device, comprising at least one processor and a memory connected to the at least one processor, wherein the at least one processor is configured to implement the steps of the method of any embodiment of the first aspect when executing a computer program stored in the memory.

[0059] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method of any embodiment of the first aspect when the computer program is executed by a processor.

[0060] It should be understood that the second to fourth aspects of the embodiments of the present invention are consistent with the technical solutions of the first aspect of the embodiments of the present invention, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.

Brief Description of the Drawings

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0062] Figure 1 A schematic diagram of a non-crowded scene provided by an embodiment of the present application;

[0063] Figure 2 A schematic diagram of a crowded scene provided in an embodiment of the present application;

[0064] Figure 3 A flowchart of a target tracking method provided in an embodiment of the present application;

[0065] Figure 4 A schematic diagram of calculating the similarity between an object to be tracked and an already tracked object provided in an embodiment of the present application;

[0066] Figure 5 A schematic diagram of the structure of a cache queue provided in an embodiment of the present application;

[0067] Figure 6 A schematic diagram of adding tracked object features to a target cache queue provided by an embodiment of the present application;

[0068] Figure 7 A schematic diagram of removing tracked object features from a target cache queue provided in an embodiment of the present application;

[0069] Figure 8 A schematic diagram of an electronic device provided in an embodiment of the present application;

[0070] Figure 9A schematic diagram of an electronic device provided in an embodiment of the present application. [Specific implementation method]

[0071] In order to better understand the technical solutions of this specification, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0072] It should be clear that the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this specification.

[0073] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a," "an," and "the" used in the examples of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0074] With the development of computer vision technology, visual object tracking has been widely used. For example, object tracking in video surveillance is achieved based on visual object tracking technology.

[0075] The inventors of this application have found that at present, the object features of the previous and next frames of the video frame are mainly extracted from adjacent video frames, and then the image overlap rate of the previous and next frames of the object features is obtained based on the IOU value calculation, and the previous and next frames are matched with the objects to be tracked according to the set overlap rate threshold, thereby realizing target tracking.

[0076] For example, see Figure 1 At this time, the object to be tracked is the object 2 to be tracked. Therefore, it is necessary to extract the object features of the object 2 to be tracked in the Nth video frame and the N+1th video frame. At this time, it is necessary to obtain the image overlap rate of the object 2 to be tracked in the two video frames through IOU value calculation, and then match the object 2 to be tracked in the Nth video frame and the N+1th video frame based on the preset overlap rate threshold, so as to achieve target tracking of the object 2 to be tracked.

[0077] See Figure 2 , is a schematic diagram of a monitoring image of video surveillance provided in an embodiment of the present application. Figure 2The changes in three video frames are described in the figure, in which the object 2 to be tracked is not obscured in the Nth video frame and the N+2th video frame, but is obscured in the N+1th video frame, that is, the object 2 to be tracked appears repeatedly in non-continuous video frames. Then, when the existing technology is applied to this type of scene, since the object 2 to be tracked in the N+1th video frame is obscured, the corresponding target features cannot be extracted in the N+1th video frame, resulting in the object 2 to be tracked in the N+2th video frame being unable to perform similarity calculation with the object 2 to be tracked in the N+1th video frame, and it is also impossible to track the object 2 to be tracked in the N+2th video frame.

[0078] In view of this, an embodiment of the present application provides a target tracking method, in which a cache queue can be considered as a cache queue formed by the tracked object features corresponding to the tracked object. If there is a certain object to be tracked in the target video frame, the target features and target position of the object to be tracked can be obtained, and then the target features are used to determine whether the object to be tracked is a tracked object in history. If it is a tracked object in history, the identity of the tracked object in history is bound to the target position of the current object to be tracked in the target video frame, that is, the object to be tracked is tracked in the target video frame. Compared with the prior art, even if the object to be tracked in the target video frame does not appear in the previous video frame, reliable target tracking can be achieved as long as it has appeared in any video frame in history.

[0079] The following is an introduction to the technical solutions provided by the embodiments of this application with reference to the accompanying drawings. Figure 2 The embodiment of the present application provides a target tracking method, which is applied to an electronic device, which may be a monitoring device. The process of the method is described as follows:

[0080] Step 101: Acquire a target video frame.

[0081] The monitoring device can be considered to collect image information of the monitoring area based on the first frequency to form video frames. If you want to know the movement trajectory of people in the above monitoring area, it is equivalent to obtaining the position of the object to be tracked in each video frame. Before that, it is necessary to obtain the target video frame formed in the above monitoring area.

[0082] For example, corresponding video frames can be extracted from the multiple video frames formed based on the second frequency to obtain the target video frame. It should be understood that the second frequency should be smaller than the first frequency. On this basis, the second frequency can be dynamically adjusted in combination with the flow of people in the monitored area and the time interval.

[0083] For example, when the flow of people in the monitoring area is large, the second frequency can be adjusted to a higher value; conversely, when the flow of people in the monitoring area is small, the second frequency can be adjusted to a lower value.

[0084] Step 102: Obtain target features of the object to be tracked and the target position of the object to be tracked based on the target video frame.

[0085] In an embodiment of the present application, after obtaining a target video frame, if one wants to track an object to be tracked in the target video frame, one needs to obtain relevant information of the object to be tracked in the target video frame so as to use the relevant information of the object to be tracked as a reference for tracking the object to be tracked.

[0086] As a possible implementation, the electronic device may obtain target features of the object to be tracked and the target position of the object to be tracked based on the target video frame.

[0087] For example, the electronic device can obtain the target position of the object to be tracked from the target video frame based on the target detection algorithm, and then obtain the target features of the object to be characterized from the target position based on the feature extraction algorithm. The target detection algorithms used here are: RCNN, SSD, YOLO, etc., and there is no particular limitation on the target detection algorithms used here. The feature extraction algorithms used here can be: LBP algorithm, HOG feature extraction algorithm, SIFT operator, etc., and there is no particular limitation on the feature extraction algorithms used in this application.

[0088] Step 103: Calculate the similarity between the target feature and the N tracked object features included in each of the M pre-stored cache queues to obtain the similarity between the object to be tracked and each tracked object. The tracked object feature is the feature of the tracked object. Each cache queue in the M cache queues corresponds to the identity of the tracked object one by one, and M and N are positive integers not less than 1.

[0089] Considering that in the prior art, the object to be tracked is tracked by correlating the features of the same object to be tracked between two adjacent video frames, if the object to be tracked does not appear in the previous video frame of the target video frame, for example, is blocked, it will result in the inability to track the object to be tracked in the current target video frame.

[0090] In an embodiment of the present application, the tracked objects appearing in video frames that are not adjacent to the current target video frame can be used as a basis for continuously tracking the current object to be tracked, thereby avoiding the adverse effects caused by tracking the same object when it appears non-continuously in multiple video frames.

[0091] As a possible implementation, the electronic device may be considered to have pre-stored M cache queues corresponding to M tracked objects, that is, one tracked object identity corresponds to one cache queue, M is a positive integer not less than 1; each cache queue stores N tracked object features, N is a positive integer not less than 1, and the N tracked object features here can be considered to be extracted from historical video frames, where the historical video frames can be continuous or discontinuous, and the number of tracked object features included in each cache queue can be the same or different, and there is no special restriction here.

[0092] On this basis, the electronic device can calculate the similarity between the target feature of the current object to be tracked and the features of the N tracked objects included in each cache queue in the above M cache queues, thereby obtaining the similarity between the above object to be tracked and each tracked object in history.

[0093] See Figure 4 , which is a schematic diagram of calculating the similarity between a to-be-tracked object and previously tracked objects. Suppose the electronic device stores cache queue 1 corresponding to tracked object 1, which contains tracked object feature AB of tracked object 1; cache queue 2 corresponding to tracked object 2, which contains tracked object feature AC of tracked object 2; and cache queue 3 corresponding to tracked object 3, which contains tracked object feature AD of tracked object 3.

[0094] When it is necessary to calculate the similarity between the object to be tracked and the tracked object 1, a first similarity between the target feature of the object to be tracked and the tracked object feature A of the tracked object 1, and a second similarity between the target feature and the tracked object feature B of the tracked object 1 can be calculated. The similarity between the object to be tracked and the tracked object 1 is then determined based on the first similarity and the second similarity.

[0095] When calculating the similarity between the object to be tracked and the tracked object 2, a first similarity between the target feature of the object to be tracked and the tracked object feature A of the tracked object 2, a second similarity between the target feature and the tracked object feature B of the tracked object 2, and a third similarity between the target feature and the tracked object C of the tracked object 2 can be calculated. The similarity between the object to be tracked and the tracked object 2 is then determined based on the first, second, and third similarities.

[0096] When it is necessary to calculate the similarity between the object to be tracked and the tracked object 3, a first similarity between the target feature of the object to be tracked and the tracked object feature A of the tracked object 3, a second similarity between the target feature and the tracked object feature B of the tracked object 3, a third similarity between the target feature and the tracked object C of the tracked object 3, and a fourth similarity between the target feature and the tracked object D of the tracked object 3 can be calculated respectively. Then, the similarity between the object to be tracked and the tracked object 3 is determined based on the first similarity, the second similarity, the third similarity, and the fourth similarity.

[0097] In some embodiments, considering that target tracking is a continuous process, in this process, the features of the same object to be tracked extracted from video frames corresponding to different frame sequences may undergo some slight changes, and the closer the frame sequences between different video frames are, the closer the features of the same object to be tracked extracted therefrom can be considered. Therefore, in an embodiment of the present application, when calculating the similarity between the object to be tracked and each tracked object in history, the similarity between the features of the tracked object extracted from the historical video frame that is closest to the frame sequence of the target video frame and the target features is used as the main basis for judging whether the current object to be tracked is an object that has been tracked in history, thereby improving the accuracy of the judgment.

[0098] As a possible implementation, the electronic device may calculate the target feature of the object to be tracked and the feature T of the nth tracked object in the i-th cache queue among the M cache queues. (i,n) The similarity S (i,n) , i is a positive integer not exceeding M, n is a positive integer not exceeding N. Here, the object feature T is tracked (i,n) It can be considered that the j-th frame video frame X j , and n is positively correlated with j, j is a positive integer not less than n, that is, the tracked object feature T (i,n) The order of arrangement in the corresponding cache queue is related to the frame sequence of the video frame from which it comes. The tracked object features T (i,n) The smaller the frame order of the video frame, the more the object features T have been tracked. (i,n) The higher the order in the corresponding cache queue, the higher the order; on the contrary, the higher the order of the tracked object features T (i,n) The larger the frame sequence of the video frame, the more the object features T have been tracked. (i,n) The later the order in the corresponding cache queue is.

[0099] Then, the electronic devices can respectively determine the similarity S (i,n) The corresponding weight value Q (i,n) , here, Q (i,n) It can be set to be positively correlated with n, and the weight values ​​Q of each cache queue in the same cache queue (i,n)The sum is 1.

[0100] Please continue to see Figure 4 Taking the cache queue 2 corresponding to the tracked object 2 as an example, the cache queue 2 includes the tracked object features AC of the tracked object 2. The first similarity between the target feature of the object to be tracked and the tracked object feature A of the tracked object 2, the second similarity between the target feature and the tracked object feature B of the tracked object 2, and the third similarity between the target feature and the tracked object C of the tracked object 2 are calculated respectively. On this basis, a first weight value can be configured for the first similarity, a second weight value can be configured for the second similarity, and a third weight value can be configured for the third similarity. The sum of the above-mentioned first weight value, second weight value and third weight value is 1, and increases sequentially, that is, the first weight value is less than the second weight value, and the second weight value is less than the third weight value.

[0101] For example, while ensuring that the sum of the first weight value, the second weight value and the third weight value is 1, the first weight value, the second weight value and the third weight value can increase in geometric proportion. For example, the first weight value is 1 / 7, the second weight value is 2 / 7, and the third weight value is 4 / 7. Of course, the first weight value, the second weight value and the third weight value can also increase in other ways, and there is no special restriction here.

[0102] Finally, the electronic device can be (i,n) And the corresponding weight value Q (i,n) , determine the similarity between the object to be tracked and the i-th tracked object.

[0103] The similarity between the object to be tracked and the i-th tracked object is obtained based on the following calculation formula (1):

[0104] S i =Q (i,n) ×S (i,n) (1)

[0105] S i Indicates the similarity between the object to be tracked and the i-th tracked object; Q (i,n) Represents the similarity S between the target feature and the nth tracked object feature in the i-th cache queue (i,n) The corresponding weight value; S (i,n) Represents the similarity between the target feature and the feature of the nth tracked object in the i-th cache queue.

[0106] In some embodiments, considering that the features of the same tracked object extracted from historical video frames may be facial features or body features, this depends on whether the facial features or body features of the tracked object in the historical video frames are more prominent. In other words, the cache queue corresponding to the tracked objects stored in the electronic device can be considered to include two subqueues, namely a facial feature subqueue and a body feature subqueue, each facial feature subqueue includes A facial features of the tracked object, and each body feature subqueue includes B body features of the tracked object, where A and B are both positive integers not less than 1.

[0107] In the embodiment of the present application, the electronic device calculates the target feature of the object to be tracked and the feature T of the nth tracked object in the i-th cache queue in the M cache queues. (i,n) The similarity S (i,n) When the target feature of the object to be tracked is extracted from the current target video frame, the corresponding sub-queue can be selected to provide a reliable basis for subsequent target tracking.

[0108] As a possible implementation, see Figure 5 Schematic diagram of any cache queue including a facial feature subqueue and a human feature subqueue. When the electronic device determines that the target feature is a facial feature, the similarity S between the target feature and the nth tracked object facial feature included in the i-th facial feature subqueue can be calculated. (i,n) When the electronic device determines that the target feature is a human feature, the similarity S between the target feature and the human feature of the nth tracked object included in the i-th human feature sub-queue can be calculated. (i,n) .

[0109] Step 104: If the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, the target position of the object to be tracked is bound to the identity identifier of the target tracked object.

[0110] In an embodiment of the present application, based on the similarity between the object to be tracked and the objects that have been tracked in history, it can be determined whether the object to be tracked has appeared in history. If it is determined that the above-mentioned object to be tracked has appeared in history, the identity information of the tracked object that has appeared in history is assigned to the position of the current object to be tracked, thereby realizing tracking of the object to be tracked in the current target video frame.

[0111] As a possible implementation method, if the electronic device determines that the similarity between the object to be tracked and the target tracked object among the historically tracked objects exceeds a set threshold, it indicates that the object to be tracked and the target tracked object are the same object. At this time, the identity of the above-mentioned target tracked object can be bound to the target position of the object to be tracked in the current target video frame.

[0112] In some embodiments, when it is determined that the object to be tracked is a target tracked object among the historically tracked objects, the relevant feature information of the object to be tracked can also be added to the cache queue corresponding to the above-mentioned target tracked object, thereby providing a reference basis for subsequent tracking of the same object to be tracked.

[0113] As a possible implementation method, after determining that the object to be tracked is the target tracked object among the historically tracked objects, the target features of the object to be tracked can be used as the tracked object features of the target tracked object, and the above target features can be added to the target cache queue corresponding to the target tracked object.

[0114] See Figure 6 , which is a schematic diagram of adding target features to the target cache queue. Taking the target tracked object as tracked object 1 as an example, the cache queue 1 corresponding to tracked object 1 includes N tracked object features. At this time, the target features of the object to be tracked can be used as the tracked object features of tracked object 1, and the above target features are added to the cache queue 1, so the cache queue 1 includes N+1 tracked object features.

[0115] In some embodiments, if a tracked object stays in the monitoring area corresponding to the monitoring device for a long time, the monitoring device will continuously collect the tracked object features of the tracked object, and continuously store the collected tracked object features in the cache queue corresponding to the tracked object. On the one hand, the number of tracked object features stored in the cache queue of the tracked object is large, which occupies too much storage space of the monitoring device. On the other hand, when it is necessary to calculate the similarity between the target feature and the tracked object feature of the tracked object, the calculation amount of the similarity will increase, affecting the efficiency of target tracking. Therefore, it is necessary to control the number of features in the cache queue within a certain range of numbers to ensure that the storage space of the monitoring device is not occupied too much while accurate similarity calculation can be performed.

[0116] As a possible implementation method, after the target feature of the object to be tracked is added to the target cache queue corresponding to the target tracked object, if the number of updated tracked object features in the target cache queue is greater than the set threshold, at this time, since the first tracked object feature in the target cache queue and the target feature of the current object to be tracked are similarly calculated, the impact on determining whether the object to be tracked and the historically tracked object are the same tracked object is minimal. Therefore, the first tracked object feature in the target cache queue can be removed from the cache queue, thereby streamlining the tracked object features stored in the target cache queue and improving the efficiency of target tracking.

[0117] See Figure 7 , taking the target tracked object as tracked object 1 and the target cache queue as cache queue 1 as an example, the set threshold for the number of tracked object features in cache queue 1 is N. After the target feature is added to cache queue 1 as a tracked object feature, there are N+1 tracked object features in cache queue 1, which exceeds the set threshold. Then, the first tracked object feature in cache queue 1 (that is, tracked object 1) needs to be removed from cache queue 1. After removal, the cache queue includes tracked object feature 2 to tracked object feature N+1, a total of N tracked object features.

[0118] When a monitoring device is monitoring, there may be situations where a mobile target (such as a pedestrian or an animal) accidentally passes through the monitoring area. Because the time it stays in the monitoring area is too short, it is not necessary to track it. In this case, these mobile targets can be considered invalid tracking targets. By deleting the cache queue corresponding to these invalid tracking targets, the monitoring device can save storage space.

[0119] As a possible implementation method, if the difference between the first frame sequence of the target video frame and the second frame sequence of the video frame corresponding to the first appearance of the tracked object feature of the target tracked object in the target cache queue exceeds a preset difference, then it can be considered that the time interval between the moment when the target feature appears and the moment when the tracked object feature first appears is long. At this time, if the number of updated tracked object features in the target cache queue is less than the set threshold, it can be said that the target tracked object only appears briefly in the monitoring area by chance. In this case, it can be considered that the target tracked object has no value in tracking, and the target cache queue can be deleted.

[0120] See Figure 8 Based on the same inventive concept, an embodiment of the present application provides an electronic device, which includes: a first acquisition unit 201, a second acquisition unit 202, a calculation unit 203, and a tracking unit 204.

[0121] A first acquisition unit 201 is configured to acquire a target video frame;

[0122] A second acquiring unit 202 is configured to acquire target features of the object to be tracked and a target position of the object to be tracked based on the target video frame;

[0123] A calculation unit 203 is configured to calculate similarity between the target feature and N previously stored tracked object features in each of the M pre-stored cache queues to obtain similarities between the target feature and each tracked object, where the tracked object feature is a feature of the tracked object, and each cache queue in the M cache queues corresponds to an identity identifier of the tracked object, where M and N are positive integers not less than 1.

[0124] The tracking unit 204 is configured to bind the target position of the object to be tracked with the identity identifier of the target tracked object when the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold.

[0125] Optionally, similarity calculation is performed between the target feature and N tracked object features included in each of the pre-stored M cache queues to obtain similarity between the object to be tracked and each tracked object. The calculation unit 203 includes:

[0126] Similarity calculation subunit, used to calculate the target feature and the nth tracked object feature T in the i-th cache queue in the M cache queues (i,n) similarity, i is a positive integer not exceeding M, n is a positive integer not exceeding N;

[0127] Weight determination subunit, used to determine the similarity S (i,n) The corresponding weight value Q (i,n) , Q (i,n) From the j-th video frame X j , n is positively correlated with j, Q (i,n) Positively correlated with n, j is a positive integer not less than n, and the weight values ​​Q in the same cache queue are (i,n) The sum is 1;

[0128] The weighted similarity calculation subunit is used to calculate the weighted similarity according to the similarity S (i,n) And the corresponding weight value Q (i,n) , determine the similarity between the object to be tracked and the i-th tracked object.

[0129] Optionally, the similarity between the object to be tracked and the i-th tracked object is obtained based on the following calculation formula:

[0130] S i =Q (i,n) ×S (i,n)

[0131] S i Indicates the similarity between the object to be tracked and the i-th tracked object;

[0132] Q (i,n) Represents the target feature and the nth tracked object feature T in the i-th cache queue (i,n) The similarity S (i,n) The corresponding weight value;

[0133] S (i,n) Represents the target feature and the nth tracked object feature T in the i-th cache queue (i,n) The corresponding similarity.

[0134] Optionally, each of the M cache queues includes a face feature subqueue and a body feature subqueue; the face feature subqueue includes A face features of the tracked objects, and the body feature subqueue includes B body features of the tracked objects, where A and B are both positive integers not less than 1. The target feature is calculated and the feature T of the nth tracked object in the i-th cache queue in the M cache queues is calculated. (i,n) The similarity calculation subunit is specifically used for:

[0135] When the target feature is a face feature, the similarity S between the target feature and the face feature of the nth tracked object included in the i-th face feature sub-queue is calculated. (i,n) ;

[0136] When the target feature is a human feature, the similarity S between the target feature and the human feature of the nth tracked object included in the i-th human feature sub-queue is calculated. (i,n) .

[0137] Optionally, if the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, after binding the target position of the object to be tracked with the identity identifier of the target tracked object, the tracking unit 204 is further configured to:

[0138] The target feature is used as a tracked object feature of the target tracked object, and the target feature is added to a target cache queue corresponding to the target tracked object.

[0139] Optionally, after taking the target feature as a tracked object feature of the target tracked object and adding the target feature to the target cache queue corresponding to the target tracked object, the tracking unit 204 is further configured to:

[0140] If the number of updated tracked object features in the target cache queue is greater than a set threshold, the first tracked object feature in the target cache queue is removed from the target cache queue.

[0141] Optionally, after taking the target feature as a tracked object feature of the target tracked object and adding the target feature to the target cache queue corresponding to the target tracked object, the tracking unit 204 is further configured to:

[0142] If the difference between the first frame sequence of the target video frame and the second frame sequence of the video frame corresponding to the first appearance of the tracked object feature of the target tracked object exceeds a preset difference, and the number of updated tracked object features in the target cache queue is less than a set threshold, the target cache queue is deleted.

[0143] See Figure 9Based on the same inventive concept, an embodiment of the present application provides an electronic device, which includes at least one processor 301, and the processor 301 is used to execute a computer program stored in a memory to implement the embodiment of the present application. Figure 3 The steps of the target tracking method are shown.

[0144] Optionally, the processor 301 may specifically be a central processing unit, a specific ASIC, or one or more integrated circuits for controlling program execution.

[0145] Optionally, the electronic device may further include a memory 302 connected to the at least one processor 301. The memory 302 may include ROM, RAM, and disk storage. The memory 302 is used to store data required by the processor 301 when it is running, that is, it stores instructions that can be executed by the at least one processor 301. The at least one processor 301 executes the instructions stored in the memory 302 to execute the following operations: Figure 3 The method shown. Wherein, the number of memory 302 is one or more. Wherein, the memory 302 is Figure 9 It is shown together with the figure, but it should be noted that the memory 302 is not a required functional module, so Figure 9 Shown in dashed lines.

[0146] The physical devices corresponding to the first acquisition unit 201, the second acquisition unit 202, the calculation unit 203, and the tracking unit 204 can all be the aforementioned processor 301. The electronic device can be used to execute Figure 3 Therefore, for the functions that can be realized by each functional module in the electronic device, please refer to Figure 3 The corresponding description in the illustrated embodiment will not be repeated here.

[0147] The embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores computer instructions, which, when executed on a computer, cause the computer to execute the following Figure 3 method.

[0148] The above are only preferred embodiments of this specification and are not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A target tracking method, characterized in that: The method comprises: Get the target video frame; Obtaining target features of an object to be tracked and a target position of the object to be tracked based on the target video frame; Calculating similarity between the target feature and N tracked object features included in each of the pre-stored M cache queues to obtain similarity between the object to be tracked and each tracked object, where the tracked object feature is a feature of the tracked object, and each cache queue in the M cache queues corresponds to an identity identifier of the tracked object, where M and N are positive integers not less than 1; If the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, the target position of the object to be tracked is bound to the identity identifier of the target tracked object.

2. The method according to claim 1, characterized in that Calculating similarity between the target feature and N tracked object features included in each of the pre-stored M cache queues to obtain similarity between the object to be tracked and each tracked object includes: Calculate the target feature and the nth tracked object feature T in the i-th cache queue in the M cache queues (i,n) The similarity S (i,n) , i is a positive integer not exceeding M, n is a positive integer not exceeding N; Determine the similarity S (i,n) The corresponding weight value Q (i,n) , the T (i,n) From the j-th video frame X j , n is positively correlated with j, Q (i,n) Positively correlated with n, j is a positive integer not less than n, and the weight values ​​Q in the same cache queue are (i,n) The sum is 1; According to the similarity S (i,n) And the corresponding weight value Q (i,n) , determine the similarity between the object to be tracked and the i-th tracked object.

3. The method according to claim 2, characterized in that The similarity between the object to be tracked and the i-th tracked object is calculated based on the following formula: S i =Q (i,n) ×S (i,n) The S i Represents the similarity between the object to be tracked and the i-th tracked object; The Q (i,n) Represents the similarity S between the target feature and the nth tracked object feature in the i-th cache queue (i,n) The corresponding weight value; The S (i,n) Indicates the similarity between the target feature and the nth tracked object feature in the i-th cache queue.

4. The method according to claim 2, characterized in that Each of the M cache queues includes a face feature subqueue and a body feature subqueue; the face feature subqueue includes A face features of the tracked object, and the body feature subqueue includes B body features of the tracked object, where A and B are both positive integers not less than 1. The target feature and the nth tracked object feature T in the i-th cache queue of the M cache queues are calculated. (i,n) The similarity S (i,n) include: When the target feature is a face feature, the similarity S between the target feature and the face feature of the nth tracked object included in the i-th face feature sub-queue is calculated. (i,n) ; When the target feature is a human feature, the similarity S between the target feature and the nth human feature of the tracked object included in the i-th human feature sub-queue is calculated. (i,n) .

5. The method according to claim 1, wherein If the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold, after binding the target position of the object to be tracked with the identity identifier of the target tracked object, the method further includes: The target feature is used as a tracked object feature of the target tracked object, and the target feature is added to a target cache queue corresponding to the target tracked object.

6. The method according to claim 5, characterized in that After using the target feature as a tracked object feature of the target tracked object and adding the target feature to a target cache queue corresponding to the target tracked object, the method further includes: If the number of updated tracked object features in the target cache queue is greater than a set threshold, the first tracked object feature in the target cache queue is removed from the target cache queue.

7. The method according to claim 5, characterized in that After using the target feature as a tracked object feature of the target tracked object and adding the target feature to a target cache queue corresponding to the target tracked object, the method further includes: If the difference between the first frame sequence of the target video frame and the second frame sequence of the video frame corresponding to the first appearance of the tracked object feature of the target tracked object exceeds a preset difference, and the number of updated tracked object features in the target cache queue is less than a set threshold, the target cache queue is deleted.

8. An electronic device, characterized in that: The electronic device comprises: A first acquisition unit, configured to acquire a target video frame; a second acquiring unit, configured to acquire a target feature of an object to be tracked and a target position of the object to be tracked based on the target video frame; a calculation unit, configured to calculate a similarity between the target feature and N tracked object features included in each of the pre-stored M cache queues, to obtain a similarity between the object to be tracked and each tracked object, where the tracked object feature is a feature of the tracked object, each cache queue in the M cache queues corresponds to an identity identifier of the tracked object, and M and N are positive integers not less than 1; The tracking unit is configured to bind the target position of the object to be tracked with the identity identifier of the target tracked object when the similarity between the object to be tracked and the target tracked object exceeds a preset similarity threshold.

9. An electronic device, characterized in that: The electronic device includes at least one processor and a memory connected to the at least one processor, and the at least one processor is configured to implement the steps of the method according to any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

Images