Trajectory tracking method, device and computer storage medium

Abstract

The present application provides a trajectory tracking method, device and computer storage medium, including associating the base library feature of each target object with at least one trajectory feature to construct the identity association information of each target object; identifying each detection feature obtained from different target videos according to at least one trajectory feature of each target object, and determining the target object corresponding to each detection feature; according to the identity association information of each target object and the target object of each detection feature, using any one of the trajectory code and the base library code of the object to be queried, all detection features of the object to be queried are obtained, and the trajectory tracking result of the object to be queried is generated accordingly. Accordingly, the present application can support the trajectory tracking of multiple surveillance videos and can effectively improve the accuracy and completeness of the trajectory tracking result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image recognition technology, and in particular to a trajectory tracking method, device and computer storage medium. Background Art

[0002] In recent years, monitoring equipment has become widely used in our lives. In many scenes such as road traffic and shopping malls, multiple monitoring devices are often installed at different locations to cover the entire area.

[0003] Current trajectory tracking technologies are mostly implemented based on a single surveillance video. However, in actual applications, in order to find and locate a specific target, it is usually necessary to view multiple surveillance videos to determine the target. Therefore, a lot of time and manpower costs are required to integrate the tracking results of multiple surveillance videos.

[0004] In view of this, a trajectory tracking technology that can integrate multiple surveillance videos is needed to overcome the problems existing in the prior art. Summary of the invention

[0005] In view of the above problems, the present application provides a trajectory tracking method, device and computer storage medium, which can support trajectory tracking of multiple surveillance videos and improve the accuracy of trajectory tracking results.

[0006] In a first aspect, the present application provides a trajectory tracking method, comprising: associating a base library feature of each target object with at least one trajectory feature to construct identity association information of each target object; identifying each detection feature obtained from different target videos according to at least one trajectory feature of each target object, and determining the target object corresponding to each detection feature; according to the identity association information of each target object and the target object corresponding to each detection feature, using the trajectory feature of the object to be queried and any one of the base library features, obtaining all detection features of the object to be queried, and generating a trajectory tracking result of the object to be queried.

[0007] The second aspect of the present application provides a trajectory tracking device, including: an association module, used to associate the base library features of each target object with at least one trajectory feature to construct identity association information of each target object; an identification module, used to identify each detection feature obtained from different target videos according to at least one trajectory feature of each target object, and determine the target object corresponding to each detection feature; a tracking module, used to obtain all detection features of the object to be queried according to the identity association information of each target object and the target object corresponding to each detection feature, using the trajectory feature of the object to be queried and any one of the base library features, so as to generate a trajectory tracking result of the object to be queried.

[0008] A third aspect of the present application provides a computer storage medium, wherein the computer storage medium stores instructions for executing the steps in the method described in the first aspect.

[0009] In summary, the trajectory tracking method provided by each embodiment of the present application constructs identity association information for identifying the association relationship between the base library feature of the target object and at least one trajectory feature, and determines the target object corresponding to each detection feature obtained from different target videos according to the different trajectory features of the target object, and can use any one of the trajectory code and base library code of the object to be queried to obtain all the detection features of the object to be queried, thereby generating the trajectory tracking result of the object to be queried. Accordingly, the present application obtains the trajectory tracking results of the object to be queried corresponding to different target videos, thereby improving the accuracy and completeness of the trajectory tracking results. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0011] Figure 1 It is a flowchart of a trajectory tracking method according to an exemplary embodiment of the present application.

[0012] Figure 2 This is a schematic diagram of an application example of the trajectory tracking method of the present application.

[0013] Figure 3 It is a flowchart of a trajectory tracking method according to another exemplary embodiment of the present application.

[0014] Figure 4 It is a flowchart of a trajectory tracking method according to another exemplary embodiment of the present application.

[0015] Figure 5 It is a flowchart of a trajectory tracking method according to another exemplary embodiment of the present application.

[0016] Figure 6 It is a flowchart of a trajectory tracking method according to another exemplary embodiment of the present application.

[0017] Figure 7 It is a flowchart of a trajectory tracking method according to another exemplary embodiment of the present application.

[0018] Figure 8 It is a structural block diagram of a trajectory tracking device of an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0020] Most of the current trajectory tracking technologies are based on a single surveillance video. In practical applications, in order to find and locate a specific target, it is usually necessary to view multiple surveillance videos to determine. In view of this, the present application provides a trajectory tracking method that can integrate multiple surveillance videos to generate trajectory tracking results for the object to be queried.

[0021] The following will describe in detail various embodiments of the present application in conjunction with the accompanying drawings.

[0022] Figure 1 The following is a flow chart of a trajectory tracking method according to an exemplary embodiment of the present application. As shown in the figure, the trajectory tracking method according to the present embodiment mainly includes the following steps:

[0023] Step S102: Associating the base database feature of each target object with at least one trajectory feature to construct identity association information of each target object.

[0024] In this embodiment, each base stock feature has a corresponding base stock code, and each track feature has a corresponding track code.

[0025] In this embodiment, a target object may have a base database feature (base database code), and a base database feature (base database code) may be associated with one or more track features (track codes), that is, the identity association information of the target object may be a one-to-one corresponding association relationship or a one-to-many corresponding association relationship.

[0026] Specifically, when the same target object appears in multiple target areas, the target object may have multiple track features (track codes).

[0027] For example, in Figure 2In the example shown, the trajectory feature a and the trajectory feature d in the trajectory pool both match the base database feature a of the target object a. Therefore, the trajectory code a of the trajectory feature a and the trajectory code d of the trajectory feature d can be associated with the base database code a of the base database feature a to form the identity association information of the target object a; the trajectory feature b and the trajectory feature c in the trajectory pool both match the base database feature b of the target object b. Therefore, the trajectory code b of the trajectory feature b and the trajectory code c of the trajectory feature c can be correspondingly associated with the base database code b of the base database feature b to form the identity association information of the target object b; the trajectory feature e in the trajectory pool matches the base database feature c of the target object c. Therefore, the trajectory code e of the trajectory feature e can be correspondingly associated with the base database code c of the base database feature c to form the identity association information of the target object c.

[0028] Step S104, identifying each detection feature obtained from different target videos according to at least one trajectory feature of each target object, and determining the target object corresponding to each detection feature.

[0029] Optionally, each monitoring device set in each target area can be used to obtain each target video corresponding to each target area.

[0030] For example, in Figure 2 In the example shown, monitoring device A set in target area A can be used to obtain target video A corresponding to target area A, and monitoring device B set in target area B can be used to obtain target video B corresponding to target area B, and so on.

[0031] Optionally, an unknown object in a target video may be identified, detection features of the unknown object may be obtained, and then each track feature in a track pool may be used to identify the unknown object in the detection feature to determine a track code corresponding to the detection feature.

[0032] Step S106, according to the identity association information of each target object and the target object corresponding to each detection feature, all detection features of the object to be queried are obtained by using any one of the trajectory features and the base database features of the object to be queried, so as to generate the trajectory tracking result of the object to be queried.

[0033] In this embodiment, based on the identity association information of each target object constructed, by inputting any one of the trajectory features (or trajectory code) and base database features (or base database code) of the object to be queried, all the detection features of the object to be queried can be obtained, and based on each detection feature obtained, the various target areas that the object to be queried has visited can be inferred, thereby obtaining the complete movement trajectory of the object to be queried.

[0034] In summary, the trajectory tracking method provided in this embodiment associates the underlying database features of the target object with at least one trajectory feature, and identifies the identity of the unknown object in different target videos based on each trajectory feature to determine each target object corresponding to each target video, and by inputting the trajectory feature of the object to be queried and any one of the underlying database features, the movement trajectory of all target areas that have been visited by the query can be queried. Therefore, the trajectory tracking solution of the present application can support the trajectory tracking of multiple surveillance videos to improve the integrity and accuracy of the trajectory tracking results.

[0035] Figure 3 The processing flow chart of the trajectory tracking method of another exemplary embodiment of the present application is shown. This embodiment is a specific implementation of the above step S104. As shown in the figure, this embodiment mainly includes the following steps:

[0036] Step S302: determine a target video as the current video.

[0037] For example, you can Figure 2 The target video (eg, any one of target video A to target video D) input by a monitoring device (eg, any one of monitoring device A to monitoring device D) shown is determined as the current video.

[0038] Step S304: acquiring one video frame in the current video in sequence according to a preset frame interval as the current video frame.

[0039] Optionally, one frame of video image may be captured sequentially from the current video stream according to a preset frame interval (eg, 5 seconds / frame) to serve as the current video frame.

[0040] Step S306, detecting each unknown object in the current video frame, and obtaining detection features of each unknown object.

[0041] In this embodiment, a current video frame may include one or more unknown objects.

[0042] In this embodiment, for example, an SSD detector (monocular multi-target detector) can be used to detect each unknown object in the current video frame, and an object detection frame of each unknown object can be obtained. The trained ReID (person re-identification) model can be used to extract image information within each object detection frame, and a feature vector with a fixed dimension can be output as a feature representation of the unknown object, thereby obtaining the detection features of each unknown object.

[0043] For example, refer to Figure 2, detection can be performed based on the current video frame obtained from the target video A to obtain the detection feature a of the unknown object a and the detection feature b of the unknown object b; detection can be performed based on the current video frame obtained from the target video B to obtain the detection feature c of the unknown object c, and so on.

[0044] Step S308 , matching each detection feature of the unknown object with each trajectory feature to determine the trajectory code of each detection feature.

[0045] Optionally, the detection feature of an unknown object may be acquired in sequence as the current detection feature, the current detection feature may be matched with each trajectory feature in the trajectory pool, and the trajectory code of the current detection feature may be determined based on the trajectory code of the trajectory feature matched with the current detection feature.

[0046] For example, if the current detection feature is Figure 2 The detection feature d shown is obtained by matching the detection feature d with each trajectory feature in the trajectory pool, and based on the trajectory code d of the trajectory feature d, the trajectory code d of the trajectory feature d (that is, the unknown object d) is determined.

[0047] Step S310, determining whether all video frames of the current video have been detected, if so, proceeding to step S312, if not, returning to step S304 to continue execution.

[0048] Step S312, determine whether all target videos have been detected, if so, end this process, otherwise return to step S302 to continue execution.

[0049] In summary, the trajectory tracking method of this embodiment matches the detection features of the unknown object with each trajectory feature to identify the identity information of the unknown object, and has the advantages of accurate recognition results and high recognition processing efficiency.

[0050] Figure 4 The processing flow of the trajectory tracking method of another exemplary embodiment of the present application is shown. This embodiment is a specific implementation of the above step S308. As shown in the figure, this embodiment mainly includes the following steps:

[0051] Step S402 , based on the first matching threshold and the second matching threshold, two matches are performed for each detection feature of the unknown object and each trajectory feature, respectively, to obtain a first matching result and a second matching result for each detection feature.

[0052] Optionally, the two matches performed based on the detection feature of each unknown object and each trajectory feature may be Hungarian matches.

[0053] Optionally, the similarity between each detection feature and each trajectory feature can be calculated to obtain the similarity value between each detection feature and each trajectory feature, and a similarity cost matrix can be obtained based on the similarity value between each detection feature and each trajectory feature. Then, based on the similarity cost matrix and a first matching threshold, a first Hungarian match is performed to obtain a first matching result, and based on the similarity cost matrix and a second matching threshold, a second Hungarian match is performed to obtain a second matching result.

[0054] For example, assuming that the number of trajectory features is M and the number of detection features is N, an M×N similarity cost matrix can be constructed according to the similarity value between each detection feature and each trajectory feature.

[0055] Optionally, the cosine similarity between each detection feature and each trajectory feature can be calculated to obtain the cosine similarity value between each detection feature and each trajectory feature, and based on the cosine similarity value between each detection feature and each trajectory feature, the similarity cost value between each detection feature and each trajectory feature can be obtained to construct each matrix element in the similarity cost matrix.

[0056] In this embodiment, the similarity cost between each detection feature and each trajectory feature can be expressed as:

[0057]

[0058] in, Represents the cosine similarity value between a detection feature and a trajectory feature.

[0059] It should be noted that other similarity algorithms, such as the Euclidean distance algorithm, may also be used to obtain the similarity value between each detection feature and each trajectory feature.

[0060] In this embodiment, the second matching threshold is higher than the first matching threshold.

[0061] Optionally, the first matching threshold may be between 0.2 and 0.45. Preferably, the first matching threshold may be set to 0.4.

[0062] Optionally, the second matching threshold may be between 0.6 and 0.9. Preferably, the second matching threshold may be set to 0.65.

[0063] Step S404, obtaining a first matching result and a second matching result of a detection feature, and continuing to execute one of steps S4061, S4071, and S4081.

[0064] Step S4061: if the same trajectory feature matching the detection feature is obtained according to the first matching result and the second matching result of the detection feature, the trajectory code of the obtained trajectory feature is determined as the trajectory code of the detection feature.

[0065] Specifically, if the first Hungarian matching result and the second Hungarian matching result obtain the same trajectory feature that matches the detection feature, it means that the unknown object in this detection feature has appeared in the previous target video, and the trajectory code of the matched trajectory feature is determined as the trajectory code of this detection feature.

[0066] For example, for detection feature a, if the first matching result and the second matching result of detection feature a are both matched with the trajectory feature a in the trajectory pool, then the trajectory code a of the trajectory feature a is determined as the trajectory code of the detection feature a.

[0067] In this embodiment, different detection features obtained in different target areas of the same target object may match the same trajectory feature. Figure 2 In the example shown, the detection feature b obtained by the target object b in the target area A and the detection feature c obtained in the target area B both match the trajectory feature b. This may be due to the small differences in the clothing features or shooting angles of the target object b when it appears in the target area A and the target area B.

[0068] In this embodiment, different detection features obtained in different areas of the same target object may also match different trajectory features. Figure 2 In the example shown, the detection feature e obtained for the target object c in the target area C matches the trajectory feature d, while the detection feature f obtained in the target area D matches the trajectory feature c. This may be due to the large differences in factors such as the clothing features or shooting angles of the target object c when it appears in the target areas C and D.

[0069] Step S4062, performing feature fusion on the detection features and the trajectory features, and updating the trajectory features based on the feature fusion results.

[0070] In this embodiment, when a trajectory feature matching the detection feature is found in the trajectory pool, the detection feature can be fused with the trajectory feature to update the trajectory feature in the trajectory pool, so that the updated trajectory feature can more accurately identify the unknown object in the detection feature.

[0071] For example, when the target object in the target video makes a large movement, such as turning around, the front detection feature, side detection feature, and back detection feature of the target object may be acquired in sequence. In this case, if the same trajectory feature is used, only the target object in the front detection feature can be identified, but the target object in the back detection feature cannot be identified. Based on this, this embodiment dynamically merges and updates the matched trajectory features with the detection features, which is conducive to continuous identification of different detection features of the target object with large movements, thereby improving the tracking and identification effect of the target object.

[0072] Step S4071: if a trajectory feature matching the detection feature is not obtained according to the first matching result and the second matching result of the detection feature, a new trajectory feature and a trajectory code of the new trajectory feature are generated according to the detection feature.

[0073] In this embodiment, if the first matching result and the second matching result of the detection feature do not find a trajectory feature matching the detection feature in the trajectory pool, it means that the object is a newly appeared object. Then, the detection feature can be added to the trajectory pool to generate a new trajectory feature, and a corresponding trajectory code is assigned to the newly generated trajectory feature.

[0074] For example, in Figure 2 In the example shown, if a trajectory feature matching the detection feature d is not found in the trajectory pool, the detection feature d is added to the trajectory pool to generate a trajectory feature d and a trajectory code d in the trajectory pool.

[0075] Step S4072: determine the trajectory code of the new trajectory feature as the trajectory code of the detection feature.

[0076] Step S4081: If the first matching result does not obtain a trajectory feature that matches the detection feature, and the second matching result obtains a trajectory feature that matches the detection feature, the detection feature is discarded.

[0077] In summary, in the prior art, the single threshold method is used for identification, which easily causes the same pedestrian to be assigned multiple labels. In view of this, the present embodiment adopts a dual threshold method, performing two matches for each detection feature and each trajectory feature, which can effectively filter out ambiguous detection features, greatly reducing the possibility of trajectory interruption and concatenation.

[0078] Furthermore, by performing feature fusion on the matched detection features and trajectory features to update the trajectory features, the target object performing large-scale movements can be continuously identified, thereby improving the trajectory tracking effect of the target object.

[0079] Figure 5FIG. 2 is a flowchart of a trajectory tracking method according to another embodiment of the present invention. This embodiment is a specific implementation plan of the above step S102.

[0080] Step S502 : Match the base database feature of each target object with each trajectory feature to determine the base database feature that matches each trajectory feature.

[0081] Optionally, based on the third matching threshold and a fourth matching threshold higher than the third matching threshold, Hungarian matching may be performed twice on each trajectory feature and each base database feature to obtain a third matching result and a fourth matching result of each trajectory feature.

[0082] In this embodiment, for each trajectory feature, if the same base database feature matching the trajectory feature is obtained based on the third matching result and the fourth matching result of the trajectory feature, the obtained base database feature is matched with the trajectory feature.

[0083] Optionally, the third matching threshold may be between 0.2 and 0.45. Preferably, the third matching threshold may be set to 0.4.

[0084] Optionally, the fourth matching threshold may be between 0.6 and 0.9. Preferably, the fourth matching threshold may be set to 0.65.

[0085] In this embodiment, the two Hungarian matches performed on the base database features of each target object and each trajectory feature are similar to the above Figure 4 The Hungarian matching is performed twice for each unknown object detection feature and each trajectory feature as shown, which is basically the same, so it will not be repeated here.

[0086] It should be noted that in this embodiment, if multiple trajectory features are newly generated for a video frame, a cost matrix needs to be established for each newly generated trajectory feature and each base library feature, and a preset algorithm (such as a greedy algorithm) is used to match each newly generated trajectory feature, first matching the two features with the highest similarity, then matching the two features with the second highest similarity, and so on, until the remaining feature similarity is lower than the preset threshold. Among them, each time a match is made, the base library feature (base library code) that is successfully matched in the base library will no longer participate in subsequent matching operations, so as to avoid multiple pedestrians appearing in the same video frame of the same target video being matched by the same base library feature, thereby ensuring the accuracy of the base library data.

[0087] Step S504: for each trajectory feature, the base stock code of the base stock feature that matches the trajectory feature is associated with the trajectory code of the trajectory feature.

[0088] In this embodiment, the base database code of the base database feature that matches the trajectory feature found in the base database may be associated with the trajectory code of the trajectory feature.

[0089] In this embodiment, for each trajectory feature, if a base database feature matching the trajectory feature is not obtained based on the third matching result and the fourth matching result of the trajectory feature, a new base database feature and a base database code of the new base database feature are generated in the base database based on the trajectory feature, and the base database code of the newly generated base database feature is correspondingly associated with the trajectory code of the trajectory feature.

[0090] For example, in Figure 2 In the example shown, if a base database feature matching the trajectory feature e is not found in the base database, the trajectory feature e is added to the base database to generate a base database feature c and its corresponding base database code c.

[0091] Step S506: construct identity association information of the target object corresponding to the base stock code according to at least one track code associated with the base stock code of the same base stock feature.

[0092] For example, in Figure 2 In the example shown, the track code a and the track code d associated with the base library code a can be integrated to form the identity association information of the target object a corresponding to the base library code a.

[0093] In summary, the present application uses the Hungarian algorithm to perform matching on trajectory features and base database features, which can improve the accuracy of feature matching results.

[0094] Figure 6 FIG. 2 shows a processing flow of a trajectory tracking method according to another embodiment of the present application. This embodiment is a specific implementation of the above step S106. As shown in the figure, this embodiment mainly includes the following steps:

[0095] Step S602, obtaining all track codes of the object to be queried according to the base code of the object to be queried and the identity association information of the object to be queried.

[0096] For example, when the object to be queried is target object a, all track codes of target object a, namely track code a and track code d, can be obtained by inputting the base inventory code a of target object a and according to the identity association information of base inventory code a and target object a.

[0097] Step S604: according to all the trajectory codes of the object to be queried, all detection features corresponding to each trajectory code are obtained.

[0098] For example, according to all the trajectory codes of the target object a, namely, trajectory code a and trajectory code d, all the detection features corresponding to each trajectory code, namely, detection feature a and detection feature d, can be obtained.

[0099] Step S606: Generate a trajectory tracking result of the object to be queried according to all detection features corresponding to each trajectory code.

[0100] For example, the trajectory tracking result of the target object a can be generated according to the detection feature a and the detection feature d. According to the detection feature a and the detection feature d in the trajectory tracking result, it can be known that the target object a has appeared in the target area A and the target area C.

[0101] Figure 7 The processing flow of the trajectory tracking method of another embodiment of the present application is shown. This embodiment is another specific implementation of the above step S106. As shown in the figure, this embodiment mainly includes the following steps:

[0102] Step S702: According to any track code of the object to be queried and the identity association information of each target object, a base stock code associated with the track code is obtained.

[0103] For example, by inputting a track code b, and based on the input track code b and the identity association information of each target object, the base stock code b associated with the track code b can be queried, and based on the base stock code b, it can be determined that the object to be queried is the target object b.

[0104] Step S704, obtaining all track codes of the object to be queried according to the base code of the object to be queried and the identity association information of the object to be queried.

[0105] For example, according to the queried base code b and the identity association information of the target object b corresponding to the base code b, all the track codes of the target object b, namely, the track code b and the track code c, can be obtained.

[0106] Step S706: according to all the trajectory codes of the object to be queried, all detection features corresponding to each trajectory code are obtained.

[0107] For example, based on all trajectory codes of target object b, namely trajectory code b and trajectory code c, all detection features corresponding to each trajectory code can be obtained, namely detection feature b corresponding to trajectory code b, detection feature c and detection feature f corresponding to trajectory code c.

[0108] Step S708: Generate a trajectory tracking result of the object to be queried according to all detection features corresponding to each trajectory code.

[0109] For example, the trajectory tracking result of the target object b can be generated according to the detection feature b, the detection feature c and the detection feature f. According to the detection feature b, the detection feature c and the detection feature f in the trajectory tracking result, it can be known that the target object b has appeared in the target area A, the target area B and the target area D.

[0110] In summary, the present application can directly query the trajectory code of the object to be queried in all target videos by directly inputting the base code of the object to be queried based on the identity association information of each target object constructed, so as to obtain the complete trajectory tracking result of the object to be queried. Alternatively, it is also possible to query the base code of the object to be queried by inputting any trajectory code of the object to be queried, and then associate the trajectory code of the object to be queried that appears in other target videos, so as to obtain the complete trajectory tracking result of the object to be queried. Thereby, the present application can not only improve the flexibility of trajectory tracking query of the target object, but also obtain the trajectory tracking results of all target videos associated with the object to be queried, so as to improve the accuracy and completeness of the trajectory tracking results.

[0111] Figure 8 The structural block diagram of the trajectory tracking device of the exemplary embodiment of the present application is shown. As shown in the figure, the trajectory tracking device 800 of the present embodiment mainly includes: an association module 802, an identification module 804, and a tracking module 806.

[0112] The association module 802 is used to associate the base database feature of each target object with at least one trajectory feature to construct identity association information of each target object.

[0113] The recognition module 804 is used to recognize each detection feature obtained from different target videos according to at least one trajectory feature of each target object, and determine the target object corresponding to each detection feature.

[0114] The tracking module 806 is used to obtain all detection features of the object to be queried based on the identity association information of each target object and the target object corresponding to each detection feature, using any one of the trajectory features and the base library features of the object to be queried, so as to generate a trajectory tracking result of the object to be queried.

[0115] Optionally, the identification module 804 is also used to: determine a target video as the current video; obtain a video frame in the current video in sequence according to a preset frame interval as the current video frame; detect each unknown object in the current video frame and obtain the detection feature of each unknown object; match the detection feature of each unknown object with each trajectory feature to determine the trajectory code of each detection feature.

[0116] Optionally, each monitoring device set in each target area can be used to obtain each target video corresponding to each target area.

[0117] Optionally, the identification module 804 is also used to: based on the first matching threshold and the second matching threshold, perform two matches for each detection feature of the unknown object and each trajectory feature, respectively, to obtain a first matching result and a second matching result for each detection feature; for each detection feature, if the same trajectory feature matching the detection feature is obtained based on the first matching result and the second matching result of the detection feature, the trajectory code of the trajectory feature is determined as the trajectory code of the detection feature.

[0118] Optionally, the recognition module 804 is further configured to: perform feature fusion on the detection feature and the trajectory feature, and update the trajectory feature based on a feature fusion result.

[0119] Optionally, the identification module 804 is also used to: if a trajectory feature matching the detection feature is not obtained according to the first matching result and the second matching result of the detection feature, generate a new trajectory feature and a trajectory code of the new trajectory feature according to the detection feature; and determine the trajectory code of the new trajectory feature as the trajectory code of the detection feature.

[0120] Optionally, the identification module 804 is further configured to: discard the detection feature if the first matching result does not obtain a trajectory feature matching the detection feature, and the second matching result obtains a trajectory feature matching the detection feature.

[0121] Optionally, the two matches are Hungarian matches respectively, and the second matching threshold is higher than the first matching threshold.

[0122] Optionally, the identification module 804 is further used to: calculate the similarity between each detection feature and each trajectory feature to obtain the similarity value between each detection feature and each trajectory feature; obtain a similarity cost matrix according to the similarity value between each detection feature and each trajectory feature; perform a first Hungarian match according to the similarity cost matrix and a first matching threshold to obtain a first matching result; perform a second Hungarian match according to the similarity cost matrix and a second matching threshold to obtain a second matching result.

[0123] Optionally, the association module 802 is also used to: match the base database feature of each target object with each trajectory feature to determine the base database feature that matches each trajectory feature; for each trajectory feature, associate the base database code of the base database feature that matches the trajectory feature with the trajectory code of the trajectory feature; and construct identity association information of the target object corresponding to the base database code based on at least one trajectory code associated with the base database code of the same base database feature.

[0124] Optionally, the association module 802 is further used to: based on a third matching threshold and a fourth matching threshold higher than the third matching threshold, perform Hungarian matching twice on each trajectory feature and each base database feature, respectively, to obtain a third matching result and a fourth matching result of each trajectory feature; for each trajectory feature, if the same base database feature matching the trajectory feature is obtained according to the third matching result and the fourth matching result of the trajectory feature, the base database feature is matched with the trajectory feature.

[0125] Optionally, the association module 802 is also used for: for each trajectory feature, if a base database feature matching the trajectory feature is not obtained based on the third matching result and the fourth matching result of the trajectory feature, a new base database feature and a base database code of the new base database feature are generated based on the trajectory feature, and the base database code of the new base database feature is correspondingly associated with the trajectory code of the trajectory feature.

[0126] Optionally, the first matching threshold or the third matching threshold may be between 0.2 and 0.45, preferably, the first matching threshold or the third matching threshold is 0.4; the second matching threshold or the fourth matching threshold may be between 0.6 and 0.9, preferably, the second matching threshold or the fourth matching threshold is 0.65.

[0127] Optionally, the tracking module 806 is also used to: obtain all track codes of the object to be queried based on the base library code of the object to be queried and the identity association information of the object to be queried; obtain all detection features corresponding to each track code based on all the track codes of the object to be queried; and generate a track tracking result of the object to be queried based on all the detection features corresponding to each track code.

[0128] Optionally, the tracking module 806 is also used to: obtain a base library code associated with the trajectory code according to any trajectory code of the object to be queried and the identity association information of each target object; obtain all trajectory codes of the object to be queried according to the base library code of the object to be queried and the identity association information of the object to be queried; obtain all detection features corresponding to each trajectory code according to all trajectory codes of the object to be queried; and generate a trajectory tracking result of the object to be queried according to all detection features corresponding to each trajectory code.

[0129] The exemplary embodiments of the present application further provide a computer storage medium, in which instructions for executing the steps in the methods described in the above embodiments are stored.

[0130] In summary, the trajectory tracking solutions provided in the embodiments of the present application can support trajectory tracking of multiple surveillance videos to effectively improve the accuracy and completeness of trajectory tracking results.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A trajectory tracking method, characterized in that: include: Associating the base database feature of each target object with at least one trajectory feature to construct identity association information of each target object; wherein the base database feature has a corresponding base database code, and the trajectory feature has a corresponding trajectory code; According to at least one trajectory feature of each target object, each detection feature obtained from different target videos is identified, and each detection feature is matched twice with each trajectory feature to determine the target object corresponding to each detection feature; According to the identity association information of each target object and the target object corresponding to each detection feature, all detection features of the object to be queried are obtained by using any one of the trajectory features and the base database features of the object to be queried, so as to generate the trajectory tracking result of the object to be queried; the object to be queried is at least one of the target objects; The step of associating the base database feature of each target object with at least one trajectory feature to construct identity association information of each target object includes: Match the base database feature of each target object with each trajectory feature twice to determine the base database feature that matches each trajectory feature; for each trajectory feature, associate the base database code of the base database feature that matches the trajectory feature with the trajectory code of the trajectory feature; and construct identity association information of the target object corresponding to the base database code based on at least one trajectory code associated with the base database code of the same base database feature.

2. The method according to claim 1, characterized in that: According to at least one trajectory feature of each target object, each detection feature obtained from different target videos is identified, and each detection feature is matched twice with each trajectory feature to determine the target object corresponding to each detection feature, including: Determine a target video as the current video; According to a preset frame interval, sequentially obtaining a video frame in the current video as the current video frame; Detecting each unknown object in the current video frame and obtaining a detection feature of each unknown object; Match each unknown object's detection feature with each trajectory feature twice to determine the trajectory code of each detection feature; The corresponding target object is determined according to the trajectory code of each detection feature.

3. The method according to claim 2, characterized in that The target videos corresponding to the target areas can be obtained by utilizing the monitoring devices arranged in the target areas.

4. The method according to claim 2, characterized in that: The step of matching each unknown object's detection feature with each trajectory feature twice to determine the trajectory code of each detection feature includes: Based on the first matching threshold and the second matching threshold, performing two matches for each detection feature of the unknown object and each trajectory feature, respectively, to obtain a first matching result and a second matching result for each detection feature; For each detection feature, if the same trajectory feature matching the detection feature is obtained according to the first matching result and the second matching result of the detection feature, the trajectory code of the trajectory feature is determined as the trajectory code of the detection feature.

5. The method according to claim 4, characterized in that The method further comprises: Feature fusion is performed on the detection feature and the trajectory feature, and the trajectory feature is updated based on the feature fusion result.

6. The method according to claim 4, characterized in that The method further comprises: If a trajectory feature matching the detection feature is not obtained according to the first matching result and the second matching result of the detection feature, a new trajectory feature and a trajectory code of the new trajectory feature are generated according to the detection feature; The trajectory code of the new trajectory feature is determined as the trajectory code of the detection feature.

7. The method according to claim 4, characterized in that The method further comprises: If the first matching result does not obtain a trajectory feature that matches the detection feature, and the second matching result obtains a trajectory feature that matches the detection feature, the detection feature is discarded.

8. The method according to claim 4, characterized in that The two matches are Hungarian matches respectively, and the second matching threshold is higher than the first matching threshold.

9. The method according to claim 8, characterized in that The detection feature of each unknown object and each trajectory feature are matched twice, including: Calculate the similarity between each detection feature and each trajectory feature to obtain the similarity value between each detection feature and each trajectory feature; According to the similarity value between each detection feature and each trajectory feature, a similarity cost matrix is ​​obtained; Perform a first Hungarian match according to the similarity cost matrix and the first matching threshold to obtain a first matching result; A second Hungarian matching is performed according to the similarity cost matrix and the second matching threshold to obtain a second matching result.

10. The method according to claim 4, characterized in that The step of matching the base database feature of each target object with each trajectory feature twice to determine the base database feature that matches each trajectory feature includes: Based on a third matching threshold and a fourth matching threshold higher than the third matching threshold, perform Hungarian matching twice on each trajectory feature and each base database feature, respectively, to obtain a third matching result and a fourth matching result for each trajectory feature; For each trajectory feature, if the same base database feature matching the trajectory feature is obtained according to the third matching result and the fourth matching result of the trajectory feature, the base database feature is matched with the trajectory feature.

11. The method according to claim 10, characterized in that The method further comprises: For each trajectory feature, if a base database feature matching the trajectory feature is not obtained according to the third matching result and the fourth matching result of the trajectory feature, a new base database feature and a base database code of the new base database feature are generated according to the trajectory feature, and the base database code of the new base database feature is correspondingly associated with the trajectory code of the trajectory feature.

12. The method according to claim 10, characterized in that The first matching threshold or the third matching threshold is greater than 0.2 and less than 0.45; The second matching threshold or the fourth matching threshold is greater than 0.6 and less than 0.

9.

13. The method according to claim 1, characterized in that According to the identity association information of each target object and the target object corresponding to each detection feature, all detection features of the object to be queried are obtained by using any one of the trajectory feature and the base database feature of the object to be queried, so as to generate the trajectory tracking result of the object to be queried, including: Obtain all track codes of the object to be queried according to the base code of the object to be queried and the identity association information of the target object corresponding to the object to be queried; According to all the trajectory codes of the object to be queried, all detection features corresponding to each trajectory code are obtained; According to all the detection features corresponding to each trajectory code, a trajectory tracking result of the object to be queried is generated.

14. The method according to claim 1, characterized in that According to the identity association information of each target object and the target object corresponding to each detection feature, all detection features of the object to be queried are obtained by using any one of the trajectory feature and the base database feature of the object to be queried, so as to generate the trajectory tracking result of the object to be queried, including: According to any track code of the object to be queried and the identity association information of each target object, a base library code associated with the track code is obtained; Obtain all track codes of the object to be queried according to the base code of the object to be queried and the identity association information of the target object corresponding to the object to be queried; According to all the trajectory codes of the object to be queried, all detection features corresponding to each trajectory code are obtained; According to all the detection features corresponding to each trajectory code, a trajectory tracking result of the object to be queried is generated.

15. A trajectory tracking device, characterized in that: include: An association module, used to associate the base database feature of each target object with at least one trajectory feature to construct identity association information of each target object; wherein the base database feature has a corresponding base database code, and the trajectory feature has a corresponding trajectory code; an identification module, for identifying each detection feature obtained from different target videos according to at least one trajectory feature of each target object, and matching each detection feature with each trajectory feature twice to determine the target object corresponding to each detection feature; A tracking module, used to obtain all detection features of the object to be queried based on the identity association information of each target object and the target object corresponding to each detection feature, and to generate a trajectory tracking result of the object to be queried based on any one of the trajectory features and the base database features of the object to be queried; the object to be queried is at least one of the target objects; The step of associating the base database feature of each target object with at least one trajectory feature to construct identity association information of each target object includes: Match the base database feature of each target object with each trajectory feature twice to determine the base database feature that matches each trajectory feature; for each trajectory feature, associate the base database code of the base database feature that matches the trajectory feature with the trajectory code of the trajectory feature; and construct identity association information of the target object corresponding to the base database code based on at least one trajectory code associated with the base database code of the same base database feature.

16. A computer storage medium, characterized in that: The computer storage medium stores instructions for executing the steps in the method according to any one of claims 1 to 14.

Images