A target tracking method, device, apparatus and storage medium

Abstract

Embodiments of the present application disclose a target tracking method, device and equipment, and a storage medium. The method proposes a tracking algorithm framework based on trajectory segment filling, and then uses the tracking algorithm framework to track each target to be tracked in a target scene. The method first constructs a seed trajectory segment set based on appearance features of trajectory segments in a candidate trajectory segment set. Since the seed trajectory segment set is constructed based on appearance features, it is not limited by time and space. Therefore, each trajectory segment belonging to the same seed trajectory segment set can span a long period of time and a wide spatial area. Then, based on time information and spatial information of other trajectory segments in the candidate trajectory segment set, the seed trajectory segment set is subjected to trajectory segment filling processing to further ensure time and space continuity of the trajectory segments in the seed trajectory segment set. Thus, long-time and large-area accurate tracking of the target to be tracked in the target scene is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a target tracking method and device, equipment and a storage medium. BACKGROUND

[0002] In recent years, target tracking technology has been one of the research hotspots in the field of computer vision, and has a wide application prospect in security monitoring and other aspects. The target tracking technology can locate the position of the target in each frame of video image, and generate a trajectory segment of the target motion based thereon.

[0003] Due to the limited shooting range of a single camera, multiple cameras are generally installed in large and medium-sized scenes (such as shopping malls, parking lots, airports, amusement parks, etc.) to ensure that the entire scene can be monitored comprehensively. When multiple targets are tracked in such a scene, the trajectory segments of each target under each camera in the scene need to be determined first, and then the activity trajectory of each target in the entire scene is determined based on the trajectory segments of each target.

[0004] However, for complex scenes with dense objects, the need to cross multiple cameras, and occlusion between target objects and between target objects and the environment, there is still a high technical difficulty in realizing long-time and large-area tracking of multiple targets. SUMMARY

[0005] The embodiments of the present application provide a target tracking method, device, equipment and storage medium, which can realize accurate tracking of multiple targets for a long time and a large area in various scenes.

[0006] Therefore, the first aspect of the present application provides a target tracking method, which comprises:

[0007] The seed trajectory segment set forming step: based on the appearance features corresponding to each trajectory segment in the candidate trajectory segment set and the appearance similarity constraint condition, identifying the trajectory segments belonging to the target to be tracked in the candidate trajectory segment set, and forming a seed trajectory segment set corresponding to the target to be tracked; the candidate trajectory segment set includes the trajectory segments of each target under each camera in the target scene;

[0008] The candidate trajectory segment set updating step: deleting the trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked in the candidate trajectory segment set, and obtaining an updated candidate trajectory segment set;

[0009] The trajectory segment filling step comprises: determining a time gap and a space gap corresponding to the seed trajectory segment set corresponding to the to-be-tracked target according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target; selecting a target trajectory segment corresponding to the time information satisfying a time constraint condition with the time gap and corresponding to the space information satisfying a space constraint condition with the space gap from the updated candidate trajectory segment set; adding the target trajectory segment to the seed trajectory segment set corresponding to the to-be-tracked target; and deleting the target trajectory segment from the updated candidate trajectory segment set.

[0010] The active trajectory splicing step comprises: splicing each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target to obtain an active trajectory of the to-be-tracked target in the target scene.

[0011] Optionally, the selecting a target trajectory segment corresponding to the time information satisfying a time constraint condition with the time gap and corresponding to the space information satisfying a space constraint condition with the space gap from the updated candidate trajectory segment set comprises:

[0012] selecting a target candidate trajectory segment corresponding to the time information satisfying the time constraint condition with the time gap and corresponding to the space information satisfying the space constraint condition with the space gap from the updated candidate trajectory segment set.

[0013] determining whether the similarity between the appearance feature corresponding to the target candidate trajectory segment and the appearance feature corresponding to the to-be-tracked target satisfies a similarity determination condition, and if so, determining the target candidate trajectory segment as the target trajectory segment.

[0014] Optionally, the determining a time gap and a space gap corresponding to the seed trajectory segment set corresponding to the to-be-tracked target according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target comprises:

[0015] splicing each trajectory segment in the seed trajectory segment set to obtain a seed trajectory corresponding to the to-be-tracked target according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set.

[0016] taking a time discontinuous hole in the seed trajectory as the time gap and taking a space discontinuous hole in the seed trajectory as the space gap.

[0017] Optionally, the seed trajectory segment set forming step comprises:

[0018] acquiring N reference appearance features corresponding to each of the to-be-tracked targets respectively; N is a positive integer;

[0019] For each of the to-be-tracked targets, determining the trajectory segments belonging to the to-be-tracked target from the candidate trajectory segment set according to the similarity between the appearance features corresponding to each of the trajectory segments in the candidate trajectory segment set and the reference appearance features corresponding to the to-be-tracked target, and composing a seed trajectory segment set corresponding to the to-be-tracked target.

[0020] Optionally, the step of composing the seed trajectory segment set comprises:

[0021] performing clustering processing on the trajectory segments in the candidate trajectory segment set based on the appearance features corresponding to the trajectory segments, determining M to-be-tracked targets in the target scene and the trajectory segments belonging to each of the to-be-tracked targets; M is a positive integer;

[0022] For each of the to-be-tracked targets, composing a seed trajectory segment set corresponding to the to-be-tracked target by using the trajectory segments belonging to the to-be-tracked target.

[0023] Optionally, the step of performing clustering processing on the trajectory segments in the candidate trajectory segment set based on the appearance features corresponding to the trajectory segments, determining the number M of to-be-tracked targets in the target scene and the trajectory segments belonging to each of the to-be-tracked targets, comprises:

[0024] performing clustering processing on the trajectory segments in the candidate trajectory segment set based on the strong identification features corresponding to the trajectory segments, determining the M to-be-tracked targets included in the target scene and the trajectory segments corresponding to the M to-be-tracked targets respectively, and composing a seed trajectory segment set corresponding to each of the M to-be-tracked targets by using the trajectory segments corresponding to the M to-be-tracked targets respectively; the strong identification features are features with unique identification;

[0025] deleting the trajectory segments belonging to the seed trajectory segment sets corresponding to the M to-be-tracked targets respectively from the candidate trajectory segment set, to obtain an updated candidate trajectory segment set;

[0026] For each of the to-be-tracked targets, determining a reference appearance feature corresponding to the to-be-tracked target according to the trajectory segments in the seed trajectory segment set corresponding to the to-be-tracked target, determining a target trajectory segment of the to-be-tracked target from the updated candidate trajectory segment set according to the similarity between the appearance features corresponding to each of the trajectory segments in the updated candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, and adding the target trajectory segment to the seed trajectory segment set corresponding to the to-be-tracked target.

[0027] Optionally, the seed trajectory segment set forming step, the candidate trajectory segment set updating step and the trajectory segment filling step are taken as a target processing link;

[0028] The target processing link is executed in multiple rounds in a loop; after each round of the target processing link is executed, the updated candidate trajectory segment set in which the trajectory segments belonging to the seed trajectory segment set of the target to be tracked have been deleted is taken as a new candidate trajectory segment set to replace the candidate trajectory segment set;

[0029] The appearance similarity constraint condition on which the seed trajectory segment set forming step in the target processing link in the next round is based is lower than the appearance similarity constraint condition on which the seed trajectory segment set forming step in the target processing link in the previous round is based; the time constraint condition and the space constraint condition on which the trajectory segment filling step in the target processing link in the next round is based are respectively lower than the time constraint condition and the space constraint condition on which the trajectory segment filling step in the target processing link in the previous round is based.

[0030] The second aspect of the present application provides a target tracking device, which comprises:

[0031] A seed trajectory segment set forming module is configured to identify trajectory segments belonging to a target to be tracked in a candidate trajectory segment set based on appearance features corresponding to each trajectory segment in the candidate trajectory segment set and an appearance similarity constraint condition, and form a seed trajectory segment set corresponding to the target to be tracked; the candidate trajectory segment set comprises trajectory segments of each target under each camera in a target scene;

[0032] A candidate trajectory segment set updating module is configured to delete trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked in the candidate trajectory segment set, and obtain an updated candidate trajectory segment set;

[0033] A trajectory segment filling module is configured to determine a time gap and a space gap corresponding to the seed trajectory segment set corresponding to the target to be tracked based on time information and space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked; select a target trajectory segment in the updated candidate trajectory segment set, which satisfies a time constraint condition in terms of corresponding time information and the time gap, and satisfies a space constraint condition in terms of corresponding space information and the space gap; add the target trajectory segment to the seed trajectory segment set corresponding to the target to be tracked; and delete the target trajectory segment in the updated candidate trajectory segment set;

[0034] The activity trajectory splicing module is configured to splice each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target to obtain an activity trajectory of the to-be-tracked target in the target scene.

[0035] The third aspect of the present application provides a trajectory filling method, the method comprising:

[0036] According to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target, the time gap and the space gap corresponding to the seed trajectory segment set are determined; the seed trajectory segment set includes trajectory segments belonging to the to-be-tracked target;

[0037] In the candidate trajectory segment set, a target trajectory segment is selected, which satisfies a time constraint condition with the time gap corresponding to the time information and satisfies a space constraint condition with the space gap corresponding to the space information; the candidate trajectory segment set includes trajectory segments that are not divided into the seed trajectory segment set;

[0038] The target trajectory segment is added to the seed trajectory segment set.

[0039] Optionally, the selecting, in the candidate trajectory segment set, the target trajectory segment which satisfies the time constraint condition with the time gap corresponding to the time information and satisfies the space constraint condition with the space gap corresponding to the space information, comprises:

[0040] In the candidate trajectory segment set, a target candidate trajectory segment is selected, which satisfies the time constraint condition with the time gap corresponding to the time information and satisfies the space constraint condition with the space gap corresponding to the space information;

[0041] It is judged whether the similarity between the appearance feature corresponding to the target candidate trajectory segment and the appearance feature corresponding to the to-be-tracked target satisfies a similarity determination condition, and if so, the target candidate trajectory segment is determined as the target trajectory segment.

[0042] Optionally, the determining, according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target, the time gap and the space gap corresponding to the seed trajectory segment set, comprises:

[0043] According to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set, each trajectory segment in the seed trajectory segment set is spliced to obtain a seed trajectory corresponding to the to-be-tracked target;

[0044] The time discontinuous hole in the seed trajectory is taken as the time gap, and the space discontinuous hole in the seed trajectory is taken as the space gap.

[0045] The fourth aspect of the present application provides a trajectory filling device, the device comprising:

[0046] a gap determination module configured to determine a time gap and a space gap corresponding to a seed trajectory segment set according to time information and space information corresponding to each trajectory segment in the seed trajectory segment set, wherein the seed trajectory segment set comprises trajectory segments belonging to the target to be tracked;

[0047] a segment selection module configured to select a target trajectory segment from a candidate trajectory segment set, wherein the target trajectory segment satisfies a time constraint condition with the time gap and satisfies a space constraint condition with the space gap, and the candidate trajectory segment set comprises trajectory segments not divided into the seed trajectory segment set;

[0048] a segment filling module configured to add the target trajectory segment to the seed trajectory segment set.

[0049] The fifth aspect of the present application provides a device, the device comprising: a processor and a memory;

[0050] the memory is configured to store a computer program;

[0051] the processor is configured to call the computer program to execute the target tracking method of the first aspect or the trajectory filling method of the third aspect.

[0052] The sixth aspect of the present application provides a computer readable storage medium for storing a computer program, wherein the computer program is used to execute the target tracking method of the first aspect or the trajectory filling method of the third aspect.

[0053] From the above technical solutions, the embodiments of the present application have the following advantages:

[0054] This application provides a target tracking method, which proposes a tracking algorithm framework based on trajectory fragment filling, and uses this tracking algorithm framework to track the target to be tracked in the target scene. Specifically, firstly, based on the appearance features and appearance similarity constraints of each trajectory segment in the candidate trajectory segment set, trajectory segments belonging to the target to be tracked are identified in the candidate trajectory segment set, and a seed trajectory segment set corresponding to the target to be tracked is formed. Here, the candidate trajectory segment set includes trajectory segments of each target under each camera in the target scene. Next, trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked are deleted from the candidate trajectory segment set, resulting in an updated candidate trajectory segment set. Then, based on the time and spatial information of each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked, the time gap and spatial gap corresponding to the seed trajectory segment set corresponding to the target to be tracked are determined. In the updated candidate trajectory segment set, the target trajectory segment whose corresponding time information and time gap satisfy the time constraint condition and whose corresponding spatial information and spatial gap satisfy the spatial constraint condition is selected, and the target trajectory segment is added to the seed trajectory segment set corresponding to the target to be tracked, and then deleted from the updated candidate trajectory segment set. Finally, the trajectory segments in the seed trajectory segment set corresponding to the target to be tracked are spliced ​​together to obtain the activity trajectory of the target to be tracked in the target scene. Since the seed trajectory fragment set is not limited by time and space when constructing a set of seed trajectory fragments based on appearance features, the trajectory fragments belonging to the same set of seed trajectory fragments can span long time periods and wide spatial regions. Furthermore, trajectory fragment filling can further ensure the spatiotemporal continuity of trajectory fragments in the seed trajectory fragment set. In this way, accurate tracking of the target to be tracked in the target scene over a long period of time and over a large area can be achieved. Attached Figure Description

[0055] Figure 1 A flowchart illustrating a target tracking method provided in an embodiment of this application;

[0056] Figure 2 A schematic flowchart illustrating a trajectory filling method provided in an embodiment of this application;

[0057] Figure 3 This is a schematic diagram of the structure of a target tracking device provided in an embodiment of this application;

[0058] Figure 4 This is a schematic diagram of the structure of a trajectory filling device provided in an embodiment of this application;

[0059] Figure 5 This is a schematic diagram of the server structure provided in an embodiment of this application;

[0060] Figure 6A structural schematic diagram of a terminal device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0061] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work fall within the scope of protection of the present application.

[0062] The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0063] In view of the problem that it is difficult to achieve long-time and large-area tracking of a target in the prior art, an embodiment of the present application provides a target tracking method.

[0064] In the target tracking method provided in the embodiments of the present application, first, based on the appearance feature and appearance characteristic similarity constraint conditions corresponding to each trajectory segment in the candidate trajectory segment set, trajectory segments belonging to the target to be tracked are identified in the candidate trajectory segment set, and a seed trajectory segment set corresponding to the target to be tracked is formed. Here, the candidate trajectory segment set includes trajectory segments of each target under each camera in the target scene. Then, the trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked are deleted from the candidate trajectory segment set, to obtain an updated candidate trajectory segment set. Further, based on the time information and spatial information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked, the time gap and the spatial gap corresponding to the seed trajectory segment set corresponding to the target to be tracked are determined. In the updated candidate trajectory segment set, a target trajectory segment that satisfies the time constraint condition in terms of the corresponding time information and the time gap, and satisfies the spatial constraint condition in terms of the corresponding spatial information and the spatial gap, is selected, and the target trajectory segment is added to the seed trajectory segment set corresponding to the target to be tracked, and the target trajectory segment in the updated candidate trajectory segment set is deleted. Finally, each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked is spliced, to obtain the activity trajectory of the target to be tracked in the target scene.

[0065] The above method proposes a tracking algorithm framework based on trajectory segment filling, and uses the tracking algorithm framework to track each target to be tracked in the target scene. Since the seed trajectory segment set is constructed based on the appearance feature, which is not limited in time and space, each trajectory segment belonging to the same seed trajectory segment set can span a long period of time and a wide spatial area. Furthermore, the trajectory segment filling can further ensure the time and space continuity of the trajectory segments in the seed trajectory segment set. In this way, long-time and large-area accurate tracking of the target to be tracked in the target scene is achieved.

[0066] It should be noted that the target tracking method provided in the embodiments of the present application can be applied to various devices with data processing capabilities, such as terminal devices and servers. The terminal device can include a computer, a tablet computer, a personal digital assistant (PDA), a smart phone, and the like. The server can be an application server or a Web server. When deployed, the server can be a standalone server or a cluster server.

[0067] The target tracking method provided in the present application will be described in detail below through embodiments.

[0068] Reference is made to Figure 1 , Figure 1A flowchart of a target tracking method provided by an embodiment of the present application is shown. In order to facilitate description, the following embodiments are introduced by taking a server as an execution subject. As shown in Figure 1 The target tracking method includes the following steps:

[0069] Step 101: Based on the appearance features and appearance similarity constraints of each trajectory segment in the trajectory segment set, identify trajectory segments belonging to a target to be tracked in the candidate trajectory segment set, and form a seed trajectory segment set corresponding to the target to be tracked; the candidate trajectory segment set includes trajectory segments of each target under each camera in a target scene.

[0070] When the server needs to perform target tracking (which can be single-target tracking or multi-target tracking) on a target scene, the server can obtain trajectory segments of each target under each camera deployed in the target scene, and form a candidate trajectory segment set by using the obtained trajectory segments.

[0071] In some embodiments, when the server needs to perform target tracking on a target scene within a time period, the server can obtain videos captured by each camera deployed in the target scene within the time period, and then identify the moving trajectories of each target in each obtained video to obtain trajectory segments of each target under each camera, and further form a candidate trajectory segment set by using the trajectory segments.

[0072] In some embodiments, when the server needs to perform target tracking on a target scene within a time period, the server can directly obtain trajectory segments of each target under each camera in the target scene within the time period from a related device. That is, the related device can preform multi-target identification on videos captured by each camera in the target scene to obtain trajectory segments of each target under each camera; when the server needs to perform a multi-target tracking task, it can directly obtain trajectory segments of each target under each camera in the target scene pre-determined by the related device, and form a candidate trajectory segment set by using the trajectory segments.

[0073] It should be understood that each trajectory segment is actually position information of the same tracking target within a time period, which can be obtained by any tracking technology, and the present application does not make any limitation on the way of determining the trajectory segment.

[0074] It should be noted that in different application scenarios, the above target is a different object, for example, in the application scenario of pedestrian recognition, the above target can be a person to be tracked, and for example, in the application scenario of vehicle recognition, the above target can be a vehicle to be tracked. The target to be tracked in the present application is not limited specifically.

[0075] After obtaining the candidate trajectory segment set, the server can identify trajectory segments belonging to the to-be-tracked target according to the appearance features of the trajectory segments in the candidate trajectory segment set, and form a seed trajectory segment set corresponding to the to-be-tracked target according to a preset appearance similarity constraint condition.

[0076] It should be understood that, in actual applications, there can be a single to-be-tracked target in the target scene, or there can be multiple to-be-tracked targets. When there is a single to-be-tracked target, the server only needs to select trajectory segments belonging to the to-be-tracked target from the candidate trajectory segment set to form a seed trajectory segment set corresponding to the to-be-tracked target. When there are multiple to-be-tracked targets, the server needs to select trajectory segments belonging to the multiple to-be-tracked targets from the candidate trajectory segment set, and for each to-be-tracked target, the trajectory segments belonging to the to-be-tracked target are used to form a seed trajectory segment set corresponding to the to-be-tracked target.

[0077] In a possible implementation, the server can first obtain reference appearance features corresponding to N (N is a positive integer) to-be-tracked targets, and then for each to-be-tracked target, the server can determine trajectory segments belonging to the to-be-tracked target from the candidate trajectory segment set according to the similarity between the appearance features of each trajectory segment in the candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, and form a seed trajectory segment set corresponding to the to-be-tracked target.

[0078] That is, when the number N of to-be-tracked targets and the reference appearance features of the to-be-tracked targets are known, the server can calculate the similarity between the appearance features of each trajectory segment in the candidate trajectory segment set and the reference appearance feature corresponding to each to-be-tracked target. If the similarity between the appearance features of a trajectory segment and the reference appearance feature corresponding to a to-be-tracked target satisfies the appearance similarity constraint condition, for example, the similarity exceeds a preset appearance similarity threshold, it can be determined that the trajectory segment is a trajectory segment of the to-be-tracked target, and the trajectory segment can be added to the seed trajectory segment set corresponding to the to-be-tracked target.

[0079] It should be noted that the reference appearance feature corresponding to the to-be-tracked target can generally completely reflect the appearance of the to-be-tracked target. For example, in a pedestrian tracking scene, the reference appearance feature corresponding to the to-be-tracked target can include any combination of the following features: facial features, head features, body features, clothing features, etc. The appearance feature corresponding to the trajectory segment includes any feature that can represent the appearance of the target included in the trajectory segment, and any combination of these features. The similarity can be determined by any calculation method that can measure the similarity between two appearance features.

[0080] Optionally, in order to make the seed trajectory segment set corresponding to the to-be-tracked target include as many trajectory segments as possible, after the server determines the seed trajectory segment set corresponding to the to-be-tracked target based on the reference appearance feature corresponding to the to-be-tracked target, the server can further determine a reference appearance feature corresponding to the to-be-tracked target according to the appearance features corresponding to each trajectory segment included in the seed trajectory segment set, where the reference appearance feature is more generalized than the reference appearance feature corresponding to the to-be-tracked target; then, the server can relax the appearance similarity constraint condition, further filter the trajectory segments belonging to the to-be-tracked target from the remaining trajectory segments in the candidate trajectory segment set based on the appearance feature corresponding to the to-be-tracked target, and add the filtered trajectory segments to the seed trajectory segment set corresponding to the to-be-tracked target.

[0081] It should be understood that in actual application, the server can perform the above operation multiple times in a loop to make the seed trajectory segment set include as many trajectory segments as possible.

[0082] In another possible implementation, the server can perform clustering processing on the trajectory segments in the candidate trajectory segment set based on the appearance features corresponding to the trajectory segments in the candidate trajectory segment set, to determine M (M is a positive integer) to-be-tracked targets existing in the target scene and the trajectory segments belonging to each to-be-tracked target; then, for each to-be-tracked target, the trajectory segments belonging to the to-be-tracked target are used to form a seed trajectory segment set corresponding to the to-be-tracked target.

[0083] Specifically, when performing the first clustering processing, the server can perform clustering processing on the trajectory segments in the candidate trajectory segment set according to the strong identification features corresponding to the trajectory segments in the candidate trajectory segment set; that is, the trajectory segments with strong identification features in the candidate trajectory segment set are clustered according to their respective strong identification features, so as to determine the number M of to-be-tracked targets existing in the target scene. Meanwhile, the trajectory segments corresponding to the M to-be-tracked targets are used to form M seed trajectory segment sets corresponding to the M to-be-tracked targets respectively; that is, the trajectory segments with strong identification features corresponding to the M to-be-tracked targets are used to form M seed trajectory segment sets corresponding to the M to-be-tracked targets respectively. Then, the trajectory segments belonging to the M seed trajectory segment sets are deleted from the candidate trajectory segment set to obtain an updated candidate trajectory segment set.

[0084] It should be noted that the strong identification feature refers to a feature with unique identification, such as a facial feature.

[0085] When the server performs the processing of expanding each seed trajectory segment set in a subsequent round, for each to-be-tracked target, the server can determine a reference appearance feature of the to-be-tracked target according to the appearance features corresponding to each trajectory segment in the seed trajectory segment set of the to-be-tracked target determined in the previous round. Then, in the updated candidate trajectory segment set obtained after the previous round of processing, the server selects a target trajectory segment whose corresponding appearance feature and the reference appearance feature of the to-be-tracked target have a similarity that exceeds a preset similarity threshold, and adds the target trajectory segment to the seed trajectory segment set of the to-be-tracked target, and deletes the target trajectory segment in the updated candidate trajectory segment set.

[0086] Taking the scenario of pedestrian tracking as an example, the server can perform clustering processing on the trajectory segments in the candidate trajectory segment set based on the face features and the pedestrian re-identification features. That is, the server can first perform clustering processing on the trajectory segments in the candidate trajectory segment set according to the face features corresponding to the trajectory segments, to obtain M seed trajectory segment sets. Then, the server deletes the trajectory segments in the candidate trajectory segment set that belong to the M seed trajectory segment sets, to obtain an updated candidate trajectory segment set. Then, the server divides the trajectory segments in the updated candidate trajectory segment set into the M seed trajectory segment sets according to the pedestrian re-identification features corresponding to the M seed trajectory segment sets and the pedestrian re-identification features corresponding to the trajectory segments in the updated candidate trajectory segment set.

[0087] Specifically, the server can determine the face features corresponding to the trajectory segments in the candidate trajectory segment set by using face detection technology and face recognition technology. For some trajectory segments corresponding to the back or side of a pedestrian, no face information is included in the trajectory segments, and thus the server can default that the trajectory segments have no corresponding face features. When performing clustering processing, the server can perform clustering processing on the trajectory segments in the candidate trajectory segment set that have face features, to obtain M seed trajectory segment sets corresponding to M to-be-tracked targets.

[0088] After the server completes the clustering processing to obtain the M seed trajectory segment sets based on the face features corresponding to the trajectory segments in the candidate trajectory segment set, the server can divide the trajectory segments in the updated candidate trajectory segment set into the M seed trajectory segment sets according to the pedestrian re-identification features corresponding to the trajectory segments in the updated candidate trajectory segment set. The updated candidate trajectory segment set here is a trajectory segment set composed of the remaining trajectory segments after the trajectory segments in the candidate trajectory segment set that belong to the M seed trajectory segment sets are deleted. The pedestrian re-identification feature here generally refers to the overall appearance feature of a target corresponding to a trajectory segment, which can be determined by the head, limbs, and torso of a human body.

[0089] Specifically, for each trajectory segment in the updated candidate trajectory segment set, the server can determine whether the trajectory segment belongs to any of the M seed trajectory segment sets according to the pedestrian re-identification feature corresponding to the trajectory segment and the pedestrian re-identification features corresponding to the M seed trajectory segment sets, for example, the server can determine whether the similarity between the pedestrian re-identification feature corresponding to the trajectory segment and the pedestrian re-identification features corresponding to the seed trajectory segment sets satisfies a preset similarity constraint condition, if yes, the trajectory segment is divided into the seed trajectory segment set, and the trajectory segment in the updated candidate trajectory segment set is deleted. Here, the pedestrian re-identification features corresponding to the seed trajectory segment set are determined according to the pedestrian re-identification features corresponding to each trajectory segment belonging to the seed trajectory segment set.

[0090] It should be noted that when the method provided by the embodiments of the present application is applied to other scenarios, the server can select the appearance features suitable for the scenario, and process the trajectory segments in the candidate trajectory segment set accordingly, for example, when the method provided by the embodiments of the present application is applied to the scenario of vehicle tracking, the server can process the trajectory segments in the candidate trajectory segment set based on the appearance features, license plate number features, etc. of the vehicle.

[0091] Step 102: deleting the trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked in the candidate trajectory segment set, to obtain an updated candidate trajectory segment set.

[0092] As introduced in step 101, after the trajectory segments in the candidate trajectory segment set are divided into the seed trajectory segment set corresponding to the target to be tracked, the server can delete the trajectory segments from the candidate trajectory segment set accordingly, so that after step 101 is completed, the trajectory segments in the candidate trajectory segment set that have been divided into the seed trajectory segment set corresponding to the target to be tracked will be deleted, at this time, the trajectory segments in the candidate trajectory segment set that have not been deleted will constitute the updated candidate trajectory segment set.

[0093] Of course, in actual application, the server can also delete the trajectory segments in the candidate trajectory segment set that have been divided into the seed trajectory segment set corresponding to the target to be tracked after completing step 101, to obtain the updated candidate trajectory segment set, and the present application does not limit the way to obtain the updated candidate trajectory segment set.

[0094] Step 103: determining time gap and space gap corresponding to the seed trajectory segment set corresponding to the target to be tracked according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked; selecting a target trajectory segment in the updated candidate trajectory segment set, which satisfies the time constraint condition with the time gap corresponding thereto and satisfies the space constraint condition with the space gap corresponding thereto, and adding the target trajectory segment to the seed trajectory segment set corresponding to the target to be tracked and deleting the target trajectory segment in the updated candidate trajectory segment set.

[0095] After the server obtains the seed trajectory segment set corresponding to the target to be tracked, the server can further perform trajectory filling processing on the seed trajectory segment set corresponding to the target to be tracked, i.e., filling the trajectory segments in the updated candidate trajectory segment set to the seed trajectory segment set corresponding to the target to be tracked.

[0096] The server can determine the time gap and the space gap corresponding to the seed trajectory segment set corresponding to each target to be tracked according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set. Further, the server can select a target trajectory segment in the updated candidate trajectory segment set, which satisfies the time constraint condition with the time gap corresponding thereto and satisfies the space constraint condition with the space gap corresponding thereto, and add the target trajectory segment to the seed trajectory segment set corresponding to the target to be tracked and delete the target trajectory segment in the updated candidate trajectory segment set.

[0097] In a specific implementation, the server can first arrange and splice each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set, to obtain a seed trajectory corresponding to the seed trajectory segment set, i.e., a seed trajectory of the to-be-tracked target. Since the seed trajectory segment set is constructed based on the appearance features without considering the time-space constraints, the trajectory segments included in the seed trajectory segment set can span a long time period in time and cover a wide spatial area in space, and there can be discontinuity in time and space between the trajectory segments. If such a situation exists, there will be discontinuous holes in time and space in the seed trajectory obtained by arranging and splicing each trajectory segment in the seed trajectory segment set, i.e., there is a time difference between the trajectory segments that exceeds a preset time threshold (i.e., a time hole), and there is a position distance difference between the trajectory segments that exceeds a preset space threshold (i.e., a space hole). Here, the time difference between the trajectory segments refers to the difference between the end time of a first trajectory segment and the start time of a second trajectory segment arranged in sequence, and the position distance difference between the trajectory segments refers to the position distance between the end position of the first trajectory segment and the start position of the second trajectory segment arranged in sequence. In this case, the server can regard the time hole as a time gap corresponding to the cluster, and regard the space hole as a space gap corresponding to the cluster.

[0098] The server can take the duration of each trajectory segment in the candidate trajectory segment set as the time information of the trajectory segment, and take the spatial range covered by each trajectory segment as the space information. Then, for each trajectory segment in the candidate trajectory segment set, the server can calculate the intersection of the trajectory segment with the time gap and the intersection of the trajectory segment with the space gap according to the time information and the space information of the trajectory segment. When the intersection of the trajectory segment with the time gap satisfies the time constraint condition, and the intersection of the trajectory segment with the space gap satisfies the space constraint condition, the server can determine that the trajectory segment is a target trajectory segment, and can add the trajectory segment to the seed trajectory segment set corresponding to the to-be-tracked target.

[0099] To ensure that the trajectory segments filled into the seed trajectory segment set correspond to the same target to be tracked as the seed trajectory segment set, the server can take the trajectory segments in the updated candidate trajectory segment set that correspond to time information satisfying the time constraint condition and spatial information satisfying the spatial constraint condition as target candidate trajectory segments. Then, the server determines whether the similarity between the appearance feature corresponding to the target candidate trajectory segment and the appearance feature corresponding to the target to be tracked satisfies a similarity determination condition, for example, whether it exceeds a preset similarity threshold. If yes, the server determines that the target to be tracked corresponding to the target candidate trajectory segment is consistent with the target to be tracked corresponding to the seed trajectory segment set, and can determine the target candidate trajectory segment as a target trajectory segment, fill the target trajectory segment into the seed trajectory segment set, and delete the target trajectory segment in the updated candidate trajectory segment set. If not, the server determines that the target to be tracked corresponding to the target candidate trajectory segment is not consistent with the target to be tracked corresponding to the seed trajectory segment set, and cannot fill the target candidate trajectory segment into the seed trajectory segment set.

[0100] It should be noted that the appearance feature can be any one or more of the appearance features. Taking the method provided by the embodiments of the present application as an example, the appearance feature can be any one or more of a face feature, a pedestrian re-identification feature, and a head feature. The appearance feature corresponding to the target to be tracked can be the appearance feature corresponding to each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked, or can be a processing result obtained by processing the appearance feature corresponding to each trajectory segment in the seed trajectory segment set.

[0101] It should be noted that in actual applications, the above steps 101, 102, and 103 can be used as target processing links. When target tracking is performed for a target scene, the above target processing links can be executed in multiple rounds. After each round of target processing links is executed, the updated candidate trajectory segment set in which the trajectory segments in the deleted seed trajectory segment set are replaced by the new candidate trajectory segment set is used to replace the candidate trajectory segment set in the previous round.

[0102] In addition, to ensure that the seed trajectory segment set can include more trajectory segments, the appearance similarity constraint condition used in the step 101 in the next round of target processing links needs to be lower than the appearance similarity constraint condition used in the step 101 in the previous round. The time constraint condition and the spatial constraint condition used in the step 103 in the next round of target processing links need to be lower than the time constraint condition and the spatial constraint condition used in the step 103 in the previous round, respectively.

[0103] It should be understood that the specific number of rounds in the target processing stage can be set according to actual needs, and this application does not impose any restrictions here.

[0104] Step 104: Piece together the trajectory segments in the seed trajectory segment set corresponding to the target to be tracked to obtain the activity trajectory of the target to be tracked in the target scene.

[0105] After the trajectory segment filling process in step 103, the seed trajectory segment set corresponding to the target to be tracked will include several spatiotemporally continuous trajectory segments. At this time, the server can perform splicing processing on the seed trajectory segment set corresponding to the target to be tracked, according to the time and space information corresponding to each trajectory segment, so as to splice the trajectory segments into a complete long trajectory, which is the activity trajectory of the target to be tracked in the target scene.

[0106] It should be understood that if after processing in steps 101-103, a set of seed trajectory segments corresponding to each of the multiple targets to be tracked is obtained, then in step 104, the activity trajectory corresponding to each of these multiple targets to be tracked will be obtained.

[0107] The target tracking method provided in this application proposes a tracking algorithm framework based on trajectory fragment filling, and uses this framework to track various targets in a target scene. Since constructing a seed trajectory fragment set based on appearance features is not limited by time and space, trajectory fragments belonging to the same seed trajectory fragment set can span long time periods and wide spatial regions. Furthermore, trajectory fragment filling further ensures the spatiotemporal continuity of trajectory fragments in the seed trajectory fragment set; thus, accurate tracking of targets in a target scene over a long period and over a large area is achieved.

[0108] In addition, this application also provides a trajectory filling method, see [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart illustrating the trajectory filling method provided in the embodiments of this application. The following embodiments still use a server as the execution entity for description. Figure 2 As shown, the method includes the following steps:

[0109] Step 201: Determine the temporal and spatial gaps corresponding to the seed trajectory fragment set based on the temporal and spatial information of each trajectory fragment in the seed trajectory fragment set corresponding to the target to be tracked; the seed trajectory fragment set includes trajectory fragments belonging to the target to be tracked.

[0110] After obtaining the seed trajectory segment set corresponding to the target to be tracked, the server can perform trajectory filling processing on the seed trajectory segment set corresponding to the target to be tracked, i.e., filling the trajectory segments in the candidate trajectory segment set into the seed trajectory segment set corresponding to the target to be tracked, where the trajectory segments in the candidate trajectory segment set include trajectory segments that are not divided into the seed trajectory segment set.

[0111] Specifically, the server can first determine the time gap and the space gap corresponding to the seed trajectory segment set according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set.

[0112] The server can first arrange and splice the trajectory segments in the seed trajectory segment set corresponding to the target to be tracked according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set, to obtain a seed trajectory corresponding to the seed trajectory segment set, i.e., a seed trajectory of the target to be tracked. Since the seed trajectory segment set is usually constructed based on appearance features, and the construction of the seed trajectory segment set based on appearance features does not consider time and space constraints, the trajectory segments included in the seed trajectory segment set can span a long time period in time and cover a wide spatial area in space, and there can be discontinuity in time and space between trajectory segments.

[0113] If such a situation exists, there will be time discontinuity holes and space discontinuity holes in the seed trajectory obtained by arranging and splicing the trajectory segments in the seed trajectory segment set, i.e., there is a time range in which the time difference between trajectory segments exceeds a preset time threshold (i.e., a time hole), and there is a space range in which the position distance difference between trajectory segments exceeds a preset space threshold (i.e., a space hole), where the time difference between trajectory segments refers to the difference between the end time of a previous trajectory segment and the start time of a next trajectory segment in the arrangement order, and the position distance difference between trajectory segments refers to the position distance between the end position of a previous trajectory segment and the start position of a next trajectory segment in the arrangement order. In this case, the server can regard the above-mentioned time hole as the time gap corresponding to the cluster, and regard the above-mentioned space hole as the space gap corresponding to the cluster.

[0114] Step 202: In the candidate trajectory segment set, a target trajectory segment is selected, which satisfies a time constraint condition with respect to the time gap and satisfies a space constraint condition with respect to the space gap, where the candidate trajectory segment set includes trajectory segments that are not divided into the seed trajectory segment set.

[0115] Step 203: The target trajectory segment is added to the seed trajectory segment set.

[0116] Further, the server can select, from the candidate trajectory segment set, a target trajectory segment whose corresponding time information satisfies the time constraint condition with the time gap and whose corresponding space information satisfies the space constraint condition with the space gap, add the target trajectory segment to the seed trajectory segment set corresponding to the to-be-tracked target, and delete the target trajectory segment from the candidate trajectory segment set.

[0117] Specifically, the server can take the duration of each trajectory segment in the candidate trajectory segment set as the time information of the trajectory segment, and take the space range covered by each trajectory segment as the space information. Further, the server can calculate, for each trajectory segment in the candidate trajectory segment set, the intersection of the trajectory segment with the time gap and the intersection of the trajectory segment with the space gap according to the time information and the space information of the trajectory segment. When the intersection of the trajectory segment with the time gap satisfies the time constraint condition and the intersection of the trajectory segment with the space gap satisfies the space constraint condition, the trajectory segment can be determined as a target trajectory segment, and the target trajectory segment can be added to the seed trajectory segment set corresponding to the to-be-tracked target.

[0118] To ensure that the to-be-tracked target corresponding to the trajectory segment filled into the seed trajectory segment set is consistent with the to-be-tracked target corresponding to the seed trajectory segment set, the server can take, as a target candidate trajectory segment, a trajectory segment in the candidate trajectory segment set whose corresponding time information satisfies the time constraint condition with the time gap and whose corresponding space information satisfies the space constraint condition with the space gap. Further, the server can determine whether the similarity between the appearance feature corresponding to the target candidate trajectory segment and the appearance feature corresponding to the to-be-tracked target satisfies a similarity determination condition, for example, whether the similarity exceeds a preset similarity threshold. If yes, the server can determine that the to-be-tracked target corresponding to the target candidate trajectory segment is consistent with the to-be-tracked target corresponding to the seed trajectory segment set, determine the target candidate trajectory segment as a target trajectory segment, fill the target trajectory segment into the seed trajectory segment set, and delete the target trajectory segment from the candidate trajectory segment set. If not, the server can determine that the to-be-tracked target corresponding to the target candidate trajectory segment is inconsistent with the to-be-tracked target corresponding to the seed trajectory segment set, and cannot fill the target candidate trajectory segment into the seed trajectory segment set.

[0119] It should be noted that the above appearance feature can be any one or more appearance features. Taking the scenario of applying the method provided by the embodiments of the present application to pedestrian tracking as an example, the appearance feature can be any one or more of a face feature, a pedestrian re-identification feature, and a head feature. The appearance feature corresponding to the to-be-tracked target can be the appearance feature corresponding to each trajectory segment in the seed trajectory segment set of the to-be-tracked target, or can be a processing result obtained by processing the appearance feature corresponding to each trajectory segment in the seed trajectory segment set.

[0120] It should be noted that in actual application, the server can perform the above operation multiple times in a loop, and appropriately relax the time constraint condition and / or the space preset condition each time to enable the seed trajectory segment set to include more trajectory segments.

[0121] The embodiments of the present application also provide a target tracking device. Referring to Figure 3 , Figure 3 FIG. 1 is a structural schematic diagram of a target tracking device provided by the embodiments of the present application. As shown in the figure, the device includes: Figure 3

[0122] The seed trajectory segment set forming module 301 is configured to identify, based on the appearance feature corresponding to each trajectory segment in the candidate trajectory segment set and the appearance similarity constraint condition, a trajectory segment belonging to a to-be-tracked target in the candidate trajectory segment set, and form a seed trajectory segment set corresponding to the to-be-tracked target; the candidate trajectory segment set includes trajectory segments of each target under each camera in a target scene;

[0123] The candidate trajectory segment set updating module 302 is configured to delete a trajectory segment belonging to the seed trajectory segment set corresponding to the to-be-tracked target in the candidate trajectory segment set, and obtain an updated candidate trajectory segment set;

[0124] The trajectory segment filling module 303 is configured to determine a time gap and a space gap corresponding to the seed trajectory segment set corresponding to the to-be-tracked target according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target; select, in the updated candidate trajectory segment set, a target trajectory segment whose corresponding time information satisfies a time constraint condition with the time gap and whose corresponding space information satisfies a space constraint condition with the space gap; add the target trajectory segment to the seed trajectory segment set corresponding to the to-be-tracked target; and delete the target trajectory segment in the updated candidate trajectory segment set;

[0125] ​The activity trajectory splicing module 304 is configured to splice each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target to obtain an activity trajectory of the to-be-tracked target in the target scene.

[0126] Optionally, the trajectory segment filling module 303 is specifically configured to:

[0127] select, from the updated candidate trajectory segment set, a target candidate trajectory segment that satisfies the time constraint condition in that the corresponding time information of the target candidate trajectory segment and the time gap satisfy the time constraint condition and satisfies the space constraint condition in that the corresponding space information of the target candidate trajectory segment and the space gap satisfy the space constraint condition.

[0128] determine whether a similarity between an appearance feature corresponding to the target candidate trajectory segment and an appearance feature corresponding to the to-be-tracked target satisfies a similarity determination condition, and if so, determine the target candidate trajectory segment as the target trajectory segment.

[0129] Optionally, the trajectory segment filling module 303 is specifically configured to:

[0130] splice each trajectory segment in the seed trajectory segment set according to the respective time information and space information of each trajectory segment in the seed trajectory segment set to obtain a seed trajectory corresponding to the to-be-tracked target.

[0131] take a hole in time that is discontinuous on the seed trajectory as the time gap and take a hole in space that is discontinuous on the seed trajectory as the space gap.

[0132] Optionally, the seed trajectory segment set forming module 301 is specifically configured to:

[0133] obtain N reference appearance features respectively corresponding to the to-be-tracked targets; the N is a positive integer.

[0134] For each to-be-tracked target, determine, according to a similarity between the respective appearance features of each trajectory segment in the candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, the trajectory segments in the candidate trajectory segment set that belong to the to-be-tracked target to form a seed trajectory segment set corresponding to the to-be-tracked target.

[0135] Optionally, the seed trajectory segment set forming module 301 is specifically configured to:

[0136] perform clustering processing on the trajectory segments in the candidate trajectory segment set based on the appearance features corresponding to the trajectory segments in the candidate trajectory segment set to determine M to-be-tracked targets in the target scene and the trajectory segments belonging to each to-be-tracked target; the M is a positive integer.

[0137] For each of the to-be-tracked targets, a set of seed trajectory segments corresponding to the to-be-tracked target is formed by using the trajectory segments belonging to the to-be-tracked target.

[0138] Optionally, the seed trajectory segment set forming module 301 is specifically configured to:

[0139] According to the strong identification features corresponding to the trajectory segments in the candidate trajectory segment set, the trajectory segments in the candidate trajectory segment set are clustered to determine the M to-be-tracked targets included in the target scene and the trajectory segments corresponding to the M to-be-tracked targets respectively, and the trajectory segments corresponding to the M to-be-tracked targets are used to form the seed trajectory segment sets corresponding to the M to-be-tracked targets respectively; the strong identification features are features with unique identification.

[0140] The trajectory segments belonging to the seed trajectory segment sets corresponding to the M to-be-tracked targets in the candidate trajectory segment set are deleted to obtain an updated candidate trajectory segment set.

[0141] For each of the to-be-tracked targets, a reference appearance feature corresponding to the to-be-tracked target is determined according to the trajectory segments in the seed trajectory segment set corresponding to the to-be-tracked target; and a target trajectory segment of the to-be-tracked target in the updated candidate trajectory segment set is determined according to the similarity between the appearance features corresponding to the trajectory segments in the updated candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, and the target trajectory segment is added to the seed trajectory segment set corresponding to the to-be-tracked target.

[0142] Optionally, the operations performed by the seed trajectory segment set forming module 301, the candidate trajectory segment set updating module 302, and the trajectory segment filling module 303 are taken as a target processing link.

[0143] The target processing link is executed in multiple rounds in a loop; after each round of the target processing link is executed, the updated candidate trajectory segment set in which the trajectory segments in the seed trajectory segment set have been deleted is used to replace the candidate trajectory segment set as a new candidate trajectory segment set.

[0144] The appearance similarity constraint condition on which the seed trajectory segment set forming step in the target processing link in a later round is based is lower than the appearance similarity constraint condition on which the seed trajectory segment set forming step in the target processing link in a previous round is based; the time constraint condition and the space constraint condition on which the trajectory segment filling step in the target processing link in a later round is based are respectively lower than the time constraint condition and the space constraint condition on which the trajectory segment filling step in the target processing link in a previous round is based.

[0145] The target tracking device provided by the embodiments of the present application proposes a tracking algorithm framework based on trajectory segment filling, and uses the tracking algorithm framework to realize tracking of each target to be tracked in a target scene. Since the seed trajectory segment set is constructed based on appearance features and is not limited in time and space, each trajectory segment belonging to the same seed trajectory segment set can span a long time period in time and a wide spatial area in space, and the trajectory segment filling can further ensure the time and space continuity of the trajectory segments in the seed trajectory segment set. In this way, long-time and large-area accurate tracking of the target to be tracked in the target scene is realized.

[0146] The embodiments of the present application also provide a trajectory filling device. Referring to Figure 4 , Figure 4 FIG. 1 is a structural schematic diagram of the trajectory filling device provided by the embodiments of the present application. As shown in the figure, the device comprises: Figure 4

[0147] a gap determination module 401 configured to determine a time gap and a space gap corresponding to a seed trajectory segment set of a target to be tracked according to time information and space information corresponding to each trajectory segment in the seed trajectory segment set; the seed trajectory segment set comprises trajectory segments belonging to the target to be tracked;

[0148] a segment selection module 402 configured to select, in a candidate trajectory segment set, a target trajectory segment whose corresponding time information satisfies a time constraint condition with the time gap and whose corresponding space information satisfies a space constraint condition with the space gap; the candidate trajectory segment set comprises trajectory segments that are not divided into the seed trajectory segment set;

[0149] a segment filling module 403 configured to add the target trajectory segment to the seed trajectory segment set.

[0150] Optionally, the segment selection module 402 is specifically configured to:

[0151] select, in the candidate trajectory segment set, a target candidate trajectory segment whose corresponding time information satisfies the time constraint condition with the time gap and whose corresponding space information satisfies the space constraint condition with the space gap;

[0152] determine whether a similarity between an appearance feature corresponding to the target candidate trajectory segment and an appearance feature corresponding to the target to be tracked satisfies a similarity determination condition, and if so, determine that the target candidate trajectory segment is the target trajectory segment.

[0153] Optionally, the gap determination module 401 is specifically configured to:

[0154] ​According to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set, the seed trajectory segment set is spliced to obtain a seed trajectory corresponding to the target to be tracked.

[0155] The time discontinuous hole in the seed trajectory is taken as the time gap, and the space discontinuous hole in the seed trajectory is taken as the space gap.

[0156] The embodiments of the present application also provide a device for target tracking or trajectory filling. The device can be a server or a terminal device. The server and the terminal device provided by the embodiments of the present application will be introduced from the perspective of hardware realization.

[0157] Referring to Figure 5 , Figure 5 A structure diagram of a server 500 provided by the embodiments of the present application is shown. The server 500 can be quite different due to different configurations or performances, and can include one or more central processing units (CPUs) 522 (for example, one or more processors) and a memory 532, one or more storage media 530 (for example, one or more mass storage devices) for storing application programs 542 or data 544. The memory 532 and the storage media 530 can be temporary storage or persistent storage. The programs stored in the storage media 530 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the server. Further, the central processing unit 522 can be configured to communicate with the storage media 530 and execute the series of instruction operations in the storage media 530 on the server 500.

[0158] The server 500 can also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input and output interfaces 558, and / or one or more operating systems 541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0159] The steps performed by the server in the above embodiments can be based on the Figure 5 The server structure is shown.

[0160] The CPU 522 is configured to perform the following steps:

[0161] The seed trajectory segment set composition step comprises: identifying trajectory segments belonging to the target to be tracked from the candidate trajectory segment set based on the appearance features and the appearance similarity constraint conditions corresponding to each trajectory segment in the candidate trajectory segment set, and composing a seed trajectory segment set corresponding to the target to be tracked; the candidate trajectory segment set comprises trajectory segments of each target under each camera in the target scene;

[0162] The candidate trajectory segment set updating step comprises: deleting trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked from the candidate trajectory segment set, to obtain an updated candidate trajectory segment set;

[0163] The trajectory segment filling step comprises: determining time gaps and space gaps corresponding to the seed trajectory segment set corresponding to the target to be tracked based on the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked; selecting a target trajectory segment from the updated candidate trajectory segment set, wherein the corresponding time information satisfies a time constraint condition with the time gaps, and the corresponding space information satisfies a space constraint condition with the space gaps; adding the target trajectory segment to the seed trajectory segment set corresponding to the target to be tracked; and deleting the target trajectory segment from the updated candidate trajectory segment set;

[0164] The activity trajectory splicing step comprises: splicing each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked, to obtain an activity trajectory of the target to be tracked in the target scene.

[0165] Alternatively,

[0166] The time gaps and the space gaps corresponding to the seed trajectory segment set are determined based on the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked; the seed trajectory segment set comprises trajectory segments belonging to the target to be tracked;

[0167] The target trajectory segment is selected from the candidate trajectory segment set, wherein the corresponding time information satisfies a time constraint condition with the time gaps, and the corresponding space information satisfies a space constraint condition with the space gaps; the candidate trajectory segment set comprises trajectory segments that are not divided into the seed trajectory segment set;

[0168] The target trajectory segment is added to the seed trajectory segment set.

[0169] Optionally, the CPU 522 is further configured to execute steps of any one of the target tracking method or the trajectory filling method provided in the embodiments of the present application.

[0170] Referring to Figure 6 ,Figure 6 A structural schematic diagram of a terminal device provided by an embodiment of the present application is shown. For ease of illustration, only parts related to the embodiments of the present application are shown, and for technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a computer, a tablet computer, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), etc. Take the computer as an example:

[0171] Figure 6 A block diagram of part of the structure of the computer related to the terminal provided by the embodiments of the present application is shown. Referring to Figure 6 , the computer includes radio frequency (English full name: Radio Frequency, English abbreviation: RF) circuit 610, memory 620, input unit 630, display unit 640, sensor 650, audio circuit 660, wireless fidelity (English full name: wireless fidelity, English abbreviation: WiFi) module 670, processor 680, and power supply 690, etc. Those skilled in the art can understand that Figure 6 the computer structure shown in the above is not a limitation on the computer, and can include more or less components than the diagram, or combine certain components, or different component arrangements.

[0172] The memory 620 can be used to store software programs and modules, and the processor 680 executes various functions of the computer and data processing by running the software programs and modules stored in the memory 620. The memory 620 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the computer (such as audio data, a phone book, etc.), etc. In addition, the memory 620 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0173] The processor 680 is the control center of the computer, which connects the various parts of the computer through various interfaces and lines, performs various functions of the computer and processes data by running or executing software programs and / or modules stored in the memory 620 and calling data stored in the memory 620, thereby overall monitoring the computer. Alternatively, the processor 680 can include one or more processing units; preferably, the processor 680 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 680.

[0174] In the embodiment of the present application, the processor 680 included in the terminal further has the following functions:

[0175] The seed trajectory segment set composition step: based on the appearance features and appearance similarity constraints corresponding to each trajectory segment in the candidate trajectory segment set, identifying trajectory segments belonging to the target to be tracked in the candidate trajectory segment set, and composing a seed trajectory segment set corresponding to the target to be tracked; the candidate trajectory segment set includes trajectory segments of each target under each camera in the target scene;

[0176] The candidate trajectory segment set updating step: deleting the trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked in the candidate trajectory segment set, and obtaining an updated candidate trajectory segment set;

[0177] The trajectory segment filling step: determining the time gap and the space gap corresponding to the seed trajectory segment set corresponding to the target to be tracked according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked; selecting a target trajectory segment in the updated candidate trajectory segment set, which satisfies the time constraint condition with the corresponding time information of the time gap and satisfies the space constraint condition with the corresponding space information of the space gap, adding the target trajectory segment to the seed trajectory segment set corresponding to the target to be tracked, and deleting the target trajectory segment in the updated candidate trajectory segment set;

[0178] The activity trajectory splicing step: splicing each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked to obtain the activity trajectory of the target to be tracked in the target scene.

[0179] Or,

[0180] determine a time gap and a space gap corresponding to the seed trajectory segment set according to time information and space information corresponding to each trajectory segment in the seed trajectory segment set;

[0181] select a target trajectory segment from the candidate trajectory segment set, which satisfies a time constraint condition with the time gap and a space constraint condition with the space gap, wherein the candidate trajectory segment set includes trajectory segments that are not divided into the seed trajectory segment set;

[0182] add the target trajectory segment to the seed trajectory segment set.

[0183] Optionally, the processor 680 is further configured to perform the steps of any one of the implementation manners of the target tracking method or the trajectory filling method provided in the embodiments.

[0184] The embodiments of the present application further provide a computer readable storage medium for storing program codes, the program codes being used for performing any one of the implementation manners of the target tracking method or the trajectory filling method provided in the foregoing embodiments.

[0185] The embodiments of the present application further provide a computer program product including instructions, which, when executed on a computer, cause the computer to perform any one of the implementation manners of the target tracking method or the trajectory filling method provided in the foregoing embodiments.

[0186] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0187] In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the device embodiments described above are only schematic, and for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0188] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0189] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0190] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or all or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), a random access memory (English full name: Random Access Memory, English abbreviation: RAM), a magnetic disk or an optical disk, and various computer program storage media.

[0191] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association relationship between the associated objects, which means that there can be three kinds of relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0192] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A target tracking method characterized by, The method comprises: a seed trajectory segment set forming step: identifying trajectory segments belonging to a to-be-tracked target from a candidate trajectory segment set based on appearance features corresponding to each trajectory segment in the candidate trajectory segment set and appearance similarity constraint conditions, and forming a seed trajectory segment set corresponding to the to-be-tracked target; the candidate trajectory segment set comprises trajectory segments of each target under each camera in a target scene; a candidate trajectory segment set updating step: deleting trajectory segments belonging to the seed trajectory segment set corresponding to the to-be-tracked target from the candidate trajectory segment set to obtain an updated candidate trajectory segment set; a trajectory segment filling step: determining time gaps and space gaps corresponding to the seed trajectory segment set corresponding to the to-be-tracked target based on time information and space information corresponding to each trajectory segment in the seed trajectory segment set; selecting a target trajectory segment from the updated candidate trajectory segment set, wherein the corresponding time information of the target trajectory segment satisfies a time constraint condition with the time gaps, and the corresponding space information satisfies a space constraint condition with the space gaps; adding the target trajectory segment to the seed trajectory segment set corresponding to the to-be-tracked target, and deleting the target trajectory segment from the updated candidate trajectory segment set; an active trajectory splicing step: splicing each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target to obtain an active trajectory of the to-be-tracked target in the target scene; the seed trajectory segment set forming step, the candidate trajectory segment set updating step, and the trajectory segment filling step are target processing links; the target processing links are executed in multiple rounds in a loop; after each round of the target processing links is executed, the updated candidate trajectory segment set from which trajectory segments in the seed trajectory segment set have been deleted is used to replace the candidate trajectory segment set as a new candidate trajectory segment set; the appearance similarity constraint condition used in the seed trajectory segment set forming step in a later round of the target processing links is lower than the appearance similarity constraint condition used in the seed trajectory segment set forming step in a previous round of the target processing links; the time constraint condition and the space constraint condition used in the trajectory segment filling step in a later round of the target processing links are respectively lower than the time constraint condition and the space constraint condition used in the trajectory segment filling step in a previous round of the target processing links.

2. The method of claim 1, wherein, the target trajectory segment is selected from the updated candidate trajectory segment set, wherein the corresponding time information of the target trajectory segment satisfies the time constraint condition with the time gaps, and the corresponding space information satisfies the space constraint condition; the target candidate trajectory segment is selected from the updated candidate trajectory segment set, wherein the corresponding time information of the target candidate trajectory segment satisfies the time constraint condition with the time gaps, and the corresponding space information satisfies the space constraint condition with the space gaps; it is determined whether the similarity between the appearance feature corresponding to the target candidate trajectory segment and the appearance feature corresponding to the to-be-tracked target satisfies a similarity determination condition; if yes, the target candidate trajectory segment is determined as the target trajectory segment.

3. The method according to claim 1 or 2, characterized in that, The time gap and the space gap corresponding to the seed trajectory segment set corresponding to the to-be-tracked target are determined according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target, and the seed trajectory segment set corresponding to the to-be-tracked target is obtained by splicing the trajectory segments in the seed trajectory segment set. The time gap and the space gap corresponding to the seed trajectory segment set corresponding to the to-be-tracked target are determined according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target, and the seed trajectory segment set corresponding to the to-be-tracked target is obtained by splicing the trajectory segments in the seed trajectory segment set. The time gap and the space gap corresponding to the seed trajectory segment set corresponding to the to-be-tracked target are determined according to the time information and the space information corresponding to each trajectory segment in the seed trajectory segment set corresponding to the to-be-tracked target, and the seed trajectory segment set corresponding to the to-be-tracked target is obtained by splicing the trajectory segments in the seed trajectory segment set.

4. The method of claim 1, wherein, The seed trajectory segment set composition step includes: N reference appearance features corresponding to the to-be-tracked targets are obtained; N is a positive integer; For each to-be-tracked target, the trajectory segments belonging to the to-be-tracked target in the candidate trajectory segment set are determined according to the similarity between the appearance features corresponding to each trajectory segment in the candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, and a seed trajectory segment set corresponding to the to-be-tracked target is formed.

5. The method of claim 1, wherein, The seed trajectory segment set composition step includes: Based on the appearance features corresponding to the trajectory segments in the candidate trajectory segment set, the trajectory segments in the candidate trajectory segment set are clustered to determine M to-be-tracked targets in the target scene and the trajectory segments belonging to each to-be-tracked target; M is a positive integer; For each to-be-tracked target, the trajectory segments belonging to the to-be-tracked target in the candidate trajectory segment set are determined according to the similarity between the appearance features corresponding to each trajectory segment in the candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, and a seed trajectory segment set corresponding to the to-be-tracked target is formed.

6. The method of claim 5, wherein, The seed trajectory segment set composition step includes: Based on the appearance features corresponding to the trajectory segments in the candidate trajectory segment set, the trajectory segments in the candidate trajectory segment set are clustered to determine M to-be-tracked targets in the target scene and the trajectory segments belonging to each to-be-tracked target; M is a positive integer; For each to-be-tracked target, the trajectory segments belonging to the to-be-tracked target in the candidate trajectory segment set are determined according to the similarity between the appearance features corresponding to each trajectory segment in the candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, and a seed trajectory segment set corresponding to the to-be-tracked target is formed. The seed trajectory segment set composition step includes: Based on the appearance features corresponding to the trajectory segments in the candidate trajectory segment set, the trajectory segments in the candidate trajectory segment set are clustered to determine M to-be-tracked targets in the target scene and the trajectory segments belonging to each to-be-tracked target; M is a positive integer; For each to-be-tracked target, the trajectory segments belonging to the to-be-tracked target in the candidate trajectory segment set are determined according to the similarity between the appearance features corresponding to each trajectory segment in the candidate trajectory segment set and the reference appearance feature corresponding to the to-be-tracked target, and a seed trajectory segment set corresponding to the to-be-tracked target is formed.

7. A target tracking device, characterized by, The device comprises: a seed trajectory segment set forming module, configured to identify trajectory segments belonging to a target to be tracked from a candidate trajectory segment set based on appearance features and appearance similarity constraints corresponding to each trajectory segment in the candidate trajectory segment set, and form a seed trajectory segment set corresponding to the target to be tracked; the candidate trajectory segment set comprises trajectory segments of each target under each camera in a target scene; a candidate trajectory segment set updating module, configured to delete trajectory segments belonging to the seed trajectory segment set corresponding to the target to be tracked from the candidate trajectory segment set, and obtain an updated candidate trajectory segment set; a trajectory segment filling module, configured to determine time gaps and space gaps corresponding to the seed trajectory segment set corresponding to the target to be tracked based on time information and space information corresponding to each trajectory segment in the seed trajectory segment set; select a target trajectory segment from the updated candidate trajectory segment set, wherein the target trajectory segment satisfies time constraints corresponding to time information of the target trajectory segment and space constraints corresponding to space information of the target trajectory segment; add the target trajectory segment to the seed trajectory segment set corresponding to the target to be tracked; and delete the target trajectory segment from the updated candidate trajectory segment set; an active trajectory splicing module, configured to splice each trajectory segment in the seed trajectory segment set corresponding to the target to be tracked, and obtain an active trajectory of the target to be tracked in the target scene; the seed trajectory segment set forming module, the candidate trajectory segment set updating module, and the trajectory segment filling module perform operations as a target processing link; the target processing link is executed in multiple rounds in a loop; after each round of the target processing link is executed, the updated candidate trajectory segment set from which trajectory segments in the seed trajectory segment set have been deleted is used to replace the candidate trajectory segment set as a new candidate trajectory segment set; appearance similarity constraints used in a seed trajectory segment set forming step in a later round of the target processing link are lower than appearance similarity constraints used in a seed trajectory segment set forming step in a previous round of the target processing link; time constraints and space constraints used in a trajectory segment filling step in the later round of the target processing link are respectively lower than time constraints and space constraints used in a trajectory segment filling step in the previous round of the target processing link.

8. An apparatus, comprising: The device comprises a processor and a memory; The memory is configured to store a computer program; The processor is configured to call the computer program to execute the target tracking method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store a computer program, and the computer program is configured to execute the target tracking method in any one of claims 1 to 6.

Images