Method and device for target tracking, and storage medium

Abstract

To provide a method and a device for target tracking, and a storage medium.SOLUTION: A method includes: determining a plurality of local target loci having local target identifiers, on the basis of a plurality of current frames at a current time point by a plurality of cameras; updating a first feature pool including a sub-locus segment feature of a latest sub-locus segment of each of the local target loci, and a second feature pool including a sub-locus segment feature of an early sub-locus segment of each of the local target loci, on the basis of the plurality of local target loci; clustering a feature in a union of the updated first feature pool and the updated second feature pool, when the current time point satisfies a time requirement for a predetermined clustering cycle, and thereby determining a plurality of current anchor points having the corresponding current cluster appearance feature; and determining a global target identifier of a target detected in the plurality of current frames, on the basis of the plurality of current anchor points.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates generally to image processing, and more particularly to a method for target tracking, an apparatus for target tracking, and a computer-readable non-transitory storage medium having a program stored thereon. [Background technology]

[0002] With the development of computer science, it has become increasingly common and effective to use computers to run image processing programs to obtain useful information. Multi-target tracking is one of the important application areas of image processing.

[0003] Multi-target tracking, commonly referred to simply as MTT (Multiple Target Tracking) (sometimes abbreviated as MOT: Multiple Object Tracking), is used to detect multiple targets of interest in a video, such as pedestrians, vehicles, and / or animals, determine their trajectories, and assign them identifiers (IDs) for tracking. The desired tracking result is to assign different IDs to different targets after multi-target tracking to achieve tasks such as accurate tracking and accurate search. MOT is one of the key technologies in the field of computer vision and is widely applied in areas such as autonomous driving, intelligent surveillance, and behavior recognition.

[0004] Multi-target tracking includes single-camera tracking and cross-camera tracking. For single-camera tracking, the input video of the tracker (e.g., an application program that realizes tracking) is acquired from only one camera.

[0005] The input video for cross-camera tracking is acquired from multiple cameras. To identify the same target with the same identifier in images from multiple cameras, the local tracking trajectories output by each single camera must be correlated. For example, if the correlation process determines that the target corresponding to the local trajectory L-Tr1 from the first camera and the local trajectory L-Tr2 from the second camera is the same person, the target identifiers of the trajectories L-Tr1 and L-Tr2 are both expressed as a unique global identifier "IDXXXX." That is, the trajectories L-Tr1 and L-Tr2 are correlated to form the global trajectory GTr of the target "IDXXXX." The global trajectory GTr may be a trajectory obtained by connecting multiple local trajectories in chronological order. The trajectory output by target tracking includes multiple trajectory points corresponding to multiple image acquisition times.

[0006] Reference 1: Huang, Hsiang-Wei, et al. "Enhancing Multi-Camera People Tracking with Anchor-Guided Clustering and Spatio-Temporal Consistency ID Re-Assignment." Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2023.

[0007] Reference 1 provides a cross-camera multi-target tracking method based on anchor points, which mainly includes the steps of: (1) generating cross-camera anchor points (with global IDs) through hierarchical clustering using appearance features in downsampled frames; (2) calculating a cost matrix between the anchor points across different cameras and the features of each frame, and assigning different global IDs to a single camera trajectory using the Hungarian algorithm; and (3) performing majority voting using a sliding window to further optimize the final trajectory.

[0008] In the field of cross-camera multi-target tracking, there is a demand for realizing online tracking (i.e., real-time tracking). The method described in Reference 1 cannot be applied to online tracking. Compared with non-online tracking (where a complete video of a predetermined period is acquired, e.g., Reference 1), online tracking requires limited data (where only the current image and trajectory information from the previous period are available) to ensure tracking accuracy and ensure faster processing speeds to achieve real-time tracking. Summary of the Invention [Problem to be solved by the invention]

[0009] The following presents a simplified summary of the disclosure in order to provide a basic understanding of aspects of the disclosure. However, this summary is not an exhaustive overview of the disclosure, and it is not intended to identify key or important portions of the disclosure or to limit the scope of the disclosure. Rather, it is intended to merely introduce concepts in a simplified form as a prelude to the more detailed description that is presented later.

[0010] The inventor of the present invention has conducted research and testing on cross-camera multi-target online tracking, and has provided a cross-camera multi-target online tracking technology, which has good effects in terms of accuracy, and has proposed the technology disclosed in this disclosure. [Means for solving the problem]

[0011] In one aspect of the present disclosure, there is provided a method for target tracking, the method including: determining a plurality of local target trajectories having a local target identifier based on a plurality of current frames at a current time point provided by a plurality of cameras; updating a first feature pool including sub-trajectory segment features of a most recent sub-trajectory segment of each local target trajectory and a second feature pool including sub-trajectory segment features of an earlier sub-trajectory segment of each local target trajectory based on the plurality of local target trajectories; if the current time point satisfies a time requirement for a predetermined clustering period, determining a plurality of current anchor points having corresponding current cluster appearance features by clustering features in a union of the updated first feature pool and the updated second feature pool; and determining a global target identifier of a detected target in the plurality of current frames based on the plurality of current anchor points.

[0012] In one aspect of the present disclosure, there is provided an apparatus for target tracking, the apparatus including: a memory having instructions stored therein; and at least one processor coupled to the memory and configured to execute the instructions to implement the method for target tracking described above.

[0013] Another aspect of the present disclosure provides a computer-readable non-transitory storage medium having a program stored thereon, the program, when executed by a computer, causing the computer to perform the following steps: determining a plurality of local target trajectories having local target identifiers based on a plurality of current frames at a current time point provided by a plurality of cameras; updating a first feature pool including sub-trajectory segment features of a most recent sub-trajectory segment of each local target trajectory and a second feature pool including sub-trajectory segment features of an earlier sub-trajectory segment of each local target trajectory based on the plurality of local target trajectories; if the current time point satisfies a time requirement for a predetermined clustering period, determining a plurality of current anchor points having corresponding current cluster appearance features by clustering features in a union of the updated first feature pool and the updated second feature pool; and determining a global target identifier of a detected target in the plurality of current frames based on the plurality of current anchor points.

[0014] The advantageous effects of the method, apparatus, and storage medium of the present disclosure include at least one of the following advantageous effects: Real-time online target tracking and better target tracking accuracy can be achieved. [Brief explanation of the drawings]

[0015] In order to make the above and other objects, features, and advantages of the present disclosure more easily understandable, the following describes embodiments of the present disclosure with reference to the drawings. It should be noted that the drawings are merely for illustrating the principles of the present disclosure. The drawings do not necessarily depict the size and relative positions of each part according to scale. The same reference numerals may represent the same features. [Figure 1] 1 is a flowchart illustrating an example of a method for target tracking according to one embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram illustrating multiple trajectories according to one embodiment of the present disclosure. [Figure 3]1 is a flowchart illustrating an example of a method for target tracking according to one embodiment of the present disclosure. [Figure 4] FIG. 1 is a block diagram illustrating an example of an apparatus for target tracking according to one embodiment of the present disclosure. [Figure 5] FIG. 1 is a block diagram illustrating an example of an information processing device according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a block diagram illustrating an example of an apparatus for target tracking according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0016] The following describes exemplary embodiments of the present disclosure with reference to the drawings. For the sake of convenience, the specification does not show all features of the actual embodiments. However, when implementing the embodiments, those skilled in the art may make certain decisions to implement the embodiments, and these decisions may vary depending on the embodiment.

[0017] It should be noted that, for clarity of the present disclosure, the drawings only show device components and / or process steps closely related to the present disclosure, and omit details unrelated to the present disclosure.

[0018] It should be noted that the present disclosure is not limited to the described embodiments, as will be described below with reference to the accompanying drawings. In this specification, where feasible, the embodiments may be combined with each other, features of different embodiments may be substituted or utilized, or one or more features may be omitted in one embodiment.

[0019] Computer program code for carrying out operations of aspects of the exemplary embodiments disclosed herein may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0020] The methods of the present disclosure may be implemented by circuits having corresponding functional configurations, including circuits for a processor.

[0021] One aspect of the present disclosure relates to a method for target tracking. The inventors of the present disclosure have developed a target tracking method using the concept of anchor points. The method is particularly applicable to online cross-camera multi-target tracking. The method may be implemented by a computer. The method includes a local trajectory determination process, a feature pool update process, a clustering process, and a global target identifier determination process. Here, to determine multiple anchor points for assigning a global identifier to a trajectory, a clustering step is performed with a predetermined clustering period T (i.e., clustering is performed every time T, for example, clustering is performed every 5 seconds). The method of the present disclosure is applicable to fields such as autonomous driving, intelligent surveillance, and behavior recognition.

[0022] Fig. 1 is a flowchart illustrating an example of a method 100 for target tracking according to one embodiment of the present disclosure. The method 100 includes performing a clustering process with a predetermined clustering period T to determine a number of anchor points for assigning global identifiers for trajectories. A specific embodiment is illustrated in Fig. 1. Fig. 2 shows a schematic diagram of multiple trajectories for understanding the method 100.

[0023] In step Op101 (also referred to as "local trajectory determination process"), a plurality of local target trajectories having a local target identifier are determined based on a plurality of current frames at a current time tc provided by a plurality of cameras. One camera among the plurality of cameras may be represented as C[n], n∈[1,N], where N is a natural number representing the number of cameras. A frame provided by camera C[n] among the plurality of current frames may be represented as f[n][i], where i is a frame number corresponding to the acquisition time of the frame. For example, the current frame provided by camera C[n] may be represented as f[n][itc]. "Lid" in FIG. 2 indicates a local identifier.

[0024] FIG. 2 shows three local trajectories LTr[1][1], LTr[1][2], and LTr[2][1] obtained when N=2. Trajectory points P1 to P9 on the local trajectory LTr[1][2] are shown, with P9 corresponding to the trajectory point of the current target "L102." When target Tgx moves from region Rge[1] to Rge[2], it forms trajectories Tr1 and Tr2 in Rge[1] and Rge[2], respectively. Tr1 and Tr2 in the local target trajectory set have different local target identifiers.

[0025] The local target trajectory may be determined using a single-camera multi-target tracking model. The basic determination method may include detecting the position of a target of a certain type in the current frame f[n][itc] and associating the target at that position with one of the previously determined trajectories (i.e., local target ID) or initializing it as the starting trajectory point of the local trajectory of a new local target. The target position may be represented using a bounding box surrounding the target. For example, relevant parameters of the bounding box include width h, height w, coordinates (x, y) of the midpoint of the bottom edge, detection confidence c, target appearance features F, and frame acquisition time t. The bounding box has a one-to-one correspondence with the trajectory points. Therefore, each trajectory point has a corresponding appearance feature. Based on the coordinates, the target's coordinates in the world coordinate system can be obtained. By determining the local target identifier of the bounding box of each detected target in the current frame using a method such as position prediction, the trajectory points corresponding to each bounding box may be associated with the previously determined local target trajectory or a new local trajectory assigned a new local target identifier may be generated. That is, step Op101 may determine the local target identifier of the bounding box of each detected target in the current frame, and thereby add the trajectory points corresponding to these bounding boxes to the corresponding local trajectories according to their local target identifiers to obtain updated local trajectories. Therefore, step Op101 may be regarded as a process of updating the local trajectory set of each camera. The local trajectory point P may be described by a multidimensional number (h, w, x, y, c, F, t, Lid). Once the global identifier Gid of the trajectory point is determined, the global trajectory point may be described by a multidimensional number (h, w, x, y, c, F, t, Gid). The goal of method 100 is to determine the global target identifier Gid of each local trajectory point and realize cross-camera multi-target tracking. Single-camera multi-target tracking, which generates local target trajectories, is a conventional technique and will not be described here.

[0026] In step Op103 (also referred to as "feature pool update processing"), a first feature pool B1 and a second feature pool B2 are updated based on multiple local target trajectories. The first feature pool B1 includes sub-trajectory segment features Fr of the most recent sub-trajectory segment rsTr (e.g., the sub-trajectory segment formed by P7 to P8) of each local target trajectory (e.g., LTr[1][2]). The second feature pool B2 includes sub-trajectory segment features Fe of the earlier sub-trajectory segment esTr (e.g., the sub-trajectory segment formed by P4 to P6) of each local target trajectory. The sub-trajectory segment features may be representative features of the sub-trajectory segments. The representative features may be average features of the appearance features of each trajectory point on the sub-trajectory segment, such as a weighted average feature based on detection confidence or a feature determined by calculating the arithmetic mean value of the appearance features of multiple trajectory points. In other words, both the first feature pool and the second feature pool are collections of appearance features. The first feature pool contains appearance features associated with the most recent sub-trajectory segment and is also referred to as the "new appearance feature pool." The second feature pool contains appearance features associated with the earlier sub-trajectory segment and is also referred to as the "original feature pool."

[0027] In step Op105 (also referred to as "determination process"), it is determined whether the current time tc satisfies a time requirement for a predetermined clustering period T (e.g., 3, 5 seconds, or more). If the determination result is "YES," the clustering process shown in step Op107 is executed. For example, if tc - tcu ≥ T, the determination result is "YES," where tcu is the time when the previous clustering operation was performed. Alternatively, if |tc - tcu - T| ≥ δ, the determination result is "YES," where δ is a positive constant close to zero. An example of the time requirement is that the time interval between the current time and the time immediately before the previous clustering of features in the union of the first feature pool and the second feature pool to determine multiple preceding anchor points having corresponding preceding cluster appearance features is equal to or greater than the predetermined clustering period T. When performing real-time multi-target tracking, tracking results need to be obtained in real time or near real time, so a high processing speed is required. On the other hand, the clustering process takes time. If the clustering operation is performed too frequently, real-time tracking cannot be ensured. Therefore, the clustering period is set so that the clustering process is performed at regular intervals, rather than every time an image frame is received. The period T may be set based on at least one of the hardware processing capacity of the device, the number of cameras, the density of targets in the monitored area, and tracking accuracy requirements. One example of a clustering method is "hierarchical clustering."

[0028] In step Op107 (also referred to as "clustering process"), a plurality of current anchor points having corresponding current cluster appearance features are determined by clustering features in the union of the updated first feature pool B1 and the updated second feature pool B2. Each current anchor point A corresponds to one cluster. The current cluster appearance feature cF[j] of the current anchor point A[j] may be a representative appearance feature of the cluster CL[j], for example, an appearance feature corresponding to the cluster center of the cluster CL[j], or an appearance feature in the cluster CL[j] that is closest to the cluster center. Note that in the field of target tracking, clustering of appearance features is a conventional process.

[0029] In step Op109 (one aspect of the "global target identifier determination process"), global target identifiers of detected targets in the multiple current frames are determined based on the determined multiple current anchor points. In one example, the process includes: configuring the multiple current anchor points as a global anchor point set by assigning corresponding global target identifiers to the multiple current anchor points based on correspondence between the multiple current anchor points and the multiple previous anchor points (i.e., realizing mapping of global IDs between the previous anchor points and the current anchor points); and determining global target identifiers of detected targets in the multiple current frames based on the global anchor point set. In one example, the Hungarian algorithm is used to determine the correspondence between the multiple current anchor points and the multiple previous anchor points. In one example, the correspondence between the multiple current anchor points and the multiple previous anchor points is determined based on the cosine distance of pairs of cluster appearance features of the multiple current anchor points and the multiple previous anchor points. In another example, the correspondence between a plurality of current anchor points and a plurality of previous anchor points is determined based on the overlap between a set of features (S) from the updated second feature pool corresponding to the plurality of current anchor points and a set of features (S') from the pre-updated second feature pool corresponding to the plurality of previous anchor points. For example, a pair of anchor points corresponding to two sets with the highest overlap is selected as a corresponding anchor point pair, and the global target identifier of the current anchor point therein is set to the global identifier of the previous anchor point. In another example, if a first anchor point among the plurality of current anchor points and a second anchor point among the plurality of previous anchor points satisfy either one or both of a first condition and a second condition, the first anchor point is determined to correspond to a second anchor point. The first condition is that the cosine distance between the pair of cluster appearance features of the first anchor point and the second anchor point is equal to or less than a cosine distance threshold.The second condition is that the overlap between the set of multiple features from the updated second feature pool corresponding to the first anchor point and the set of multiple features from the pre-updated second feature pool corresponding to the second anchor point is greater than or equal to an overlap threshold.

[0030] If the determination result in step Op105 is "NO", execute step Op111 (another aspect of "global target identifier determination process"), that is, determine the global target identifier of the detected target in multiple current frames based on multiple previous anchor points. In one example, determine which anchor point in the anchor point set the detected target corresponds to (i.e., which global identifier the detected target corresponds to) by comparing the appearance features of the anchor points with the appearance features of the detected target.

[0031] A method for target tracking according to the present disclosure may include a step of determining whether to terminate target tracking. FIG. 3 is a flowchart illustrating an example of a method 300 for target tracking according to an embodiment of the present disclosure. As shown in FIG. 3, in addition to steps Op101 to Op111 identical to those of method 100, method 300 further includes step Op301 of determining whether an image is input within a predetermined waiting period. If an image is input, step Op101 is executed with the input image as the current frame. If no image is input, method 300 terminates. As can be seen from the flow of FIG. 3, method 300 periodically executes a process of updating an anchor point set for determining a global target identifier. Here, to achieve global association of different local trajectories and cross-camera target tracking, the global target identifier of the anchor point set before the update is inherited by the updated anchor point set.

[0032] In one embodiment, the nearest sub-trajectory fragment and the early sub-trajectory fragment of the local target trajectory are determined based on trajectory points where the appearance features suddenly change. Specifically, the start trajectory point of the nearest sub-trajectory segment is the trajectory point where the appearance features suddenly change relative to the early sub-trajectory segment. For example, in FIG. 2, the appearance feature F7 of the start trajectory point P7 of the nearest sub-trajectory segment rsTr suddenly changes relative to the early sub-trajectory segment esTr. Specifically, the difference between the appearance feature F7 and the sub-trajectory segment feature Fe of the early sub-trajectory segment esTr is greater than a predetermined level. For example, the cosine distance between F7 and Fe is greater than a predetermined level, or the Euclidean distance between F7 and Fe is greater than a predetermined level. An example of a sudden change in appearance features is as follows: The target Tgx moves early toward the camera C[1], and the corresponding process corresponds to the early sub-trajectory segment esTr. Then, at the time point corresponding to point P7, when the target Tgx suddenly turns its back to camera C[1] and camera C[1] captures the back image of the target Tgx for the first time, the appearance feature F7 of the target Tgx at the time point corresponding to point P7 (i.e., the appearance feature corresponding to the back image) suddenly changes from the appearance feature corresponding to the front image of Tgx. In other words, P7 is the trajectory point where the appearance feature suddenly changes. Then, as the target Tgx continues to move with its back to camera C[1], the corresponding process corresponds to the nearest sub-trajectory segment rsTr, and the starting point of this nearest sub-trajectory segment rsTr is P7.

[0033] In one embodiment, the sub-trajectory segment feature is the average feature of the trajectory points of the sub-trajectory segment. Therefore, when updating the first feature pool and the second feature pool, for each local target trajectory (e.g., LTr[1][2] in FIG. 2 ) among the multiple local target trajectories, the sub-trajectory segment feature Fr of the nearest sub-trajectory segment is the average feature of the appearance features from the nearest abrupt change trajectory point (P7) of the local target trajectory to the trajectory point (P8) immediately before the current trajectory point (P9) of the local target trajectory, and the sub-trajectory segment feature Fe of the early sub-trajectory segment (esTr) is the average feature of the appearance features from the early abrupt change trajectory point (P4) of the local target trajectory to the trajectory point (P6) immediately before the nearest abrupt change trajectory point (P7).

[0034] [Table 1]

[0035] [Table 2] Regarding the feature pool update process, an example of updates to local target trajectories is shown in Table 1 and Table 2. Here, the local target trajectories are LTr[1][2] in FIG. 2, and Table 1 shows the case where the difference exceeds a threshold, and Table 2 shows the case where the difference does not exceed the threshold. In one embodiment, the step of updating the first feature pool and the second feature pool based on the multiple local target trajectories includes the following steps: For the local target trajectory LTr, if the difference between the appearance feature Fc (corresponding to F9 in Table 1) of the current trajectory point Pc (corresponding to P9 in Table 1 and Figure 2) and the sub-trajectory segment feature Fr0 of the most recent sub-trajectory segment rsTr is greater than a predetermined level, the sub-trajectory segment feature Fr0 of the most recent sub-trajectory segment rsTr is removed from the first feature pool B1 and added to the second feature pool B2, the appearance feature Fc of the current trajectory point is added to the first feature pool B1, the most recent sub-trajectory segment rsTr of the trajectory LTr is updated to the trajectory segment containing the current trajectory point, and the earlier sub-trajectory segment esTr of the trajectory LTr is updated to the preceding most recent sub-trajectory segment (e.g., the trajectory segment indicated by P7-P8 in Table 1 and Figure 2). If the difference between the appearance feature Pc of the current trajectory point and the sub-trajectory segment feature Fr of the nearest sub-trajectory segment rsTr is equal to or less than a predetermined level, the nearest sub-trajectory segment rsTr of the local target trajectory LTr is updated to the trajectory segment to which the current trajectory point Pc was added (e.g., the trajectory segments P7 to P9 in Table 2 and FIG. 2), and the sub-trajectory segment feature of the nearest sub-trajectory segment rsTr in the first feature pool B1 is updated to the trajectory segment feature of the updated nearest sub-trajectory segment (e.g., as shown in Table 2, the appearance feature Fr0 in B1 is updated to Fr1). The sub-trajectory segment feature may be, for example, the feature average value (i.e., average characteristic) of the features of the trajectory points on the corresponding sub-trajectory segment.

[0036] If the second feature pool B2 contains too many features, the clustering process will take too long, which is unfavorable for real-time online tracking. In one embodiment, if the number of sub-trajectory segment features in the second feature pool B2 is greater than the feature number threshold Thn, after clustering the features in the union of the first feature pool B1 and the second feature pool B2, pairs of features in the second feature pool B2 whose feature similarity is equal to or greater than the feature similarity threshold are merged. For example, the two features are merged into a weighted average feature or an arithmetic average feature.

[0037] The method for target tracking according to the present disclosure uses a first feature pool and a second feature pool when determining a global target identifier. In the initial stage of the method according to the present disclosure, the reliability and accuracy of target tracking are reduced due to the small number of features in the first feature pool and the second feature pool. Therefore, a preparatory stage may be set for the method according to the present disclosure. In one embodiment, the method for target tracking includes a preparatory stage. The length of a period corresponding to the preparatory stage is at least twice the predetermined clustering period T (e.g., 3T, 4T, 5T, 6T). Furthermore, during the preparatory stage, updating the first feature pool B1 and the second feature pool B2 is performed, but clustering of features in the union of the updated first feature pool and the updated second feature pool is not performed (i.e., even if the current time meets the time requirement for the predetermined clustering period). The length of the period corresponding to the preparatory stage may be determined experimentally or empirically.

[0038] The method for target tracking according to the present disclosure uses a clustering threshold when performing clustering. The clustering threshold and the number of features in the feature pool both affect the performance of the method. To improve the performance of the method, the clustering threshold may be appropriately adjusted as the method is executed (the number of times the clustering process is executed increases). In one embodiment, as the number of features in the union of the first feature pool and the second feature pool increases, the clustering threshold for clustering is gradually increased from a base threshold to a predetermined upper threshold that is greater than the base threshold.

[0039] In one embodiment of the present disclosure, an apparatus for target tracking is provided. Figure 4 is a block diagram illustrating an example of an apparatus 400 for target tracking according to one embodiment of the present disclosure.

[0040] The apparatus 400 includes a memory 401 having instructions Inst stored therein, and at least one processor 403 coupled to the memory 401 and configured to execute the instructions to implement a method for target tracking (e.g., method 100 or 300).

[0041] Another aspect of the present disclosure provides a computer-readable non-transitory storage medium having a program stored thereon. When the program is executed by a computer, the computer performs the following steps: determining a plurality of local target trajectories having a local target identifier based on a plurality of current frames at a current time provided by a plurality of cameras; updating a first feature pool including sub-trajectory segment features of a most recent sub-trajectory segment of each local target trajectory and a second feature pool including sub-trajectory segment features of an earlier sub-trajectory segment of each local target trajectory based on the plurality of local target trajectories; if the current time satisfies a time requirement for a predetermined clustering period, determining a plurality of current anchor points having corresponding current cluster appearance features by clustering features in a union of the updated first feature pool and the updated second feature pool; and determining a global target identifier of a target detected in the plurality of current frames based on the plurality of current anchor points. For further details of the program, please refer to the description of the method for target tracking according to the present disclosure (e.g., method 100 or 300).

[0042] One aspect of the present disclosure further provides an information processing device.

[0043] Fig. 5 is a block diagram showing an example of an information processing device 500 according to an embodiment of the present disclosure. In Fig. 5, a central processing unit (CPU) 501 executes various processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 stores data necessary for the CPU 501 to execute various processes as needed.

[0044] The CPU 501, ROM 502, and RAM 503 are connected to one another via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0045] An input unit 506 (including a keyboard, a mouse, etc.), an output unit 507 (including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), a storage unit 508 (including, for example, a hard disk, etc.), and a communication unit 509 (including a network interface card, such as a LAN card, a modem, etc.) are connected to the input / output interface 505. The communication unit 509 performs communication processing via a network, such as the Internet, a local area network, a mobile network, or a combination thereof.

[0046] The driver 510 may be connected to the input / output interface 505 as needed. The removable medium 511 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is set up in the driver 510 as needed, and a computer program read from the removable medium 511 is installed in the storage unit 508 as needed.

[0047] The CPU 501 may execute a program corresponding to a method for target tracking.

[0048] An embodiment of the present disclosure further provides an apparatus for target tracking. Hereinafter, an exemplary description will be given with reference to FIG. 6. FIG. 6 is a block diagram illustrating an example of an apparatus 600 for target tracking according to an embodiment of the present disclosure. The apparatus 600 includes a local trajectory determiner 601, an updater 603, a determiner 605, a clustering unit 607, and a global target identifier determiner 609. The local trajectory determiner 601 determines multiple local target trajectories having local target identifiers based on multiple current frames at a current time point provided by multiple cameras. The updater 603 updates a first feature pool including sub-trajectory segment features of recent sub-trajectory segments of each local target trajectory and a second feature pool including sub-trajectory segment features of earlier sub-trajectory segments of each local target trajectory based on the multiple local target trajectories. The determiner 605 determines whether the current time point satisfies a time requirement for a predetermined clustering period. If the determination result of the determination unit 605 is “YES,” the clustering unit 607 determines a plurality of current anchor points having corresponding current cluster appearance features by clustering features in the union of the updated first feature pool and the updated second feature pool. The global target identifier determination unit 609 determines global target identifiers of targets detected in a plurality of current frames based on the plurality of current anchor points. Here, after the plurality of current anchor points are determined by clustering, the global anchor point set is updated by assigning corresponding global target identifiers to the plurality of current anchor points based on the correspondence between the plurality of current anchor points and a plurality of previous anchor points, and the anchor points in the global anchor point set are the anchor points generated by the clustering process. For further configuration of the device 600, please refer to the description of the method for target tracking according to the present disclosure (e.g., method 100 or 300).

[0049] [Table 3]

[0050] [Table 4] To verify the effectiveness of the target tracking method of the present disclosure, the inventors tested the performance of the target tracking method of the present disclosure using the public test datasets AICity Challenge (AIC) and FRJ video dataset, as well as the self-created video dataset FRDCCrowd (including FRDCCrowd20, FRDCCrowd40, and FRDCCrowd60), and compared it with the performance of the prior art, as shown in Tables 3 and 4. The test results shown in Table 3 indicate that the online method of the present disclosure maintains accuracy compared to offline methods and can run in real time (300 FPS). The test results shown in Table 4 indicate that the online method of the present disclosure improves accuracy compared to the original tracking method. When using the FRDCCrowd dataset, the online method of the present disclosure slightly degraded in IDF1 metric compared to the offline method because it did not use the entire video during tracking, but this performance was still within an acceptable range.

[0051] The advantageous effects of the method, apparatus, and storage medium according to the present disclosure include at least one of the following advantageous effects: Real-time online target tracking and better target tracking accuracy can be achieved.

[0052] As described above, the present disclosure has described the principles of online tracking. However, the advantages of the present disclosure are not necessarily limited to the above advantages, and any of the advantages set forth herein may be obtained or other advantages may be understood from the present disclosure in addition to or instead of the advantages described in the above paragraphs.

[0053] Although the specific embodiments of the present disclosure have been described above, those skilled in the art may make various modifications (combining or substituting the features of the respective embodiments in the case of modifications), improvements, or equivalents to the present disclosure within the spirit and scope of the appended claims. These modifications, improvements, or equivalents fall within the scope of protection of the present disclosure.

[0054] It should be noted that the terms "comprise" and "have" refer to the presence of features, elements, steps or components described in this specification, but do not exclude the presence or addition of one or more other features, elements, steps or components.

[0055] Furthermore, the methods of each embodiment of the present invention are not limited to being performed in the chronological order described in the specification or shown in the drawings, and may be performed in other chronological orders, or may be performed in parallel or independently. Therefore, the order of performing the methods described herein does not limit the technical scope of the present invention.

[0056] Furthermore, the following supplementary notes are also disclosed regarding the embodiments including the above-described examples, but the present invention is not limited to these supplementary notes. (Appendix 1) 1. A method for target tracking, comprising: determining a plurality of local target trajectories having local target identifiers based on a plurality of current frames at a current time provided by a plurality of cameras; updating a first feature pool including sub-trajectory segment features of a recent sub-trajectory segment of each local target trajectory and a second feature pool including sub-trajectory segment features of an earlier sub-trajectory segment of each local target trajectory based on the plurality of local target trajectories; If the current time satisfies the time requirement for a given clustering period, determining a plurality of current anchor points having corresponding current cluster appearance features by clustering features in a union of the updated first feature pool and the updated second feature pool; determining global target identifiers of detected targets in the plurality of current frames based on the plurality of current anchor points. (Appendix 2) 2. The method of claim 1, wherein the starting trajectory point of the most recent sub-trajectory segment is a trajectory point where a sudden change occurs in appearance features relative to the earlier sub-trajectory segment. (Appendix 3) 2. The method of claim 1, wherein the time requirement is that the time interval between the current time point and the time point immediately before the previous time when features in the union of the first feature pool and the second feature pool were clustered to determine a plurality of preceding anchor points having corresponding preceding cluster appearance features is equal to or greater than the predetermined clustering period. (Appendix 4) determining global target identifiers of detected targets in the current frames based on the current anchor points; configuring the plurality of current anchor points as a global anchor point set by assigning corresponding global target identifiers to the plurality of current anchor points based on a correspondence relationship between the plurality of current anchor points and a plurality of previous anchor points; and determining global target identifiers for detected targets in the current frames based on the global anchor point set. (Appendix 5) For each local target trajectory among the plurality of local target trajectories, The sub-trajectory segment feature of the nearest sub-trajectory segment is an average feature of appearance features from the nearest sudden change trajectory point of the local target trajectory to the trajectory point immediately before the current trajectory point; The sub-trajectory segment feature of the early sub-trajectory segment is an average feature of appearance features from an early abrupt change trajectory point of the local target trajectory to a trajectory point immediately before the latest abrupt change trajectory point; 2. The method of claim 1, wherein a difference between the appearance feature of the most recent abrupt change trajectory point and the sub-trajectory segment feature of the early sub-trajectory segment is greater than a predetermined level. (Appendix 6) The step of updating the first feature pool and the second feature pool includes, for one local target trajectory among the plurality of local target trajectories: If a difference between the appearance feature of the current trajectory point and the sub-trajectory segment feature of the nearest sub-trajectory segment of the local target trajectory is greater than a predetermined level, moving the sub-trajectory segment feature of the nearest sub-trajectory segment from the first feature pool to the second feature pool, and adding the appearance feature of the current trajectory point to the first feature pool; and updating the sub-trajectory segment feature of the nearest sub-trajectory segment of the local target trajectory to the trajectory segment to which the current trajectory point was added, and updating the sub-trajectory segment feature of the nearest sub-trajectory segment of the local target trajectory in the first feature pool to the trajectory segment feature of the updated nearest sub-trajectory segment, if a difference between the appearance feature of the current trajectory point and the sub-trajectory segment feature of the nearest sub-trajectory segment of the local target trajectory is less than or equal to a predetermined level. (Appendix 7) 2. The method of claim 1, wherein if the number of sub-trajectory segment features in the second feature pool is greater than a feature count threshold, after clustering the features in the union of the first feature pool and the second feature pool, pairs of features in the second feature pool whose feature similarity is greater than or equal to a feature similarity threshold are merged. (Appendix 8) 5. The method of claim 4, wherein the correspondence between the current anchor points and the previous anchor points is determined using a Hungarian algorithm. (Appendix 9) 5. The method of claim 4, wherein the correspondence between the current anchor points and the previous anchor points is determined based on cosine distances between pairs of cluster appearance features of the current anchor points and the previous anchor points. (Appendix 10) 5. The method of claim 4, wherein a correspondence relationship between the current anchor points and the previous anchor points is determined based on an overlap between a set of features from the updated second feature pool corresponding to the current anchor points and a set of features from the pre-updated second feature pool corresponding to the previous anchor points. (Appendix 11) determining that a first anchor point among the plurality of current anchor points and a second anchor point among the plurality of previous anchor points corresponds to the second anchor point if either one or both of a first condition and a second condition are satisfied for the first anchor point and the second anchor point; the first condition is that a cosine distance between a pair of cluster appearance features of the first anchor point and the second anchor point is equal to or less than a cosine distance threshold; 5. The method of claim 4, wherein the second condition is that an overlap between a set of features from the updated second feature pool corresponding to the first anchor point and a set of features from the pre-updated second feature pool corresponding to the second anchor point is greater than or equal to an overlap threshold. (Appendix 12) The method further comprises a preparation step: the length of the period corresponding to the preparation stage is longer than twice the predetermined clustering period; 2. The method of claim 1, wherein, in the preparation step, updating the first feature pool and the second feature pool is performed, and clustering features in the union of the updated first feature pool and the updated second feature pool is not performed. (Appendix 13) 2. The method of claim 1, wherein a clustering threshold for clustering is gradually increased from a base threshold to a predetermined upper threshold that is greater than the base threshold as the number of features in the union of the first feature pool and the second feature pool increases. (Appendix 14) 1. An apparatus for target tracking, comprising: a memory in which instructions are stored; at least one processor coupled to the memory and configured to execute the instructions to implement a method according to any of claims 1 to 13. (Appendix 15) A computer-readable non-transitory storage medium storing a program, the program being executed by a computer to determining a plurality of local target trajectories having local target identifiers based on a plurality of current frames at a current time provided by a plurality of cameras; updating a first feature pool including sub-trajectory segment features of a recent sub-trajectory segment of each local target trajectory and a second feature pool including sub-trajectory segment features of an earlier sub-trajectory segment of each local target trajectory based on the plurality of local target trajectories; If the current time satisfies the time requirement for a given clustering period, determining a plurality of current anchor points having corresponding current cluster appearance features by clustering features in a union of the updated first feature pool and the updated second feature pool; determining global target identifiers of detected targets in the plurality of current frames based on the plurality of current anchor points. (Appendix 16) 16. The storage medium of claim 15, wherein the start trajectory point of the most recent sub-trajectory segment is a trajectory point where a sudden change occurs in appearance features relative to the earlier sub-trajectory segment. (Appendix 17) 16. The storage medium of claim 15, wherein the time requirement is that the time interval between the current time point and the time point immediately before the previous time when features in the union of the first feature pool and the second feature pool were clustered to determine a plurality of preceding anchor points having corresponding preceding cluster appearance features is equal to or greater than the predetermined clustering period. (Appendix 18) determining global target identifiers of detected targets in the current frames based on the current anchor points; configuring the plurality of current anchor points as a global anchor point set by assigning corresponding global target identifiers to the plurality of current anchor points based on a correspondence relationship between the plurality of current anchor points and a plurality of previous anchor points; determining global target identifiers for detected targets in the current frames based on the global anchor point set. (Appendix 19) For each local target trajectory among the plurality of local target trajectories, The sub-trajectory segment feature of the nearest sub-trajectory segment is an average feature of appearance features from the nearest sudden change trajectory point of the local target trajectory to the trajectory point immediately before the current trajectory point; The sub-trajectory segment feature of the early sub-trajectory segment is an average feature of appearance features from an early abrupt change trajectory point of the local target trajectory to a trajectory point immediately before the latest abrupt change trajectory point; 16. The storage medium of claim 15, wherein a difference between the appearance feature of the most recent abrupt change trajectory point and the sub-trajectory segment feature of the early sub-trajectory segment is greater than a predetermined level. (Appendix 20) 16. The storage medium of claim 15, wherein if the number of sub-trajectory segment features in the second feature pool is greater than a feature count threshold, after clustering the features in the union of the first feature pool and the second feature pool, pairs of features in the second feature pool whose feature similarity is greater than or equal to a feature similarity threshold are merged.

Claims

1. 1. A method for target tracking, comprising: determining a plurality of local target trajectories having local target identifiers based on a plurality of current frames at a current time provided by a plurality of cameras; updating a first feature pool including sub-trajectory segment features of a recent sub-trajectory segment of each local target trajectory and a second feature pool including sub-trajectory segment features of an earlier sub-trajectory segment of each local target trajectory based on the plurality of local target trajectories; If the current time satisfies the time requirement for a given clustering period, determining a plurality of current anchor points having corresponding current cluster appearance features by clustering features in a union of the updated first feature pool and the updated second feature pool; determining global target identifiers of detected targets in the plurality of current frames based on the plurality of current anchor points.

2. The method of claim 1 , wherein the starting trajectory point of the most recent sub-trajectory segment is a trajectory point where a sudden change occurs in appearance features relative to the earlier sub-trajectory segment.

3. 2. The method of claim 1, wherein the time requirement is that a time interval between the current time point and a time point immediately before a previous time when features in a union of the first feature pool and the second feature pool are clustered to determine a plurality of preceding anchor points having corresponding preceding cluster appearance features is equal to or greater than the predetermined clustering period.

4. determining global target identifiers of detected targets in the current frames based on the current anchor points; configuring the plurality of current anchor points as a global anchor point set by assigning corresponding global target identifiers to the plurality of current anchor points based on a correspondence relationship between the plurality of current anchor points and a plurality of previous anchor points; and determining global target identifiers for detected targets in the plurality of current frames based on the set of global anchor points.

5. For each local target trajectory among the plurality of local target trajectories, The sub-trajectory segment feature of the nearest sub-trajectory segment is an average feature of appearance features from the nearest sudden change trajectory point of the local target trajectory to the trajectory point immediately before the current trajectory point; The sub-trajectory segment feature of the early sub-trajectory segment is an average feature of appearance features from an early abrupt change trajectory point of the local target trajectory to a trajectory point immediately before the latest abrupt change trajectory point; The method of claim 1 , wherein a difference between the appearance feature of the most recent abrupt change trajectory point and the sub-trajectory segment feature of the early sub-trajectory segment is greater than a predetermined level.

6. 2. The method of claim 1, wherein if the number of sub-trajectory segment features in the second feature pool is greater than a feature count threshold, after clustering features in the union of the first feature pool and the second feature pool, pairs of features in the second feature pool whose feature similarity is greater than or equal to a feature similarity threshold are merged.

7. determining that a first anchor point among the plurality of current anchor points and a second anchor point among the plurality of previous anchor points corresponds to the second anchor point if either one or both of a first condition and a second condition are satisfied; the first condition is that a cosine distance between a pair of cluster appearance features of the first anchor point and the second anchor point is equal to or less than a cosine distance threshold; 5. The method of claim 4, wherein the second condition is that an overlap between a set of features from the updated second feature pool corresponding to the first anchor point and a set of features from the pre-updated second feature pool corresponding to the second anchor point is greater than or equal to an overlap threshold.

8. 2. The method of claim 1, wherein a clustering threshold for clustering is gradually increased from a base threshold to a predetermined upper threshold that is greater than the base threshold as the number of features in the union of the first feature pool and the second feature pool increases.

9. 1. An apparatus for target tracking, comprising: a memory in which instructions are stored; and at least one processor coupled to said memory and configured to execute said instructions to implement a method according to any one of claims 1 to 8.

10. A computer-readable non-transitory storage medium storing a program, the program being executed by a computer to determining a plurality of local target trajectories having local target identifiers based on a plurality of current frames at a current time provided by a plurality of cameras; updating a first feature pool including sub-trajectory segment features of a recent sub-trajectory segment of each local target trajectory and a second feature pool including sub-trajectory segment features of an earlier sub-trajectory segment of each local target trajectory based on the plurality of local target trajectories; If the current time satisfies the time requirement for a given clustering period, determining a plurality of current anchor points having corresponding current cluster appearance features by clustering features in a union of the updated first feature pool and the updated second feature pool; determining global target identifiers of detected targets in the plurality of current frames based on the plurality of current anchor points.

Images