Multi-target tracking method and apparatus

Abstract

This invention relates to the field of computer vision technology and discloses a multi-target tracking method and apparatus. The method includes: acquiring a video frame sequence; performing target detection on each frame in the video frame sequence to obtain a target detection box sequence; and extracting the ReID feature vector corresponding to each target detection box sequence; constructing a spatiotemporally adaptive threshold surface based on the target detection box sequence; forming an initial trajectory group based on the target detection box sequence and the corresponding ReID feature vector; performing iterative graph clustering and merging on the initial trajectory group based on the spatiotemporally adaptive threshold surface to obtain an iterative trajectory group, and using the final trajectory group as the target tracking result. This invention achieves dynamic adaptive adjustment of similarity requirements based on the spatiotemporal distance between targets by constructing a spatiotemporally adaptive threshold surface instead of a fixed threshold; furthermore, it employs an iterative graph clustering and merging strategy to globally optimize and progressively merge trajectory segments, significantly enhancing the continuity and integrity of the trajectory.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a multi-target tracking method and apparatus. Background Technology

[0002] Existing multi-target tracking methods typically rely on fixed similarity thresholds for target association, which makes it difficult to adapt to the dynamic changes in feature similarity of targets at different spatiotemporal distances, easily leading to false matches and identity switching. At the same time, they lack explicit modeling of spatiotemporal physical constraints, making it impossible to effectively filter out erroneous associations caused by movement speeds exceeding reasonable ranges. Furthermore, most methods employ greedy or frame-by-frame matching strategies, lacking the ability to globally optimize trajectory segments, which can easily lead to trajectory breakage and fragmentation when the target is occluded or its appearance changes, affecting the continuity and accuracy of long-term tracking. Summary of the Invention

[0003] This invention provides a multi-target tracking method and apparatus to solve the problems of unstable tracking accuracy and discontinuous long-term tracking caused by the use of fixed similarity thresholds and lack of spatiotemporal physical constraint modeling in the prior art.

[0004] In a first aspect, the present invention provides a multi-target tracking method, the method comprising:

[0005] Obtain a video frame sequence, perform object detection on each frame in the video frame sequence to obtain an object detection box sequence, and extract the ReID feature vector corresponding to each object detection box sequence; Based on the object detection box sequence, a spatiotemporal adaptive threshold surface is constructed; An initial trajectory group is formed based on the target detection box sequence and the corresponding ReID feature vector; Based on the spatiotemporal adaptive threshold surface, iterative graph clustering and merging are performed on the initial trajectory group to obtain the iterative trajectory group, and the final trajectory group is used as the target tracking result.

[0006] This invention provides a multi-target tracking method that, by constructing a spatiotemporally adaptive threshold surface to replace a fixed threshold, achieves dynamic adaptive adjustment of similarity requirements based on the spatiotemporal distance between targets, effectively improving matching accuracy. Furthermore, it employs an iterative graph clustering and merging strategy to globally optimize and progressively merge trajectory segments, significantly enhancing the continuity and integrity of the trajectory. Ultimately, this method achieves more stable and accurate long-term multi-target tracking in complex scenarios, solving the problems of unstable tracking accuracy and discontinuous long-term tracking caused by the use of fixed similarity thresholds and lack of spatiotemporal physical constraint modeling in existing technologies.

[0007] In one optional implementation, a spatiotemporally adaptive threshold surface is constructed based on the target detection box sequence, including: Determine the rules for constraining abnormal speeds; Based on the abnormal velocity constraint rules, a set of data points covering different combinations of spatial distance and time interval is generated using a pre-defined bivariate function model. An interpolator is constructed based on the data point set to generate a spatiotemporally adaptive threshold surface.

[0008] This invention provides a multi-target tracking method that, by introducing abnormal speed constraint rules and using physical feasibility as a hard judgment condition, effectively eliminates erroneous matching caused by unreasonable situations such as instantaneous movement. Simultaneously, it utilizes a bivariate function model to generate data points covering all spatiotemporal combinations, enabling the similarity threshold to continuously and smoothly adapt to changes in spatial distance and time intervals, overcoming the poor adaptability of fixed thresholds. Finally, an interpolator transforms this model into a quickly queried surface, ensuring the accuracy of decision-making.

[0009] In one alternative implementation, determining the abnormal velocity constraint rules includes: Determine the spatial distance and time interval between the two target detection boxes whose association relationship is to be determined; Based on spatial distance and time interval, a dynamic similarity threshold is calculated using a pre-defined bivariate function model. When the ratio of spatial distance to time interval exceeds the preset maximum speed threshold, the corresponding dynamic similarity threshold is set to a predetermined value for forcibly rejecting association.

[0010] The present invention provides a multi-target tracking method that directly integrates abnormal velocity constraint rules into the similarity threshold calculation process. Based on traditional appearance matching, it introduces physical reachability criteria to effectively identify and filter erroneous associated candidates caused by the instantaneous displacement of the target exceeding a reasonable range. Thus, in the matching stage, it actively avoids associating detection boxes that are physically impossible to be the same target.

[0011] In one optional implementation, constructing a spatiotemporally adaptive threshold surface based on the target detection box sequence further includes: Based on a preset time similarity configuration table, a corresponding similarity threshold is configured for a specified time interval; the similarity threshold in the preset time similarity configuration table increases with the increase of the time interval.

[0012] The present invention provides a multi-target tracking method that effectively compensates for the matching uncertainty caused by the decay or change of target appearance over time by presetting higher similarity requirements for longer time intervals, thereby significantly improving the success rate of trajectory association over long time spans and the robustness of long-term tracking.

[0013] In one optional implementation, an initial trajectory group is formed based on the target detection box sequence and the corresponding ReID feature vector, including: Based on the temporal order of image frames in the target detection box sequence, all target detection boxes belonging to the same image frame and their corresponding ReID feature vectors are divided into independent initial groups. Each initial group is assigned an independent group identifier to generate an initial trajectory group. The multi-target tracking method provided by this invention organizes detection results in chronological order and constructs initial trajectory groups in units of image frames. From the very beginning of processing, it strictly maintains the basic spatiotemporal continuity of the target, providing a clearly structured and spatiotemporally defined processing unit for subsequent global optimization. This effectively avoids erroneous associations and trajectory fragmentation caused by loose data organization in the initial stage.

[0014] In one optional implementation, based on a spatiotemporal adaptive threshold surface, iterative graph clustering and merging are performed on the initial trajectory group to obtain an iterative trajectory group, including: Calculate the feature similarity between each trajectory group and select trajectory pairs with similarity higher than the current iteration threshold; Spatiotemporal conflict detection is performed on each trajectory pair based on the spatiotemporal adaptive threshold surface, and a valid association edge is established between the two trajectory pairs based on the conflict detection results. Construct a graph structure with all trajectory groups as nodes and effective associated edges as connections, and extract the connected components in the graph structure; the graph structure is an undirected weighted graph. Perform a merging operation on the trajectory groups within each connected component, update the trajectory groups after each iteration, and use them as input for the next iteration.

[0015] This invention provides a multi-target tracking method that progressively filters candidate associations by using iterative thresholds from high to low, and uses a spatiotemporally adaptive threshold surface to verify the physical rationality of each candidate association pair, effectively ensuring the reliability of each merging. Furthermore, it achieves global optimization and merging of trajectory groups through graph connectivity component analysis, breaking the limitations of traditional frame-by-frame matching, thereby significantly reducing trajectory fragmentation and identity switching caused by occlusion and appearance changes, and ultimately forming a longer and more complete continuous trajectory.

[0016] In one optional implementation, spatiotemporal conflict detection is performed on each trajectory pair based on a spatiotemporal adaptive threshold surface, and a determination is made based on the conflict detection results as to whether a valid association edge is established between two trajectory pairs in the trajectory pair, including: Obtain the time interval and spatial distance between temporally adjacent target detection boxes in trajectory pairs; Based on the spatiotemporal adaptive threshold surface, the required similarity threshold corresponding to the time interval and spatial distance is determined; If the feature similarity of the trajectory pair reaches the required similarity threshold, and the time interval and spatial distance do not trigger the abnormal speed constraint rules, then a valid association edge is established between the two trajectory pairs in the trajectory pair.

[0017] The present invention provides a multi-target tracking method that achieves a precise correspondence between the matching standard and the actual motion state of the target by querying adaptive thresholds with precisely quantified spatiotemporal parameters, and simultaneously performs strict physical velocity verification. This directly eliminates erroneous candidates with unreasonable spatiotemporal relationships at the feature matching level, fundamentally avoiding the trajectory mis-merging problem caused by ignoring physical constraints in traditional methods.

[0018] In one optional implementation, a merging operation is performed on the trajectory groups within each connected component to update the trajectory groups after each iteration, including: Merge members of all trajectory groups within the same connected component; Based on the group identifier of the members, all members after merging are regrouped, and the ReID feature vector of each new group is extracted; The ReID feature vectors are subjected to secondary clustering, and new trajectory groups are generated based on the secondary clustering results, which serve as the trajectory groups updated in each iteration.

[0019] The present invention provides a multi-target tracking method that, before merging trajectory groups within connected components, first regroups and performs secondary clustering based on the original member identifiers, and performs rigorous verification and fine segmentation on the preliminary association results. This effectively avoids the problem of excessive merging that may be caused by single graph clustering, thereby enhancing trajectory continuity while ensuring a high degree of consistency of features within each final trajectory group.

[0020] In one optional implementation, a merging operation is performed on the trajectory groups within each connected component to update the trajectory groups after each iteration, further comprising: After performing a merging operation on the trajectory groups within the connected components, the representative ReID feature vector of the merged trajectory group is updated. The updated representative ReID feature vector is the mean vector calculated based on the feature vectors of multiple filtered members within the merged trajectory group, without normalization.

[0021] The present invention provides a multi-target tracking method that dynamically updates the representative features of the trajectory group based on the selected high-quality member features after each merging, and adopts a non-normalized mean fusion strategy, so that the updated feature vector can more robustly represent the common appearance of the target in the recent motion process. At the same time, it utilizes the natural formation of modulus difference to suppress low-quality matching, thereby effectively improving the consistency within the trajectory group in subsequent iterative matching.

[0022] In a second aspect, the present invention provides a multi-target tracking device, the device comprising: The target detection and feature extraction module is used to acquire video frame sequences, perform target detection on each frame in the video frame sequence to obtain a sequence of target detection boxes, and extract the ReID feature vector corresponding to each target detection box sequence. The threshold surface construction module is used to construct a spatiotemporally adaptive threshold surface based on the target detection box sequence; The initial grouping module is used to form initial trajectory groups based on the target detection box sequence and the corresponding ReID feature vectors; The iterative graph clustering processing module is used to perform iterative graph clustering and merging on the initial trajectory group based on the spatiotemporal adaptive threshold surface, to obtain the iterative trajectory group, and the final trajectory group is used as the target tracking result.

[0023] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the multi-target tracking method described in the first aspect or any corresponding embodiment thereof.

[0024] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the multi-target tracking method described in the first aspect or any of its corresponding embodiments.

[0025] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the multi-target tracking method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0026] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first process of a multi-target tracking method according to an embodiment of the present invention; Figure 3 This is a second flowchart illustrating a multi-target tracking method according to an embodiment of the present invention; Figure 4This is a schematic diagram of the third process of the multi-target tracking method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the fourth process of the multi-target tracking method according to an embodiment of the present invention; Figure 6 This is a structural block diagram of a multi-target tracking device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0030] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0031] As an optional application scenario of this invention, such as Figure 1 As shown, application 101 is installed in terminal device 110, and user 130 can interact with application 101 through terminal device 110 and / or access device of terminal device 110.

[0032] For example, application 101 can be any application that provides question-and-answer related services. For instance, application 101 could be a question-and-answer interactive application, such as a text-to-text application, an image-to-text application, etc. Figure 1 In the application scenario shown, if application 101 is active, the terminal device 110 can display the interface 102 of application 101. The interface 102 may include various pages that application 101 can provide, such as interactive pages, settings pages, query pages, etc.

[0033] In some embodiments, terminal device 110 is communicatively connected to server 120 to provide services to application 101. Terminal device 110 may be a mobile terminal, fixed terminal, or portable terminal, etc., including but not limited to mobile phones, desktop computers, laptop computers, multimedia tablets, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 may also support any type of interface, and server 120 may be various types of computing systems or servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0034] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this invention.

[0035] The embodiments of the present invention will now be described with reference to the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and various page designs are possible in practice. The various graphic elements on the page may have different arrangements and different visual representations; one or more elements may be omitted or replaced, and one or more other elements may also be present, without any limitation in the embodiments of the present invention. Furthermore, the embodiments described below primarily pertain to terminal device 110. It should be understood that the actions described relative to terminal device 110 can be performed by application 101 on terminal device 110, or can be performed by application 101 in conjunction with its server (e.g., server 120).

[0036] According to an embodiment of the present invention, a multi-target tracking method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] This embodiment provides a multi-target tracking method, which can be used in the aforementioned electronic devices or terminal devices. Figure 2 This is a flowchart of a multi-target tracking method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the video frame sequence, perform target detection on each frame in the video frame sequence to obtain the target detection box sequence, and extract the ReID feature vector corresponding to each target detection box sequence.

[0038] Among them, a video frame sequence refers to multiple image frames captured by one or more video acquisition devices (such as surveillance cameras) and arranged in chronological order.

[0039] ReID feature vectors are highly discriminative, high-dimensional numerical codes extracted from target images using deep learning models. For example, when a target (such as a pedestrian) is detected, the system crops its image region and inputs it into a pre-trained re-identification neural network. This network acts as a feature extractor, converting the image into a fixed-length (e.g., 512-dimensional) numerical vector.

[0040] First, the input video frame sequence is processed chronologically. A target detection model (such as the YOLO detection model) is used to locate and select all targets to be tracked from each image frame, forming a sequence of target detection boxes. Then, for the target image region cropped from each detection box, a re-identification network (such as a ReID network) is used to extract its high-dimensional ReID feature vector. This process provides structured data that combines accurate spatiotemporal location information with highly discriminative appearance features.

[0041] Step S202: Construct a spatiotemporal adaptive threshold surface based on the target detection box sequence.

[0042] Among them, the spatiotemporal adaptive threshold surface is a pre-generated mathematical surface that can be quickly queried. Its function is to dynamically determine the minimum similarity threshold that must be reached for a current association for any pair of target detection boxes that are spatially separated by a specific distance and temporally separated by a specific time interval.

[0043] Step S203: Based on the target detection box sequence and the corresponding ReID feature vector, an initial trajectory group is formed.

[0044] Specifically, the input to this step is the sequence of target detection boxes and the corresponding ReID feature vectors. The specific operation is as follows: by strictly following the timestamp order of the video frames, all targets detected in each frame and their ReID feature vectors are automatically aggregated into an independent initial trajectory group, thereby establishing the basic unit of trajectory segments with time as the axis at the beginning of the algorithm.

[0045] Step S204: Based on the spatiotemporal adaptive threshold surface, perform iterative graph clustering and merging on the initial trajectory group to obtain the iterative trajectory group, and use the final trajectory group as the target tracking result.

[0046] Specifically, a progressive global optimization strategy is adopted, which iterates through multiple rounds by gradually relaxing the similarity threshold from high to low. In each iteration: first, the feature similarity between trajectory groups is calculated and candidate pairs are selected; then, a spatiotemporal adaptive threshold surface is used to verify physical rationality to establish reliable associations; subsequently, a graph model is constructed and connected components are extracted to identify merging trajectory group clusters; finally, fine-grained merging and feature updates are performed on each cluster. This process is repeated cyclically, gradually integrating the fragmented initial trajectories into fewer but more complete and reliable continuous trajectories, thereby outputting accurate and stable long-term multi-target tracking results.

[0047] The multi-target tracking method provided in this embodiment achieves dynamic adaptive adjustment of similarity requirements based on the spatiotemporal distance between targets by constructing a spatiotemporally adaptive threshold surface instead of a fixed threshold, effectively improving matching accuracy. Furthermore, it employs an iterative graph clustering and merging strategy to globally optimize and progressively merge trajectory segments, significantly enhancing the continuity and integrity of the trajectory. This method ultimately achieves more stable and accurate long-term multi-target tracking in complex scenarios, solving the problems of unstable tracking accuracy and discontinuous long-term tracking caused by the use of fixed similarity thresholds and lack of spatiotemporal physical constraint modeling in existing technologies.

[0048] This embodiment provides a multi-target tracking method, which can be used in the aforementioned electronic devices or terminal devices. Figure 3 This is a flowchart of a multi-target tracking method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the video frame sequence, perform object detection on each frame in the video frame sequence to obtain the object detection box sequence, and extract the ReID feature vector corresponding to each object detection box sequence. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0049] Step S302: Construct a spatiotemporal adaptive threshold surface based on the target detection box sequence.

[0050] Specifically, step S302 includes: Step S3021: Determine the abnormal speed constraint rules.

[0051] In some optional implementations, step S3021 above includes: Step a1: Determine the spatial distance and time interval between the two target detection boxes whose association relationship is to be determined.

[0052] Specifically, define two variables: Spatial distance (ds): The normalized spatial distance between two object detection boxes (e.g., in human body length units).

[0053] Difference Time (dt): The time interval (in seconds) between the appearance of two object detection boxes.

[0054] Scene Classification: Low-traffic scenarios (number of people < 10): MIN = 0.60, MAX = 0.75 - K = 0.4 (more emphasis on space) - number of iterations = 30; Medium-sized crowds (10 ≤ number of people < 50): MIN = 0.63, MAX = 0.80 - K = 0.5 (space-time balance) - number of iterations = 40; High-traffic scenarios (number of people ≥ 50): MIN=0.65, MAX=0.82 - K=0.6 (more emphasis on time) - number of iterations=50.

[0055] First, based on the object detection results, the center pixel coordinates and their corresponding timestamps of the two detection boxes whose relationship needs to be determined are obtained. Spatial distance is typically calculated by obtaining the raw pixel distance from the Euclidean distance between the two center point coordinates, and then normalized to a physical distance in body length units according to a predefined reference scale (such as average human height or scene calibration information). The time interval is directly obtained by calculating the difference between the two timestamps, usually in seconds. This process provides accurate and quantified spatiotemporal relationship input for subsequent queries of the spatiotemporal adaptive threshold surface.

[0056] Step a2: Based on spatial distance and time interval, a preset bivariate function model is used to calculate the dynamic similarity threshold.

[0057] Specifically, the presupposed bivariate function model is a bivariate threshold function model built based on the Sigmoid function. The specific formulas for the spatial distance Sigmoid function fds, the time interval Sigmoid function fdt, and the bivariate threshold function f(ds, dt) are expressed as follows: fds=(1-K) / (1+exp(-SCALE×ds))(1); fdt=K / (1+exp(-SCALE×dt))(2); f(ds, dt)=MIN+((fds+fdt)-0.5)×2×(MAX-MIN)(3); Wherein: the spatiotemporal weight coefficient K = 0.4 to 0.6, preferably 0.5; the kurtosis coefficient SCALE = 0.8 to 1.5, preferably 1.0; the minimum threshold MIN = 0.60 to 0.65, preferably 0.63; and the maximum threshold MAX = 0.75 to 0.85, preferably 0.80.

[0058] Specifically, MIN is the minimum similarity threshold (e.g., 0.63), MAX is the maximum similarity threshold (e.g., 0.8), K is the spatiotemporal weight coefficient (e.g., 0.5, indicating spatiotemporal balance), SCALE is the Sigmoid steepness coefficient (e.g., 1.0), ds is the spatial distance, and dt is the time interval.

[0059] The specific operation is as follows: The spatial distance (ds) and time interval (dt) are received as inputs, and the spatial component (fds) and time component (fdt) are calculated using two Sigmoid functions respectively. Then, the two are combined and mapped to a preset minimum threshold (MIN) and maximum threshold (MAX) to obtain the basic dynamic similarity threshold f(ds, dt).

[0060] The physical meaning of the parameter is: The larger ds is, the higher the required similarity threshold (stronger feature evidence is needed); the larger dt is, the higher the required similarity threshold (feature decay needs to be compensated); K controls the relative importance of spatial distance and time interval.

[0061] Step a3: When the ratio of spatial distance to time interval exceeds the preset maximum speed threshold, the corresponding dynamic similarity threshold is set to a predetermined value for forcibly rejecting association.

[0062] The reference basis for the preset maximum speed threshold is as follows: Normal walking speed: 1.0-1.5 meters per second; Fast walking speed: 2.0-2.5 meters per second; Maximum speed threshold setting: 3.3 m / s (with margin to account for situations such as jogging).

[0063] The preset maximum speed threshold V_max is set according to the application scenario: Retail store scenario: V_max = 3.0 to 3.5 m / s; Transportation hub scenario: V_max = 4.5 to 5.5 m / s; Park monitoring scenario: V_max = 3.0 to 4.0 m / s.

[0064] Specifically, the rationality of trajectory association is ensured by introducing hard physical constraints. After calculating the spatial distance and time interval between two detection boxes, the system compares the ratio of the two as an instantaneous velocity estimate with a preset maximum velocity threshold. If the ratio exceeds the preset maximum velocity threshold (e.g., a pedestrian's movement speed exceeds 3.3 m / s), the association is deemed physically impossible, and a constraint rule is triggered, forcibly rewriting the dynamic similarity threshold originally calculated based on spatiotemporal relationships to a pre-set, extremely high rejection value (e.g., 1.0). This ensures that regardless of the visual feature similarity between two targets, they will be automatically rejected by the system in subsequent matching judgments because they cannot reach this raised threshold, effectively filtering out physically abnormal matches caused by data errors or coincidental appearances.

[0065] The code implementation for this step is as follows: if ds / dt > V_ma: # For example, 3.3 m / s; f(ds, dt)=1.0 # Force rejection of matching.

[0066] Step S3022: Based on the abnormal velocity constraint rules, a set of data points covering different combinations of spatial distance and time interval is generated using a preset bivariate function model.

[0067] Specifically, dense sampling is performed within a preset range of spatial distance and time interval values. For each sampling point, its basic similarity threshold is first calculated based on a bivariate function model. Then, anomaly velocity constraint rules are immediately applied for verification: if the spatiotemporal relationship of the point indicates that the velocity exceeds the maximum threshold, the threshold of the point is forcibly overwritten to a predetermined value (such as 1.0) indicating absolute rejection. By traversing all sampling points and performing the above calculation-overwriting operation, a data point set that integrates continuous mathematical modeling and hard physical rules is finally generated.

[0068] Step S3023: Based on the preset time similarity configuration table, configure the corresponding similarity threshold for the specified time interval; the similarity threshold in the preset time similarity configuration table increases with the increase of the time interval.

[0069] Specifically, the specified time interval is a preset specific time interval, and stricter similarity requirements are set for this specific time interval. The design rationale for the similarity threshold in the preset time similarity configuration table increasing with the time interval is as follows: Over long intervals, a pedestrian's appearance may change (e.g., putting on or taking off a coat, changing a hat). A higher feature similarity is needed to confirm that they are the same target; Segmented configuration is more in line with actual business needs than continuous functions.

[0070] The code representation of this step is as follows: TIME_SIM_CONFIG = [ [10 seconds, 0.88], # A similarity score of 0.88 or higher is required after 10 seconds; [15 minutes, 0.90], # Similarity of 0.90 or higher is required after 15 minutes; [30 minutes, 0.92], # A similarity score of 0.92 or higher is required after 30 minutes; [45 minutes, 0.94], # A similarity score of 0.94 or higher is required after 45 minutes; [60 minutes, 0.96], # A similarity score of 0.96 or higher is required after 60 minutes; ].

[0071] Step S3024: Construct an interpolator based on the data point set to generate a spatiotemporal adaptive threshold surface.

[0072] Specifically, by inputting a data point set that integrates mathematical models and physical rules into a multidimensional linear interpolator (such as LinearNDInterpolator), a query surface covering a continuous spatiotemporal domain is constructed. This interpolator can quickly calculate the corresponding dynamic similarity threshold based on any input spatial distance and time interval, thereby transforming a discrete, rule-based dataset into a continuous, real-time queryable decision function. This ultimately generates a spatiotemporally adaptive threshold surface that provides accurate, smooth, and physically consistent matching criteria for any possible target association within milliseconds.

[0073] For example, the following code demonstrates how to construct a 2D interpolated surface using scipy's LinearNDInterpolator: # Generate training data points data_points = [] for ds in linspace(0, 20, 40): # Spatial distance: 0-20 body lengths for dt in linspace(0.0001, 5, 40): # Time difference: 0-5 seconds f = calculate_threshold(ds, dt) # Use the above formula data_points.append([ds, dt, f]) # Add long time interval configuration points: for t, sim in TIME_SIM_CONFIG: data_points.append([ds, t, sim]) # Create interpolator: interpolator = LinearNDInterpolator(points=[(ds, dt) for ds, dt, f in data_points], values=[f for ds, dt, f in data_points]) # Use the interpolator to query: threshold = interpolator(ds_query, dt_query).

[0074] The code above works as follows: First, it densely samples on a spatiotemporal two-dimensional grid using a double loop, and combines this with pre-defined long-time interval rules to generate a set of data points containing the basic model and business rules. Then, it uses this set (coordinates (ds, dt) and corresponding values ​​f) to construct a multidimensional linear interpolator. Finally, this interpolator becomes a fully functional spatiotemporal adaptive threshold surface. During tracking, simply inputting the spatiotemporal distance (ds_query, dt_query) of any pair of bounding boxes allows for instant querying of the most matching dynamic similarity threshold.

[0075] The advantages of multidimensional linear interpolators are: fast query speed (O(log n) time complexity); smooth transition (avoiding threshold abrupt changes); and flexible expansion (allowing for easy addition of new constraint points).

[0076] Step S303: Based on the target detection box sequence and the corresponding ReID feature vectors, an initial trajectory group is formed. For details, please refer to [link to relevant documentation]. Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0077] Step S304: Based on the spatiotemporal adaptive threshold surface, iterative graph clustering and merging are performed on the initial trajectory group to obtain the iterative trajectory group, and the final trajectory group is used as the target tracking result. For details, please refer to [link to relevant documentation]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0078] The multi-target tracking method provided in this embodiment introduces abnormal speed constraint rules, using physical feasibility as a hard judgment condition, effectively eliminating erroneous matches caused by unreasonable situations such as instantaneous movement. At the same time, it uses a bivariate function model to generate data points covering the entire spatiotemporal combination, enabling the similarity threshold to continuously and smoothly adapt to changes based on spatial distance and time interval, overcoming the poor adaptability of fixed thresholds. Finally, an interpolator transforms this model into a surface that can be quickly queried, ensuring the accuracy of decision-making.

[0079] This embodiment provides a multi-target tracking method, which can be used in the aforementioned electronic devices or terminal devices. Figure 4 This is a flowchart of a multi-target tracking method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps: Step S401: Obtain the video frame sequence, perform object detection on each frame in the video frame sequence to obtain the object detection box sequence, and extract the ReID feature vector corresponding to each object detection box sequence. For details, please refer to [link to relevant documentation]. Figure 3 Step S301 of the illustrated embodiment will not be described again here.

[0080] Step S402: Construct a spatiotemporally adaptive threshold surface based on the target detection box sequence. See details below. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0081] Step S403: Based on the target detection box sequence and the corresponding ReID feature vector, an initial trajectory group is formed.

[0082] Specifically, step S403 includes: Step S4031: Sort the image frames in the target detection box sequence according to their temporal order, and divide all target detection boxes and their corresponding ReID feature vectors belonging to the same image frame into independent initial groups.

[0083] Step S4032: Assign an independent group identifier to each initial group to generate an initial trajectory group.

[0084] Specifically, steps S4031 and S4032 are described as follows: First, all detection results are strictly sorted according to the timestamp of the image frame to which they belong, thus establishing a globally unified timeline at the beginning of the algorithm. Then, taking each frame as the basic unit, all target detection boxes and their corresponding ReID feature vectors appearing in that frame are aggregated together to form an independent initial trajectory group.

[0085] Step S404: Based on the spatiotemporal adaptive threshold surface, perform iterative graph clustering and merging on the initial trajectory group to obtain the iterative trajectory group, and use the final trajectory group as the target tracking result.

[0086] Specifically, step S404 includes: Step S4041: Calculate the feature similarity between each trajectory group and select trajectory pairs with similarity higher than the current iteration threshold.

[0087] Specifically, the trajectory groups are represented using the TkGp data structure, and the element codes of each trajectory group are represented as follows: TkGp { k: str# group identifier (key of representative image); v: List[str] # List of members (all image keys); reid: The representative ReID feature vector of the ndarray group; rows: DataFrame # Data of all check boxes in the group; }

[0088] Calculating the feature similarity between each trajectory group includes: The ReID features of all trajectory groups are extracted, as shown in the following code: reids = [gp.reid for gp in group_list]# shape: (N, 512).

[0089] TopK similarity pairs, i.e. trajectory pairs, are computed using FAISS (Facebook AI Similarity Search, a high-performance similarity search library and dense vector clustering library): # FAISS builds the index: index = faiss.IndexFlatIP(512) # Inner product index; index.add(reids).

[0090] #Search the TopK most similar groups for each group: similarities, indices = index.Search(reids, k=10).

[0091] # Generate a list of similar pairs: pairs = [] for i, (sims, idxs) in enumerate(zip(similarities, indices)): for sim, j in zip(sims, idxs): if i<j and sim> = edge_sim: # Avoid duplicates and meet the threshold; pairs.append(i, j, sim).

[0092] FAISS efficiently identifies all unique trajectory pairs that meet the current iteration's similarity threshold (edge_sim). The output pair list forms the basis for subsequent graph structure construction and connectivity component analysis. Specifically, edge_sim decreases progressively during iteration (e.g., from 0.99 to 0.6) to achieve gradual trajectory merging. The iteration similarity threshold is set between 0.4 and 0.6.

[0093] The iteration parameters are set as follows: High similarity edge_sim_high = 0.95 to 0.99, preferably 0.99; Low similarity edge_sim_low = 0.55 to 0.65, preferably 0.60; The number of iterations N = 30 to 50, preferably 40.

[0094] The trajectory pair filtering provided in this embodiment reduces the time complexity from O(N²) to O(N×K), supports GPU acceleration, and automatically filters low similarity pairs.

[0095] Step S4042: Perform spatiotemporal conflict detection on each trajectory pair based on the spatiotemporal adaptive threshold surface, and determine whether to establish a valid association edge between the two trajectory pairs based on the conflict detection results.

[0096] In an optional implementation, the time interval and spatial distance of each spatiotemporal jump point extracted during the conflict detection phase are used as input to query the spatiotemporal adaptive threshold surface in real time to obtain the minimum similarity threshold required under the current spatiotemporal relationship. Subsequently, this dynamic threshold is compared with the actual appearance feature similarity of the two trajectory groups. If the actual similarity is lower than the threshold, it is determined to be a physically unreasonable association and rejected. At the same time, the query process will synchronously check whether abnormal speed constraints are triggered, directly rejecting any speeding match. The above step S4042 includes: Step b1: Obtain the time interval and spatial distance between the target detection boxes that are temporally adjacent between trajectory pairs.

[0097] For each trajectory pair (gp1, gp2), perform collision detection as follows: Step b11: All bounding box data in the two candidate trajectory groups (gp1 and gp2) are merged into one list; Step b12: Sort the merged list in ascending order according to the timestamp field; Step b13: Traverse the sorted data, check each pair of adjacent detection boxes, and determine whether the two adjacent detection boxes come from two different groups (i.e., whether they are on both sides of the "group boundary"). Step b14, calculate the time interval: convert the millisecond timestamp difference to seconds, and set the minimum time difference to 1 second to avoid noise caused by instantaneous calculation; Step b15, calculate spatial distance: call the function to calculate the normalized spatial distance between the two detection boxes, including: inputting the coordinates of the previous detection box and the coordinates of the next detection box; limit the spatial distance to between 1 and 10 body lengths to avoid the influence of extreme values; Step b16: Add the calculated pair (dt, ds) to the list as a conflict detection metric.

[0098] The code for performing conflict detection is shown below: #Merge the data from the two groups and sort by time: rows = gp1.rows + gp2.rows rows.sort_by('timestamp') # Extract the spatiotemporal differences between adjacent frames: for i in range(len(rows) - 1): if rows[i].group_id != rows[i+1].group_id: #Across group boundaries; #Time difference (seconds): dt=(rows[i+1].timestamp - rows[i].timestamp) / 1000 dt=max(dt, 1.0) # Minimum 1 second #Spatial distance (normalized avatar bits): ds=calculate_normalized_distance( rows[i].bbox, rows[i+1].bbox ) ds = clip(ds, 1.0, 10.0) # Limit to positions 1-10 body lengths; conflict_metrics.append(dt, ds).

[0099] Step b2: Based on the spatiotemporal adaptive threshold surface, determine the required similarity threshold corresponding to the time interval and spatial distance.

[0100] Specifically, the calculated time interval and spatial distance are used as two-dimensional coordinate inputs to query a pre-generated spatiotemporal adaptive threshold surface in real time. This surface, as a continuous function mapping, can immediately output a dynamic and strictly corresponding desired similarity threshold based on the input spatiotemporal relationship.

[0101] Step b3: If the feature similarity of the trajectory pair reaches the required similarity threshold, and the time interval and spatial distance do not trigger the abnormal speed constraint rule, then a valid association edge is established between the two trajectory pairs in the trajectory pair.

[0102] First, the system verifies whether the feature similarity between the two trajectory groups meets the dynamic requirements specified by the spatiotemporal adaptive threshold surface. Second, it confirms that their spatiotemporal relationship does not violate the abnormal velocity constraint rules. Only when both of the above conditions are met simultaneously will the system determine that the association is reasonable in both visual appearance and physical spatiotemporal aspects, thereby establishing a valid association edge between the two trajectory groups. This edge will serve as the key connection basis for achieving global trajectory merging in subsequent graph clustering analysis.

[0103] The code representation for querying threshold surfaces to determine conflicts is as follows: # Use an interpolator to query the threshold for each conflict location: thresholds = interpolator(ds_array, dt_array) # Determine if a strong conflict exists: if any(thresholds == 1.0): # Abnormal speed constraint triggered; return (is_relation=False, is_conflict=True) #Determine if a relationship exists: if group_similarity>min(thresholds): return (is_relation=True, is_conflict=False) #Unable to determine; return (is_relation=False, is_conflict=False).

[0104] This embodiment also provides a conflict caching mechanism, using trajectory group features as the cache key to store conflict detection results in the cache. Subsequent detections prioritize querying the cache; if a match is found, the result is returned directly. The specific operation is as follows: First, the identifiers and the number of members of the two trajectory groups are sorted and combined according to rules to generate a unique cache key. Whenever spatiotemporal conflict detection is required, the cache dictionary is first checked to see if the key exists. If it exists, the cached result is returned directly, thus completely avoiding repeated complex calculations. If it does not exist, the complete conflict detection process is executed, and the final judgment result (whether there is a conflict) is associated with the cache key and stored in the cache dictionary for direct use when querying the same group pair in the future. This significantly improves the overall execution efficiency of the iterative graph clustering merging algorithm.

[0105] The code representation of the conflict caching mechanism is as follows: #Use group features as cache keys to avoid duplicate calculations: conflict_key = tuple(sorted([ (gp1.k, len(gp1.v)), (gp2.k, len(gp2.v)) ])) if conflict_key in conflict_cache: return conflict_cache[conflict_key] #Calculate and store in cache: result = calculate_conflict(gp1, gp2) conflict_cache[conflict_key] = result Return result.

[0106] Step S4043: Construct a graph structure with all trajectory groups as nodes and effective associated edges as connections, and extract the connected components in the graph structure; the graph structure is an undirected weighted graph.

[0107] Specifically, each iteration treats all trajectory groups as nodes, and valid associations verified by spatiotemporal conflict detection are used as edges connecting these nodes (the weight of each edge is typically the similarity of features between groups), thus constructing a global undirected weighted graph. Subsequently, a connected component analysis algorithm from graph theory is applied to identify all subgraphs (i.e., connected components) within this graph that are internally connected by edges. Each connected component represents a potential cluster of trajectory groups that should be merged.

[0108] The code representation of constructing an undirected weighted graph is as follows: import networkx as nx G = nx.Graph() G.add_nodes_from(range(len(group_list))) #Add related edges: for i, j, sim in valid_pairs: if is_relation and not is_conflict: G.add_edge(i, j, weight=sim).

[0109] The code for extracting connected components is represented as follows: #Each connected component represents a cluster: components = list(nx.connected_components(G)) #Example: [[0,5,12], [1,3], [2,8,9], ...].

[0110] The graph clustering provided in this embodiment has the following advantages: Transitive closure: AB association, BC association → AC automatic association; Global consistency: Avoiding local optima in greedy strategies; Conflict constraint: Conflicting edges are not connected and are automatically split into different components.

[0111] Step S4044 involves performing a merging operation on the trajectory groups within each connected component, updating the trajectory groups after each iteration, and using them as input for the next iteration. Specifically, for each identified connected component, the system performs an internal trajectory merging operation: first, it merges the member and feature data of all trajectory groups within the component; then, it generates a new, more complete trajectory group to represent the cluster using strategies such as secondary clustering. After processing all components, the system replaces the old set from the previous iteration with this newly generated set of trajectory groups and uses it as input for the next iteration. This process allows the algorithm to progressively reduce trajectory fragments and fuse related segments in each iteration, ultimately generating fewer but more complete and accurate long-term tracking trajectories through multiple iterations.

[0112] In an optional implementation, step S4044 includes: Step c1: Merge the members of all trajectory groups within the same connected component; regroup all merged members based on the group identifier of the members, and extract the ReID feature vector of each new group.

[0113] First, the member lists of all trajectory groups within the same connected component are merged to form a set containing all observation data for that component. This operation breaks down the initial boundaries based on trajectory groups, bringing together all detection boxes and their features originating from different time points and different initial groups within the component. This provides a complete data foundation for analyzing the true attribution relationships between members at a finer granular level.

[0114] After obtaining the merged data for all members, the method regroups them based on each member's original, unique group identifier, grouping members belonging to the same original trajectory group together. Then, for each newly formed group, the ReID feature vectors of all its members are extracted. This step restores the structure of the members in the initial clustering, ensuring that subsequent secondary clustering is performed at the level of the initial group features, thus preparing the data for more accurate cluster partitioning.

[0115] For each connected component, the code representation of performing quadratic clustering is as follows: def conditional_merge(component_groups, threshold=0.5): #Merge all members within the group: merged_members = sum([gp.v for gp in component_groups]) #Regroup by member: member_groups = group_by_member(merged_members) # Perform secondary clustering on the ReID features of each member group member_reids = [get_reid(members) for members in member_groups] clusters = FAISS_clustering(member_reids, threshold) #Each cluster generates a new trajectory group: new_groups = [] for cluster in clusters: new_gp = merge_and_update_reid(cluster) new_groups.append(new_gp) return new_groups.

[0116] Step c2 involves performing secondary clustering on the ReID feature vectors and generating new trajectory groups based on the secondary clustering results, which serve as the trajectory groups updated in each iteration.

[0117] Specifically, the feature vector sets obtained after regrouping are subjected to secondary clustering, and these groups are re-divided into more precise clusters based on feature similarity. Each newly generated cluster represents a group of members with highly consistent features who should belong to the same target. The system then merges the data of all members of each cluster to form a new, cleaner trajectory group. These newly generated trajectory groups serve as the final output of this iteration and become the input of the next iteration, thus realizing the round-by-round optimization and integration of trajectories.

[0118] The core algorithm of the iterative update strategy is represented as follows: def iterative_merge(group_list, sim_range): # Iteration parameters: from high similarity to low similarity; sim_values ​​= linspace(0.99, 0.6, 40) # 40 iterations for iter_idx, edge_sim in enumerate(sim_values): print(f"Iteration {iter_idx+1} / 40, edge_sim={edge_sim:.2f}") #Perform a single graph cluster merge: group_list = graph_clustering_merge( group_list, edge_sim, conflict_checker=spatiotemporal_conflic, conditional_merger = secondary clustering ) print(f" Number of groups: {org_count}->{len(group_list)}") return group_list.

[0119] Step c3: After performing a merging operation on the trajectory groups within the connected components, the representative ReID feature vector of the merged trajectory group is updated; wherein, the updated representative ReID feature vector is the mean vector calculated based on the feature vectors of multiple members within the merged trajectory group after filtering, and no normalization processing is performed.

[0120] Specifically, after each graph clustering merge, the representative ReID features of the trajectory group are updated, i.e., the ReID feature adaptive fusion step, which includes: Cluster the ReID features of all members within the trajectory group; Sort by cluster size, prioritizing larger clusters; A hierarchical quality control strategy is applied: the process stops when the number of selected clusters is greater than 1 and the current cluster size is 1; when the number of selected clusters is greater than 2 and the current cluster size is 2; and when the number of selected clusters is greater than 3 and the current cluster size is 3. Select the optimal representative feature for each cluster; The mean of the selected features is calculated as the new representative ReID feature without normalization.

[0121] The code is represented as: def merge_reid(group, similarity_threshold=0.8): #Step 1: Cluster the ReID features of all members within the group: member_reids = [get_reid(member) for member in group.v] clusters = FAISS_clustering(member_reids, similarity_threshold) #Step 2: Sort by cluster size, prioritizing larger clusters: clusters.sort(key=lambda c: len(c), reverse=True) #Step 3: Graded Quality Control: selected_reids = [] for cluster in clusters[:8]: # Maximum of 8 clusters; #Singleton cluster: has only 1 member: if len(selected_reids)>1 and len(cluster) == 1: break #Binary cluster: has only 2 members: if len(selected_reids)>2 and len(cluster) == 2: break #Trivarian cluster: Only 3 members: if len(selected_reids)>3 and len(cluster) == 3: break #Optimal representative feature of the selected cluster: cluster_reids = [member_reids[i] for i in cluster] best_idx = argmax_quality(cluster_reids) selected_reids.append(cluster_reids[best_idx]) #Step 4: Calculate the mean but do not normalize: group.reid = mean(selected_reids, axis=0) #Note: group.reid / = norm(group.reid) is not executed. return group.reid.

[0122] This embodiment also includes: a cross-camera trajectory association step: Perform steps S401 to S404 for each camera to generate a single camera trajectory; Merge all single-camera tracks; Step S404 is executed again using a stricter similarity threshold range (0.90 ~ 0.65); Generate global cross-camera trajectories.

[0123] It should be noted that the hardware environment provided in the embodiments of the present invention includes; Minimum requirements: CPU: Intel i5-8400 or equivalent; RAM: 16GB DDR4; GPU: NVIDIA GTX 1660 (6GB VRAM); Storage: 256GB SSD; Recommended configuration: CPU: Intel i7-10700K or AMD Ryzen 7 5800X; RAM: 32GB DDR4; GPU: NVIDIA RTX 3060 (12GB VRAM); Storage: 512GB NVMe SSD; High-performance configuration (multi-camera scenarios): CPU: Intel i9-12900K or AMD Ryzen 9 5950X; RAM: 64GB DDR4; GPU: NVIDIA RTX 4090 (24GB VRAM); Storage: 1TB NVMe SSD.

[0124] The software environment provided in this embodiment of the invention includes: operating system: Ubuntu 20.04 / 22.04 LTS; Windows 10 / 11 (WSL2 required); macOS 12+ (CPU inference only); Development environment: Python 3.8 ~ 3.11; CUDA 11.7 / 11.8; cuDNN 8.6+; Core dependency libraries: NumPy >= 1.21.0; scipy>=1.7.0; pandas>=1.3.0; opencv-Python>=4.6.0; torch>=1.12.0; torchvision>=0.13.0; Faiss-gpu>=1.7.2; networkx>=2.8.0; scikit-learn >= 1.0.0; Model file: Object detection: YOLOv8n.pt (~6MB) or YOLOv8x.pt (~136MB); ReID characteristics: OSNet_x1_0.pth (~9MB) or ResNet50-ibn.pth (~98MB).

[0125] The multi-target tracking method provided in this embodiment filters candidate associations step by step through iterative thresholds from high to low, and uses a spatiotemporal adaptive threshold surface to verify the physical rationality of each candidate association pair, effectively ensuring the reliability of each merging. Furthermore, it achieves global optimization and merging of trajectory groups through graph connectivity component analysis, breaking the limitations of traditional frame-by-frame matching, thereby significantly reducing trajectory fragmentation and identity switching caused by occlusion and appearance changes, and ultimately forming a longer and more complete continuous trajectory.

[0126] As one or more specific application embodiments of the present invention, combined with Figure 5 The multi-target tracking method provided by this invention will be further described in detail, such as... Figure 5 As shown, the specific process is as follows: Step S1: Obtain the video frame sequence, perform target detection on each frame to obtain the target detection box sequence; extract the ReID feature vector of each detection box; Step S2: Construct a spatiotemporal adaptive threshold surface, specifically including: S2.1: Define two variables: spatial distance ds and time difference dt, where ds is the normalized spatial distance between the two detection boxes and dt is the time interval between the two detection boxes; S2.2: The similarity threshold is calculated using the bivariate Sigmoid function. For details, please refer to formulas (1) to (3). S2.3: For cases where DS / DT > V_max, set f(ds, dt) = 1.0 to implement abnormal speed constraints; S2.4: Based on the preset time-similarity configuration list TIME_SIM_CFG, set stricter threshold requirements at specific time intervals; S2.5: Based on the data points calculated above, construct a two-dimensional linear interpolator to form a spatiotemporal adaptive threshold surface; Step S3: Group the detection boxes according to image frames to form an initial trajectory group list; Step S4: Perform iterative graph clustering merging, specifically including: S4.1: Set the iteration range for the similarity threshold, from high similarity edge_sim_high to low similarity edge_sim_low, and perform N iterations; S4.2: In each iteration: (a) Calculate the ReID feature similarity of all trajectory pairs using FAISS; (b) For trajectory pairs (gp1, gp2) with a similarity greater than the current edge_sim, perform spatiotemporal conflict detection: merge the detection box data of the two groups and sort them by time; extract the temporal difference dt and spatial distance DS between adjacent frames across group boundaries; use the interpolator constructed in step S2 to query the threshold f(ds,dt); if f(ds, dt) = 1.0 exists, it is determined to be a conflict and marked as a conflict edge; if the inter-group similarity > min(f(ds, dt)), it is determined to be a valid associated edge; (c) Construct an undirected weighted graph, where nodes are trajectories and effective associated edges are the edges of the graph; (d) Extract the connected components of the graph; (e) Perform conditional merging for each connected component: regroup by member and perform secondary clustering; (f) Update the list of trajectory groups; S4.3: Repeat step S4.2 until N iterations are completed.

[0127] Step S5: Output the final trajectory groups, assign a global ID to each group, and generate target tracking results.

[0128] Example 1: Customer tracking in retail stores, the scenario description includes: Scenario: A 200-square-meter retail store; Cameras: 4 1080P cameras, covering the entrance, shelf area, and checkout area; Objective: To track customers' complete behavior throughout the store; Challenges: Shelf obstruction, similar appearance, prolonged stay.

[0129] Step 1, Data Acquisition and Preprocessing: First, video streams are simultaneously read from four cameras deployed in different locations (entrance, shelf A, shelf B, and checkout). Then, a YOLOv8 object detection model is used to detect pedestrians in each frame, resulting in a sequence of bounding boxes. Next, for each detected pedestrian bounding box, the corresponding image region is cropped, and a 512-dimensional feature vector is extracted using the OSNet re-identification model as the pedestrian's ReID feature. Finally, the system obtains the detection boxes and corresponding feature vectors for all pedestrians from all cameras.

[0130] Step 2, construct the spatiotemporal threshold surface: Based on the characteristics of the retail store scenario, key parameters of the threshold surface are configured as follows: the lower limit of the similarity threshold is set to 0.63, and the upper limit to 0.80; the spatiotemporal balance coefficient is 0.5; the skewing of the sigmoid function is 1.0; and the maximum reasonable walking speed for pedestrians is 3.3 m / s. Simultaneously, strict matching requirements for long time intervals are defined, for example, the similarity must be higher than 0.88 after 10 seconds and higher than 0.96 after 60 minutes. Using these configurations and a bivariate function model, a two-dimensional interpolator is generated, which constitutes a spatiotemporally adaptive threshold surface that can be quickly queried.

[0131] Step 3, Trajectory generation within a single camera: The detection data of each camera is processed independently. The system calls an iterative graph clustering merging function, performing 40 iterations with a similarity threshold range of 0.99 to 0.6, and uses the threshold surface constructed in step 2 for conflict detection, gradually merging fragmented detection boxes belonging to the same pedestrian within the same camera into complete trajectories. The processing results show that the number of detection boxes is significantly reduced to the number of coherent trajectories; for example, one camera merged 2341 detection boxes into 134 trajectories.

[0132] Step 4, cross-camera trajectory association: The individual camera trajectories generated by all cameras are merged into a single set. Then, an iterative graph clustering merging process is executed again, but this time with a stricter threshold range (0.90 to 0.65) to more rigorously determine whether trajectories from different cameras belong to the same pedestrian. This step successfully achieves cross-camera pedestrian identity association, for example, merging 401 individual camera trajectories into 156 global trajectories.

[0133] Step 5, Results Output and Analysis: Each global trajectory is assigned a unique ID, and its behavior is statistically analyzed. Output information includes: total dwell time, the sequence of cameras traversed, and the main areas the customer spent time in the store. For example, the output shows that customer ID 0023 had a total dwell time of 12.3 minutes, passing through the entrance, shelf A, shelf C, and the checkout area. This structured trajectory data provides direct evidence for customer flow analysis and behavioral understanding.

[0134] The implementation results of Example 1 are shown in Table 1 below: Table 1 shows the implementation results of Example 1.

[0135] Example 2: Pedestrian flow tracking at transportation hubs, the scenario is described as follows: Scene: Subway station entrance / exit; Cameras: 8 cameras covering turnstiles, passageways, and station hall; Objective: To analyze passenger flow routes and optimize signage; Challenges: Dense crowds, rapid movement, and multiple obstructions.

[0136] To address the specific characteristics of transportation hub scenarios, the parameters were adjusted as follows: Step 1: Adjust the basic threshold range to accommodate high-density pedestrian flow: Due to the high density of people and frequent occlusion in transportation hub scenarios, the upper and lower limits of the similarity threshold were first lowered: the minimum threshold (MIN) was set to 0.60 and the maximum threshold (MAX) to 0.75. This adjustment relaxes the absolute requirements for matching, allowing the algorithm to initiate associations more flexibly in complex scenarios, while avoiding the loss of a large number of potential correct matches due to excessively high standards.

[0137] Step 2: Optimize the spatiotemporal weights and the form of the decision function: To address the high speed of pedestrian movement, the weight of the time factor was increased (K=0.6), giving the time interval a more significant role in threshold calculation. Simultaneously, the steepness coefficient of the Sigmoid function was increased (SCALE=1.2), making the threshold curve more sensitive to changes in time and space distance, thus facilitating more decisive association decisions in fast-moving scenarios.

[0138] Step 3: Relax the physical velocity constraint: To accommodate passengers' potential jogging or running behavior, the maximum threshold (V_max) for abnormal speed constraints has been increased to 5.0 m / s. This ensures that the system does not mistakenly classify reasonable rapid movement as a physically impossible association.

[0139] Step 4: Reconstruct the strictness rules for long-term associations: Given the generally short dwell times of passengers at transportation hubs, the time-similarity configuration list (TIME_SIM_CFG) has been redesigned. This list focuses on shorter time windows (e.g., 5 seconds, 30 seconds, 1 minute, 3 minutes) and assigns progressively increasing similarity requirements for each time point (e.g., 0.82 after 5 seconds, 0.94 after 3 minutes). This ensures reliable identity association for pedestrians who reappear after brief separation.

[0140] The implementation results of Example 2 are shown in Table 2 below: Table 2 shows the implementation results of Example 2.

[0141] Example 3: Park security monitoring, the scenario is described as follows: Scenario: Corporate park; Cameras: 20 cameras covering buildings, roads, and parking lots; Objective: To track suspicious individuals over a long period and generate their movement trajectories; Challenges: Wide-ranging, long-term, and complex paths.

[0142] A hierarchical tracking strategy is adopted: Layer 1, tracking within the region: The 20 cameras were divided into 5 areas; Each region independently executes single-camera + cross-border tracking; Generate regional-level trajectories.

[0143] Layer 2, Global Tracking: Merge the trajectories of the five regions; Use a more stringent threshold surface; Generate a global trajectory.

[0144] Layer 3, long-term association: For trajectories with time intervals > 5 minutes; Use a combination of appearance and behavioral characteristics for judgment; Manual confirmation is required.

[0145] The implementation results of Example 3 are shown in Table 3 below: Table 3 shows the implementation effect of Example 3.

[0146] This embodiment provides a multi-target tracking method that achieves smooth adaptation through a spatiotemporal adaptive threshold surface, enables global optimization through graph clustering, and significantly reduces trajectory fragmentation through iterative merging. Ultimately, it achieves more stable and accurate long-term multi-target tracking in complex scenarios, solving the problems of unstable tracking accuracy and discontinuous long-term tracking caused by the use of fixed similarity thresholds and lack of spatiotemporal physical constraint modeling in existing technologies.

[0147] This embodiment also provides a multi-target tracking device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0148] This embodiment provides a multi-target tracking device, such as Figure 6 As shown, it includes: The target detection and feature extraction module 601 is used to acquire a video frame sequence, perform target detection on each frame in the video frame sequence to obtain a target detection box sequence, and extract the ReID feature vector corresponding to each target detection box sequence.

[0149] The threshold surface construction module 602 is used to construct a spatiotemporally adaptive threshold surface based on the target detection box sequence.

[0150] The initial grouping module 603 is used to form an initial trajectory group based on the target detection box sequence and the corresponding ReID feature vector.

[0151] The iterative graph clustering processing module 604 is used to perform iterative graph clustering and merging on the initial trajectory group based on the spatiotemporal adaptive threshold surface to obtain the iterative trajectory group, and use the final trajectory group as the target tracking result.

[0152] In some alternative implementations, the threshold surface construction module 602 includes: Abnormal speed constraint unit, used to determine abnormal speed constraint rules; The bivariate function calculation unit is used to generate a set of data points covering different combinations of spatial distances and time intervals based on abnormal velocity constraint rules and a preset bivariate function model. A two-dimensional linear interpolator unit is used to construct an interpolator based on a set of data points to generate a spatiotemporally adaptive threshold surface.

[0153] In some optional implementations, the abnormal velocity constraint unit includes: The spatiotemporal parameter determination subunit is used to determine the spatial distance and time interval between two target detection boxes whose correlation is to be judged.

[0154] The dynamic similarity threshold calculation subunit is used to calculate the dynamic similarity threshold based on spatial distance and time interval using a preset bivariate function model.

[0155] The judgment sub-unit is used to set the corresponding dynamic similarity threshold to a predetermined value for forcibly rejecting association when the ratio of spatial distance to time interval exceeds a preset maximum speed threshold.

[0156] In some alternative implementations, the threshold surface construction module 602 further includes: The time-similarity configuration unit is used to configure the corresponding similarity threshold for a specified time interval based on a preset time similarity configuration table; the similarity threshold in the preset time similarity configuration table increases with the increase of the time interval.

[0157] In some alternative implementations, the initial grouping module 603 includes: The grouping unit is used to sort the image frames in the target detection box sequence according to their temporal order, dividing all target detection boxes and their corresponding ReID feature vectors belonging to the same image frame into independent initial groups.

[0158] The initial trajectory group generation unit is used to assign an independent group identifier to each initial group and generate the initial trajectory group.

[0159] In some alternative implementations, the iterative graph clustering processing module 604 includes: The FAISS similarity calculation unit is used to calculate the feature similarity between each trajectory group and to filter out trajectory pairs with a similarity higher than the current iteration threshold.

[0160] The spatiotemporal conflict detection unit is used to perform spatiotemporal conflict detection on each trajectory pair based on the spatiotemporal adaptive threshold surface, and to determine whether to establish a valid association edge between the two trajectory pairs based on the conflict detection results.

[0161] The undirected weighted graph construction and connected component extraction unit is used to construct a graph structure with all trajectory groups as nodes and effective associated edges as connections, and to extract the connected components in the graph structure; the graph structure is an undirected weighted graph.

[0162] The conditional merging and iterative control unit is used to perform merging operations on the trajectory groups within each connected component, update the trajectory groups after each iteration, and use them as input for the next iteration.

[0163] In some optional implementations, the spatiotemporal conflict detection unit includes: The spatiotemporal parameter acquisition subunit is used to acquire the time interval and spatial distance between target detection boxes that are temporally adjacent between trajectory pairs.

[0164] The required similarity threshold calculation subunit is used to determine the required similarity threshold corresponding to the time interval and spatial distance based on the spatiotemporal adaptive threshold surface.

[0165] The spatiotemporal relationship construction sub-unit is used to establish a valid association edge between two trajectory groups if the feature similarity of the trajectory pair reaches the required similarity threshold and the time interval and spatial distance do not trigger the abnormal speed constraint rule.

[0166] In some alternative implementations, the condition merging and iteration control unit includes: The member merging sub-unit is used to merge members of all trajectory groups within the same connected component.

[0167] The ReID feature vector extraction subunit is used to regroup all merged members based on their group identifiers and extract the ReID feature vector for each new group.

[0168] The secondary clustering subunit is used to perform secondary clustering on the ReID feature vectors and generate new trajectory groups based on the secondary clustering results, which serve as the trajectory groups updated in each iteration.

[0169] In some alternative implementations, the condition merging and iteration control unit further includes: The ReID feature adaptive fusion subunit is used to update the representative ReID feature vector of the merged trajectory group after performing a merging operation on the trajectory group within the connected components. The updated representative ReID feature vector is the mean vector calculated based on the feature vectors of multiple members in the merged trajectory group after filtering, and is not normalized.

[0170] The multi-target tracking device provided in this embodiment of the invention can execute the multi-target tracking method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0171] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0172] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0173] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0174] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the multi-target tracking method of the embodiments of the present invention.

[0175] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0176] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the multi-target tracking method shown in the above embodiments is implemented.

[0177] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0178] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A multi-target tracking method, characterized in that, The method includes: A video frame sequence is acquired, and target detection is performed on each frame in the video frame sequence to obtain a target detection box sequence. The ReID feature vector corresponding to each target detection box sequence is then extracted. Based on the target detection box sequence, a spatiotemporal adaptive threshold surface is constructed; Based on the target detection box sequence and the corresponding ReID feature vector, an initial trajectory group is formed; Based on the spatiotemporal adaptive threshold surface, iterative graph clustering and merging are performed on the initial trajectory group to obtain the iterative trajectory group, and the final trajectory group is used as the target tracking result.

2. The method according to claim 1, characterized in that, The construction of a spatiotemporally adaptive threshold surface based on the target detection box sequence includes: Determine the rules for constraining abnormal speeds; Based on the aforementioned abnormal velocity constraint rules, a set of data points covering different combinations of spatial distance and time interval is generated using a preset bivariate function model. An interpolator is constructed based on the set of data points to generate the spatiotemporal adaptive threshold surface.

3. The method according to claim 2, characterized in that, The rules for determining abnormal speed constraints include: Determine the spatial distance and time interval between the two target detection boxes whose association relationship is to be determined; Based on the spatial distance and time interval, a dynamic similarity threshold is calculated using a preset bivariate function model. When the ratio of spatial distance to time interval exceeds the preset maximum speed threshold, the corresponding dynamic similarity threshold is set to a predetermined value for forcibly rejecting association.

4. The method according to claim 3, characterized in that, Based on the target detection box sequence, a spatiotemporal adaptive threshold surface is constructed, which further includes: Based on a preset time similarity configuration table, a corresponding similarity threshold is configured for a specified time interval; the similarity threshold in the preset time similarity configuration table increases with the increase of the time interval.

5. The method according to claim 4, characterized in that, Based on the target detection box sequence and the corresponding ReID feature vector, an initial trajectory group is formed, including: Based on the temporal order of image frames in the target detection box sequence, all target detection boxes belonging to the same image frame and their corresponding ReID feature vectors are divided into independent initial groups. Each initial group is assigned an independent group identifier to generate the initial trajectory group.

6. The method according to claim 5, characterized in that, Based on the spatiotemporal adaptive threshold surface, iterative graph clustering and merging are performed on the initial trajectory group to obtain the iterative trajectory group, including: Calculate the feature similarity between each trajectory group and select trajectory pairs with similarity higher than the current iteration threshold; Based on the spatiotemporal adaptive threshold surface, spatiotemporal conflict detection is performed on each trajectory pair, and based on the conflict detection results, it is determined whether a valid association edge is established between the two trajectory pairs in the trajectory pair. A graph structure is constructed using all trajectory groups as nodes and the effective associated edges as connections, and the connected components in the graph structure are extracted; the graph structure is an undirected weighted graph. Perform a merging operation on the trajectory groups within each connected component, update the trajectory groups after each iteration, and use them as input for the next iteration.

7. The method according to claim 6, characterized in that, Based on the spatiotemporal adaptive threshold surface, spatiotemporal conflict detection is performed on each trajectory pair, and based on the conflict detection results, it is determined whether a valid association edge is established between two trajectory pairs, including: Obtain the time interval and spatial distance between the target detection boxes that are temporally adjacent between the trajectory pairs; Based on the spatiotemporal adaptive threshold surface, determine the required similarity threshold corresponding to the time interval and spatial distance; If the feature similarity of the trajectory pair reaches the required similarity threshold, and the time interval and spatial distance do not trigger the abnormal speed constraint rule, then a valid association edge is established between the two trajectory pairs in the trajectory pair.

8. The method according to claim 6, characterized in that, The step of performing a merging operation on the trajectory groups within each connected component to update the trajectory groups after each iteration includes: Merge members of all trajectory groups within the same connected component; Based on the group identifier of the members, all members after merging are regrouped, and the ReID feature vector of each new group is extracted; The ReID feature vectors are subjected to secondary clustering, and new trajectory groups are generated based on the secondary clustering results, which serve as the trajectory groups updated in each iteration.

9. The method according to claim 6, characterized in that, The step of performing a merging operation on the trajectory groups within each connected component to update the trajectory groups after each iteration also includes: After performing a merging operation on the trajectory groups within the connected components, the representative ReID feature vector of the merged trajectory group is updated. The updated representative ReID feature vector is the mean vector calculated based on the feature vectors of multiple filtered members within the merged trajectory group, without normalization.

10. A multi-target tracking device, characterized in that, The device includes: The target detection and feature extraction module is used to acquire a video frame sequence, perform target detection on each frame in the video frame sequence to obtain a target detection box sequence, and extract the ReID feature vector corresponding to each target detection box sequence; The threshold surface construction module is used to construct a spatiotemporally adaptive threshold surface based on the target detection box sequence; The initial grouping module is used to form an initial trajectory group based on the target detection box sequence and the corresponding ReID feature vector; The iterative graph clustering processing module is used to perform iterative graph clustering and merging on the initial trajectory group based on the spatiotemporal adaptive threshold surface to obtain the iterative trajectory group, and use the final trajectory group as the target tracking result.

Images