A method for long video target trajectory integration and identity preservation with quality constraint

Abstract

The present application relates to the technical field of computer vision, and particularly relates to a quality-constrained long video target track integration and identity maintenance method, comprising the following steps: inputting a current video frame, a current frame target observation set and an existing active target track set; calculating appearance similarity; determining a historical identity recovery result for a target track in a dormant state; calculating intersection over union and minimum containing overlap rate, and combining area difference to distinguish ordinary competition and cross-granularity containing relationship; forming a competition track group; calculating competition scores of each target track, and executing results of different competition tracks; calculating a difference value of reliability of a component track and an overall track to form a preliminary cross-granularity decision; and outputting an active target track set and a target identity label according to the historical identity recovery result, the competition track processing result and the cross-granularity track processing result. The present application aims to solve the technical problem that a target identity is difficult to be stably maintained for a long time in a long video target segmentation or target tracking process.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, video target segmentation, and multi-target tracking, and particularly to a quality-constrained method for integrating and preserving the identity of targets in long videos. Background Technology

[0002] Existing video object segmentation and tracking methods typically focus on object propagation between adjacent frames or object association within a short timeframe. Video object segmentation methods generally propagate the object mask to subsequent frames using features from historical frames, current frames, and memory mechanisms, given an initial object mask. These methods perform well when the object is continuously visible, the video is short, or the scene changes minimally. However, when the object is occluded for extended periods, reappears after going out of frame, undergoes drastic changes in appearance, or when mask propagation errors accumulate, issues such as trajectory breakage, identity switching, or duplicate registration can easily occur.

[0003] Multi-target tracking methods typically employ a detection-association framework, where targets are first detected in each frame, and then the target observations in different frames are associated as trajectories based on appearance features, spatial location, motion prediction, and temporal continuity. While this type of method can handle cross-frame identity matching to some extent, most methods primarily use bounding boxes or center points as representations, and are insufficient in handling pixel-level mask variations, inconsistent segmentation granularity, and part-to-whole relationships.

[0004] Furthermore, existing trajectory post-processing methods typically perform pairwise trajectory retention, deletion, or merging based on the intersection-union ratio (IU) or appearance similarity between trajectories. When three or more trajectories simultaneously cover the same or adjacent areas, the results of pairwise processing are easily affected by the processing order, making it difficult to determine a more reliable target identity from the overall group of competing trajectories. On the other hand, high trajectory overlap may indicate that two trajectories repeatedly cover the same object, or it may indicate that a component trajectory is contained within the overall trajectory. If only the IU or a single inclusion ratio is used for judgment, it is easy to misjudge cross-granularity inclusion relationships as ordinary repeated trajectory relationships. Especially in long videos, partial areas and overall areas may alternate, requiring further judgment based on trajectory stability within the time window, mask quality, and time support length.

[0005] Meanwhile, existing methods typically use appearance similarity directly for trajectory updates or identity recovery without fully considering the quality of the current mask. Once a low-quality mask is written into the appearance feature memory, it may cause subsequent identity recovery errors and continuously pollute the trajectory state.

[0006] Therefore, a processing method for long video trajectory sets is needed, which controls historical memory updates through quality constraints, distinguishes between ordinary competitive relationships and cross-granularity inclusion relationships through dual overlap indicators, performs group scoring through competitive relationship graphs, and performs bidirectional cross-granularity trajectory decision-making based on multidimensional reliability evidence within the time window. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a quality-constrained method for integrating and preserving the identity of long video target trajectories. This method aims to solve the technical problems of low-quality target observation contaminating appearance feature memory, the pair processing results of more than three overlapping trajectories being affected by the processing order, the difficulty in distinguishing between ordinary repeated trajectories and cross-granularity included trajectories, and the unstable recovery of historical identity after a long period of target loss during long video target segmentation or multi-target tracking.

[0008] To achieve the above objectives, this invention proposes a quality-constrained long video target trajectory integration and identity preservation method, comprising the following steps:

[0009] S1. Input the current video frame, the target observation set of the current frame, and the existing target trajectory set;

[0010] S2. Extract appearance features from target observations in the current frame and calculate mask quality scores. Determine update weights based on the appearance similarity between the current target observation and the corresponding historical trajectory. For dormant target trajectories, it is necessary to determine the historical identity recovery results.

[0011] S3. Calculate the intersection-union ratio and minimum overlap rate between the current masks of active trajectories to determine the spatial overlap relationship; combine area differences to distinguish between ordinary competitive relationships and cross-granularity inclusion relationships; construct a competitive relationship graph with active trajectories as nodes and competitive relationships as edges, and form competitive trajectory groups according to connected components;

[0012] S4. Calculate the competition score of each target trajectory in the competition trajectory group, and perform preservation, suppression or merging processing on the competition trajectories based on the competition score, appearance similarity and the persistence of spatial overlap.

[0013] S5. Within a preset time window, statistically analyze the stability of cross-granularity inclusion relationship candidates, calculate the difference in reliability between the component trajectory and the overall trajectory to form a preliminary cross-granularity decision, thereby deciding whether to absorb, split, or postpone.

[0014] S6. Based on the historical identity recovery results in S2, the competitive trajectory processing results in S4, and the cross-granularity trajectory processing results in S5, update the active, dormant, or terminated status of each trajectory, update, compress, and prune the appearance feature memory and historical mask sequence, and output the integrated target trajectory set and target identity identifier.

[0015] Preferably, in S2, appearance features are extracted based on the mask region. The appearance feature extraction is represented as follows: ;in, This indicates the visual feature extraction model for the current frame. Extracted image feature map; Indicates the first Mask for each target observation; This represents a mask weighting operation; Indicates pooling operation; This indicates a feature normalization operation; the visual feature extraction model employs a convolutional neural network, a visual Transformer, a self-supervised visual model, or a multimodal visual encoder.

[0016] Preferably, the mask quality score in S2 is determined by at least two of the following: model confidence, boundary stability, area reasonableness, internal consistency of the region, and conflict penalty; the area reasonableness score is calculated by the change between the current mask area and the historical average area of ​​the trajectory; when the mask quality score is lower than the mask quality threshold, the current observation is not directly used to update the historical trajectory.

[0017] Preferably, the method for determining the update weight based on appearance similarity in S2 includes the following steps: calculating the cosine similarity between the appearance features of the current target observation and the appearance feature memory of the corresponding target trajectory, and determining a similarity gate value based on the cosine similarity; combining the similarity gate value with the mask quality score to obtain the appearance feature memory update weight; pausing the current target observation and writing to the appearance feature memory when the appearance feature memory update weight is lower than the update threshold; and updating the appearance feature memory using an exponential moving average method when the appearance feature memory update weight is not lower than the update threshold, the mask quality score of the current target observation is not lower than the mask quality threshold, and the corresponding target trajectory is not in the current frame suppression, absorption, or termination state.

[0018] Preferably, in step S2, for a target trajectory in a dormant state, a historical identity recovery score is calculated based on the appearance similarity between the current target observation and the appearance feature memory, the spatial context consistency score between the current target observation position and the historical motion prediction position, the mask quality score of the current target observation, and the trajectory dormancy time penalty term; when the historical identity recovery score reaches the identity recovery threshold, the corresponding target trajectory is restored from a dormant state to an active state.

[0019] Preferably, the spatial overlap relationship is confirmed when the cross-union ratio reaches the cross-union ratio threshold, or when the minimum containment overlap rate reaches the containment overlap threshold. The requirement for marking the spatial overlap relationship as a candidate for ordinary competition relationship and cross-granularity containment relationship is that the minimum containment overlap rate reaches the containment overlap threshold and the area ratio between active trajectories satisfies the area difference condition.

[0020] As a preferred option, the competition score in S4 is uniformly calculated within each connected component of the competition relationship graph, and a group-level multi-factor score is performed on each trajectory in the competition trajectory group based on appearance consistency score, historical duration score, recent stability score, recent mask quality score, spatial coverage rationality score, trajectory anomaly penalty item and corresponding weight coefficient.

[0021] As a preferred method, the competitive scoring method is determined by weighting the appearance consistency score, historical duration score, recent stability score, recent mask quality score, spatial coverage rationality score, and trajectory anomaly penalty item.

[0022] Preferably, the method for performing retention, suppression, or merging of competitive trajectories in S4 is as follows: The highest-scoring competitive trajectory and the second-highest-scoring competitive trajectory are determined in the competitive trajectory group; when the difference between the highest-scoring competitive trajectory and the second-highest-scoring competitive trajectory reaches a score difference threshold, the highest-scoring competitive trajectory is retained, and at least one of the following processes is performed on the remaining competitive trajectories: current frame suppression, reduction of subsequent competitive weights, or suspension of current frame memory updates; when the score difference between the highest-scoring competitive trajectory and the second-highest-scoring competitive trajectory is less than a score difference threshold, and their appearance similarity, crossover ratio, and number of consecutive overlapping frames reach their respective thresholds, trajectory merging is performed; during merging, the identity of the target trajectory with a longer historical duration or higher stability is retained, and the historical mask sequence and appearance feature memory are fused according to the mask quality.

[0023] Preferably, step S5 includes: statistically analyzing the proportion of candidate component trajectories being contained within the candidate overall trajectory within a preset time window to determine the stability of cross-granularity inclusion relationships; calculating the reliability of the component trajectory and the overall trajectory based on the average recent stability, average mask quality, and time support length normalized score of the target trajectory within the time window; when the overall trajectory reliability is higher than the component trajectory reliability and the difference between the two reaches an absorption threshold, absorbing the component trajectory into the overall trajectory; when the component trajectory reliability is higher than the overall trajectory reliability and the difference between the two reaches a retention threshold, retaining the component trajectory and suppressing or splitting the overall trajectory; when the overall trajectory... If the difference between reliability and component trajectory reliability does not meet either of the aforementioned absorption and retention conditions, absorption or splitting is temporarily suspended, and observation continues. After completing the aforementioned reliability difference judgment, when the same candidate overall trajectory corresponds to multiple candidate component trajectories, the joint coverage of the multiple candidate component trajectories to the candidate overall trajectory reaches the coverage threshold, and the reliability of each candidate component trajectory is higher than the reliability of the candidate overall trajectory, the aforementioned processing result is updated to suppress or split the candidate overall trajectory and retain the candidate component trajectory set. When the joint coverage condition is not met, the processing result determined based on the aforementioned reliability difference is maintained.

[0024] As a preferred method, the output of the integrated target trajectory set and target identity in S6 includes: participating active target trajectories in the current frame observation matching, memory update and competition relationship judgment; retaining the appearance feature memory of dormant target trajectories and participating in historical identity recovery judgment; removing terminated target trajectories from subsequent matching and output; when the number of historical appearance features exceeds the preset capacity, prioritizing the retention of observation features with high mask quality, observation features that differ significantly from existing features, observation features when the target undergoes significant viewpoint or scale changes, the first stable observation feature after trajectory recovery, and observation features before and after trajectory competition or merging, while deleting low-quality features whose similarity reaches the redundancy threshold.

[0025] Compared with existing technologies, the quality-constrained long video target trajectory integration and identity preservation method provided by the present invention has at least the following beneficial effects:

[0026] (1) By using both mask quality and appearance similarity to control appearance feature memory updates, the pollution of historical identity memory by low-quality masks and anomalous observations can be reduced, and the reliability of historical identity recovery after long-term loss can be improved.

[0027] (2) By simultaneously calculating the intersection-union ratio and the minimum overlap rate, and combining the trajectory area difference to divide the overlap relationship, it is possible to distinguish between ordinary repeated trajectory competition and cross-granularity inclusion relationship, thereby reducing the situation of misjudging component trajectories as repeated trajectories.

[0028] (3) By establishing a competitive relationship graph and performing group scoring within the connected component range based on appearance consistency, historical duration, recent stability, recent mask quality, spatial coverage rationality and trajectory anomaly penalty, the impact of pair processing order on trajectory preservation results can be reduced.

[0029] (4) By comparing the reliability of component trajectories and the reliability of the overall trajectory within the time window, the component absorption, overall splitting, trajectory retention or delay judgment can be performed bidirectionally based on actual evidence, thereby reducing the granularity of errors caused by unidirectional absorption.

[0030] (5) By compressing historical memory based on observation quality, feature differences and trajectory key events, it is possible to retain historical observations that are more valuable for identity recovery and conflict resolution while controlling the storage scale. Attached Figure Description

[0031] Figure 1 This is a flowchart of the overall process for the long video target trajectory integration and identity preservation method with quality constraints according to an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram of the target trajectory data structure according to an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of quality coupling memory update according to an embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram illustrating the restoration of historical identity according to an embodiment of the present invention.

[0035] Figure 5 This is a schematic diagram of overlap detection, dual-path splitting, and competition graph construction according to an embodiment of the present invention.

[0036] Figure 6 This is a schematic diagram of multi-factor scoring and conflict resolution of the competitive trajectory group according to an embodiment of the present invention.

[0037] Figure 7 This is a schematic diagram of time-evidence-driven bidirectional cross-granularity trajectory decision-making according to an embodiment of the present invention.

[0038] Figure 8 This is a schematic diagram of trajectory state transition and event-driven memory compression according to an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0040] See Figure 1 The present invention provides a method for quality-constrained long video target trajectory integration and identity preservation, as illustrated in the following steps:

[0041] S1. Input the current video frame, the target observation set of the current frame, and the existing target trajectory set;

[0042] S2. Extract appearance features from target observations in the current frame and calculate mask quality scores. Determine update weights based on the appearance similarity between the current target observation and the corresponding historical trajectory. For dormant target trajectories, it is necessary to determine the historical identity recovery results.

[0043] S3. Calculate the intersection-union ratio and minimum overlap rate between the current masks of active trajectories to determine the spatial overlap relationship; combine area differences to distinguish between ordinary competitive relationships and cross-granularity inclusion relationships; construct a competitive relationship graph with active trajectories as nodes and competitive relationships as edges, and form competitive trajectory groups according to connected components;

[0044] S4. Calculate the competition score of each target trajectory in the competition trajectory group, and perform preservation, suppression or merging processing on the competition trajectories based on the competition score, appearance similarity and the persistence of spatial overlap.

[0045] S5. Within a preset time window, statistically analyze the stability of cross-granularity inclusion relationship candidates, calculate the difference in reliability between the component trajectory and the overall trajectory to form a preliminary cross-granularity decision, thereby deciding whether to absorb, split, or postpone.

[0046] S6. Based on the historical identity recovery results in S2, the competitive trajectory processing results in S4, and the cross-granularity trajectory processing results in S5, update the active, dormant, or terminated status of each trajectory, update, compress, and prune the appearance feature memory and historical mask sequence, and output the integrated target trajectory set and target identity identifier.

[0047] In long video scenarios, targets may temporarily disappear due to occlusion, going out of frame, changes in viewpoint, or changes in lighting, and then reappear. Simultaneously, video segmentation models may produce low-quality or granularly unstable masking results in different frames, leading to the same target being split into multiple trajectories, or multiple trajectories repeatedly covering the same area. This invention first uses mask quality constraints to write historical memory and restore identity, then uses intersection-union ratio, minimum overlap rate, and area difference to separate overlapping relationships; for ordinary competitive relationships, multi-factor group scoring is performed within the connected components of the competitive relationship graph; for cross-granularity inclusion relationship candidates, the reliability of component trajectories and the overall trajectory are compared within a time window, and bidirectional decision-making is performed; finally, historical memory is compressed by combining trajectory status and key events, thereby reducing false identity restoration, false suppression of group trajectories, and false merging of cross-granularity trajectories.

[0048] Furthermore, S1, input the current video frame, the target observation set of the current frame, and the existing target trajectory set, and read the identity identifier, historical mask sequence, appearance feature memory, trajectory stability, recent mask quality, historical duration, number of consecutive missing frames, and trajectory status of each target trajectory.

[0049] Specifically, the processing procedure for step S1 is as follows:

[0050] First, read the long video sequence: ;in, Represents the image of frame t. This indicates the total number of frames in the video.

[0051] See Figure 2 The schematic diagram of the trajectory data structure shows that each trajectory includes an identity identifier, historical mask sequence, historical appearance feature set, trajectory appearance feature memory, stability score, recent mask quality score, historical duration, number of consecutive missing frames, and trajectory status.

[0052] Specifically, in frame t, the existing set of target trajectories is read: .in, Indicates the first Target trajectory, This indicates the number of tracks maintained at the current moment. Since targets may enter, leave, hibernate, or terminate in a long video, the number of tracks will change over time. Each target track... Includes the following information: .in, Indicates the identity of the trajectory; Represents the historical mask sequence, Represents a set of historical appearance features; Indicates up to the number Memorizing the trajectory appearance features at each frame time; Indicates the trajectory stability score; This indicates the recent mask quality score of the trajectory; Indicates the duration of the trajectory's history; Indicates the number of consecutively missing frames; This indicates the trajectory status, which includes active, dormant, and terminated.

[0053] An active trajectory is a target trajectory that is in an active state. Active trajectories participate in the detection of overlap relationships and the construction of the competition relationship graph in the current frame; dormant trajectories do not participate in regular spatial competition, but retain appearance feature memory and participate in historical identity recovery and discrimination.

[0054] The target observation set for the current frame is denoted as: .in, This indicates the number of target observations in the current frame. Each observation... include: .in, Indicates the current frame number The mask for each observation; This describes the external characteristics of the observation; This indicates the mask quality score for the observation; Indicates the spatial location or bounding box of the observation; This indicates the center point of the observation.

[0055] The correspondence between the current target observation and the active trajectory can be determined by the association results output by the upstream target detection, video mask propagation, or multi-target tracking modules. When the upstream module does not provide association results, a candidate correspondence can be established based on at least one of the following: appearance similarity, spatial location consistency, and mask overlap between the current target observation and the active trajectory. This invention does not limit the specific method of establishing the correspondence.

[0056] If active trajectory If a matching observation exists in the current frame based on the above correspondence, then this observation is denoted as... ,in Representation and Trajectory The index of the current frame observation that matches; if no reliable matching observation exists, the trajectory is considered an unmatched trajectory or a missing trajectory in the current frame.

[0057] The aforementioned current frame target observation set can be generated by any target detection model, image segmentation model, video mask propagation model, multi-target tracking model, or manually labeled results; this invention does not limit its specific source. This invention is deployed after the upstream target observation generation module, and its processing objects are the current frame target observation set output by the upstream module and the existing target trajectory status.

[0058] Furthermore, S2 extracts appearance features from the target observation in the current frame and calculates the mask quality score. Based on the mask quality score and appearance similarity, it controls the appearance feature memory update and performs multi-factor historical identity recovery judgment on the dormant trajectory.

[0059] Specifically, the processing procedure in step S2 is as follows:

[0060] S2 includes current observation appearance feature extraction, mask quality assessment, quality-coupled trajectory memory update control, and multi-factor historical identity recovery discrimination.

[0061] S2.1 The current method for extracting appearance features is as follows:

[0062] For each target observation in the current frame First, the appearance features are extracted based on the mask region. The appearance feature extraction can be represented as: .

[0063] in, This indicates the visual feature extraction model for the current frame. Extracted image feature map; Indicates the first Mask for each target observation; This represents a mask weighting operation; Indicates pooling operation; This represents the feature normalization operation. Visual feature extraction models can employ convolutional neural networks, visual Transformers, self-supervised visual models, or multimodal visual encoders.

[0064] S2.2 The current observation mask quality score is calculated as follows:

[0065] Calculate the mask quality score for the current target observation. The mask quality score is determined by a weighted average of at least two of the following: model confidence, boundary stability, area reasonableness, internal region consistency, and conflict penalty. Each quality indicator is normalized to the [0,1] interval before being included in the calculation; when a quality indicator cannot be obtained from the upstream target observation generation module, the weight corresponding to that indicator is set to zero, and the weights of the remaining available quality indicators are normalized. The mask quality score is expressed as: .in, This indicates the confidence level of the model output; Indicates the boundary stability score; The score indicates the reasonableness of the area. Indicates the consistency score within the region; Indicates penalties for conflicts with other targets or abnormal areas; to This represents the weighting coefficient; the weights are set as follows: This means that the score will be limited to the interval [0,1].

[0066] The area reasonableness score is calculated by comparing the current mask area with the historical average area of ​​the trajectory. .in, Indicates the current observation mask area. Indicates the corresponding historical trajectory The average mask area, To prevent small constants from being divided by zero.

[0067] when Below the mask quality threshold At this time, current observations are not directly used to update historical trajectories. Preferably, It can be between 0.4 and 0.6, for example, 0.5. The method for determining the reliable mask is as follows: statistically analyze the reliable mask and the abnormal mask on representative verification videos. The distribution scans candidate thresholds within the range of 0.4 to 0.6, with selection criteria including an abnormal observation write rate not exceeding a preset upper limit and an effective observation retention rate not falling below a preset lower limit; 0.5 is a set of reproducible example values. The mask quality score is also used to control whether the current observation participates in historical identity recovery and competitive score calculation, thereby ensuring that quality constraints are applied throughout the trajectory update, identity recovery, and group conflict resolution processes.

[0068] S2.3 The trajectory memory update control method of quality coupling is as follows:

[0069] See Figure 3 , Figure 3This is a schematic diagram of quality-coupled memory update. The diagram shows that after the current observation undergoes feature extraction and mask quality assessment, a similarity gating value is obtained based on appearance similarity. The similarity gating value is then combined with the mask quality score to determine the memory update weight. For dormant trajectories, the historical identity recovery score is further calculated based on appearance, spatial context, mask quality, and dormancy duration.

[0070] For active trajectory and matching current target observation Calculate the current observed appearance features With appearance feature memory Cosine similarity between them: .

[0071] in, Representing the trajectory Compared with current observations The appearance similarity. The cosine similarity value ranges from [-1, 1], and can be mapped to the [0, 1] interval in the following way: .

[0072] Set a similarity gating function based on appearance similarity:

[0073] .in, and These represent the low similarity threshold and the high similarity threshold, respectively, in typical video object segmentation scenarios. , This segmented gating corresponds to three processing segments: when the similarity is lower than... Write refused; located and When writing between them, the weight is increased linearly based on similarity; not less than Full write privileges are allowed at this time. and It can be determined by the similarity distribution between positive samples on the same trajectory and negative samples on different trajectories or by the identity switching rate on the validation set.

[0074] The current observations' update weights for the trajectory are: .

[0075] when Less than the update threshold At that time, the appearance feature memory of the trajectory is not updated. Preferably, A value of 0.35 to 0.50 is acceptable. Acting on and The product can be determined by a validation set scan in the range of 0.35 to 0.50, with the selection based on a trade-off between the historical memory contamination rate and the effective update retention rate.

[0076] when When the update conditions are met, the memory of the trajectory's appearance features can be updated using an exponential moving average method: The update conditions include at least the following: Not less than , Not less than And trajectory It is not in a state of current frame contention suppression, absorption, or termination. Among them, The historical memory retention coefficient is preferably set between 0.7 and 0.95.

[0077] S2.4 The method for multi-factor historical identity recovery judgment is as follows:

[0078] See Figure 4 , Figure 4 This is a schematic diagram of historical identity restoration. For target trajectories in a dormant state, the system does not directly accept the upstream propagation results. Instead, it comprehensively considers the appearance similarity between the current target observation and the trajectory's appearance feature memory, spatial context consistency, mask quality, and dormancy duration to determine whether to restore the original identity.

[0079] When a historical trajectory is in a dormant state, the system determines whether a newly appearing observation in the current frame represents the recovery of the historical identity of the target corresponding to that historical trajectory. For dormant trajectories... and current observations Calculate the historical identity restoration score: .in, Indicates the visual similarity between the current observation and historical trajectories; The spatial context consistency score is represented by the following: This represents the mask quality score for the current observation; This indicates the penalty for track sleep time; This represents the weighting coefficient. In a preferred embodiment, the following can be set: Spatial context consistency score can be calculated based on the distance between the current observation location and the historical motion prediction location: .in, Indicates the current observation center point. This represents the predicted center point estimated based on historical motion. This represents the image scale normalization parameter. The contributions of appearance, spatial context, mask quality, and sleep duration are controlled respectively; each input is first normalized to [0,1], and then grid search, cross-validation, or constrained learning is performed with identity false recovery rate and recurrence recall rate as indicators. The listed values ​​are the initial values ​​of the example.

[0080] The penalty for hibernation time is: .in, Indicates the last visible frame number of the trajectory. This indicates the maximum sleep window. Preferably, 100 to 500 frames can be selected.

[0081] If the following conditions are met: If so, the current observation is determined to be a dormant trajectory. The reappearance of the trajectory restores its dormant state to active. Preferably, A value between 0.65 and 0.80 is acceptable. Among them, Indicates the hibernation trajectory Compared with current observations The historical identity recovery score for frame t. Indicates the threshold for restoring historical identity; The accuracy can be determined based on the false identity recovery rate and identity recovery recall rate in the verification video.

[0082] Furthermore, S3, based on the double overlap index, divides the overlap relationship between active trajectories and constructs a competition relationship graph.

[0083] Specifically, step S3 is used to identify ordinary repeating trajectory competition and cross-granularity inclusion relationship candidates, organizes ordinary competition relationships into a competition relationship graph, and sends cross-granularity inclusion relationship candidates to step S5 for time window verification. The specific processing procedure is as follows:

[0084] See Figure 5 , Figure 5 The diagram illustrates the process of overlapping relationship detection, dual-path splitting, and competitive relationship graph construction. The diagram shows the two-path overlapping indices of intersection-union ratio and minimum overlap rate, area difference condition, ordinary competitive relationship branches, cross-granularity inclusion relationship candidate branches, and the process of forming competitive trajectory groups based on connected components.

[0085] S3.1 The method for trajectory overlap detection and dual-path splitting is as follows:

[0086] For trajectory and trajectory Current mask and Calculate the intersection-union ratio: Simultaneously calculate the minimum overlap rate: .

[0087] An overlapping correlation is determined between two trajectories when any of the following conditions are met:

[0088] or .in, The value can be between 0.45 and 0.65, preferably 0.5; The value can be between 0.70 and 0.90, with 0.8 being preferred. Used to measure symmetrical overlap Used to measure the extent to which a smaller mask is covered; and The selection can be based on the verification distribution of repeated trajectory pairs, component-to-whole trajectory pairs, and non-associated trajectory pairs.

[0089] Furthermore, the requirements for labeling spatial overlap relationships as candidates for ordinary competition relationships and cross-granularity inclusion relationships are that the minimum inclusion overlap rate reaches the inclusion overlap threshold and the area ratio between active trajectories satisfies the area difference condition, i.e. When the area difference between the two trajectories is high and significant, it indicates that the trajectory pair may simultaneously exhibit both ordinary competition and cross-granularity inclusion relationships. In this case, in addition to including the trajectory pair in the competition detection, the system can also simultaneously send the trajectory pair to the cross-granularity inclusion relationship time verification process in step S5 to avoid misjudging the cross-granularity inclusion relationship as an ordinary repeating trajectory competition relationship.

[0090] In the current frame, the system constructs a competition relationship graph using the active trajectories as nodes and the competition relationships as edges, and obtains one or more groups of competition trajectories based on the connected components: .in, Indicates the first A group of competing trajectories This indicates the number of competing trajectory groups in the current frame. A set of competing trajectories can be represented as: .in, This indicates the number of trajectories in the competing trajectory group. The relationship splitting does not directly determine the trajectory merging or absorption result, but instead sends the trajectory pairs to the group conflict resolution process and the cross-granularity inclusion relationship time verification process respectively based on the overlap pattern.

[0091] Intersection over Union (IoU) ratio is primarily used to identify overlapping relationships between trajectories with similar areas, while minimum overlap ratio (MINUR) is primarily used to identify situations where a smaller trajectory is covered by a larger trajectory. Combining IoU ratio with dual-path splitting avoids directly deleting smaller trajectories simply because of a higher overlap ratio.

[0092] S3.2 The multi-factor scoring method for the competition trajectory group is as follows:

[0093] Calculate a score for each trajectory in the competing trajectory group: .in, Indicates the score for appearance consistency; The score indicates the duration of the historical period. Indicates recent stability score; This indicates the recent mask quality score; Indicates the score for the reasonableness of spatial coverage; This indicates the penalty for abnormal trajectory. This represents the weighting coefficient. In a preferred embodiment, the following can be set: If the trajectory Observation corresponding to the current frame ,but: , If the trajectory If no reliable observation is available in the current frame, the corresponding index from the previous frame can be used for attenuation, or it can be set to a lower value. All scores are normalized to [0,1] before weighting; weight coefficients can be determined through grid search, cross-validation, or constrained learning on representative verification videos, with identity switching rate, repeated trajectory rate, and false suppression rate as joint evaluation indicators. The competitive score is calculated uniformly within the connected components of the competitive relationship graph, rather than comparing two trajectories independently in pairs.

[0094] The historical duration score is: .in, This indicates the number of frames the trajectory has remained visible. This represents the normalization constant. Preferably, 30 to 100 frames can be selected.

[0095] Recent stability score: .in, This represents the historical stability retention coefficient, which can be taken as 0.7 to 0.9; This represents the current frame stability observation value.

[0096] The current frame stability observation can be determined by both mask temporal consistency and area change consistency: .in, This represents the current frame mask predicted from the previous frame. A value of 0.5 to 0.8 is acceptable. This indicates the weight of the mask time consistency term. Indicates the weight of the area change consistency term; The selection can be based on verification videos that include occlusion, scale changes, and mask jitter.

[0097] Set up a set of competitive trajectories The union of the covered areas is: .

[0098] The spatial coverage rationality score is: .

[0099] The first term represents the degree of matching between the current trajectory mask and the competing region, while the second term represents the consistency between the current trajectory area and the historical average area. This design avoids simply rewarding large-area masks, ensuring that spatial coverage rationality considers both coverage relationships and historical shape stability.

[0100] Track anomaly penalty item It can be determined by a combination of factors such as the number of consecutive contention failures, the number of consecutive low-quality frames, abnormal area jumps, and historical identity instability. For example: .

[0101] in, This represents the normalized score indicating consecutive failed competitions. This indicates a series of low-quality normalized scores. Indicates abnormal jumps. These are the weighting coefficients. Specifically, It can be obtained by dividing the number of consecutive competition failures by the failure count window and truncating it to 1. It can be obtained by dividing the number of consecutive low-quality frames by the low-quality window and truncating it to 1. It can be obtained by normalizing the logarithmic ratio of the mask areas of adjacent frames or the center position jump variable; This can be determined through validation set scanning or learning.

[0102] Furthermore, S4, based on the multi-factor scoring results of the competing trajectory groups, performs group-level resolution of trajectory conflicts and performs trajectory merging when appearance, spatial and temporal conditions are met.

[0103] Specifically, step S4 compares the competition scores of each trajectory within each connected component of the competition graph, performing trajectory preservation, current frame suppression, pausing memory updates, or conditional merging. The processing procedure for step S4 is as follows:

[0104] See Figure 5 and Figure 6 Figure 6 is a schematic diagram of multi-factor scoring and conflict resolution for competing trajectory groups. The figure shows a competing trajectory group formed by three or more overlapping trajectories. After multi-factor scoring, the highest-scoring trajectory is retained, other trajectories are temporarily suppressed, or similar trajectories that meet the conditions are merged.

[0105] For the competitive trajectory group Calculation of all trajectories Then, select the highest-rated trajectory: .set up This indicates the highest score in the competitive group. This indicates the second highest score. If the difference between the highest and second highest scores exceeds a threshold... Then, the highest-scoring trajectory will be retained, while other competing trajectories will be suppressed: .in, The value can be from 0.10 to 0.25, preferably 0.15.

[0106] For suppressed trajectories, their subsequent competition weights can be reduced, the current frame mask output can be paused, or the current frame appearance feature memory update can be paused. Current frame suppression is not the same as immediately deleting the trajectory. Whether to switch to a dormant or terminated state is determined by step S6 based on the number of consecutive competition failures and trajectory quality.

[0107] If two trajectories have similar scores, but high appearance similarity and overlap over multiple consecutive frames, then trajectory merging is performed. First, the appearance similarity between the two trajectories is defined: The trajectory merging condition is: .and and .

[0108] And the above conditions are met continuously for no less than Frame. Preferably, A value of 0.75 to 0.90 is acceptable. A value of 0.55 to 0.75 is acceptable. 3 to 10 frames can be selected. , and Constraints on appearance consistency, spatial overlap, and temporal persistence can be determined using a joint index of false merging rate and duplicate trajectory elimination rate within the specified range.

[0109] When the system determines two trajectories and When merging is required, generate merge trajectories according to the following rules:

[0110] (1) Retain the trajectory identity with a longer historical duration or higher stability score;

[0111] (2) Merge the historical mask sequences of the two trajectories in chronological order;

[0112] (3) For masks in overlapping frames, retain the one with higher mask quality, or perform weighted fusion based on mask quality;

[0113] (4) Merge the memory of the appearance features of the two trajectories;

[0114] (5) Mark the merged trajectory as terminated and record the source of the merge.

[0115] The merged appearance features are remembered as follows: .

[0116] in: . .in, , , , , , All scores are normalized scores for stability, quality, and historical duration, with values ​​ranging from [0,1]. , Representing trajectories and The proportion of appearance feature memory in the merged results.

[0117] If a certain trajectory is continuous If the score is lower than other trajectories in each competition, then step S6 adjusts its state from active to dormant or terminated. Preferably, A value of 5 to 15 is acceptable.

[0118] Furthermore, S5 performs time-based evidence verification on cross-granularity inclusion relationship candidates and performs bidirectional cross-granularity trajectory decision-making based on the reliability of component trajectories and overall trajectories.

[0119] Specifically, S5 is used to identify the trajectory representations of the same physical object at different segmentation granularities, and determines the representation granularity to be retained based on multidimensional reliability evidence within the time window, rather than defaulting to retaining the entire trajectory or only performing unidirectional component absorption. The processing procedure of step S5 is as follows:

[0120] See Figure 7 , Figure 7 This is a schematic diagram of time-evidence-driven bidirectional cross-granularity trajectory decision-making. For cross-granularity inclusion candidates verified by a time window, a preliminary cross-granularity decision is first formed based on the difference between the reliability of the component trajectory and the reliability of the overall trajectory. The preliminary cross-granularity decision includes the overall trajectory absorbing the component trajectory, retaining the component trajectory and suppressing or splitting the overall trajectory, and temporarily suspending processing and continuing observation. When the same candidate overall trajectory corresponds to multiple candidate component trajectories, a multi-component joint coverage verification is further performed after the preliminary cross-granularity decision. When the multi-component joint coverage condition is met, the aforementioned preliminary cross-granularity decision is updated with the multi-component joint decision; otherwise, the aforementioned preliminary cross-granularity decision is maintained.

[0121] For two trajectories and ,like The mask is mostly located in Inside the mask, and The area is significantly smaller ,but It can be regarded as a candidate component trajectory. It can be considered as a candidate overall trajectory.

[0122] Define the proportions of the components: .

[0123] The area ratio is defined as follows: .

[0124] When the following conditions are met: and At that time, it was believed and There are candidate cross-granularity inclusion relationships. Preferably, The value can be between 0.75 and 0.95, preferably 0.85; The value can be from 0.05 to 0.65, preferably 0.60.

[0125] To avoid misjudgment in a single frame, the present invention further incorporates a time window... Stability of internal statistical cross-granularity inclusion relationships:

[0126] .

[0127] in, This indicates an indicator function that takes the value 1 if the condition is true and 0 otherwise. Indicates a time window; Representing the trajectory Trajectory The proportion of stable inclusion.

[0128] If the following conditions are met: This confirms the existence of a stable cross-granularity inclusion relationship. Preferably, a time window... 5 to 30 frames can be selected. A value of 0.5 to 0.8 is acceptable.

[0129] When component trajectory is detected and overall trajectory Then, a comprehensive comparison of the two within the time window was made. Within this range, consider recent stability, mask quality, and time support length. Define component trajectory reliability: .

[0130] Define the overall trajectory reliability: .in, and These represent the component trajectory and the overall trajectory within the time window, respectively. Average short-term stability within the range; and These represent the component trajectory and the overall trajectory within the time window, respectively. Average mask quality within; and These represent the component trajectory and the overall trajectory within the time window, respectively. The time within the time frame supports length-normalized scores. Preferably, Based on the difference between the overall trajectory reliability and the component trajectory reliability, a preliminary cross-granularity decision is first formed regarding the relationship between the individual component trajectory and the overall trajectory.

[0131] If the following conditions are met: , This is the absorption threshold.

[0132] The overall trajectory is considered more reliable, and cross-granularity decision-making is performed on the overall trajectory absorbing component trajectory. Preferably, The value can be between 0.10 and 0.25. The absorption operation includes: preserving the overall trajectory identity; using the valid historical frames of the component trajectory as auxiliary evidence for the overall trajectory; canceling the independent identity output of the component trajectory; marking the component trajectory status as terminated or absorbed; and recording the absorption relationship between the component trajectory and the overall trajectory.

[0133] If the following conditions are met: , To retain the threshold.

[0134] If the reliability of a component trajectory is higher than that of the overall trajectory, and the difference between the two reaches a retention threshold, then the component trajectory is considered more reliable, the component trajectory is retained, and the overall trajectory is suppressed or split. Preferably, A value of 0.10 to 0.25 is acceptable.

[0135] When neither the conditions for absorbing the component trajectory nor the conditions for retaining the component trajectory are met, it is considered that the reliability evidence within the current time window is insufficient. The system will not perform absorption, suppression, or splitting for the time being, but will retain the relevant trajectory and continue to observe for several frames.

[0136] After forming the aforementioned preliminary cross-granularity decision, if the same candidate overall trajectory corresponds to two or more candidate component trajectories, then a joint coverage verification is further performed on multiple component trajectories. For a candidate overall trajectory... With the corresponding candidate component trajectory set Calculate the joint coverage of the component trajectory set with respect to the candidate overall trajectory. .

[0137] When the combined coverage rate reaches the coverage threshold When the reliability of each candidate component trajectory is higher than that of the overall candidate trajectory, it indicates that multiple component trajectories can collectively replace the overall trajectory with a more reliable fine-grained representation. In this case, the aforementioned preliminary cross-granularity decision is updated to: retain the set of component trajectories and suppress or split the overall trajectory. Preferably, A value of 0.60 to 0.85 is acceptable.

[0138] When there are no two or more candidate component trajectories, or when the joint coverage and component trajectory reliability do not simultaneously meet the above conditions, the aforementioned preliminary cross-granularity decision is not updated, and the preliminary cross-granularity decision is executed according to the aforementioned overall trajectory absorbing component trajectories, retaining component trajectories and suppressing or splitting the overall trajectory, or temporarily suspending processing and continuing to observe.

[0139] Furthermore, S6 updates the active, dormant, or terminated status of each trajectory based on the historical identity recovery results, group conflict resolution results, and bidirectional cross-granularity trajectory decision results, performs event-driven historical memory compression, and outputs the integrated target trajectory set and target identity identifier.

[0140] Specifically, S6 is used to uniformly update the trajectory lifecycle state and, based on observation quality, differences in appearance features, changes in perspective, and key trajectory events, to perform capacity control, compression, and pruning on the appearance feature memory and historical mask sequence. The processing procedure for step S6 is as follows:

[0141] See Figure 8 , Figure 8 This diagram illustrates the trajectory state transition and event-driven memory compression. It shows the transition relationship between the trajectory and the three states of active, dormant and terminated, as well as the memory retention process of high-quality frames, perspective change frames, the first stable frame after historical identity restoration, and key frames before and after group conflict resolution or merging.

[0142] S6.1 Trajectory State Definition

[0143] This invention sets each trajectory state into three categories: active, dormant, and terminated.

[0144] An active state indicates that the target is currently visible and the trajectory quality meets the requirements. Active trajectories participate in current frame observation matching, quality coupling memory updates, competition graph construction, group conflict resolution, and cross-granularity relationship determination.

[0145] A dormant state indicates that the target is currently invisible or temporarily missing, but still retains memories of its appearance features and the possibility of identity recovery. Dormant trajectories do not participate in regular spatial competition, but participate in multi-factor historical identity recovery discrimination.

[0146] The terminated state indicates that the target is not visible for a long time, its quality is consistently insufficient, it has been absorbed or merged by other trajectories, and it no longer participates in subsequent matching and output.

[0147] S6.2, The state transition rules are as follows:

[0148] The conditions for transitioning from an active state to a dormant state include: And the historical stability of the trajectory satisfies: .in, Indicates the number of consecutive missing frames. 5 to 30 frames can be selected. A value of 0.5 to 0.7 is acceptable. The conditions for transitioning from a dormant state to an active state are: That is, at least one observation in the current observation set has a historical identity recovery score for a dormant trajectory that exceeds a threshold. The condition for transitioning from a dormant state to a terminated state is: or .in, Indicates the number of frames the sleep lasts. 100 to 500 frames can be selected; This represents the minimum hibernation observation window, which can be 20 to 50 frames. Indicates the most recent time window The dormant trajectory is compared with the maximum historical identity recovery score of all candidate observations; A value of 0.4 to 0.6 is acceptable. This design can prevent premature termination of observations simply because a reliable identity cannot be found within a short period of time after the trajectory has just entered a dormant state.

[0149] The conditions under which an active state can directly transition to a terminated state include:

[0150] (1) The mask quality of multiple consecutive frames is lower than the threshold;

[0151] (2) It is repeatedly suppressed in the process of resolving group conflicts;

[0152] (3) Absorbed by the stable overall trajectory;

[0153] (4) Merging with another highly stable trajectory;

[0154] (5) It is determined to be a short-term noise trajectory or a repetitive trajectory.

[0155] The condition of continuous low quality can be expressed as: The condition for continuous competition failure can be expressed as: .in, Indicates the number of consecutive low-quality frames. This indicates the number of consecutive failed attempts.

[0156] S6.3 The method for event-driven trajectory-level historical memory compression and pruning is as follows:

[0157] To accommodate long video processing, this invention performs capacity control on the memory of appearance features and historical mask sequences for each trajectory. For each trajectory, a maximum of [missing information] is retained. Key appearance features. Preferably, A value of 5 to 30 can be selected.

[0158] When the number of historical features exceeds When the time is right, features are retained according to the following priority: the retention priority is not determined solely by the time, but is determined comprehensively based on observation quality, appearance differences, perspective changes and the importance of identity events.

[0159] (1) Frames with high mask quality;

[0160] (2) Frames that differ significantly from existing preserved features;

[0161] (3) Frames in which the target undergoes significant changes in viewpoint or scale;

[0162] (4) The first stable observation frame after the restoration of historical identity;

[0163] (5) Keyframes before and after group conflict resolution or trajectory merging.

[0164] For historical features that are too similar, only one representative feature is retained. If two historical features are... and The cosine similarity between them is higher than If the quality is low, then the lower-quality entries will be deleted. Preferably, A value between 0.90 and 0.98 is acceptable. For historical mask sequences, priority is given to retaining key masks corresponding to high-quality observations, significant viewpoint changes, significant scale changes, historical identity restoration, trajectory competition, trajectory merging, component trajectory absorption, overall trajectory splitting, and trajectory state transition events. For continuous masks with high overlap between adjacent frames, small area changes, and small center position changes, they are identified as redundant masks. Representative masks are retained or non-critical redundant masks are deleted at fixed time intervals, thereby achieving compression and pruning of historical mask sequences.

[0165] S6.4 Output Results.

[0166] After completing the quality-coupled memory update control, multi-factor historical identity recovery, group conflict resolution, and bidirectional cross-granularity trajectory decision-making and state management, the system outputs the integrated target trajectory set: .in, This indicates the number of tracks after integration, which is usually less than or equal to the number of tracks before integration. .

[0167] The output includes: the identity of each target trajectory, the mask sequence of each target in each frame of the video, trajectory status, trajectory merging record, cross-granularity trajectory decision record, historical identity recovery record, appearance feature memory update and compression record, and optional trajectory quality statistics.

[0168] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0169] It is understood that the relevant features in the above methods and systems can be referenced interchangeably. Furthermore, the terms "first," "second," etc., in the above embodiments are used to distinguish between embodiments and do not represent the superiority or inferiority of any particular embodiment.

[0170] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0171] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0172] In addition, the memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0173] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0175] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0177] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0178] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0179] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0180] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0181] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for integrating and preserving the identity of target trajectories in long videos with quality constraints, characterized in that, Includes the following steps: S1. Input the current video frame, the target observation set of the current frame, and the existing target trajectory set; S2. Extract appearance features from the target observation in the current frame and calculate the mask quality score. Determine the update weight based on the appearance similarity between the current target observation and the corresponding historical trajectory. For target trajectories in a dormant state, it is necessary to determine the results of historical identity recovery; S3. Calculate the intersection-union ratio and minimum overlap rate between the current masks of active trajectories to determine the spatial overlap relationship; combine area differences to distinguish between ordinary competitive relationships and cross-granularity inclusion relationships; construct a competitive relationship graph with active trajectories as nodes and competitive relationships as edges, and form competitive trajectory groups according to connected components; S4. Calculate the competition score of each target trajectory in the competition trajectory group, and perform preservation, suppression or merging processing on the competition trajectories based on the competition score, appearance similarity and the persistence of spatial overlap. S5. Within a preset time window, statistically analyze the stability of cross-granularity inclusion relationship candidates, calculate the difference in reliability between the component trajectory and the overall trajectory to form a preliminary cross-granularity decision, thereby deciding whether to absorb, split, or postpone. S6. Based on the historical identity recovery results in S2, the competitive trajectory processing results in S4, and the cross-granularity trajectory processing results in S5, update the active, dormant, or terminated status of each trajectory, update, compress, and prune the appearance feature memory and historical mask sequence, and output the integrated target trajectory set and target identity identifier.

2. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 1, characterized in that, In S2, appearance features are extracted based on the mask region. The appearance feature extraction is represented as follows: ;in, This indicates the visual feature extraction model for the current frame. Extracted image feature map; Indicates the first Mask for each target observation; This represents a mask weighting operation; Indicates pooling operation; This indicates a feature normalization operation; the visual feature extraction model employs a convolutional neural network, a visual Transformer, a self-supervised visual model, or a multimodal visual encoder.

3. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 1, characterized in that, In S2, the mask quality score is determined by at least two of the following: model confidence, boundary stability, area reasonableness, internal region consistency, and conflict penalty; the area reasonableness score is calculated by the change between the current mask area and the historical average area of ​​the trajectory. When the mask quality score is lower than the mask quality threshold, the current observation is not directly used to update the historical trajectory.

4. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 1, characterized in that, The method for determining update weights based on appearance similarity in S2 includes the following steps: calculating the cosine similarity between the appearance features of the current target observation and the appearance feature memory of the corresponding target trajectory, and determining a similarity gate value based on the cosine similarity; combining the similarity gate value with the mask quality score to obtain the appearance feature memory update weight; pausing the current target observation and writing to the appearance feature memory when the appearance feature memory update weight is lower than the update threshold; updating the appearance feature memory using an exponential moving average method when the appearance feature memory update weight is not lower than the update threshold, the mask quality score of the current target observation is not lower than the mask quality threshold, and the corresponding target trajectory is not in the current frame suppression, absorption, or termination state.

5. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 4, characterized in that, In step S2, for a target trajectory in a dormant state, a historical identity recovery score is calculated based on the appearance similarity between the current target observation and the appearance feature memory, the spatial context consistency score between the current target observation position and the historical motion prediction position, the mask quality score of the current target observation, and the trajectory dormancy time penalty term. When the historical identity recovery score reaches the identity recovery threshold, the corresponding target trajectory is restored from a dormant state to an active state.

6. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 1, characterized in that, The spatial overlap relationship is confirmed when the cross-union ratio reaches the cross-union ratio threshold, or when the minimum containment overlap rate reaches the containment overlap threshold. The requirements for marking the spatial overlap relationship as a candidate for ordinary competition relationship and cross-granularity containment relationship are that the minimum containment overlap rate reaches the containment overlap threshold and the area ratio between active trajectories meets the area difference condition.

7. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 6, characterized in that, The competitive scoring method is determined by weighting the appearance consistency score, historical duration score, recent stability score, recent mask quality score, spatial coverage rationality score, and trajectory anomaly penalty item.

8. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 7, characterized in that, The methods for handling race tracks in S4, such as preservation, suppression, or merging, are as follows: In the competitive trajectory group, the highest competitive score trajectory and the second highest competitive score trajectory are determined. When the difference between the highest and second highest competitive scores reaches the score difference threshold, the highest competitive score trajectory is retained, and at least one of the following processes is performed on the remaining competitive trajectories: current frame suppression, reduction of subsequent competitive weights, or suspension of current frame appearance feature memory update. When the score difference between the highest and second highest competitive score trajectories is less than the score difference threshold, and their appearance similarity, crossover ratio, and number of consecutive overlapping frames reach the corresponding thresholds, trajectory merging is performed. During merging, the identity of the target trajectory with a longer historical duration or higher stability is retained, and the historical mask sequence and appearance feature memory are fused according to the mask quality.

9. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 8, characterized in that, Step S5 includes: Statistically determining the stability of cross-granularity inclusion relationships by statistically analyzing the proportion of candidate component trajectories continuously contained within the candidate overall trajectory within a preset time window; calculating the reliability of the component trajectory and the overall trajectory based on the target trajectory's average recent stability, average mask quality, and time support length normalized score within the time window; absorbing the component trajectory into the overall trajectory when the overall trajectory reliability is higher than the component trajectory reliability and the difference between them reaches an absorption threshold; retaining the component trajectory and suppressing or splitting the overall trajectory when the component trajectory reliability is higher than the overall trajectory reliability and the difference between them reaches a retention threshold; temporarily refraining from absorption or splitting and continuing observation when the difference between the overall trajectory reliability and the component trajectory reliability does not meet any of the aforementioned absorption and retention conditions; updating the aforementioned processing result to suppress or split the candidate overall trajectory and retain the candidate component trajectory set after completing the aforementioned reliability difference judgment, when the same candidate overall trajectory corresponds to multiple candidate component trajectories, the joint coverage rate of the multiple candidate component trajectories to the candidate overall trajectory reaches a coverage threshold, and the reliability of each candidate component trajectory is higher than the reliability of the candidate overall trajectory; maintaining the processing result determined based on the aforementioned reliability difference when the joint coverage condition is not met.

10. The method for quality-constrained long video target trajectory integration and identity preservation as described in claim 1, characterized in that, The method for outputting the integrated target trajectory set and target identity in S6 includes: participating active target trajectories in current frame observation matching, memory update, and competition relationship judgment; retaining appearance feature memory for dormant target trajectories and participating in historical identity recovery judgment; removing terminated target trajectories from subsequent matching and output; and when the number of historical appearance features exceeds the preset capacity, prioritizing the retention of observation features with high mask quality, observation features that differ significantly from existing features, observation features when the target undergoes significant viewpoint or scale changes, the first stable observation feature after trajectory recovery, and observation features before and after trajectory competition or merging, while deleting low-quality features whose similarity reaches the redundancy threshold.

Images