A multi-target tracking optimization method and system for dense occlusion scenarios

Abstract

The application discloses a multi-target tracking optimization method and system for a dense occlusion scene, and the method comprises the following steps: detecting a video frame sequence to obtain a current frame detection box set; based on the historical motion state sequence of a survival trajectory set and the boundary box coordinates in the current frame detection box set, performing motion state prediction on the survival trajectory set in a space-time context, to obtain a predicted boundary box set; an occlusion state counter performs occlusion-aware feature aggregation on a dynamic feature memory queue, to obtain a trajectory representative feature vector set; the predicted boundary box set and the trajectory representative feature vector set perform adaptive data association on the current frame detection box set in an occlusion-aware manner, to obtain a matched trajectory pair set, an unmatched trajectory set and an unmatched detection box set; and the unmatched trajectory set is subjected to trajectory recovery based on a cross-attention mechanism, to generate a recovery trajectory set. The application overcomes motion deviation and feature pollution under dense occlusion, and realizes accurate matching and recovery tracking.

Description

Technical Field

[0001] This invention belongs to the field of video tracking technology, specifically a multi-target tracking optimization method and system for densely occluded scenes. Background Technology

[0002] With the rapid development of intelligent video surveillance technology, multi-target tracking technology has emerged. This technology can detect moving targets in a continuous sequence of video frames and assign them unique identifiers, and has the characteristics of continuously recording and analyzing the target's behavioral trajectory.

[0003] Traditional multi-object tracking techniques typically employ a detection-based tracking framework. This involves independently extracting the bounding box coordinates and appearance features of each target in each frame, then using the motion state of historical trajectories for linear prediction, and combining this with the similarity of appearance features between targets for data association. When a target's detection box is temporarily missing due to occlusion, the trajectory is preserved, and matching is performed only after the target reappears. However, in densely occluded scenes, mutual occlusion between targets leads to severe loss or contamination of effective appearance features. Furthermore, the crowding often results in non-linear motion, making traditional methods reliant solely on historical linear motion prediction prone to positional deviations. These methods lack isolation and protection mechanisms for features during occlusion and effective aggregation mechanisms. Consequently, when the target reappears after occlusion, historical trajectory features cannot effectively match the current local detection features, easily leading to identity switching or trajectory breakage. This results in the easy loss of target trajectories in densely occluded scenes, the inability to maintain the target's identity during brief occlusion, and the inability to accurately recover tracking after occlusion ends. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-target tracking optimization method and system for densely occluded scenarios, which can overcome motion deviation and feature contamination under dense occlusion, achieve stable maintenance of target identity during occlusion and accurate matching and recovery tracking after occlusion ends, thereby significantly improving the continuity and accuracy of multi-target tracking in complex occlusion scenarios.

[0005] This invention is achieved through the following technical solution: An optimization method for multi-target tracking in densely occluded scenes includes the following steps: S1. Obtain the video frame sequence, use the pre-trained object detection network to perform object detection on the video frame sequence, and obtain the current frame detection box set. The current frame detection box set includes bounding box coordinates, classification confidence and initial appearance feature vector. S2. Based on the historical motion state sequence of the survival trajectory set and the bounding box coordinates in the current frame detection box set, construct a spatial neighborhood graph for the survival trajectory set, and then perform spatiotemporal context-aware motion state prediction on the survival trajectory set based on the spatial neighborhood graph to obtain the predicted bounding box set. During video tracking, a set of live trajectories is continuously maintained and updated in real time. All target trajectory data that have not yet triggered the tracking termination condition and are in a continuous tracking state are filtered and retained to form a set of live trajectories, which includes historical motion state sequences, occlusion state counters and dynamic feature memory queues. S3. Based on the occlusion state counter, perform occlusion perception feature aggregation on the dynamic feature memory queue to obtain a set of trajectory representative feature vectors. S4. Based on the predicted bounding box set and the trajectory representative feature vector set, perform adaptive data association for occlusion awareness on the current frame detection box set to obtain the matching trajectory pair set, the unmatched trajectory set, and the unmatched detection box set. The process is as follows: S41. Calculate the Mahalanobis distance between the bounding box coordinates of the predicted bounding box set and the previous frame detection box set to obtain the motion matching distance; S42. Calculate the cosine distance between the initial apparent feature vector and the trajectory representative feature vector of the current frame detection box set to obtain the apparent matching distance; S43. Adaptively calculate the fusion weight value by combining the real-time value of the occlusion state counter with the classification confidence of the current frame detection box set. S44. Based on the fusion weight value, the motion matching distance and the apparent matching distance are weighted and summed to obtain the joint cost matrix; S45. Input the joint cost matrix into the bipartite graph matching algorithm to solve for the optimal matching, and obtain the set of matched trajectory pairs, the set of unmatched trajectories and the set of unmatched detection boxes. S5. Based on the trajectory representative feature vector in the unmatched trajectory set and the initial apparent feature vector in the unmatched detection box set, perform trajectory recovery based on the cross-attention mechanism on the unmatched trajectory set to generate the recovered trajectory set.

[0006] Furthermore, the process of S1 is as follows: S11. According to a fixed temporal splitting rule, the continuous video data stream transmitted by the video acquisition device is processed by frame splitting to obtain a temporally continuous and orderly video frame sequence. S12. Using a pre-trained target detection network, analyze the image target information in the video frame sequence frame by frame, extract the bounding box coordinates, classification confidence and initial appearance feature vector of the effective targets respectively, summarize the bounding box coordinates, classification confidence and initial appearance feature vector of all targets in a single frame, and generate a set of detection boxes of the current frame with a unified structure.

[0007] Furthermore, the process of S2 is as follows: S21. Extract the predicted position of each survival trajectory in the previous frame from the historical motion state sequence, and match the bounding box coordinates of the current frame detection box set. Construct a spatial neighborhood graph for the survival trajectory set to obtain a spatial neighborhood graph including the set of center nodes, the set of neighbor nodes, and spatial distance weights. S22. Based on spatial distance weights, normalize the set of central nodes and the set of neighboring nodes to obtain the normalized aggregate weights of the neighboring nodes. The specific process is as follows: read the spatial distance weights in the spatial neighborhood graph, perform global accumulation calculation on the spatial distance weights of all neighboring nodes corresponding to a single central node in the set of central nodes in the spatial neighborhood graph to obtain the sum of neighborhood weights corresponding to a single central node, then perform a ratio calculation between the spatial distance weight of each neighboring node and the sum of neighborhood weights to complete the weight normalization mapping, solve the weight values ​​corresponding to all neighboring nodes one by one, and obtain the normalized aggregate weights of all neighboring nodes. S23. Based on the normalized aggregation weights of neighboring nodes, perform graph convolution aggregation on the initial motion feature vector of the center node and the initial motion feature vectors of neighboring nodes to obtain the updated feature vector of the center node. S24. Input the updated feature vector of the center node into the pre-trained bounding box regression network, solve the current frame prediction position corresponding to each survival trajectory, and summarize the regression prediction results of all center nodes to obtain the predicted bounding box set.

[0008] Furthermore, the process of constructing the spatial neighborhood graph in S21 is as follows: S211. Set each surviving trajectory as an independent graph node, define the surviving trajectory node to be predicted as the center node, and define other surviving trajectory nodes within the preset spatial neighborhood as neighbor nodes. All center nodes are aggregated to form a center node set, and all neighbor nodes are aggregated to generate a neighbor node set. S212. Calculate the spatial positional deviation between the central node and its corresponding neighboring nodes to obtain the spatial distance weight that can characterize the degree of spatial association between nodes. S213. Integrate the central node set, the neighbor node set, and the spatial distance weight to generate a complete spatial neighborhood graph.

[0009] Furthermore, the process of S23 is as follows: S231. Perform feature concatenation on the historical bounding box coordinates, center point velocity, and width and height change rate in the historical motion state sequence to obtain the initial motion feature vector of the center node and the initial motion feature vector of the neighboring nodes. S232. Retrieve the normalized aggregation weights of neighboring nodes and use them as the quantitative basis for feature fusion. Perform weighted fusion of the initial motion feature vector of the central node and the initial motion feature vectors of neighboring nodes to obtain the aggregated feature vector that fuses the motion information of multiple nodes. S233. Perform standard matrix multiplication on the aggregated feature vector and the preset learnable weight matrix to obtain the matrix multiplication result after linear transformation. S234. Input the matrix multiplication result into the nonlinear mapping function ReLU to perform nonlinear mapping, and obtain the updated feature vector of the central node, expressed as: In the formula: For the first The center node of the frame updates the feature vector; It is a nonlinear mapping function; The weight matrix is ​​a learnable weight matrix; For the first The self-loop weight of the frame's center node; For the first The initial motion feature vector of the center node of the frame; For the set of neighboring nodes; For the first Normalized aggregate weights of the neighboring nodes of the frame; For the first The initial motion feature vectors of the neighboring nodes of the frame.

[0010] Furthermore, the process of S3 is as follows: S31. Read the count value of the occlusion state counter corresponding to each survival trajectory in the survival trajectory set in real time, compare the real-time count result of the occlusion state counter with the preset occlusion tolerance threshold, and stop the feature update operation for the corresponding survival trajectory when the real-time count result of the occlusion state counter exceeds the preset occlusion tolerance threshold, lock the currently stored valid appearance features, and generate a frozen feature queue; when the real-time count result of the occlusion state counter does not exceed the preset occlusion tolerance threshold, continue to receive new appearance features and iteratively update the feature data in the queue, generate an updated feature queue, integrate the feature queues corresponding to all survival trajectories, and obtain a complete memory queue to be aggregated. S32. Retrieve each historical appearance feature vector from the memory queue to be aggregated, perform time dimension feature decay modeling on it, and obtain the time decay weight of each historical appearance feature vector. S33. Based on the time decay weight, perform weighted average pooling on each historical apparent feature vector in the memory queue to be aggregated to obtain a set of trajectory representative feature vectors. The specific process is as follows: perform element-wise weighted operation on each historical apparent feature vector and the corresponding time decay weight to obtain the weighted feature data of each historical apparent feature vector, then solve the global mean of all weighted feature data, and then integrate the weighted feature data corresponding to all surviving trajectories to generate a set of trajectory representative feature vectors.

[0011] Furthermore, the process in S32 is as follows: S321. Extract the frame number from each historical apparent feature vector in the aggregated memory queue to obtain the historical frame number and the current frame number. S322. Perform a difference operation between the current frame number and the historical frame number to obtain the inter-frame distance; S323. Based on the inter-frame distance, an exponential decay operation is used to calculate the exponential decay weight of each historical appearance feature vector to obtain the time decay weight of each historical appearance feature vector, expressed as follows: In the formula: As a time decay weight; This is the decay rate parameter, used to uniformly control the decay rate of the overall feature weights; The current frame number; The historical frame number is the one that corresponds to the historical appearance feature vector.

[0012] Furthermore, the process of S43 is as follows: S431. Perform a difference calculation between the occlusion state counter and the preset occlusion tolerance threshold to obtain the occlusion deviation. S432. Perform a difference calculation on the classification confidence of the current frame detection box set and the preset confidence threshold to obtain the confidence deviation. S433. Apply a logistic regression function to the occlusion deviation and confidence deviation to obtain the fusion weight value, expressed as: In the formula: For fusion weight values; and To adjust the parameters; This is the real-time value of the occlusion status counter. Set the preset occlusion tolerance threshold; For classification confidence; To pre-set the reliability threshold.

[0013] Further, the process of S5 is as follows: extract the corresponding trajectory representative feature vector from the unmatched trajectory set and the initial appearance feature vector from the unmatched detection box set; use a two-way feature interaction modeling mechanism as a cross-attention mechanism; establish a two-way interactive mapping relationship between the trajectory representative feature vector and the initial appearance feature vector through the cross-attention mechanism; deeply mine the target homologous feature information weakened by noise in the occluded scene; strengthen the similarity weight of homologous features; correct the feature matching deviation caused by occlusion; rebind the identity and continue the trajectory for unmatched trajectories with broken trajectory and failed matching; and generate a complete and unique restored trajectory set.

[0014] A multi-target tracking optimization system for densely occluded scenes includes a target detection module, a motion prediction module, a feature aggregation module, a data association module, and a trajectory recovery module, wherein: The target detection module is used to acquire a video frame sequence, perform target detection on the video frame sequence, and obtain a set of detection boxes for the current frame. The set of detection boxes for the current frame includes bounding box coordinates, classification confidence, and initial appearance feature vector. The motion prediction module performs spatiotemporally context-aware motion state prediction on the survival trajectory set based on the historical motion state sequence of the survival trajectory set and the bounding box coordinates in the current frame detection box set, to obtain a predicted bounding box set, wherein the survival trajectory set includes a historical motion state sequence, an occlusion state counter and a dynamic feature memory queue. The feature aggregation module performs occlusion perception feature aggregation on the dynamic feature memory queue based on the occlusion state counter to obtain a set of trajectory representative feature vectors. The data association module performs adaptive data association for occlusion perception on the current frame detection box set based on the predicted bounding box set and the trajectory representative feature vector set, to obtain a set of matching trajectory pairs, a set of unmatched trajectories, and a set of unmatched detection boxes; The trajectory recovery module performs trajectory recovery on the unmatched trajectory set based on the trajectory representative feature vector in the unmatched trajectory set and the initial appearance feature vector in the unmatched detection box set, generating a recovered trajectory set.

[0015] The present invention has the following beneficial technical effects: This invention obtains a set of detection boxes from the previous frame through target detection, and then predicts the motion state of the survival trajectory based on a spatial neighborhood graph with spatiotemporal context awareness. This is to overcome the nonlinear offset under occlusion by utilizing the motion trend of the target in the neighborhood. Next, the dynamic feature memory queue is updated and intercepted with occlusion awareness and aggregated with temporal decay to maintain the pure identity representation during occlusion. Then, an adaptive association between the predicted position and the trajectory representative features is constructed using occlusion adaptive weights. Finally, unmatched trajectories are recovered by focusing on locally visible features using a cross-attention mechanism. That is, the cross-attention mechanism realizes track break recovery, achieving full-link occlusion scene adaptation optimization. It can stably maintain the identity of the target during short-term occlusion and accurately recover tracking after occlusion ends. This effectively overcomes the problems of motion prediction deviation and feature pollution failure under dense occlusion, and achieves stable maintenance of target identity during occlusion and accurate matching recovery after occlusion ends. It significantly improves the continuity and accuracy of multi-target tracking in complex occlusion scenes. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the multi-target tracking optimization method for densely occluded scenarios of the present invention. Figure 2 This is a schematic diagram of the structure of the multi-target tracking optimization system for densely occluded scenarios according to the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0018] like Figure 1 As shown, a multi-target tracking optimization method for densely occluded scenes includes the following steps: S1. Obtain the video frame sequence, and use a pre-trained object detection network to perform object detection on the video frame sequence to obtain the current frame detection box set. The current frame detection box set includes bounding box coordinates, classification confidence, and initial appearance feature vector. The process is as follows: S11. According to a fixed temporal splitting rule, the continuous video data stream uploaded by the video acquisition device is processed by frame splitting to obtain a temporally continuous and orderly video frame sequence. S12. Use a pre-trained target detection network to parse the image target information in the video frame sequence frame by frame, extract the bounding box coordinates, classification confidence and initial appearance feature vector of all valid targets respectively, summarize the bounding box coordinates, classification confidence and initial appearance feature vector of all targets in a single frame, and complete the regularization and integration to generate a set of detection boxes of the current frame with a unified structure. The object detection network is either Faster R-CNN (Faster Region-based Convolutional Neural Network) or YOLOv8 (You Only Look Once version 8), a single-stage object detection network. Faster R-CNN is a classic two-stage object detection network. In the second stage, it performs feature normalization on the feature map region corresponding to each candidate detection box through region of interest alignment, generating a fixed-dimensional region feature vector, which serves as the initial apparent feature vector of the corresponding detection box. YOLOv8 is a general one-stage object detection network. Its backbone and neck network output multi-scale feature maps. For each output detection box, a feature vector of the corresponding dimension is extracted through region feature cropping and global average pooling operations, serving as the initial apparent feature vector of that detection box. The target detection network completes the entire training process using a publicly available large-scale visual dataset. The training process is as follows: image samples and corresponding target annotation information from the dataset are input into the target detection network in batches, the detection results are output through forward propagation, and the weights and bias parameters of the target detection network are continuously updated iteratively based on the back propagation algorithm to gradually reduce the detection error until the target detection network converges, thereby possessing general target detection capabilities and obtaining a pre-trained target detection network. S2. Based on the historical motion state sequence of the survival trajectory set and the bounding box coordinates in the current frame detection box set, perform spatiotemporal context-aware motion state prediction on the survival trajectory set to obtain the predicted bounding box set. Specifically, retrieve the historical motion state sequence corresponding to each target in the survival trajectory set to obtain the motion change pattern of the target in past frames. Combine the real-time position information of the target corresponding to the bounding box coordinates in the current frame detection box set to carry out spatiotemporal context-aware motion state prediction. This prediction method can combine the temporal motion pattern of the target with the spatial position association information, break through the limitations of traditional single linear motion prediction, and adapt to the nonlinear motion characteristics of targets in dense scenes. Then, calculate the estimated position data of each survival trajectory in the current frame, integrate all the estimated position data, and generate a complete predicted bounding box set. During video tracking, a continuously updated set of surviving trajectories is maintained. All target trajectory data that have not yet triggered tracking termination conditions and are in continuous tracking status are selected and retained, forming the surviving trajectory set. This set includes three core data types: historical motion state sequences, occlusion state counters, and dynamic feature memory queues. These three types of data work together to completely record the motion changes, occlusion status, and appearance features of a single target trajectory. Specifically, the occlusion state counter continuously counts the number of consecutive occlusion frames for a single target trajectory, accurately reflecting the duration and severity of occlusion in the video frame sequence. The dynamic feature memory queue is a temporal feature storage queue used to continuously retain effective appearance features from each frame during target tracking, thereby avoiding interference from contaminated features on trajectory matching in occluded scenarios. The process of S2 is as follows: S21. Extract the predicted position of the previous frame corresponding to each survival trajectory in the historical motion state sequence, and at the same time match the bounding box coordinates of the current frame detection box set. Construct a spatial neighborhood graph for the survival trajectory set to obtain a spatial neighborhood graph, which includes a set of center nodes, a set of neighboring nodes, and spatial distance weights. The spatial neighborhood graph is a topological data structure used to represent the spatial correlation of multiple target trajectories, and its construction process is as follows: S211. Set each surviving trajectory as an independent graph node, define the surviving trajectory node to be predicted as the center node, and define other surviving trajectory nodes within the preset spatial neighborhood as neighbor nodes. All center nodes are aggregated to form a center node set, and all neighbor nodes are aggregated to generate a neighbor node set. S212. Calculate the spatial positional deviation between the central node and its corresponding neighboring nodes, and quantify it to obtain spatial distance weights that can characterize the degree of spatial association between nodes. S213. Integrate the central node set, the neighbor node set, and the spatial distance weight to generate a complete spatial neighborhood graph; S22. Based on spatial distance weights, normalize the set of central nodes and the set of neighboring nodes to obtain the normalized aggregate weights of the neighboring nodes. The specific process is as follows: read the spatial distance weights in the spatial neighborhood graph, perform global accumulation calculation on the spatial distance weights of all neighboring nodes corresponding to a single central node in the set of central nodes in the spatial neighborhood graph to obtain the sum of neighborhood weights corresponding to a single central node, and then perform a ratio calculation between the spatial distance weight of each neighboring node and the sum of neighborhood weights to complete the weight normalization mapping. Solve the weight values ​​corresponding to all neighboring nodes one by one to obtain the normalized aggregate weights of all neighboring nodes with a unified quantification standard. The normalization is a weight normalization processing method in the field of computer vision, which is used to map the differentiated spatial distance weights to a unified numerical range, eliminate the dimensional differences of different spatial distance values, and thus improve the rationality of the weight allocation of feature aggregation. S23. Based on the normalized aggregation weights of neighboring nodes, graph convolution aggregation is performed on the initial motion feature vector of the central node and the initial motion feature vectors of neighboring nodes to obtain the updated feature vector of the central node. Specifically, the normalized aggregation weights of neighboring nodes are obtained, and the motion features of each central node in the central node set and the motion features of each neighboring node in the neighboring node set are retrieved. Graph convolution aggregation is used to perform weighted fusion on the initial motion feature vector of the central node and the initial motion feature vector of the associated neighboring nodes, integrating all neighborhood motion information corresponding to a single central node, realizing iterative update of motion features, and outputting the updated feature vector of the central node with unified dimensions and fusion of spatiotemporal neighborhood information. The graph convolutional aggregation is a general graph structure data feature fusion technology that can mine the feature association information of neighboring nodes based on the association weight of topological nodes, and adapt to the feature fusion needs of dense multi-object scenarios. The specific process of S23 is as follows: S231. The historical bounding box coordinates, center point velocity, and width / height change rate in the historical motion state sequence are concatenated to obtain the initial motion feature vectors of the center node and neighboring nodes. Specifically: First, the historical motion state sequence in the survival trajectory set is traversed, and the three basic motion parameters corresponding to each trajectory node in the survival trajectory set are extracted in batches. The three basic motion parameters are historical bounding box coordinates, center point velocity, and width / height change rate. Among them, the historical bounding box coordinates are the bounding box position data of the target in the historical continuous video frames, which can intuitively reflect the temporal and spatial position changes of the target; the center point velocity... The motion rate data is calculated based on the displacement of the center point of the bounding box of the target in adjacent frames, which is used to characterize the speed of the target's movement; the width and height change rate is the temporal fluctuation ratio of the target's bounding box width and height, which is used to characterize the change characteristics of the target's attitude and distance; then, feature splicing is used to fuse the three types of basic motion parameters in dimensions, and the three types of basic motion parameters are uniformly mapped to the same feature dimension and then spliced ​​and combined to complete the integration of multi-dimensional motion information of the center node and neighbor nodes in the spatial neighborhood graph, respectively, to obtain the initial motion feature vector of the center node and the initial motion feature vector of the neighbor nodes with unified dimensions and complete information; S232. Retrieve the normalized aggregation weights of neighboring nodes. Using the normalized aggregation weights as the quantitative basis for feature fusion, the initial motion feature vector of the central node and the initial motion feature vectors of neighboring nodes are weighted and fused to obtain an aggregated feature vector. Specifically, a fixed self-loop weight is configured for each central node to retain the core information of the central node's own initial motion feature vector. At the same time, for each neighboring node corresponding to the central node, the normalized aggregation weight of the neighboring node is weighted element-wise with the initial motion feature vector of the neighboring node to complete the feature weighting of a single neighboring node and obtain the weighted initial motion feature vector of the neighboring node. Then, all the weighted initial motion feature vectors of the neighboring nodes are accumulated, and the accumulated result is fused with the weighted initial motion feature vector of the central node to integrate the central node's own motion information and the motion association information of the neighboring nodes to obtain an aggregated feature vector that integrates the motion information of multiple nodes. S233. Perform standard matrix multiplication on the aggregated feature vector and the preset learnable weight matrix to obtain the matrix multiplication result after linear transformation. The learnable weight matrix is ​​a parameter matrix in the field of deep learning. During the training phase, the initial matrix parameters are randomly initialized and generated. During the continuous sample iteration training process, the internal parameters of the matrix are continuously updated and optimized to obtain the learnable weight matrix. Its function is to complete the feature dimension transformation and feature information filtering. The feature dimension of the aggregated feature vector is normalized through matrix multiplication, while filtering redundant feature information and strengthening effective motion feature information. S234. Perform nonlinear mapping on the matrix multiplication result to obtain the updated feature vector of the center node. Specifically, input the matrix multiplication result into the nonlinear mapping function to achieve nonlinear activation, and solve for the updated feature vector of the center node based on the calculation formula of the updated feature vector of the center node. The formula for calculating the updated feature vector of the central node is as follows: In the formula: For the first The center node of the frame updates the feature vector; It is a nonlinear mapping function; The weight matrix is ​​a learnable weight matrix; For the first The self-loop weight of the frame's center node; For the first The initial motion feature vector of the center node of the frame; For the set of neighboring nodes; For the first Normalized aggregate weights of the neighboring nodes of the frame; For the first The initial motion feature vectors of the neighboring nodes of the frame; The nonlinear mapping function is ReLU (Rectified Linear Unit), a nonlinear activation function in the field of computer vision tracking. It has stable nonlinear mapping capabilities and can break the limitations of linear operation in feature expression. The parameters are substituted into the nonlinear mapping function one by one to complete the operation and output the center node update feature vector with nonlinear expression capabilities. In steps S231 to S234, a complete initial motion feature vector is constructed by integrating three types of basic motion parameters. The motion information fusion of multiple targets is completed by combining the normalized weights of neighboring nodes. Then, feature purification and high-order expression are completed through matrix linear transformation and nonlinear mapping. This effectively makes up for the shortcomings of the traditional single motion feature representation capability. It can accurately adapt to the nonlinear motion characteristics of targets in densely occluded scenes, and the output node features fully cover the motion law of the target itself and the motion association information of surrounding targets. This significantly improves the accuracy and scene adaptability of subsequent bounding box position prediction. From the perspective of motion feature optimization, it reduces the trajectory offset and identity confusion problem of multi-target tracking. S24. Perform bounding box regression prediction on the updated feature vector of the center node to obtain a set of predicted bounding boxes. The specific process is as follows: input the updated feature vector of the center node into the pre-trained bounding box regression network, accurately solve the current frame prediction position corresponding to each survival trajectory, summarize the regression prediction results of all center nodes, and generate a complete set of predicted bounding boxes. The bounding box regression network is a lightweight prediction network in the field of computer vision. Its training process can be completed based on a public target trajectory dataset. Specifically, the historical motion feature samples and real bounding box annotation data in the dataset are paired and input. Predicted bounding box data is generated through forward propagation. Then, the weight parameters and bias parameters of the bounding box regression network are continuously iteratively optimized according to the backpropagation mechanism until the prediction error of the bounding box regression network converges, thus completing the training of the bounding box regression network and obtaining the pre-trained bounding box regression network. S3. Based on the occlusion state counter, feature aggregation for occlusion perception is performed on the dynamic feature memory queue to obtain a set of trajectory representative feature vectors. Specifically, the count value of the occlusion state counter corresponding to each target in the surviving trajectory set is read in real time. The real-time count value is used as the core judgment basis to distinguish between the normal tracking state and the occlusion tracking state of the target. Different feature processing is performed on the dynamic feature memory queue for targets in different states. The effective appearance features in the dynamic feature memory queue are integrated and optimized. Invalid feature data generated during the occlusion period are removed. The complete and pure temporal appearance features of each target are retained. The feature dimensions are unified and information is condensed through standardized feature aggregation to generate a set of trajectory representative feature vectors that can be directly used for trajectory matching calculation. The process of S3 is as follows: S31. Read the count value of the occlusion state counter corresponding to each survival trajectory in the survival trajectory set in real time, compare the real-time count result of the occlusion state counter with the preset occlusion tolerance threshold, and stop the feature update operation for the corresponding survival trajectory when the real-time count result of the occlusion state counter exceeds the preset occlusion tolerance threshold, lock the currently stored valid appearance features, and generate a frozen feature queue; when the real-time count result of the occlusion state counter does not exceed the preset occlusion tolerance threshold, continue to receive new appearance features and iteratively update the feature data in the queue, generate an updated feature queue, integrate the feature queues corresponding to all survival trajectories, and obtain a complete memory queue to be aggregated. The occlusion status counter is used to continuously count the number of consecutively occluded video frames of a single survival trajectory, which can accurately quantify the duration of target occlusion. The preset occlusion tolerance threshold is used as a standard threshold to determine the severity of target occlusion. The dynamic feature memory queue is a structured data queue that stores the appearance features of each frame of the survival trajectory in time sequence. S32. Retrieve each historical apparent feature vector in the memory queue to be aggregated, perform time dimension feature decay modeling on it, and obtain the time decay weight of each historical apparent feature vector. Specifically, take the temporal relationship of video frames as the core basis, quantify the temporal difference between each historical apparent feature vector and the current frame, assign differentiated weights to each historical apparent feature vector according to the size of the temporal difference, realize dynamic modeling of feature effectiveness, calculate one by one, and obtain the time decay weight corresponding to each historical apparent feature vector in the memory queue to be aggregated, thereby quantitatively distinguishing between old features and recent effective features. The historical apparent features of targets in video sequences gradually lose their reference value over time. The further back the features are from the current frame, the lower their effectiveness and the more likely they are to have feature bias and information lag. Modeling feature decay in the time dimension can quantify and suppress feature bias and information lag. The specific process of S32 is as follows: S321. Extract the frame number of each historical appearance feature vector in the memory queue to be aggregated to obtain the historical frame number and the current frame number. Specifically: retrieve the complete memory queue to be aggregated, and extract the temporal information of each historical appearance feature vector stored in the memory queue to be aggregated one by one. Read the temporal identifier bound to each historical appearance feature vector and extract the historical frame number corresponding to all historical appearance feature vectors in batches. Read the temporal identifier corresponding to the currently being tracked video frame to obtain the current frame number updated in real time. Completely retain all the basic temporal data required for subsequent temporal operations. During the generation and storage process, all historical apparent feature vectors stored in the memory queue to be aggregated are synchronously bound to the unique temporal identifier of the corresponding video frame. This unique temporal identifier is used to distinguish the feature generation time of different frames, which can ensure the accurate correspondence of temporal information. S322. Perform a difference operation between the current frame number and the historical frame number to obtain the inter-frame distance. Specifically, perform an independent difference calculation on each pair of matching current frame numbers and historical frame numbers, and solve the independent inter-frame distance corresponding to each historical apparent feature vector in turn. The inter-frame distance is a special numerical indicator for quantifying the temporal interval between the historical apparent feature vector and the current processing frame. It can intuitively reflect the temporal staleness of a single historical apparent feature vector. It is the core basic parameter for realizing the temporal differential weight calculation and can realize the accurate quantification of the temporal interval of all historical apparent feature vectors. S323. Based on the inter-frame distance, calculate the exponential decay weight for each historical appearance feature vector to obtain the time decay weight of each historical appearance feature vector. Specifically: based on the inter-frame distance, use exponential decay operation to solve for the weight of each historical appearance feature vector, and generate the time decay weight corresponding to each individual historical appearance feature vector, expressed as: In the formula: As a time decay weight; This is the decay rate parameter, used to uniformly control the decay rate of the overall feature weights; The current frame number; The historical frame number matched with the corresponding historical appearance feature vector; The parameters corresponding to the inter-frame distance of each group By substituting into the above formula, the time decay weights of all historical apparent feature vectors can be solved in batches, resulting in a complete set of time decay weights. In steps S321 to S323, the time frame number corresponding to the historical appearance feature vector can be accurately extracted, the time interval between frames can be quantified, and the weight can be solved based on the exponential operation formula. This achieves refined and differentiated assignment of historical appearance features at different time series, effectively distinguishes the weight ratio of recent high-value features and old features from the past, solves the problem of weakening effective features caused by the equalization of time series weights in traditional feature aggregation methods, adapts to the effective change law of time series features during target occlusion, significantly improves the accuracy and anti-interference ability of target trajectory representative features in occlusion scenarios, and improves the stability of feature matching for multi-target tracking. S33. Based on the time decay weight, perform weighted average pooling on each historical apparent feature vector in the memory queue to be aggregated to obtain a set of trajectory representative feature vectors. Specifically, perform element-wise weighted operation on each historical apparent feature vector and its corresponding time decay weight to obtain the weighted feature data of each historical apparent feature vector. Then, perform global mean calculation on all weighted feature data to achieve the condensation and fusion of multiple time series features, unify the feature output dimension, eliminate temporal redundant feature interference, and integrate the weighted feature data corresponding to all surviving trajectories to generate a set of trajectory representative feature vectors with unified dimension and accurate features. In S31~S34, the occlusion state determination is used to realize the differentiated update and freezing interception of the feature queue, effectively protecting the effective target features during the occlusion period and isolating contaminated features. Then, the time decay weight of different time-series features is quantified by the time dimension feature decay modeling. Finally, the weighted average pooling is used to realize accurate feature condensation. The weighted average pooling can realize the adaptive fusion of multiple sets of features according to the differentiated weights, avoid the problem of effective feature weakening caused by fixed mean pooling, and can adaptively adapt to the feature change rules of dense occlusion scene, condense highly robust trajectory representative features, greatly reduce the probability of target feature matching failure and trajectory breakage in occlusion scene, and effectively solve the problem that traditional feature aggregation methods cannot distinguish between occlusion state and the interference of mixed old and new features. S4. Based on the predicted bounding box set and the trajectory representative feature vector set, adaptive data association for occlusion perception is performed on the current frame detection box set to obtain a set of matched trajectory pairs, a set of unmatched trajectories, and a set of unmatched detection boxes. Specifically, the predicted bounding box set and the trajectory representative feature vector set are retrieved to obtain the estimated motion position information and the refined core appearance feature information of the target, respectively. These are used as the basis for matching, and adaptive data association for occlusion perception is performed on the current frame detection box set. This adaptive data association method is different from the traditional fixed weight matching mechanism. It can adaptively adjust the matching weight of motion information and appearance feature information according to the real-time occlusion status of the target, effectively adapting to the target matching requirements in dense occlusion scenarios. By performing correlation matching verification between the target in the current frame detection box set and the historical surviving trajectory one by one, the successfully matched target trajectory combination, the unmatched historical surviving trajectory, and the new detection target without a corresponding historical trajectory are accurately identified. The above three types of results are classified, regulated, and independently divided to obtain a set of matched trajectory pairs, a set of unmatched trajectories, and a set of unmatched detection boxes that are non-overlapping and have complete data. The process of S4 is as follows: S41. Perform Mahalanobis distance calculation on the predicted bounding box set and bounding box coordinates to obtain the motion matching distance. Specifically: extract all predicted bounding box position data contained in the predicted bounding box set, and extract the bounding box coordinate data corresponding to the current frame detection box set. Use Mahalanobis distance operation to pair the predicted bounding box of each live trajectory with the bounding box coordinates of each detection box in the current frame one by one. Complete the deviation quantification calculation of the two sets of position data to obtain the position deviation values ​​of different trajectory and detection box combinations. Organize all position deviation values ​​to obtain the motion matching distance used to characterize the motion dimension matching cost. The Mahalanobis distance can eliminate the influence of the dimension and the interference of dimensional correlation of the location data, accurately reflect the true offset between the predicted target location and the actual detection location, and has stronger spatial location adaptability compared with the traditional Euclidean distance. S42. Perform cosine distance calculation on the initial apparent feature vector and trajectory representative feature vector set of the current frame detection box set to obtain the apparent matching distance. Specifically: extract the initial apparent feature vector corresponding to the current frame detection box set, and extract the trajectory representative feature vector corresponding to the trajectory representative feature vector set. Use cosine distance operation to pair the trajectory representative feature vector corresponding to the live trajectory with the initial apparent feature vector corresponding to the current frame detection box one by one, quantify the size of the appearance feature difference of different pairing combinations, integrate and summarize all feature difference values ​​to obtain the apparent matching distance used to characterize the appearance dimension matching cost. The cosine distance calculation is based on the spatial angle between feature vectors to determine feature similarity. It is not affected by the difference in the magnitude of feature vectors and can stably represent the similarity between two sets of high-dimensional feature vectors. S43. Adaptive fusion weight calculation is performed on the real-time value of the occlusion state counter and the classification confidence of the current frame detection box set to obtain the fusion weight value. Specifically, the real-time value of the occlusion state counter corresponding to each survival trajectory in the survival trajectory set is retrieved, and the classification confidence value corresponding to each detection box in the current frame detection box set is retrieved. Adaptive fusion weight calculation is performed on the real-time value of the occlusion state counter and the classification confidence value to obtain the fusion weight value. By combining the severity of target occlusion with the reliability of detection results, a dynamic weight mapping relationship is constructed. For each set of trajectory and detection box matching combinations, the corresponding weight parameters are solved independently. Unlike the traditional fixed weight matching mechanism, it can adapt to the reliability change law of target motion information and appearance features in dense occlusion scenarios and obtain the fusion weight value corresponding to each matching combination. The fusion weight value is a dynamic adjustment parameter used to balance the ratio of motion matching distance to apparent matching distance, and it can be adaptively adjusted according to the real-time occlusion status of the target and the detection reliability. The specific process of S43 is as follows: S431. Perform a difference calculation on the occlusion state counter and the preset occlusion tolerance threshold to obtain the occlusion deviation. Specifically: retrieve the real-time value of the occlusion state counter corresponding to a single survival trajectory, retrieve the predefined preset occlusion tolerance threshold, perform a difference calculation on the two sets of values ​​to obtain the occlusion deviation of the corresponding survival trajectory. The preset occlusion tolerance threshold is a fixed judgment threshold that is manually calibrated and stored in advance. The preset occlusion tolerance threshold is used to define the reasonable fluctuation range of the target occlusion state and is the benchmark standard for quantifying the severity of the target occlusion. The occlusion state counter can count the number of consecutive occlusion frames of a single survival trajectory in real time. By calculating the difference between the occlusion state counter value and the preset occlusion tolerance threshold, the magnitude of the deviation of the current target occlusion state from the normal tolerable occlusion range can be accurately quantified, thereby intuitively reflecting the severity level of the target's current occlusion condition. S432. Perform a difference operation on the classification confidence and the preset confidence threshold of the current frame detection box set to obtain the confidence deviation. Specifically: by retrieving the classification confidence of the corresponding detection target in the current frame detection box set and the preset confidence threshold, perform a difference operation on the two sets of values ​​to obtain the confidence deviation of the corresponding detection target. Wherein: the preset confidence threshold is a fixed confidence judgment threshold that is pre-calibrated and stored, used to distinguish the effective confidence level of the detection result; the classification confidence is a quantitative value output during the target detection process to characterize the target recognition accuracy of the current detection box. The confidence deviation degree obtained by quantifying the degree of confidence of the target detection result in the current frame by interpolation can accurately reflect the reliability level of the effective information of the current detection box. S433. Apply a logistic regression function to the occlusion deviation and confidence deviation to obtain the fusion weight value, expressed as: In the formula: For fusion weight values; and To adjust the parameters; This is the real-time value of the occlusion status counter. Set the preset occlusion tolerance threshold; For classification confidence; To pre-set the reliability threshold; The occlusion deviation and confidence deviation are input into the above logistic regression function to complete the nonlinear mapping operation. Based on the given adjustment parameters, the fusion weight value is obtained. In steps S431 to S433, the nonlinear fusion of two-dimensional parameters is achieved by quantifying the target occlusion deviation degree and the detection confidence deviation degree respectively, combined with the logistic regression function, and adaptively generating dynamic fusion weights. This completely abandons the fixed weight ratio method in the traditional multi-target tracking data association process. It can dynamically balance the weight ratio of motion matching distance and apparent matching distance according to the real-time changes in the target occlusion degree and the reliability of detection results in dense occlusion scenarios. It effectively adapts to the reliability differences of different feature dimensions under complex occlusion conditions, greatly improves the scene adaptability and matching accuracy of the data association algorithm, and effectively suppresses the problems of target identity switching and trajectory mismatch from the weight optimization dimension. S44. Based on the fusion weight value, the motion matching distance and the apparent matching distance are weighted and summed to obtain the joint cost matrix. Specifically, the fusion weight value corresponding to each matching combination is used as the weighting benchmark, and the motion matching distance and the apparent matching distance are weighted and summed group by group. That is, the dual matching costs of the motion dimension and the apparent dimension are integrated by weighted fusion, so as to achieve the effect of quantifying the matching quality with a single numerical value, complete the cost calculation of all pairing combinations, and generate a joint cost matrix with regular dimensions and complete data. The joint cost matrix is ​​a two-dimensional structured data matrix. The row dimension of the joint cost matrix corresponds to each live trajectory in the video frame, and the column dimension of the joint cost matrix corresponds to each detection box in the current frame. The value of each element inside the joint cost matrix is ​​the comprehensive matching cost of the corresponding trajectory and detection box pairing. S45. Input the joint cost matrix into the bipartite graph matching algorithm to solve for the optimal matching, and obtain the set of matched trajectory pairs, the set of unmatched trajectories and the set of unmatched detection boxes. The bipartite graph matching algorithm treats the live trajectory and the current frame detection box as two independent types of nodes. It completes the global optimal pairing selection based on the cost value of the joint cost matrix. That is, by traversing the cost values ​​of all node pairing combinations through the bipartite graph matching algorithm, the matching combination with the lowest global cost is selected. At the same time, it accurately distinguishes the historical live trajectories that cannot be paired and the new detection targets without corresponding historical trajectories. Finally, the algorithm classifies and regularizes the results to obtain a set of non-overlapping matching trajectory pairs, a set of unmatched trajectories, and a set of unmatched detection boxes. In steps S41-S45, by quantifying the motion matching distance and the apparent matching distance respectively, and dynamically adjusting the fusion weight of the motion matching distance and the apparent matching distance in combination with the target occlusion state and detection confidence, the global optimal association pairing is completed based on the bipartite graph matching algorithm. This effectively solves the problems of single matching dimension and poor weight adaptability of traditional fixed weight association algorithms in dense occlusion scenarios. It can adaptively emphasize the reliable matching dimension according to the real-time state of the target, and significantly reduce the probability of identity mismatch and association failure in the multi-target tracking process. S5. Based on the trajectory representative feature vector in the unmatched trajectory set and the initial apparent feature vector in the unmatched detection box set, perform trajectory recovery based on the cross-attention mechanism on the unmatched trajectory set to generate the recovered trajectory set; Extract the trajectory representative feature vector from the unmatched trajectory set and the initial appearance feature vector from the unmatched detection box set. Build a bidirectional interactive mapping relationship between the trajectory representative feature vector and the initial appearance feature vector through a cross-attention mechanism. Deeply mine the target homologous feature information weakened by noise in occluded scenes, strengthen the similarity weight of homologous features, correct feature matching deviations caused by occlusion, and rebind the identity and continue the trajectory of unmatched trajectories with broken trajectories and failed matching, thereby generating a complete and unique restored trajectory set. The cross-attention mechanism is a common bidirectional feature interaction modeling mechanism in the field of computer vision. It can directly realize the association mining and adaptive weight optimization of two sets of features from different sources, accurately capture the latent feature association of the same source target, and weaken the interference of dissimilar features.

[0019] The multi-target tracking optimization method for densely occluded scenarios proposed in this embodiment can be deployed independently on a tracking optimization server, or it can be deployed separately in a system containing a tracking optimization terminal and a tracking optimization server. The multi-target tracking task is executed through the interaction between the tracking optimization terminal and the tracking optimization server.

[0020] This embodiment obtains bounding boxes and appearance features through target detection, realizes motion state prediction by fusing spatiotemporal context based on spatial neighborhood graph, completes feature queue update interception and temporal decay aggregation by combining occlusion state counter, and completes data association by constructing joint cost matrix using occlusion adaptive weights. Track recovery is achieved through cross-attention mechanism, realizing full-link occlusion scene adaptation optimization. It can stably maintain the identity of the target during short-term occlusion and accurately resume tracking after occlusion ends. It effectively reduces the probability of target identity switching and trajectory breakage risk in dense occlusion scenes, significantly improves the continuity and tracking accuracy of multi-target tracking in complex and crowded scenes, and has good robustness.

[0021] like Figure 2 As shown, a multi-target tracking optimization system for densely occluded scenes includes a target detection module, a motion prediction module, a feature aggregation module, a data association module, and a trajectory recovery module, wherein: The target detection module is used to acquire a video frame sequence, perform target detection on the video frame sequence, and obtain a set of detection boxes for the current frame. The set of detection boxes for the current frame includes bounding box coordinates, classification confidence, and initial appearance feature vector. The motion prediction module performs spatiotemporally context-aware motion state prediction on the survival trajectory set based on the historical motion state sequence of the survival trajectory set and the bounding box coordinates in the current frame detection box set, to obtain a predicted bounding box set, wherein the survival trajectory set includes a historical motion state sequence, an occlusion state counter and a dynamic feature memory queue. The feature aggregation module performs occlusion perception feature aggregation on the dynamic feature memory queue based on the occlusion state counter to obtain a set of trajectory representative feature vectors. The data association module performs adaptive data association for occlusion perception on the current frame detection box set based on the predicted bounding box set and the trajectory representative feature vector set, to obtain a set of matching trajectory pairs, a set of unmatched trajectories, and a set of unmatched detection boxes; The trajectory recovery module performs trajectory recovery on the unmatched trajectory set based on the trajectory representative feature vector in the unmatched trajectory set and the initial appearance feature vector in the unmatched detection box set, generating a recovered trajectory set.

Claims

1. A multi-target tracking optimization method for densely occluded scenes, characterized in that, Includes the following steps: S1. Obtain the video frame sequence, use the pre-trained object detection network to perform object detection on the video frame sequence, and obtain the current frame detection box set. The current frame detection box set includes bounding box coordinates, classification confidence and initial appearance feature vector. S2. Based on the historical motion state sequence of the survival trajectory set and the bounding box coordinates in the current frame detection box set, construct a spatial neighborhood graph for the survival trajectory set, and then perform spatiotemporal context-aware motion state prediction on the survival trajectory set based on the spatial neighborhood graph to obtain the predicted bounding box set. During video tracking, a set of live trajectories is continuously maintained and updated in real time. All target trajectory data that have not yet triggered the tracking termination condition and are in a continuous tracking state are filtered and retained to form a set of live trajectories, which includes historical motion state sequences, occlusion state counters and dynamic feature memory queues. S3. Based on the occlusion state counter, perform occlusion perception feature aggregation on the dynamic feature memory queue to obtain a set of trajectory representative feature vectors. S4. Based on the predicted bounding box set and the trajectory representative feature vector set, perform adaptive data association for occlusion awareness on the current frame detection box set to obtain the matching trajectory pair set, the unmatched trajectory set, and the unmatched detection box set. The process is as follows: S41. Calculate the Mahalanobis distance between the bounding box coordinates of the predicted bounding box set and the previous frame detection box set to obtain the motion matching distance; S42. Calculate the cosine distance between the initial apparent feature vector and the trajectory representative feature vector of the current frame detection box set to obtain the apparent matching distance; S43. Adaptively calculate the fusion weight value by combining the real-time value of the occlusion state counter with the classification confidence of the current frame detection box set. S44. Based on the fusion weight value, the motion matching distance and the apparent matching distance are weighted and summed to obtain the joint cost matrix; S45. Input the joint cost matrix into the bipartite graph matching algorithm to solve for the optimal matching, and obtain the set of matched trajectory pairs, the set of unmatched trajectories and the set of unmatched detection boxes. S5. Based on the trajectory representative feature vector in the unmatched trajectory set and the initial apparent feature vector in the unmatched detection box set, perform trajectory recovery based on the cross-attention mechanism on the unmatched trajectory set to generate the recovered trajectory set.

2. The multi-target tracking optimization method for densely occluded scenes according to claim 1, characterized in that, The process of S1 is as follows: S11. According to a fixed temporal splitting rule, the continuous video data stream transmitted by the video acquisition device is processed by frame splitting to obtain a temporally continuous and orderly video frame sequence. S12. Using a pre-trained target detection network, analyze the image target information in the video frame sequence frame by frame, extract the bounding box coordinates, classification confidence and initial appearance feature vector of the effective targets respectively, summarize the bounding box coordinates, classification confidence and initial appearance feature vector of all targets in a single frame, and generate a set of detection boxes of the current frame with a unified structure.

3. The multi-target tracking optimization method for densely occluded scenes according to claim 1, characterized in that, The process of S2 is as follows: S21. Extract the predicted position of each survival trajectory in the previous frame from the historical motion state sequence, and match the bounding box coordinates of the current frame detection box set. Construct a spatial neighborhood graph for the survival trajectory set to obtain a spatial neighborhood graph including the set of center nodes, the set of neighbor nodes, and spatial distance weights. S22. Based on spatial distance weights, normalize the set of central nodes and the set of neighboring nodes to obtain the normalized aggregate weights of the neighboring nodes. The specific process is as follows: read the spatial distance weights in the spatial neighborhood graph, perform global summation calculation on the spatial distance weights of all neighboring nodes corresponding to a single central node in the set of central nodes in the spatial neighborhood graph to obtain the sum of neighborhood weights corresponding to a single central node, then perform a ratio calculation between the spatial distance weight of each neighboring node and the sum of neighborhood weights to complete the weight normalization mapping, solve the weight values ​​corresponding to all neighboring nodes one by one, and obtain the normalized aggregate weights of all neighboring nodes. S23. Based on the normalized aggregation weights of neighboring nodes, perform graph convolution aggregation on the initial motion feature vector of the center node and the initial motion feature vectors of neighboring nodes to obtain the updated feature vector of the center node. S24. Input the updated feature vector of the center node into the pre-trained bounding box regression network, solve the current frame prediction position corresponding to each survival trajectory, and summarize the regression prediction results of all center nodes to obtain the predicted bounding box set.

4. The multi-target tracking optimization method for densely occluded scenes according to claim 3, characterized in that, The process of constructing the spatial neighborhood graph of S21 is as follows: S211. Set each surviving trajectory as an independent graph node, define the surviving trajectory node to be predicted as the center node, and define other surviving trajectory nodes within the preset spatial neighborhood as neighbor nodes. All center nodes are aggregated to form a center node set, and all neighbor nodes are aggregated to generate a neighbor node set. S212. Calculate the spatial positional deviation between the central node and its corresponding neighboring nodes to obtain the spatial distance weight that can characterize the degree of spatial association between nodes. S213. Integrate the central node set, the neighbor node set, and the spatial distance weight to generate a complete spatial neighborhood graph.

5. The multi-target tracking optimization method for densely occluded scenes according to claim 3, characterized in that, The process in S23 is as follows: S231. Perform feature concatenation on the historical bounding box coordinates, center point velocity, and width and height change rate in the historical motion state sequence to obtain the initial motion feature vector of the center node and the initial motion feature vector of the neighboring nodes. S232. Retrieve the normalized aggregation weights of neighboring nodes and use them as the quantitative basis for feature fusion. Then, perform weighted fusion of the initial motion feature vector of the central node and the initial motion feature vectors of neighboring nodes to obtain the aggregated feature vector that fuses the motion information of multiple nodes. S233. Perform standard matrix multiplication on the aggregated feature vector and the preset learnable weight matrix to obtain the matrix multiplication result after linear transformation. S234. Input the matrix multiplication result into the nonlinear mapping function ReLU to perform nonlinear mapping, and obtain the updated feature vector of the central node, expressed as: In the formula: For the first The center node of the frame updates the feature vector; It is a nonlinear mapping function; The weight matrix is ​​a learnable weight matrix; For the first The self-loop weight of the frame's center node; For the first The initial motion feature vector of the center node of the frame; For the set of neighboring nodes; For the first Normalized aggregate weights of the neighboring nodes of the frame; For the first The initial motion feature vectors of the neighboring nodes of the frame.

6. The multi-target tracking optimization method for densely occluded scenes according to claim 1, characterized in that, The process of S3 is as follows: S31. Read the count value of the occlusion state counter corresponding to each survival trajectory in the survival trajectory set in real time, compare the real-time count result of the occlusion state counter with the preset occlusion tolerance threshold, and stop the feature update operation of the corresponding survival trajectory when the real-time count result of the occlusion state counter exceeds the preset occlusion tolerance threshold, lock the currently stored valid appearance features, and generate a frozen feature queue. When the real-time count result of the occlusion state counter does not exceed the preset occlusion tolerance threshold, new appearance features are continuously received and the feature data in the queue is iteratively updated to generate an updated feature queue. The feature queues corresponding to all survival trajectories are integrated to obtain the memory queue to be aggregated. S32. Retrieve each historical appearance feature vector from the memory queue to be aggregated, perform time dimension feature decay modeling on it, and obtain the time decay weight of each historical appearance feature vector. S33. Based on the time decay weight, perform weighted average pooling on each historical apparent feature vector in the memory queue to be aggregated to obtain a set of trajectory representative feature vectors. The specific process is as follows: perform element-wise weighted operation on each historical apparent feature vector and the corresponding time decay weight to obtain the weighted feature data of each historical apparent feature vector, then solve the global mean of all weighted feature data, and then integrate the weighted feature data corresponding to all surviving trajectories to generate a set of trajectory representative feature vectors.

7. The multi-target tracking optimization method for densely occluded scenes according to claim 6, characterized in that, The process in S32 is as follows: S321. Extract the frame number from each historical apparent feature vector in the aggregated memory queue to obtain the historical frame number and the current frame number. S322. Perform a difference operation between the current frame number and the historical frame number to obtain the inter-frame distance; S323. Based on the inter-frame distance, an exponential decay operation is used to calculate the exponential decay weight of each historical appearance feature vector to obtain the time decay weight of each historical appearance feature vector, expressed as follows: In the formula: As a time decay weight; This is the decay rate parameter, used to uniformly control the decay rate of the overall feature weights; The current frame number; The historical frame number is the one that corresponds to the historical appearance feature vector.

8. The multi-target tracking optimization method for densely occluded scenes according to claim 1, characterized in that, The process in S43 is as follows: S431. Perform a difference calculation between the occlusion state counter and the preset occlusion tolerance threshold to obtain the occlusion deviation. S432. Perform a difference calculation on the classification confidence of the current frame detection box set and the preset confidence threshold to obtain the confidence deviation. S433. Apply a logistic regression function to the occlusion deviation and confidence deviation to obtain the fusion weight value, expressed as: In the formula: For fusion weight values; and To adjust the parameters; This is the real-time value of the occlusion status counter. Set the preset occlusion tolerance threshold; For classification confidence; To pre-set the reliability threshold.

9. The multi-target tracking optimization method for densely occluded scenes according to claim 1, characterized in that, The process of S5 is as follows: extract the corresponding trajectory representative feature vector from the unmatched trajectory set and the initial appearance feature vector from the unmatched detection box set; use a two-way feature interaction modeling mechanism as a cross-attention mechanism; establish a two-way interactive mapping relationship between the trajectory representative feature vector and the initial appearance feature vector through the cross-attention mechanism; deeply mine the target homologous feature information weakened by noise in the occluded scene; strengthen the similarity weight of homologous features; correct the feature matching deviation caused by occlusion; rebind the identity and continue the trajectory for unmatched trajectories with broken trajectory and failed matching; and generate a complete and unique restored trajectory set.

10. A multi-target tracking optimization system for densely occluded scenes to implement the method of claim 1, characterized in that, It includes a target detection module, a motion prediction module, a feature aggregation module, a data association module, and a trajectory recovery module, among which: The target detection module is used to acquire a video frame sequence, perform target detection on the video frame sequence, and obtain a set of detection boxes for the current frame. The set of detection boxes for the current frame includes bounding box coordinates, classification confidence, and initial appearance feature vector. The motion prediction module performs spatiotemporal context-aware motion state prediction on the survival trajectory set based on the historical motion state sequence of the survival trajectory set and the bounding box coordinates in the current frame detection box set, and obtains a predicted bounding box set, wherein the survival trajectory set includes a historical motion state sequence, an occlusion state counter and a dynamic feature memory queue. The feature aggregation module performs occlusion perception feature aggregation on the dynamic feature memory queue based on the occlusion state counter to obtain a set of trajectory representative feature vectors. The data association module performs adaptive data association for occlusion perception on the current frame detection box set based on the predicted bounding box set and the trajectory representative feature vector set, to obtain a set of matching trajectory pairs, a set of unmatched trajectories, and a set of unmatched detection boxes; The trajectory recovery module performs trajectory recovery on the unmatched trajectory set based on the trajectory representative feature vector in the unmatched trajectory set and the initial appearance feature vector in the unmatched detection box set, generating a recovered trajectory set.

Images