An adaptive target tracking method based on multi-granularity context perception

Abstract

The application discloses a kind of based on multi-granularity situation perception's adaptive target tracking method, by calculating the normalized crowd density of current video frame, global physical occlusion rate and global appearance similarity to determine current scene mode, to set the adaptive appearance weight corresponding to current video frame, to set different adaptive appearance weight based on different scene mode, after predicting the priori prediction state corresponding to each tracking trajectory in current video frame in the last video frame that has existed, the appearance feature cost and motion space cost between the appearance feature of each detection frame and the appearance feature of each tracking trajectory are calculated, the appearance feature cost and motion space cost of each detection frame and each tracking trajectory are weighted and fused using the adaptive appearance weight set above, the total matching cost between each detection frame and each tracking trajectory is obtained, so as to realize the matching and state updating of tracking trajectory and detection frame, realize the scene generalization of full automation.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and video analysis, and more specifically, relates to an adaptive target tracking method based on multi-granularity context awareness. Background Technology

[0002] Multi-object tracking is one of the core foundational tasks in computer vision, aiming to continuously detect multiple targets of interest in a video sequence, assign a unique ID to the same target in different frames, and ultimately generate a complete motion trajectory. With the development of deep learning, the most widely adopted paradigm in industry and academia is the "tracking-by-detection" (TBD) approach. As a classic benchmark model in this field, DeepSORT and its latest advanced version, StrongSORT, represent the mainstream level of current multi-object tracking technology and have achieved excellent performance on standard datasets (such as MOT17).

[0003] However, when existing technologies are migrated from a familiar scenario to a vastly different and unfamiliar one, they generally expose the problem of "static parameters," resulting in a lack of scene awareness. This directly leads to insufficient algorithm generalization ability and poor target tracking capabilities in dynamic scenes. Specifically, most existing tracking algorithms typically use a fixed set of hyperparameters, such as fixed appearance weights and IoU thresholds, to handle all different application scenarios. But in the real world, scenes are dynamically changing. For example, in sparsely populated scenes, the appearance features of targets are usually very reliable; while in specific scenarios like a dance team performance or athletes wearing uniforms, the appearances of targets are highly similar, causing appearance features to become almost ineffective. In this case, the algorithm should rely more on motion prediction. Obviously, a fixed set of parameters cannot accommodate both extreme cases, which easily leads to serious identity switching problems. Some trackers introduce scene awareness, mainly by calculating the global crowding of the entire image. However, this "one-size-fits-all" global metric completely ignores the differences in local space. For example, in a scenario of group movement, the high global congestion is often caused by the overlap of a few pairs of targets in a localized area, while other areas may still be very open. If, based on the high global congestion, all targets are forced into a "conservative prediction" state, the behavior of individuals who are in local gaps and genuinely intend to maneuver will be incorrectly suppressed. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides an adaptive target tracking method based on multi-granularity context awareness, thereby solving the technical problem that the static parameters of the existing technology lead to poor target tracking capability in dynamic scenes.

[0005] To achieve the above objectives, according to a first aspect of the present invention, an adaptive target tracking method based on multi-granularity context awareness is provided, comprising: The system acquires the detection box information and appearance features of all detection boxes in the current video frame to calculate normalized crowd density, global physical occlusion rate, and global appearance similarity. Based on these parameters, it determines the current scene mode and sets an adaptive appearance weight for the current video frame. Specifically, the current scene mode is designated as the first mode when the global appearance similarity is higher than a first preset value; the current scene mode is designated as the second mode when the normalized crowd density is higher than a second preset value or the global physical occlusion rate is higher than a third preset value; otherwise, the current scene mode is designated as the third mode. The adaptive appearance weights set for different scene modes are different. For each tracking trajectory that already exists in the previous video frame, predict the prior prediction state corresponding to each tracking trajectory in the current video frame; For each tracking trajectory and each detection box in the current video frame, calculate the appearance feature cost between the appearance features of each detection box and the appearance features of each tracking trajectory, and calculate the motion space cost between the spatial position of each detection box and the spatial position contained in the prior prediction state corresponding to each tracking trajectory in the current video frame; wherein, the higher the similarity between the appearance features of any detection box and the appearance features of any tracking trajectory, the lower the corresponding appearance feature cost; the greater the distance between the spatial position of any detection box and the spatial position contained in the prior prediction state corresponding to any tracking trajectory in the current video frame, the greater the corresponding motion space cost; Based on the adaptive appearance weight corresponding to the current video frame, the appearance feature cost between each detection box and each tracking trajectory, and the motion space cost, calculate the total matching cost between each detection box and each tracking trajectory; Matching is performed based on the total matching cost between each detection box and each tracking trajectory to obtain the matching result of each detection box. The state is updated for the successfully matched detection boxes and tracking trajectories to obtain the posterior prediction state of each successfully matched tracking trajectory in the current video frame.

[0006] According to the above-mentioned adaptive target tracking method based on multi-granularity context awareness, the step of calculating normalized crowd density, global physical occlusion rate, and global appearance similarity based on the detection box information and appearance features of each detection box in the current video frame specifically includes: The normalized crowd density is determined based on the ratio of the total number of valid detected targets in the current video frame to the total number of pixels in the image. The cross-union ratio (CUP) between different detection boxes is calculated based on the spatial location contained in the detection box information of each detection box, and the global physical occlusion rate is calculated based on the CUP between different detection boxes. An original feature matrix is ​​constructed based on the appearance features of all detection boxes containing valid detection targets, and then normalized to obtain a normalized feature matrix. The dot product between the normalized feature matrix and its transpose is calculated to obtain a feature autocorrelation matrix. The global appearance similarity is calculated based on the upper triangular elements of the feature autocorrelation matrix.

[0007] According to the above-mentioned adaptive target tracking method based on multi-granularity context awareness, the step of determining the current scene mode based on normalized crowd density, global physical occlusion rate, and global appearance similarity, and setting the adaptive appearance weight corresponding to the current video frame based on the current scene mode, specifically includes: If the global appearance similarity If the value is greater than the first preset value, and the current scene mode is the first mode, then the target appearance weight is calculated based on the following formula. : ; If the normalized population density Greater than the second preset value or the global physical occlusion rate If the value is greater than the third preset value, and the current scene mode is the second mode, then the target appearance weight will be adjusted. Set to the preset maximum value; Otherwise, the current scene mode is the third mode, and the occlusion penalty factor is calculated based on the following formula. : And based on the global appearance similarity Calculate the target appearance weight; wherein, if the global appearance similarity... If the value is greater than the fourth preset value, the target appearance weight is calculated according to the following formula. : , Otherwise, calculate the target appearance weight according to the following formula. : ; This is the default value; If the absolute difference between the target appearance weight and the adaptive appearance weight corresponding to the previous video frame is less than or equal to a preset dead zone threshold, then the adaptive appearance weight corresponding to the previous video frame is determined as the adaptive appearance weight corresponding to the current video frame; otherwise, the target appearance weight is smoothed based on the adaptive appearance weight corresponding to the previous video frame to obtain the final adaptive appearance weight corresponding to the current video frame.

[0008] According to the above adaptive target tracking method based on multi-granularity context awareness, the prior prediction state corresponding to each tracking trajectory in the current video frame is obtained through an interactive multi-model filter; the interactive multi-model filter includes an inertial expert, a maneuvering expert, and an attenuation expert corresponding to different motion modes; the motion mode corresponding to the inertial expert is uniform motion, the motion mode corresponding to the maneuvering expert is directional motion, and the motion mode corresponding to the attenuation expert is deceleration motion. The step of predicting the prior prediction state of each tracking trajectory in the current video frame for each existing tracking trajectory in the previous video frame specifically includes: Extract the detection box and predicted velocity vector of the tracking target corresponding to each tracking trajectory from the posterior prediction state corresponding to each tracking trajectory in the previous video frame, and search for neighboring tracking targets of each tracking target within a preset search radius; For each tracked target, the intersection-union ratio (IUU) of the detection bounding box of each tracked target with the detection bounding boxes of its neighboring tracked targets is calculated. Based on the IUU of the detection bounding boxes of each tracked target with the detection bounding boxes of its neighboring tracked targets, the local congestion degree of the corresponding tracked target is determined. ; Based on the predicted velocity vector of each tracked target and the average predicted velocity vector of each target's neighboring tracked targets, the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity is determined. Then, based on the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity, the relative motion anomaly degree of each tracked target is comprehensively determined. ; The turning gain for each tracked target is calculated based on the following formula. : And calculate the attenuation gain for each tracked target based on the following formula. : ; For each tracked target, the column vector elements pointing to the maneuvering expert in the basic transfer probability matrix are multiplied by the turning gain of the corresponding tracked target, and the column vector elements pointing to the attenuation expert are multiplied by the attenuation gain of the corresponding tracked target to obtain the gain transfer probability matrix of the tracking trajectory corresponding to the tracked target. Each row of the gain transfer probability matrix of each tracking trajectory is normalized to obtain the dynamic transfer probability matrix of each tracking trajectory. The elements of the dynamic transfer probability matrix represent the probability of transfer between the motion modes corresponding to any two of the inertial expert, maneuvering expert, and attenuation expert. Based on the inertial expert, the maneuvering expert, and the attenuation expert, and combined with the dynamic transition probability matrix of each tracking trajectory, the prior prediction state of each tracking trajectory in the current video frame is predicted.

[0009] According to the above-described adaptive target tracking method based on multi-granularity context awareness, the inertial expert corresponds to the state transition matrix of each tracking trajectory. process noise covariance matrix They are represented as follows:

[0010]

[0011] in, It represents the discrete time interval between two adjacent video frames. for The identity matrix; These represent the standard deviations of process noise corresponding to the x-coordinate and y-coordinate of the center position and the height of the detection frame, respectively. These represent the standard deviations of the process noise for the corresponding velocity components. These represent the standard deviations of process noise corresponding to the aspect ratio and its rate of change, respectively. The state transition matrix of the mobile expert corresponding to each tracking trajectory process noise covariance matrix They are represented as follows:

[0012]

[0013] in, , The current heading angular velocity of the corresponding tracking trajectory, This is the default value; The attenuation expert corresponds to the state transition matrix of each tracking trajectory. process noise covariance matrix They are represented as follows:

[0014]

[0015] in, and This is the default value.

[0016] The adaptive target tracking method based on multi-granularity context awareness described above is characterized by the current heading angular velocity of any tracking trajectory. It was determined based on the following method: The predicted velocity vector of the tracked target corresponding to the tracked trajectory, which is extracted from the posterior prediction state corresponding to the tracked trajectory in the previous video frame, is decomposed into lateral velocity and longitudinal velocity, and the current absolute heading angle of the tracked trajectory is solved based on the lateral velocity and longitudinal velocity. If the number of consecutive matching frames of the tracking trajectory reaches the preset requirement and the velocity magnitudes corresponding to the lateral and longitudinal velocities are greater than the set static threshold, then the current absolute heading angle of the tracking trajectory is added to the heading angle history buffer queue corresponding to the tracking trajectory. If the length of the heading angle history buffer queue corresponding to the tracking trajectory reaches the preset requirement, then the absolute heading angle that was added earliest in the heading angle history buffer queue is extracted as the starting heading angle, and the original algebraic difference between the current absolute heading angle of the tracking trajectory and the starting heading angle is calculated. Angle jump correction is applied to the original algebraic difference: if the original algebraic difference is greater than... Then subtract the original algebraic difference value. The corrected heading angle is obtained; if the original algebraic difference is less than Then add the original algebraic difference value. The corrected heading angle is obtained; Divide the corrected heading angle by the number of frame intervals corresponding to the heading angle history buffer queue to obtain the current heading angular velocity of the tracking trajectory. The number of frame intervals corresponding to the heading angle history buffer queue is equal to the length of the heading angle history buffer queue minus 1.

[0017] According to the above-described adaptive target tracking method based on multi-granularity context awareness, the method is characterized in that, based on the inertial expert, the maneuvering expert, and the attenuation expert, and combined with the dynamic transition probability matrix of any tracking trajectory, the prior prediction state of any tracking trajectory in the current video frame is predicted, specifically including: Based on the dynamic transition probability matrix of any tracking trajectory and the posterior probability that the tracking target corresponding to any tracking trajectory in the previous video frame is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert, the prior probability that the tracking target corresponding to any tracking trajectory in the current video frame is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert is calculated using the following formula: Where i and j are the indices of each expert. Let be the posterior probability that the tracking target corresponding to any tracking trajectory in the previous video frame is in the motion mode corresponding to expert i. The element in the i-th row and j-th column of the dynamic transition probability matrix. The prior probability that the tracking target corresponding to any tracking trajectory in the current video frame is in the motion mode corresponding to expert j; The posterior state vector and posterior covariance of any tracking trajectory output by the inertial expert, the maneuvering expert, and the attenuation expert in the previous video frame are cross-mixed using the following formula to obtain the mixed initial state vector and mixed initial covariance of each expert for any tracking trajectory: ; in, For expert j, the hybrid initial state vector for any of the tracking trajectories. For expert j, the mixed initial covariance for any of the tracking trajectories, and These are the posterior state vector and posterior covariance of any tracking trajectory output by expert i in the previous video frame, respectively. Based on the state transition matrix and process noise covariance matrix of the inertial expert, the maneuvering expert and the attenuation expert for any tracking trajectory, the mixed initial state vector and mixed initial covariance of the corresponding experts for any tracking trajectory are extrapolated forward by one time step to obtain the prior state vector and prior covariance output by the corresponding experts for any tracking trajectory in the current video frame. Based on the prior probability that the tracking target corresponding to any tracking trajectory in the current video frame is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert, the prior state vector and prior covariance output by the inertial expert, the maneuvering expert, and the attenuation expert for any tracking trajectory in the current video frame are weighted and fused respectively to obtain the prior prediction state and prior prediction covariance corresponding to any tracking trajectory in the current video frame.

[0018] The adaptive target tracking method based on multi-granularity context awareness described above is characterized by updating the state of any successfully matched detection box and tracking trajectory to obtain the posterior predicted state of the tracking trajectory in the current video frame, specifically including: Using a preset observation matrix, the prior state vectors and prior covariances output by each expert for the tracking trajectory in the current video frame are converted into predicted state observations and innovation covariance matrices, and the spatial positions of the matched detection boxes are converted into the actual state observations of the tracking trajectory. Calculate the Mahalanobis distances for this tracking trajectory from various experts: ;in, For expert J, the Mahalanobis distance of this tracking trajectory, The residual between the predicted state observations of the tracked trajectory and the actual state observations of the tracked trajectory is given by expert j. For expert j, the new information covariance matrix of the track should be provided; Substituting the Mahalanobis distance and information covariance matrix of each expert for the tracking trajectory into the multivariate Gaussian probability density function, we obtain the observation likelihood of the corresponding expert. The observation likelihood of each expert is multiplied by the prior probability that the tracking target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to the corresponding expert, and then normalized to obtain the posterior probability that the tracking target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to each expert. Calculate the Kalman gain for each expert: ;in, For expert j, the Kalman gain, For expert j, the prior covariance of the tracking trajectory output in the current video frame. The preset observation matrix; Determine the posterior state vector and posterior covariance of the tracking trajectory output by each expert in the current video frame: ;in, For expert j, this is the prior state vector in the current video frame output for the tracking trajectory; The posterior state vectors of the tracking trajectory output by each expert in the current video frame are fused into the corresponding posterior predicted state of the tracking trajectory in the current video frame based on the following formula. : ; The posterior probability that the tracking target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to expert j; The posterior covariance of the tracking trajectory output by each expert in the current video frame is fused into the posterior prediction covariance of the tracking trajectory in the current video frame based on the following formula. : .

[0019] According to the above-described adaptive target tracking method based on multi-granularity context awareness, the key feature is that calculating the appearance feature cost between the appearance features of any detection box and the appearance features of any tracking trajectory specifically includes: If the first feature library contains the appearance features of the tracking trajectory and the global physical occlusion rate is less than a preset threshold, then the cosine similarity between the appearance features of the detection box and the appearance features of the tracking trajectory stored in the first feature library is calculated, and the difference between the constant 1 and the cosine similarity is calculated as the first appearance cost between the appearance features of the detection box and the appearance features of the tracking trajectory; otherwise, the first appearance cost is determined to be 1. Extract the clusters formed by the appearance features of the tracking trajectory stored in the second feature library, calculate the cosine similarity between the appearance features of the detection box and the centroid of the cluster, calculate the difference between the constant 1 and the largest cosine similarity, and multiply the difference by a preset attenuation discount coefficient to obtain the second appearance cost between the appearance features of the detection box and the appearance features of the tracking trajectory. The smaller value between the first appearance cost and the second appearance cost is determined to be the appearance feature cost between the appearance features of the detection box and the appearance features of the tracking trajectory. The appearance features of each tracking trajectory currently stored in the first feature library are obtained by smoothing the appearance features of the detection boxes matched by the corresponding tracking trajectory in the previous video frame; the appearance features of each tracking trajectory stored in the second feature library are obtained by filtering the appearance features of the detection boxes matched by the corresponding tracking trajectory each time.

[0020] According to the above-described adaptive target tracking method based on multi-granularity context awareness, the second feature library is updated in the following manner: Calculate the cosine similarity between the appearance features of all detection boxes in the current video frame and the centroids of all clusters in the second feature library, and determine the maximum cosine similarity corresponding to all detection boxes; For each detection box in the current video frame, if the maximum cosine similarity corresponding to the detection box is greater than a preset threshold, the cluster corresponding to the maximum cosine similarity is updated based on the appearance features of the detection box; otherwise, the cosine similarity between the appearance features of the detection box and the feature entries stored in the preset buffer is calculated. When the cosine similarity between the appearance features of the detection box and any feature entry is higher than the preset threshold, the counter value of any feature entry is incremented by 1. When the counter value of any feature entry reaches a preset number of entries, the any feature entry is moved from the preset buffer to the second feature library to form a new cluster corresponding to the tracking trajectory. When the cosine similarity between the appearance features of the detection box and all feature entries is lower than the preset threshold, the appearance features of the detection box are added to the preset buffer as a new feature entry.

[0021] According to a second aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0022] According to a third aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the first aspect.

[0023] According to a fourth aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0024] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: Using the detection box information and appearance features of each detection box in the current video frame, normalized crowd density, global physical occlusion rate, and global appearance similarity are calculated respectively. Based on the normalized crowd density, global physical occlusion rate, and global appearance similarity, the current scene mode is determined, and an adaptive appearance weight corresponding to the current video frame is set based on the current scene mode. Different adaptive appearance weights are set for different scene modes. After predicting the prior prediction state of each tracking trajectory that already exists in the previous video frame in the current video frame, the appearance feature cost between the appearance features of each detection box and the appearance features of each tracking trajectory is calculated. The motion between the spatial position of each detection box and the spatial position contained in the prior prediction state of each tracking trajectory in the current video frame is also calculated. Spatial cost is calculated by weighting and fusing the appearance feature cost and motion space cost of each detection box and each tracking trajectory using the aforementioned adaptive appearance weights. This yields the total matching cost between each detection box and each tracking trajectory, thus achieving matching between the tracking trajectory and the detection box. The state of successfully matched detection boxes and tracking trajectories is updated. The current scene mode is determined by features such as normalized crowd density, global physical occlusion rate, and global appearance similarity, achieving multi-granularity context awareness. It can automatically identify and adapt to various cross-domain scenes, from sparse to highly similar and highly crowded, achieving fully automatic scene generalization. Furthermore, different adaptive appearance weights are set based on different scene modes to adjust the contribution of appearance features in the total matching cost calculation process, thereby improving the target tracking capability in dynamic scenes.

[0025] Furthermore, by constructing an interactive multi-model filter that includes inertial experts, maneuvering experts, and attenuation experts corresponding to different motion modes, and using this interactive multi-model filter for prediction and state updates, an instance-level dynamic routing mechanism can be realized. This effectively solves the trajectory divergence problem in scenarios such as sharp turns and rotations, and can accurately track nonlinear moving targets, effectively reducing prediction errors at low frame rates (such as 2FPS).

[0026] Furthermore, by constructing a first feature library and a second feature library, feature matching is performed using the first and second feature libraries in this hierarchical manner when calculating the appearance feature cost. This ensures the purity and real-time performance of feature matching, effectively isolates dirty features caused by occlusion, and solves the feature lag problem under rapid pose changes. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the adaptive target tracking method based on multi-granularity context awareness provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0029] This invention provides an adaptive target tracking method based on multi-granularity context awareness, such as... Figure 1 As shown, it includes: S1. Obtain the detection box information and appearance features of all detection boxes in the current video frame to calculate the normalized crowd density, global physical occlusion rate, and global appearance similarity. Determine the current scene mode based on the normalized crowd density, global physical occlusion rate, and global appearance similarity, and set the adaptive appearance weight corresponding to the current video frame based on the current scene mode. Specifically, when the global appearance similarity is higher than a first preset value, the current scene mode is the first mode; when the normalized crowd density is higher than a second preset value or the global physical occlusion rate is higher than a third preset value, the current scene mode is the second mode; otherwise, the current scene mode is the third mode. The adaptive appearance weights set for different scene modes are different. S2, For each tracking trajectory that already exists in the previous video frame, predict the prior prediction state corresponding to each tracking trajectory in the current video frame; S3, for each tracking trajectory and each detection box in the current video frame, calculate the appearance feature cost between the appearance features of each detection box and the appearance features of each tracking trajectory, and calculate the motion space cost between the spatial position of each detection box and the spatial position contained in the prior prediction state corresponding to each tracking trajectory in the current video frame; wherein, the higher the similarity between the appearance features of any detection box and the appearance features of any tracking trajectory, the lower the corresponding appearance feature cost; the greater the distance between the spatial position of any detection box and the spatial position contained in the prior prediction state corresponding to any tracking trajectory in the current video frame, the greater the corresponding motion space cost; S4. Based on the adaptive appearance weight corresponding to the current video frame, the appearance feature cost between each detection box and each tracking trajectory, and the motion space cost, calculate the total matching cost between each detection box and each tracking trajectory. S5, perform matching based on the total matching cost between each detection box and each tracking trajectory to obtain the matching result of each detection box, and update the state of the interactive multi-model filter for the successfully matched detection boxes and tracking trajectories to obtain the posterior prediction state of each successfully matched tracking trajectory in the current video frame.

[0030] Specifically, the input video sequence is processed by an object detector (e.g., YOLOX, bytetrack, etc.) to obtain the bounding box information for all detection boxes in the current video frame. This bounding box information includes the spatial location and confidence level of each bounding box. A deep residual network is then used to extract the semantic information within each bounding box, generating high-dimensional appearance features. It should be noted that the bounding boxes obtained here can be valid bounding boxes with a confidence level higher than a preset value.

[0031] Subsequently, statistical analysis is performed on each detection box in the current video frame to calculate a multi-dimensional scene fingerprint. Specifically, based on the detection box information and appearance features of each detection box in the current video frame, normalized crowd density, global physical occlusion rate, and global appearance similarity can be calculated respectively. Adaptive appearance weights corresponding to the current video frame are then determined based on the normalized crowd density, global physical occlusion rate, and global appearance similarity.

[0032] In some embodiments, the normalized crowd density can first be determined based on the ratio of the total number of valid detected targets in the current video frame to the total number of pixels in the image. The total number of valid detected targets is obtained by counting the number of detection boxes in the current video frame with a confidence level higher than a preset value. The ratio of the total number of valid detected targets to the total number of pixels in the image is calculated and multiplied by one million to obtain the number of targets per million pixels, which is then used as the normalized crowd density.

[0033] Secondly, the global physical occlusion rate is calculated to quantify the degree of mutual occlusion between different targets. This involves calculating the intersection-union ratio (CIU) between different detection boxes based on the spatial locations contained within the detection box information, and then calculating the global physical occlusion rate based on the CIU. Specifically, assuming there are n valid detection boxes in the current video frame, the top-left corner coordinates of each detection box are first extracted. Given the width w and height h, calculate the coordinates of the bottom right corner. ,in For any two bounding boxes i and j, the coordinates of the top-left corner of their potential intersection region are calculated by taking the maximum and minimum values ​​of their corresponding coordinates. and the coordinates of the bottom right corner Based on the aforementioned intersecting boundaries, the width and height of the intersecting region are further calculated, and the two are multiplied together to obtain the intersecting area. Simultaneously, the areas of detection boxes i and j are calculated respectively. and Therefore, the combined area is obtained. Based on this, the intersection-union ratio (IUU) of any two detection boxes is calculated. ,in To minimize the value, we need to prevent the denominator from being zero. Therefore, we can construct a value of size... The global IoU matrix is ​​calculated, and its diagonal elements are forced to be set to 0 to exclude the comparison of the target itself. To avoid redundant calculations caused by the symmetric matrix, the upper triangular elements of the matrix are further extracted to obtain the cross-union ratio of all unique detection box pairs. To prevent a large number of non-overlapping targets from diluting the occlusion features, non-overlapping terms with values ​​of 0 can be removed, retaining only the effective cross-union ratios of substantial overlap. Finally, the arithmetic mean of all retained non-zero cross-union ratios is calculated and used as a quantitative indicator to measure the severity of effective occlusion in the current video frame, i.e., the global physical occlusion rate. If there are no overlapping targets in the current video frame, that is, the intersection-union ratio of all detection box pairs is 0, then the global physical occlusion rate output is 0.

[0034] Finally, global appearance similarity is calculated to automatically identify special scenarios where target appearances are highly similar. This involves constructing an original feature matrix based on the appearance features of all detection boxes containing valid targets, normalizing it to obtain a normalized feature matrix, and then calculating the dot product between this normalized feature matrix and its transpose to obtain a feature autocorrelation matrix. The global appearance similarity is then calculated based on the upper triangular elements of this feature autocorrelation matrix. Specifically, to ensure the reliability of feature extraction and to filter out noise from low-quality detections, only high-quality detection boxes with a confidence score greater than 0.6 in the current video frame (considered as detection boxes containing valid targets) are selected for calculation. Assuming K detection boxes containing valid targets are selected, the appearance features of each detection box are extracted and concatenated row-wise to construct a matrix of size [size missing]. The original feature matrix F, This represents the dimension of the appearance features. Next, the original feature matrix F undergoes row-level L2 normalization. Specifically, for any row vector in the original feature matrix... Calculate its L2 norm , row vector The d-th element is then divided by the original row vector by the L2 norm and a local minimum. The sum achieves row-level normalization, that is Introducing a local minimum This is to prevent division-by-zero anomalies caused by all features being zero, thus ensuring the absolute stability of numerical calculations. After processing, a normalized feature matrix is ​​obtained. Subsequently, matrix multiplication is used to perform a dot product operation between the normalized feature matrix and its transpose matrix to obtain the feature autocorrelation matrix. Since the feature vectors have been normalized, this dot product result is equivalent to the cosine similarity between vectors. Therefore, the cosine similarity between all pairs of targets is calculated quickly and in parallel, resulting in a matrix of size [missing value]. The feature autocorrelation matrix S is obtained. Since this matrix is ​​symmetric and its main diagonal elements are always 1, the upper triangular elements of this matrix can be extracted to avoid redundant calculations and interference from its own data. Finally, the arithmetic mean of all extracted upper triangular elements is calculated, and this arithmetic mean is used as the global appearance similarity of the current video frame. It should be noted that if fewer than two detection boxes containing valid targets are selected, the calculation results of historical video frames will be used to maintain the continuity of scene perception. If the standard deviation of the original feature matrix is ​​close to 0 (i.e., the target features have severely degraded or the appearance is extremely similar), the circuit breaker mechanism will be triggered, and the theoretical maximum value of cosine similarity, 1.0, will be forcibly output as the global appearance similarity of the current video frame.

[0035] Based on the normalized crowd density, global physical occlusion rate, and global appearance similarity of the current video frame, the scene mode of the current video frame can be determined. An adaptive appearance weight is then set for the current video frame based on this scene mode to adjust the contribution of appearance features in the subsequent trajectory matching process. Specifically, when the global appearance similarity is higher than a first preset value, the current scene mode is the first mode, corresponding to a high-similarity scene; when the normalized crowd density is higher than a second preset value or the global physical occlusion rate is higher than a third preset value, the current scene mode is the second mode, corresponding to a crowded scene; otherwise, the current scene mode is the third mode, i.e., a general scene. It is important to emphasize that the adaptive appearance weights set for different scene modes are different.

[0036] For example, when the current scene mode is mode one, it can be determined that appearance features have lost their discriminative power. In this case, the adaptive appearance weight can be reduced to mask the matching noise brought by appearance features, and the tracking task can be completed entirely by relying on the motion model. When the current scene mode is mode two, it indicates that the scene mode is a crowded mode, and the motion prediction results will fail. Therefore, the adaptive appearance weight can be increased to achieve trajectory association by leveraging the appearance differences between targets. If neither of the above two conditions is met, and the scene mode is considered to correspond to a general scene, the adaptive appearance weight can be adjusted linearly based on the global physical occlusion rate to achieve a weight balance between appearance features and motion prediction.

[0037] In some embodiments, if global appearance similarity If the value is greater than the first preset value (e.g., 0.65), the target appearance weight is calculated based on the following formula. : The target appearance weight is linearly reduced as the global appearance similarity increases. If normalized population density Greater than the second preset value (e.g., 10) or the global physical occlusion rate If the value is greater than the third preset value (e.g., 0.5), then the target appearance weight will be adjusted. Set to the preset maximum value; Otherwise, calculate the occlusion penalty factor based on the following formula. : And based on global appearance similarity Calculate the target appearance weight; where, if the global appearance similarity... If the value is greater than the fourth preset value (e.g., 0.4), the target appearance weight is calculated according to the following formula. : , Otherwise, calculate the target appearance weight according to the following formula. : ; This is a preset value. Specifically, when the average size of the detection frame is less than the preset value, Set it to 0.6, otherwise set it to 0.0.

[0038] After the calculation is completed, a physical limit can be applied to the target appearance weight. In the first mode, the lower limit of the target appearance weight is 0.0, and in other cases the lower limit is 0.2, with the upper limit uniformly set at 0.99.

[0039] To prevent minor fluctuations in scene fingerprints caused by the instability of single-frame detection, which could lead to drastic oscillations in weight parameters, a dual anti-shake mechanism is introduced. Specifically, if the absolute difference (i.e., the absolute value of the difference) between the calculated target appearance weight and the adaptive appearance weight corresponding to the previous video frame is less than or equal to a preset dead zone threshold, then the adaptive appearance weight corresponding to the previous video frame is determined as the adaptive appearance weight corresponding to the current video frame; otherwise, the target appearance weight is smoothed based on the adaptive appearance weight corresponding to the previous video frame to obtain the adaptive appearance weight corresponding to the current video frame. The smoothing process can employ an EMA filtering mechanism, and the smooth transition calculation is performed according to the following formula: .in, The preset time smoothing coefficient, The adaptive appearance weights are the values ​​corresponding to the previous video frame. This represents the final adaptive appearance weight corresponding to the current video frame.

[0040] For each tracking trajectory that already existed in the previous video frame—that is, each tracking trajectory that the tracker still maintains after processing the previous video frame—motion prediction models such as the Kalman filter can be used to predict the prior prediction state of each tracking trajectory in the current video frame based on the prediction state and prediction covariance of each tracking trajectory at the previous moment. Here, "tracking trajectories still maintained" includes confirmed trajectories and trajectories that are still in the trial period but have not yet been deleted. For confirmed trajectories, they have usually been successfully associated and assigned unique identity IDs in multiple consecutive historical video frames, and can be considered to correspond to relatively stable physical targets. If a tracking trajectory successfully matches a detection box in the previous video frame, its posterior prediction state and posterior prediction covariance obtained from the state update in the previous video frame are directly used as the prediction basis for the current video frame. When the tracking trajectory fails to match in the previous video frame but has not yet been deleted, it cannot update its state. Therefore, after entering the current video frame, it obtains the prior prediction state and prior prediction covariance in the previous video frame as the posterior prediction state and posterior prediction covariance to make predictions for the current video frame.

[0041] Considering that most existing tracking algorithms assume the target is moving in a uniform linear motion, but in real-world scenarios, the target's motion is much more complex, such as making sharp turns, running back and forth, or suddenly stopping, this single linear motion model cannot adapt to these nonlinear motion abrupt changes, leading to tracking failure. Therefore, to address the problem of nonlinear motion abrupt changes, some embodiments have constructed a multi-model interactive prediction and update mechanism. Specifically, an interactive multi-model filter can be used to predict the prior prediction state of each tracking trajectory in the current video frame. This interactive multi-model filter includes an inertial expert, a maneuvering expert, and a decay expert corresponding to different motion modes. The inertial expert corresponds to uniform motion, the maneuvering expert corresponds to directional change motion, and the decay expert corresponds to deceleration motion.

[0042] When predicting the prior prediction state corresponding to each tracking trajectory in the current video frame, the following steps are included: S31, extract the detection box and predicted velocity vector of the tracking target corresponding to each tracking trajectory from the posterior prediction state corresponding to each tracking trajectory in the previous video frame, and search for neighboring tracking targets of each tracking target within a preset search radius; S32, for each tracked target, calculate the intersection-union ratio (IUU) of the detection box of each tracked target with the detection boxes of its neighboring tracked targets, and determine the local congestion degree of the corresponding tracked target based on the IUU of the detection boxes of each tracked target with the detection boxes of its neighboring tracked targets. ; S33. Based on the predicted velocity vector of each tracked target and the average predicted velocity vector of each tracked target's neighboring tracked targets, determine the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity. Then, based on the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity, comprehensively determine the relative motion anomaly degree of each tracked target. ; S34, calculate the turning gain based on the following formula : And calculate the attenuation gain based on the following formula. : ; S35, multiply the column vector elements pointing to the maneuver expert in the basic transition probability matrix by the turning gain, and multiply the column vector elements pointing to the attenuation expert by the attenuation gain to obtain the gain transition probability matrix; normalize each row of the gain transition probability matrix to obtain the dynamic transition probability matrix; where the elements of the dynamic transition probability matrix represent the basic probability of transition between the motion modes corresponding to any two of the inertial expert, maneuver expert and attenuation expert. S36, based on inertial experts, maneuver experts, and attenuation experts, combined with a dynamic transition probability matrix, predicts the prior prediction state of each tracking trajectory in the current video frame.

[0043] Specifically, since each tracking trajectory already exists in the previous video frame, the tracking result of the previous video frame can be extracted, that is, the posterior predicted state corresponding to each tracking trajectory in the previous video frame, output by the interactive multi-model filter after state update. The posterior predicted state, prior predicted state, and the dimensions of the prior and posterior state vectors (described later) are defined consistently, all including the spatial location, size, and motion information of the detection box of the tracking target corresponding to the tracking trajectory, which will not be elaborated further. For example, it can be defined as... ,in, and ... Indicates the aspect ratio of the detection frame; Indicates the height of the detection frame; These represent the first-order rates of change corresponding to the aforementioned state variables. Based on the motion information contained in the posterior predicted state, predicted velocity vectors representing the target's motion direction and velocity can be extracted.

[0044] Based on the bounding boxes of the tracked targets extracted from each tracking trajectory in the posterior predicted state, neighboring tracked targets are searched within a preset search radius. For each tracked target, the intersection-union ratio (IUR) of the bounding box of each tracked target with the bounding boxes of its various neighboring tracked targets is calculated. Based on the IUR of the bounding boxes of each tracked target with the bounding boxes of its various neighboring tracked targets, the local congestion degree of the corresponding tracked target is determined. Specifically, for any tracked target, the intersection-union ratios (IUUs) of its detection boxes with those of all neighboring tracked targets can be accumulated. The smaller of this sum and the constant 1.0 is then used to limit outliers, and this sum is taken as the local crowding degree of the tracked target. This allows for precise quantification of the severity of physical occlusion within the target's local space.

[0045] Next, the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion speed is calculated. Specifically, for any tracked target, the average of the predicted velocity vectors of all its neighboring tracked targets is first calculated as the average predicted velocity vector, and then the difference between its motion and that of the surrounding crowd is comprehensively evaluated from the two dimensions of direction and velocity magnitude.

[0046] In the direction of motion dimension, cosine similarity is calculated by multiplying the predicted velocity vector of any tracked target by the average predicted velocity vector of its neighboring tracked targets. To transform this cosine similarity into a reasonable anomaly metric, a mapping formula is used. Calculate the directional difference penalty term. Among them, For the above cosine similarity, This represents the degree of anomalousness between the tracked target and its neighboring tracked targets in the direction of motion. When calculating cosine similarity, a minimum constant can be introduced into the denominator (i.e., the product of the magnitudes of the two vectors) to effectively prevent division-by-zero collapse anomalies caused when the target or crowd is absolutely stationary. Through this mapping mechanism, cases where the target and crowd move in the same direction are mapped to a penalty value of 0, while when the target crosses or moves in the opposite direction, the penalty value is strictly mapped to a high-anomaly range of 0.5 to 1.0, thus accurately quantifying the outlier degree of the target's direction of motion.

[0047] In terms of speed, the focus is on capturing sudden stops or accelerations of the target relative to a crowd. Here, the magnitude of the predicted velocity vector of the tracked target can be calculated. The magnitude of the average predicted velocity vector of the target being tracked by its neighbors The relative difference ratio is specifically calculated using the following formula: ,in This represents the degree of anomalousness between the tracked target and its neighboring tracked targets in the dimension of motion velocity. In this formula, a constant of 1.0 is added to the denominator as a smoothing term, which physically smooths out small perturbations in low-speed states, avoiding misjudging normal low-speed gait differences as anomalous.

[0048] Finally, the relative motion anomaly degree of each tracked target is determined by comprehensively considering the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity. Specifically, the degree of anomaly between any tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity can be linearly weighted and fused according to preset empirical weights to output the final relative motion anomaly degree of the tracked target. For example, the following formula can be used to calculate: It should be noted that if any tracked target has no neighboring tracked targets, its relative motion anomaly degree will be output as 0 by default.

[0049] To address the complex nonlinear motion and frequent occlusion of targets, this invention constructs an Interacting Multiple Model (IMM) filter that incorporates three Kalman filters in parallel. This IMM filter simultaneously retains multiple possible motion modes and dynamically adjusts the confidence weights of each model based on the current scene context and observation results, thereby improving its adaptability to complex motion states such as turning, sudden stops, and recovery after occlusion. Specifically, the IMM filter includes an inertial expert for uniform motion, a kinematic expert for axial motion, and a decay expert for deceleration motion.

[0050] In some embodiments, the three experts share the same state space and observation space, but differ in how they construct the state transition matrix F and the process noise covariance matrix Q.

[0051] In this model, the inertial expert employs the standard constant velocity (CV) model, and the state transition matrix for each tracking trajectory is defined as follows:

[0052] in, This represents the discrete time interval between two adjacent frames. for The identity matrix; in frame-by-frame video processing, it can be made This matrix indicates that the position and scale at the next moment are determined by the current position, scale, and rate of change, while the velocity-related components remain constant throughout the one-step prediction.

[0053] Inertial expert's process noise covariance matrix As a baseline process uncertainty introduced by inertial experts in the one-step state prediction process, it is used to characterize the position, scale, and velocity disturbances that may still exist in the target under the constant velocity assumption. The construction method adopts the form of "the squared standard deviations of each state component forming a diagonal covariance matrix", and adaptively scales the main position and velocity components according to the current target detection box height. Specifically, corresponding to each tracking trajectory Defined as:

[0054] in, These represent the standard deviations of process noise corresponding to the x-coordinate and y-coordinate of the center position and the height of the detection frame, respectively. These represent the standard deviations of the process noise for the corresponding velocity components. These represent the standard deviations of process noise corresponding to the aspect ratio and its rate of change, respectively.

[0055] For example, h represents the height component of the initial detection box used to initialize the new trajectory. The physical meaning of this setting is that the larger the target scale, the greater the allowable range of position and velocity fluctuations, thus enabling the filter to have better adaptability to targets of different scales; while the aspect ratio and its rate of change usually change slowly, so they are given a small constant noise.

[0056] To enhance the ability to describe maneuvers such as sharp turns, crossings, reversals, and rapid changes of direction, this embodiment introduces the current heading angular velocity of the tracking trajectory. And define the single-step rotation angle as

[0057] Based on this, this embodiment further constructs a two-dimensional rotation matrix.

[0058] in, and These represent the projection coefficients of the rotation transformation in the horizontal and vertical directions, respectively. Using the aforementioned rotation matrix, this embodiment projects the velocity subspace... The propagation method was changed from "propagating while maintaining the original direction" to "propagating after rotating at angular velocity," thereby constructing the state transition matrix for each tracking trajectory corresponding to the kinematic expert:

[0059] In the matrix above, the first and second rows are used to predict the center point position of the detection box of the tracking target corresponding to the tracking trajectory at the next moment. Essentially, this involves first analyzing the current velocity vector... The rotation is performed, and then projected onto the position increment according to the time step; rows 5 and 6 are used to predict the velocity component at the next moment, which is essentially: performing active rotation on the two-dimensional velocity vector; rows 3, 4, 7, and 8 maintain the conventional propagation form of aspect ratio, height, and their rate of change. It can be seen that the state transition matrix of the maneuvering expert for different tracking trajectories will change with the change of the current heading angular velocity of each tracking trajectory.

[0060] In other embodiments, in order to calculate the current heading angular velocity of an arbitrary tracking trajectory For the current tracking trajectory, firstly, the posterior predicted state corresponding to this trajectory in the previous video frame is obtained, and then the lateral and longitudinal velocities of the tracked target corresponding to this trajectory are extracted from it. The lateral velocity of this tracking trajectory is denoted as... The longitudinal velocity is denoted as The current absolute heading angle of the tracking trajectory is calculated based on the lateral and longitudinal velocities. Specifically, the absolute heading angle at the current moment is calculated using the arctangent function: .in, It can output the absolute heading angle with quadrant information based on the direction of the velocity vector in the two-dimensional plane, thus avoiding the directional ambiguity of the ordinary arctangent function in different quadrants.

[0061] Considering that directly using the difference between the heading angles of two adjacent frames to estimate the angular velocity is highly susceptible to single-frame detection noise, minor state fluctuations, and directional instability at low speeds, this embodiment introduces a multi-frame sliding window mechanism to smoothly statistically analyze the heading changes of the tracking target corresponding to the tracking trajectory over several consecutive frames. Specifically, a heading angle history buffer queue is maintained for each tracking trajectory. This buffer queue is used to store the absolute heading angles of the most recent frames of the tracking trajectory in chronological order. To limit the length of the history and prevent excessively long time spans from weakening the response sensitivity to short-term turning behavior, the maximum capacity of this buffer queue is set to 5. When a new absolute heading angle is pushed in, if the queue length exceeds 5, the oldest pushed-in historical absolute heading angle is deleted, and only the 5 most recent absolute heading angle samples are retained.

[0062] In terms of specific execution conditions, to avoid calculating meaningless angular velocities for trajectories that have not yet been stabilized or are approximately stationary, the validity of the trajectory is first determined. Only when the number of consecutive matching frames of the tracked trajectory reaches a preset requirement and the velocity magnitudes corresponding to the current lateral and longitudinal velocities are... Only when the absolute heading angle of the current tracking trajectory exceeds a preset static threshold will it be added to the heading angle history buffer queue corresponding to that tracking trajectory and participate in subsequent angular velocity calculations. For example, the preset requirement for the number of consecutive matching frames of the trajectory can be set to 3, and the static threshold for the current velocity modulus can be set to 1.0, that is, only when the tracking trajectory meets the requirement of having at least 3 consecutive matching frames and Only when the target's current direction of motion is considered to have a stable physical meaning is it considered to have a stable physical meaning. The reason for the above design is that if the target's current velocity is too small, its direction of motion is extremely sensitive to noise. In this case, the heading angle obtained by back-deriving from the velocity component does not have a stable kinematic interpretation, and therefore should not be used for angular velocity estimation.

[0063] Once the heading angle history buffer queue has collected enough historical data, the smoothed heading angle velocity is estimated. For example, the length of the heading angle history buffer queue can be required to be at least 4. If the length of the heading angle history buffer queue corresponding to the tracking trajectory meets this preset requirement, the absolute heading angle with the earliest addition time in the corresponding heading angle history buffer queue is extracted as the starting heading angle. Obtain the current absolute heading angle of the tracking trajectory. And calculate the original algebraic difference between the two: Meanwhile, the frame interval number corresponding to the heading angle history buffer queue is defined as: , This is the length of the history buffer queue for this heading angle.

[0064] Because the arctangent function outputs an angle with a periodic truncation property, it is usually defined as follows: Or within the equivalent interval, therefore directly from The obtained primitive algebraic difference Angle jumps may occur. For example, if the target has only undergone a slight turn, but the initial heading angle is close to... The current absolute heading angle is close to Subtracting the two directly will give a result close to... The spurious large angle difference is thus identified. Therefore, a shortest rotation path correction is applied to the original algebraic difference to obtain the corrected heading angle. :when When, it was corrected to: .when When, it was corrected to: In all other cases, the corrected heading angle is the original algebraic difference value mentioned above. This process ensures that the final angle difference always corresponds to the shortest rotation path in physical space, thus eliminating false large turns caused by angle period truncation.

[0065] After completing the angle jump correction, the heading angle will be corrected. Divide by the number of frame intervals corresponding to the heading angle history buffer queue to obtain the current heading angular velocity of the tracked trajectory. This quantity represents the average rate of change of heading of the tracked target within a unit frame interval in the most recent consecutive frames. Compared to directly estimating angular velocity using the heading angle difference between two adjacent frames, the above method can effectively suppress the impact of single-frame noise, detection perturbations, and anomaly estimations on the angular velocity results while retaining short-to-medium-term steering information, thereby improving the stability and robustness of heading angular velocity estimation.

[0066] For cases where the calculation conditions are not met, a protective rollback mechanism can be adopted. If the number of consecutive matching frames of the tracking trajectory does not reach the preset requirement, or the current velocity magnitude does not exceed the static threshold, then no effective angular velocity estimate will be output. In this case, the heading angle history buffer queue corresponding to the tracking trajectory is cleared, and the current heading angular velocity is directly set to 0. The reason for clearing the heading angle history buffer queue is to avoid low-confidence historical heading angles generated during the low-speed stationary phase and the initial trajectory establishment phase from continuing to participate in subsequent window statistics, thereby preventing old direction information from polluting the re-established angular velocity estimate, and ensuring that subsequent angular velocities are calculated only from the heading angle samples of the current continuous effective motion phase. On the other hand, if the number of consecutive matching frames of the tracking trajectory has met the preset requirement and the current velocity magnitude is greater than the static threshold, but the length of the heading angle history buffer queue has not reached the preset requirement, then the heading angle history buffer queue is not cleared, but the current absolute heading angle is retained and accumulated, while the current heading angular velocity is temporarily set to 0; after the length of the heading angle history buffer queue reaches the preset requirement, the current heading angular velocity is calculated in the above manner.

[0067] Process noise covariance matrix of the machine expert In this embodiment, it is further defined as .in, This represents the noise amplification factor for the maneuvering expert, used to improve the model's tolerance to prediction uncertainties during turns, changes of direction, and sudden maneuvers. For example, it can be taken as... =1.2. The maneuvering expert corresponds to movement modes with significant changes in heading, such as turning, lateral cutting, and turning back.

[0068] To suppress prediction drift caused by continuous extrapolation at the original velocity during sudden stops, decelerations, or prolonged obstructions of the target, this embodiment introduces an attenuation coefficient in the velocity subspace. ,in

[0069] Based on this, the state transition matrix of the attenuation expert is constructed:

[0070] in, Indicates the degree to which speed is maintained; The smaller the value, the more inclined the prediction is to actively mitigate the lingering effects of historical velocities. For example, a lower value could be considered. =0.9. This matrix indicates that position and scale are still propagated by the current state and its rate of change, but the velocity component at the next moment is multiplied by an attenuation coefficient during propagation. This gradually compresses the velocity amplitude, preventing large prediction offsets due to velocity accumulation in scenarios such as sudden stops or occlusion. The attenuation expert's process noise covariance matrix... In this embodiment, it is defined as

[0071] in, This represents the noise adjustment factor for the attenuation expert; when no additional expansion of the prediction envelope is required, it can be set as follows: =1.0. If it is necessary to appropriately relax the prediction uncertainty in severely occluded scenarios, then it can be set to 1.0. Attenuation experts have identified motion scenarios where extrapolation of the state is prone to divergence after emergency stops, deceleration, strong localized obstruction, or short-term mismatches.

[0072] Subsequently, a basic transition probability matrix is ​​established for the interactive multi-model filter. This basic transition probability matrix is ​​used to characterize the inertial tendency of the target to maintain its existing motion pattern in the absence of significant external stimuli. Its elements in the i-th row and j-th column are... This represents the base probability that the target was in expert mode i at the previous moment and is transitioning to expert mode j at the current moment. Specifically, the first row of the base transition probability matrix represents the base probability that the target was in the motion mode corresponding to inertial expert A at the previous moment and is transitioning to the motion mode corresponding to inertial expert A, maneuvering expert B, or attenuation expert C at the current moment; the second row represents the base probability that the target was in the motion mode corresponding to maneuvering expert B at the previous moment and is transitioning to the motion mode corresponding to inertial expert A, maneuvering expert B, or attenuation expert C at the current moment; and the third row represents the base probability that the target was in the motion mode corresponding to attenuation expert C at the previous moment and is transitioning to the motion mode corresponding to inertial expert A, maneuvering expert B, or attenuation expert C at the current moment. Larger diagonal elements in the matrix indicate that the target is more likely to maintain its current motion mode rather than frequently switching motion modes between adjacent frames. In some embodiments, the base transition probability matrix can be defined as...

[0073] Subsequently, the local congestion degree calculated independently for each tracking target corresponding to the above steps is obtained. With relative motion anomaly The turning gain is calculated based on the following formula. : Simultaneously, the attenuation gain is calculated based on the following formula. : For each tracked target, the column vector elements pointing to the maneuvering expert in the basic transfer probability matrix are multiplied by the turning gain of the corresponding tracked target, and the column vector elements pointing to the attenuation expert are multiplied by the attenuation gain of the corresponding tracked target, resulting in the gain transfer probability matrix of the tracking trajectory corresponding to the target. Then, each row of the gain transfer probability matrix of each tracking trajectory is normalized to ensure that the sum of probabilities in each row is strictly 1, resulting in the dynamic transfer probability matrix that allows each tracking trajectory to respond in real-time to changes in the physical environment. It can be seen that the elements of the dynamic transfer probability matrix represent the probability of the corresponding tracking trajectory transferring between the motion modes corresponding to any two of the inertial expert, maneuvering expert, and attenuation expert.

[0074] Based on inertial experts, maneuvering experts, and attenuation experts, and combined with the dynamic transition probability matrix of each tracking trajectory, the prior prediction state of each tracking trajectory in the current video frame is predicted.

[0075] In some embodiments, for any tracking trajectory, in order to ensure that experts fully share historical information before prediction, a state cross-mixing based on Markov chains is first performed. The specific calculation steps are as follows: First, extract the posterior probability of the tracking target in the motion mode corresponding to the tracking trajectory in the previous video frame, which is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert. , here The indexes represent the inertial expert, the maneuvering expert, and the attenuation expert. Simultaneously, the dynamic transition probability matrix of the tracking trajectory is obtained. Based on these two sets of data, the prior probability that the tracking target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert is calculated using the following formula: Where i and j are the indices of each expert. This is the dynamic transition probability matrix. The element in the i-th row and j-th column.

[0076] Subsequently, the posterior state vectors and posterior covariances of the tracking trajectory previously output by the inertial expert, the maneuvering expert, and the attenuation expert in the previous video frame are cross-mixed using the following formula to obtain the mixed initial state vectors and mixed initial covariances of each expert for the tracking trajectory: .

[0077] in, For expert j, the mixed initial state vector for this tracking trajectory. For expert j, the mixed initial covariance of the tracking trajectory, and These are the posterior state vector and posterior covariance of the tracking trajectory output by expert i in the previous video frame, respectively. This mechanism ensures that a single expert can absorb the global historical state distribution.

[0078] Next, based on the state transition matrix and process noise covariance matrix of the tracking trajectory corresponding to the inertial expert, the maneuvering expert, and the attenuation expert, a time step is extrapolated forward for the mixed initial state vector and mixed initial covariance of the corresponding expert for the tracking trajectory, obtaining the prior state vector and prior covariance of the corresponding expert in the current video frame for the tracking trajectory. Wherein, for expert j, the prior state vector of the tracking trajectory in the current video frame... This represents the position and velocity derived from specific kinematic equations, where Let j be the state transition matrix of the tracked trajectory; and let j be the prior covariance of the tracked trajectory in the current video frame. This represents an amplification of uncertainty in the deduction process, among which Let j be the process noise covariance matrix for the tracking trajectory.

[0079] Finally, in order to provide a unified spatial benchmark for subsequent data association and matching, the prior probability of the tracking target corresponding to the tracking trajectory in the current video frame being in the motion mode corresponding to each expert, calculated above, is used. As fusion weights, the prior state vectors and prior covariances output by the inertial expert, the maneuver expert, and the attenuation expert for the tracking trajectory in the current video frame are weighted and fused separately. The resulting prior predicted state of the tracking trajectory in the current video frame is then calculated. and prior prediction covariance They are respectively: With fusion covariance .

[0080] After obtaining the prior prediction state corresponding to each tracking trajectory in the current video frame, for the N detection boxes output by the target detector in the current video frame (i.e., the physical entities actually observed in the scene) and the M tracking trajectories maintained internally by the system, the matching cost between the two needs to be calculated and accurately paired. The specific calculation and association process is as follows: First, calculate the appearance feature cost. For any i-th detection box and any j-th tracking trajectory, calculate the appearance feature cost between the appearance features of the detection box and the appearance features of the tracking trajectory. The higher the similarity between the appearance features of the detection box and the appearance features of the tracking trajectory, the lower the corresponding appearance feature cost.

[0081] In some embodiments, for the i-th detection box and the j-th tracking trajectory, the cosine similarity between the appearance features of the detection box and the appearance features of the tracking trajectory stored in the first feature library is calculated first. This cosine similarity is then subtracted from the cosine similarity by a constant 1 to convert it into a cost metric that is as small as possible, thereby obtaining the first appearance cost between the appearance features of the detection box and the appearance features of the tracking trajectory. The appearance features of each tracking trajectory currently stored in the first feature library are obtained by smoothing the appearance features of the detection boxes matched with the corresponding tracking trajectory in the previous video frame.

[0082] If the first feature library does not contain the appearance features of the tracking trajectory or the global physical occlusion rate is less than a preset threshold (e.g., 0.5), it is considered that the regular features are severely polluted by noise and the representation ability is affected, and the first appearance cost is determined to be 1.

[0083] Furthermore, one or more clusters are formed by extracting the appearance features of the tracking trajectory stored in the second feature library. The cosine similarity between the appearance features of the detection box and the centroid of each cluster is calculated. The difference between a constant 1 and the largest cosine similarity is calculated, and this difference is multiplied by a preset attenuation discount factor (e.g., 0.9) to penalize the timeliness decay of long-term historical features, thus obtaining the second appearance cost between the appearance features of the detection box and the appearance features of the tracking trajectory. The appearance features of each tracking trajectory stored in the second feature library are obtained by filtering the appearance features of the detection boxes matched for each tracking trajectory.

[0084] In some embodiments, when updating the first feature library, the appearance features of a detection box that successfully matches any tracking trajectory in the current video frame are set as follows: The stored appearance features from historical video frames are The updated appearance features of the tracking trajectory for: Among them, it can be based on the global physical occlusion rate. Dynamically adjust feature weights: The more severe the occlusion, the higher the feature weight, thus enhancing the stability of the appearance features. To ensure numerical stability, [the following is omitted as the text is incomplete and requires further context]. Perform L2 normalization In the formula It is a very small positive number.

[0085] The second feature library is updated as follows: The cosine similarity between the appearance features of all detection boxes in the current video frame and the centroids of all clusters in the second feature library is calculated, and the maximum cosine similarity corresponding to all detection boxes is determined. For each detection box in the current video frame, if the maximum cosine similarity corresponding to the detection box is greater than a preset threshold, the cluster corresponding to the maximum cosine similarity is updated based on the appearance features of the detection box (the appearance features of the detection box are added to the corresponding cluster). Otherwise, the cosine similarity between the appearance features of the detection box and the feature entries stored in the preset buffer is calculated. When the cosine similarity between the appearance features of the detection box and any feature entry is higher than the preset threshold, the counter value of the feature entry is incremented by 1. When the counter value of the feature entry reaches the preset number of entries (e.g., 3), the feature entry is moved from the preset buffer to the second feature library, forming a new cluster corresponding to the tracking trajectory. When the cosine similarity between the appearance features of the detection box and all feature entries is lower than the preset threshold, the appearance features of the detection box are added to the preset buffer as a new feature entry. Furthermore, when the number of feature entries in the preset buffer exceeds the capacity limit, feature entries are discarded according to the first-in, first-out principle. When updating the corresponding cluster in the second feature library, only when the quality score of the newly added appearance feature (e.g.) is updated... The centroid of a cluster is updated only if its mass score is not lower than the current centroid of that cluster. When the number of clusters in the second feature library reaches its upper limit, the value score of each cluster can be calculated. ,in , For time decay, where The difference between the last updated frame number and the current frame number of the cluster is used to calculate the diversity score D, which is 1 minus the average similarity between the cluster and other clusters. This eliminates the cluster with the lowest value score.

[0086] The smaller of the first appearance cost and the second appearance cost is determined as the appearance feature cost between the appearance features of the detection box and the appearance features of the tracking trajectory, denoted as . .

[0087] Secondly, the motion space cost between the spatial location of each detection box and the spatial location contained in the prior prediction state corresponding to each tracking trajectory in the current video frame is calculated. For any i-th detection box and any j-th tracking trajectory, the dimension of the spatial location of the detection box is transformed to be consistent with the spatial location contained in the prior prediction state of the tracking trajectory, according to the observation space transformation method. It should be noted that the detection boxes directly output by the target detector are usually represented in the form of the upper left corner coordinates and width and height, but the spatial location in the prior prediction state is represented in the form of the horizontal and vertical coordinates of the center point, the aspect ratio of the detection box, and the height of the detection box. Therefore, it is necessary to transform the spatial location of the detection box so that the detection box and the prior prediction state can be compared in the same space. Subsequently, the observation residual is constructed for the spatial location of the detection box and the spatial location contained in the prior prediction state corresponding to the tracking trajectory in the current video frame:

[0088] in, This represents the spatial position of the i-th detection box after transformation. Let be the spatial location contained in the prior predicted state corresponding to the j-th tracking trajectory in the current video frame, which can be calculated in the following way: , The preset observation matrix is ​​used to reduce the dimensionality of the prior predicted state and obtain only its spatial location.

[0089] Calculate the corresponding squared Mahalanobis distance:

[0090] in, R represents the measurement noise covariance matrix. This represents the motion space cost between the i-th detection box and the j-th tracking trajectory. A smaller cost indicates a more consistent fusion prior prediction between the candidate detection box and the trajectory; a larger cost indicates a greater deviation between the prior prediction states of the detection box and the tracking trajectory. This is because the Mahalanobis distance incorporates the prediction-observation covariance matrix during calculation. Therefore, it considers not only the deviation between the detection box and the prior predicted state of the tracking trajectory, but also the uncertainty of the prediction distribution. In some embodiments, to improve the stability of numerical computation, the matrix may not be directly and explicitly obtained. Instead, first work on the matrix We perform Cholesky decomposition and then solve for the equivalent distance value using trigonometric equations to avoid numerical instability caused by directly inverting the covariance matrix when it is close to singular.

[0091] Subsequently, based on the adaptive appearance weights corresponding to the current video frame, the appearance feature cost between the appearance features of each detection box and the appearance features of each tracking trajectory, and the motion space cost between the spatial position of each detection box and the spatial position contained in the prior predicted state of each tracking trajectory in the current video frame, the total matching cost between each detection box and each tracking trajectory is calculated. For the i-th detection box and the j-th tracking trajectory, the total matching cost is defined as:

[0092] By iterating through all pairs of N detection boxes and M tracking trajectories, and calculating the total matching cost for each pair, a dataset of size [size missing] is constructed. The two-dimensional global cost matrix, where each element represents the cost of associating the i-th detection box with the j-th tracking trajectory.

[0093] Finally, a matching algorithm (such as bipartite graph matching) is used to match the two-dimensional global cost matrix to obtain the matching result for each detection box. The constructed two-dimensional global cost matrix can be input into the Hungarian algorithm. Within this algorithm framework, the M tracking trajectories and N detection boxes are treated as independent vertices on opposite sides of a bipartite graph. The elements of the two-dimensional global cost matrix serve as the edge weights connecting the two vertices. By solving for the optimal connection combination that minimizes the sum of the global edge weight costs, the accurate association between the tracking trajectories and detection boxes is achieved. A cascaded matching strategy can be used in the matching phase, processing the tracking trajectories in ascending order of the number of frames (time_since_update) since the last successful update. Detection boxes are preferentially assigned to the most recently successfully matched tracking trajectories with more reliable state estimations, followed by those with longer mismatch times and greater uncertainty. This reduces the interference of older trajectories on the current high-confidence association, minimizing false matches and identity switching. After matching is completed, the results are divided into three logical branches: For successfully matched "tracking trajectory-detection box" pairs, the detection box is extracted as the actual observation value to update the state of the interactive multi-model filter, obtaining the posterior prediction state of each successfully matched tracking trajectory in the current video frame; For unmatched tracking trajectories, the system calls the unmatched handling mechanism. If the tracking trajectory is in a trial period, it is marked for deletion. If the trajectory is in a confirmed state and the number of consecutive unmatched frames does not exceed a preset threshold, its current prior prediction state and prior prediction covariance are retained so that motion prediction and association matching can continue in subsequent frames; For unmatched detection boxes, they are regarded as new targets appearing in the picture, and a brand new tracking trajectory is created for them.

[0094] The interactive multi-model filter is updated for each successfully matched detection box and tracking trajectory to obtain the posterior prediction state of each successfully matched tracking trajectory in the current video frame. The specific process includes: Using a preset observation matrix, the prior state vectors and prior covariances output by each expert for the tracking trajectory in the current video frame are converted into predicted state observations and an innovation covariance matrix. The spatial positions of the matched detection boxes are then converted into the actual state observations of the tracking trajectory. Since the prior state vectors encompass multiple dimensions such as position and velocity, while the spatial positions of the detection boxes only include the position dimension, the preset observation matrix H primarily serves as a dimensionality reduction mapping. By eliminating unobservable velocity components, the predicted state observations of the tracking trajectory for each expert in the current video frame are obtained. The residual is calculated by subtracting the actual state observation value z from the predicted state observation value. This parameter reflects the physical deviation between the predicted positions of each expert and the actual detection positions of the detector. Simultaneously, the information covariance matrix for each expert is calculated. In the formula, R is the pre-defined detector measurement noise covariance matrix. This formula mathematically superimposes the model prediction uncertainty after the observation matrix transformation with the detector's own measurement error to represent the total acceptable range of residuals for the current video frame system.

[0095] To evaluate the prediction accuracy of different experts, it is necessary to calculate the Mahalanobis distance for each expert for the tracking trajectory: ;in, For expert j, the Mahalanobis distance for this tracking trajectory is calculated. Considering that in practical calculations, directly applying the high-dimensional information covariance matrix... Inverting the matrix can easily introduce floating-point truncation errors and can easily cause program crashes when the matrix is ​​close to singular. Therefore, a novel covariance matrix is ​​used... The mathematical property of symmetric positive definiteness is addressed by using the Cholsky decomposition method. This method transforms the unstable matrix inversion into a forward and backward substitution problem by decomposing the matrix into a product of triangular matrices.

[0096] Substituting the Mahalanobis distance and information covariance matrix of each expert for the tracking trajectory into the multivariate Gaussian probability density function yields the observation likelihood of the respective expert. :

[0097] Where d represents the dimension of the observation space, i.e., the number of components contained in the actual state observation value z; in this embodiment, the actual state observation value z consists of the x-coordinate of the center of the target detection box, the y-coordinate of the center, the aspect ratio, and the height, so d = 4. Further, for ease of numerical implementation, the above formula is rewritten in exponential form, yielding:

[0098] in, Used to measure the magnitude of the deviation between the j-th expert's predicted observation and the actual observation. Used to reflect the range of uncertainty in the expert prediction distribution. This originates from the dimension term in the normalization constant of the d-dimensional Gaussian probability density function. Therefore, the observation likelihood... It can be quantified to represent the probability of the current actual state observation z occurring under the prediction assumption of the j-th expert.

[0099] The observation likelihood of each expert The prior probability that the tracked target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to the relevant expert. After multiplication, normalization is performed to obtain the posterior probability that the tracked target corresponding to the tracking trajectory in the current video frame belongs to the motion mode corresponding to each expert. This allows for dynamic feedback, assigning the highest confidence weight to the expert with the highest prediction accuracy. If the denominator approaches zero, it indicates that the current observations lack effective discriminatory power among the experts. To avoid division-by-zero anomalies and numerical instability, the prior probability is directly retained. As a probabilistic approximation of the model in the degenerate case, let: .

[0100] Calculate the Kalman gain for each expert: ;in, For expert j, the Kalman gain, For expert j, the prior covariance of the tracking trajectory output in the current video frame. This is a preset observation matrix. Among them, Used to balance the degree of trust that expert j has in the prior state vector and the current actual state observation z: when the new information covariance matrix When the prior covariance is large, indicating strong observational uncertainty, the gain decreases relatively, and the system tends to retain the prior state vector. Conversely, when the prior covariance is large and the observations are reliable, the gain increases relatively, and the system tends to use actual state observations to correct the prior state vector. Subsequently, experts combined the actual residuals... By using the corresponding Kalman gain to proportionally correct its prior state vector, the posterior state vector and posterior covariance of the tracking trajectory output by each expert in the current video frame are obtained: ;in, This is the prior state vector output by expert j in the current video frame for the tracking trajectory.

[0101] The posterior state vectors of the tracking trajectory output by each expert in the current video frame are fused into the corresponding posterior predicted state of the tracking trajectory in the current video frame based on the following formula. : The following formula is used to fuse the posterior covariance of the tracking trajectory output by each expert in the current video frame into the posterior prediction covariance of the tracking trajectory in the current video frame. : .in, This represents the posterior probability that the tracked target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to expert j. It reflects the expert's ability to interpret the true motion state of the target after combining the current observations. In the above formula, the first term This indicates the remaining uncertainty after expert j updates itself. This represents the degree of dispersion of the updated results from different experts relative to the final fused mean. By simultaneously considering both intra-expert uncertainty and inter-expert differences, a more complete final posterior prediction covariance can be obtained.

[0102] In summary, the method provided by this invention calculates normalized crowd density, global physical occlusion rate, and global appearance similarity based on the detection box information and appearance features of each detection box in the current video frame. It then determines the current scene mode based on these parameters and sets an adaptive appearance weight for the current video frame accordingly. Different adaptive appearance weights are set for different scene modes. Furthermore, after predicting the prior prediction state of each tracking trajectory that existed in the previous video frame in the current video frame, the method calculates the appearance features of each detection box and the appearance similarity of each tracking trajectory. The system calculates the appearance feature cost between the observation features and the motion space cost between the spatial location of each detection box and the spatial location of the prior predicted state corresponding to each tracking trajectory in the current video frame. The system then uses the adaptive appearance weights set above to perform weighted fusion of the appearance feature cost and motion space cost between each detection box and each tracking trajectory to obtain the total matching cost between each detection box and each tracking trajectory. This enables the matching of tracking trajectories and detection boxes. The system also updates the state of successfully matched detection boxes and tracking trajectories, achieving fully automatic scene generalization. It can automatically identify and adapt to various cross-domain scenes, from sparse to highly similar and highly crowded.

[0103] Furthermore, by constructing an interactive multi-model filter that includes inertial experts, maneuvering experts, and attenuation experts corresponding to different motion modes, and using this interactive multi-model filter for prediction and state updates, an instance-level dynamic routing mechanism can be realized. This effectively solves the trajectory divergence problem in scenarios such as sharp turns and rotations, and can accurately track nonlinear moving targets, effectively reducing prediction errors at low frame rates (such as 2FPS).

[0104] Furthermore, by constructing a first feature library and a second feature library, feature matching is performed using the first and second feature libraries in this hierarchical manner when calculating the appearance feature cost. This ensures the purity and real-time performance of feature matching, effectively isolates dirty features caused by occlusion, and solves the feature lag problem under rapid pose changes.

[0105] This invention provides an electronic device, including: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any of the above embodiments.

[0106] This invention provides a computer-readable storage medium storing computer instructions that cause a processor to perform the method described in any of the above embodiments.

[0107] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0108] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An adaptive target tracking method based on multi-granularity context awareness, characterized in that, include: The system acquires the detection box information and appearance features of all detection boxes in the current video frame to calculate normalized crowd density, global physical occlusion rate, and global appearance similarity. Based on these parameters, it determines the current scene mode and sets an adaptive appearance weight for the current video frame. Specifically, the current scene mode is designated as the first mode when the global appearance similarity is higher than a first preset value; the current scene mode is designated as the second mode when the normalized crowd density is higher than a second preset value or the global physical occlusion rate is higher than a third preset value; otherwise, the current scene mode is designated as the third mode. The adaptive appearance weights set for different scene modes are different. For each tracking trajectory that already exists in the previous video frame, predict the prior prediction state corresponding to each tracking trajectory in the current video frame; For each tracking trajectory and each detection box in the current video frame, calculate the appearance feature cost between the appearance features of each detection box and the appearance features of each tracking trajectory, and calculate the motion space cost between the spatial position of each detection box and the spatial position contained in the prior prediction state corresponding to each tracking trajectory in the current video frame; wherein, the higher the similarity between the appearance features of any detection box and the appearance features of any tracking trajectory, the lower the corresponding appearance feature cost; the greater the distance between the spatial position of any detection box and the spatial position contained in the prior prediction state corresponding to any tracking trajectory in the current video frame, the greater the corresponding motion space cost; Based on the adaptive appearance weight corresponding to the current video frame, the appearance feature cost between each detection box and each tracking trajectory, and the motion space cost, calculate the total matching cost between each detection box and each tracking trajectory; Matching is performed based on the total matching cost between each detection box and each tracking trajectory to obtain the matching result of each detection box. The state is updated for the successfully matched detection boxes and tracking trajectories to obtain the posterior prediction state of each successfully matched tracking trajectory in the current video frame.

2. The adaptive target tracking method based on multi-granularity context awareness as described in claim 1, characterized in that, The calculation of normalized crowd density, global physical occlusion rate, and global appearance similarity based on the detection box information and appearance features of each detection box in the current video frame specifically includes: The normalized crowd density is determined based on the ratio of the total number of valid detected targets in the current video frame to the total number of pixels in the image. The cross-union ratio (CUP) between different detection boxes is calculated based on the spatial location contained in the detection box information of each detection box, and the global physical occlusion rate is calculated based on the CUP between different detection boxes. An original feature matrix is ​​constructed based on the appearance features of all detection boxes containing valid detection targets, and then normalized to obtain a normalized feature matrix. The dot product between the normalized feature matrix and its transpose is calculated to obtain a feature autocorrelation matrix. The global appearance similarity is calculated based on the upper triangular elements of the feature autocorrelation matrix.

3. The adaptive target tracking method based on multi-granularity context awareness as described in claim 2, characterized in that, The process of determining the current scene mode based on normalized crowd density, global physical occlusion rate, and global appearance similarity, and setting the adaptive appearance weight corresponding to the current video frame based on the current scene mode, specifically includes: If the global appearance similarity If the value is greater than the first preset value, and the current scene mode is the first mode, then the target appearance weight is calculated based on the following formula. : ; If the normalized population density Greater than the second preset value or the global physical occlusion rate If the value is greater than the third preset value, and the current scene mode is the second mode, then the target appearance weight will be adjusted. Set to the preset maximum value; Otherwise, the current scene mode is the third mode, and the occlusion penalty factor is calculated based on the following formula. : And based on the global appearance similarity Calculate the target appearance weight; wherein, if the global appearance similarity... If the value is greater than the fourth preset value, the target appearance weight is calculated according to the following formula. : , Otherwise, calculate the target appearance weight according to the following formula. : ; This is the default value; If the absolute difference between the target appearance weight and the adaptive appearance weight corresponding to the previous video frame is less than or equal to a preset dead zone threshold, then the adaptive appearance weight corresponding to the previous video frame is determined as the adaptive appearance weight corresponding to the current video frame; otherwise, the target appearance weight is smoothed based on the adaptive appearance weight corresponding to the previous video frame to obtain the final adaptive appearance weight corresponding to the current video frame.

4. The adaptive target tracking method based on multi-granularity context awareness as described in claim 1, characterized in that, The prior prediction state corresponding to each tracking trajectory in the current video frame is obtained through an interactive multi-model filter; the interactive multi-model filter includes an inertial expert, a maneuvering expert, and an attenuation expert corresponding to different motion modes; the motion mode corresponding to the inertial expert is uniform motion, the motion mode corresponding to the maneuvering expert is directional motion, and the motion mode corresponding to the attenuation expert is deceleration motion. The step of predicting the prior prediction state of each tracking trajectory in the current video frame for each existing tracking trajectory in the previous video frame specifically includes: Extract the detection box and predicted velocity vector of the tracking target corresponding to each tracking trajectory from the posterior prediction state corresponding to each tracking trajectory in the previous video frame, and search for neighboring tracking targets of each tracking target within a preset search radius; For each tracked target, the intersection-union ratio (IUU) of the detection bounding box of each tracked target with the detection bounding boxes of its neighboring tracked targets is calculated. Based on the IUU of the detection bounding boxes of each tracked target with the detection bounding boxes of its neighboring tracked targets, the local congestion degree of the corresponding tracked target is determined. ; Based on the predicted velocity vector of each tracked target and the average predicted velocity vector of each target's neighboring tracked targets, the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity is determined. Then, based on the degree of anomaly between each tracked target and its neighboring tracked targets in the dimensions of motion direction and motion velocity, the relative motion anomaly degree of each tracked target is comprehensively determined. ; The turning gain for each tracked target is calculated based on the following formula. : And calculate the attenuation gain for each tracked target based on the following formula. : ; For each tracked target, the column vector elements pointing to the maneuvering expert in the basic transfer probability matrix are multiplied by the turning gain of the corresponding tracked target, and the column vector elements pointing to the attenuation expert are multiplied by the attenuation gain of the corresponding tracked target to obtain the gain transfer probability matrix of the tracking trajectory corresponding to the tracked target. Each row of the gain transfer probability matrix of each tracking trajectory is normalized to obtain the dynamic transfer probability matrix of each tracking trajectory. The elements of the dynamic transfer probability matrix represent the probability of transfer between the motion modes corresponding to any two of the inertial expert, maneuvering expert, and attenuation expert. Based on the inertial expert, the maneuvering expert, and the attenuation expert, and combined with the dynamic transition probability matrix of each tracking trajectory, the prior prediction state of each tracking trajectory in the current video frame is predicted.

5. The adaptive target tracking method based on multi-granularity context awareness as described in claim 4, characterized in that, The state transition matrix of the inertial expert for each tracking trajectory process noise covariance matrix They are represented as follows: in, It represents the discrete time interval between two adjacent video frames. for The identity matrix; These represent the standard deviations of process noise corresponding to the x-coordinate and y-coordinate of the center position and the height of the detection frame, respectively. These represent the standard deviations of the process noise for the corresponding velocity components. These represent the standard deviations of process noise corresponding to the aspect ratio and its rate of change, respectively. The state transition matrix of the mobile expert corresponding to each tracking trajectory process noise covariance matrix They are represented as follows: in, , The current heading angular velocity of the corresponding tracking trajectory, This is the default value; The attenuation expert corresponds to the state transition matrix of each tracking trajectory. process noise covariance matrix They are represented as follows: in, and This is the default value.

6. The adaptive target tracking method based on multi-granularity context awareness as described in claim 5, characterized in that, The current heading angular velocity of any tracking trajectory It was determined based on the following method: The predicted velocity vector of the tracked target corresponding to the tracked trajectory, which is extracted from the posterior prediction state corresponding to the tracked trajectory in the previous video frame, is decomposed into lateral velocity and longitudinal velocity, and the current absolute heading angle of the tracked trajectory is solved based on the lateral velocity and longitudinal velocity. If the number of consecutive matching frames of the tracking trajectory reaches the preset requirement and the velocity magnitudes corresponding to the lateral and longitudinal velocities are greater than the set static threshold, then the current absolute heading angle of the tracking trajectory is added to the heading angle history buffer queue corresponding to the tracking trajectory. If the length of the heading angle history buffer queue corresponding to the tracking trajectory reaches the preset requirement, then the absolute heading angle that was added earliest in the heading angle history buffer queue is extracted as the starting heading angle, and the original algebraic difference between the current absolute heading angle of the tracking trajectory and the starting heading angle is calculated. Angle jump correction is applied to the original algebraic difference: if the original algebraic difference is greater than... Then subtract the original algebraic difference value. The corrected heading angle is obtained; if the original algebraic difference is less than Then add the original algebraic difference value. The corrected heading angle is obtained; Divide the corrected heading angle by the number of frame intervals corresponding to the heading angle history buffer queue to obtain the current heading angular velocity of the tracking trajectory. The number of frame intervals corresponding to the heading angle history buffer queue is equal to the length of the heading angle history buffer queue minus 1.

7. The adaptive target tracking method based on multi-granularity context awareness as described in claim 5, characterized in that, Based on the inertial expert, the maneuvering expert, and the attenuation expert, and combined with the dynamic transition probability matrix of any tracking trajectory, the prior prediction state of any tracking trajectory in the current video frame is predicted, specifically including: Based on the dynamic transition probability matrix of any tracking trajectory and the posterior probability that the tracking target corresponding to any tracking trajectory in the previous video frame is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert, the prior probability that the tracking target corresponding to any tracking trajectory in the current video frame is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert is calculated using the following formula: Where i and j are the indices of each expert. Let be the posterior probability that the tracking target corresponding to any tracking trajectory in the previous video frame is in the motion mode corresponding to expert i. The element in the i-th row and j-th column of the dynamic transition probability matrix. The prior probability that the tracking target corresponding to any tracking trajectory in the current video frame is in the motion mode corresponding to expert j; The posterior state vector and posterior covariance of any tracking trajectory output by the inertial expert, the maneuvering expert, and the attenuation expert in the previous video frame are cross-mixed using the following formula to obtain the mixed initial state vector and mixed initial covariance of each expert for any tracking trajectory: ； in, For expert j, the hybrid initial state vector for any of the tracking trajectories. For expert j, the mixed initial covariance for any of the tracking trajectories, and These are the posterior state vector and posterior covariance of any tracking trajectory output by expert i in the previous video frame, respectively. Based on the state transition matrix and process noise covariance matrix of the inertial expert, the maneuvering expert and the attenuation expert for any tracking trajectory, the mixed initial state vector and mixed initial covariance of the corresponding experts for any tracking trajectory are extrapolated forward by one time step to obtain the prior state vector and prior covariance output by the corresponding experts for any tracking trajectory in the current video frame. Based on the prior probability that the tracking target corresponding to any tracking trajectory in the current video frame is in the motion mode corresponding to the inertial expert, the maneuvering expert, and the attenuation expert, the prior state vector and prior covariance output by the inertial expert, the maneuvering expert, and the attenuation expert for any tracking trajectory in the current video frame are weighted and fused respectively to obtain the prior prediction state and prior prediction covariance corresponding to any tracking trajectory in the current video frame.

8. The adaptive target tracking method based on multi-granularity context awareness as described in claim 7, characterized in that, For any successfully matched detection box and tracking trajectory, the state is updated to obtain the posterior prediction state of the tracking trajectory in the current video frame, specifically including: Using a preset observation matrix, the prior state vectors and prior covariances output by each expert for the tracking trajectory in the current video frame are converted into predicted state observations and innovation covariance matrices, and the spatial positions of the matched detection boxes are converted into the actual state observations of the tracking trajectory. Calculate the Mahalanobis distances for this tracking trajectory from various experts: ;in, For expert J, the Mahalanobis distance of this tracking trajectory, The residual between the predicted state observations of the tracked trajectory and the actual state observations of the tracked trajectory is given by expert j. For expert j, the new information covariance matrix of the track should be provided; Substituting the Mahalanobis distance and information covariance matrix of each expert for the tracking trajectory into the multivariate Gaussian probability density function, we obtain the observation likelihood of the corresponding expert. The observation likelihood of each expert is multiplied by the prior probability that the tracking target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to the corresponding expert, and then normalized to obtain the posterior probability that the tracking target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to each expert. Calculate the Kalman gain for each expert: ;in, For expert j, the Kalman gain, For expert j, the prior covariance of the tracking trajectory output in the current video frame. The preset observation matrix; Determine the posterior state vector and posterior covariance of the tracking trajectory output by each expert in the current video frame: ;in, For expert j, this is the prior state vector in the current video frame output for the tracking trajectory; The posterior state vectors of the tracking trajectory output by each expert in the current video frame are fused into the corresponding posterior predicted state of the tracking trajectory in the current video frame based on the following formula. : ; The posterior probability that the tracking target corresponding to the tracking trajectory in the current video frame is in the motion mode corresponding to expert j; The posterior covariance of the tracking trajectory output by each expert in the current video frame is fused into the posterior prediction covariance of the tracking trajectory in the current video frame based on the following formula. : .

9. The adaptive target tracking method based on multi-granularity context awareness as described in claim 1, characterized in that, Calculate the appearance feature cost between the appearance features of any detection box and the appearance features of any tracking trajectory, specifically including: If the first feature library contains the appearance features of the tracking trajectory and the global physical occlusion rate is less than a preset threshold, then the cosine similarity between the appearance features of the detection box and the appearance features of the tracking trajectory stored in the first feature library is calculated, and the difference between the constant 1 and the cosine similarity is calculated as the first appearance cost between the appearance features of the detection box and the appearance features of the tracking trajectory; otherwise, the first appearance cost is determined to be 1. Extract the clusters formed by the appearance features of the tracking trajectory stored in the second feature library, calculate the cosine similarity between the appearance features of the detection box and the centroid of the cluster, calculate the difference between the constant 1 and the largest cosine similarity, and multiply the difference by a preset attenuation discount coefficient to obtain the second appearance cost between the appearance features of the detection box and the appearance features of the tracking trajectory. The smaller value between the first appearance cost and the second appearance cost is determined to be the appearance feature cost between the appearance features of the detection box and the appearance features of the tracking trajectory. The appearance features of each tracking trajectory currently stored in the first feature library are obtained by smoothing the appearance features of the detection boxes matched by the corresponding tracking trajectory in the previous video frame; the appearance features of each tracking trajectory stored in the second feature library are obtained by filtering the appearance features of the detection boxes matched by the corresponding tracking trajectory each time.

10. The adaptive target tracking method based on multi-granularity context awareness as described in claim 9, characterized in that, The second feature library is updated in the following way: Calculate the cosine similarity between the appearance features of all detection boxes in the current video frame and the centroids of all clusters in the second feature library, and determine the maximum cosine similarity corresponding to all detection boxes; For each detection box in the current video frame, if the maximum cosine similarity corresponding to the detection box is greater than a preset threshold, the cluster corresponding to the maximum cosine similarity is updated based on the appearance features of the detection box; otherwise, the cosine similarity between the appearance features of the detection box and the feature entries stored in the preset buffer is calculated. When the cosine similarity between the appearance features of the detection box and any feature entry is higher than the preset threshold, the counter value of any feature entry is incremented by 1. When the counter value of any feature entry reaches a preset number of entries, the any feature entry is moved from the preset buffer to the second feature library to form a new cluster corresponding to the tracking trajectory. When the cosine similarity between the appearance features of the detection box and all feature entries is lower than the preset threshold, the appearance features of the detection box are added to the preset buffer as a new feature entry.

Images