A multi-object tracking method for noise control
By introducing a priori noise reduction module and adaptive Kalman filtering into the multi-objective tracking algorithm, combined with gradient acceleration trajectory reconnection, the problem of insufficient noise control in traditional methods is solved, and higher tracking accuracy and trajectory stability are achieved.
Patent Information
- Application Number
- CN202210960897.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-11
- Publication Date
- 2025-07-11
- Estimated Expiration
- 2042-08-11
AI Technical Summary
Traditional detector-based multi-objective tracking algorithms are likely to lead to tracking target loss and trajectory interruption under the influence of factors such as occlusion, lighting and deformation. In addition, traditional Kalman filtering cannot effectively combine tracking system information to obtain motion information, and the trajectory reconnection lacks the true value trend, resulting in low tracking accuracy.
A priori noise reduction module is introduced to remove the thermogram fusion redundant noise, combine Kalman filtering to adaptively observe noise to obtain smooth gain, design a gradient acceleration trajectory reconnection module, obtain target motion information through Gaussian function and detection result confidence, and perform trajectory reconnection.
It improves the tracking accuracy in small-target tracking and complex environments, alleviates the trajectory drift problem, and improves the accuracy of motion information acquisition and trajectory reconnection.
Smart Images
Figure CN115482250B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a multi - target tracking method for noise control, belonging to the technical field of image processing. Background Art
[0002] With the rapid development of artificial intelligence technology, intelligent monitoring systems and autonomous driving fields based on computer vision technology have made significant breakthroughs compared with the past, further reducing the waste of human resources and improving the security in the fields of security and transportation. Visual multi - target tracking technology is one of the key fundamental technologies in these fields. The accuracy and robustness of visual target tracking algorithms are of great significance for further enhancing the safety and effectiveness of high - level intelligent applications. The multi - target tracking task needs to ensure the real - time performance and accuracy of tracking. Traditional detector - based tracking algorithms rely too much on detectors. Slight errors caused by factors such as occlusion, illumination, and deformation in the external environment during detection will greatly affect the data association between consecutive frames, resulting in the loss of tracking targets and the interruption of tracking trajectories. Summary of the Invention
[0003] The technical problem to be solved by the present invention is to provide a multi - target tracking method for noise control, which improves the tracking accuracy in scenarios of small - target tracking and complex - environment tracking, thereby solving the above problems.
[0004] The technical solution of the present invention is: a multi - target tracking method for noise control, which obtains a smooth adaptive observation noise matrix by combining a Gaussian function with the confidence of detection results, further obtains a smooth gain, and obtains the motion information of the target. The specific steps are as follows:
[0005] Step1: Obtain video data through a camera.
[0006] Step2: Extract features and perform addition fusion on the current frame, the previous frame of tracking data, and the current - frame heat map. Remove redundant fusion noise by introducing a prior denoising module.
[0007] Step3: Extract features from the fusion result, obtain heat - map fusion features with different receptive fields, calculate the gradient of the center point of the heat map in the fusion result, and obtain the target center point for target detection.
[0008] Step4: Perform smooth - gain Kalman filtering on the detection result. Obtain a smooth adaptive observation noise matrix by combining a Gaussian function with the confidence of detection results, further obtain a smooth gain, and obtain the motion information of the target.
[0009] Step5: Combine the motion information to perform data association on the inter - frame targets and obtain the target motion trajectory.
[0010] Step6: Design a gradient-accelerated trajectory reconnection module to reconnect fragmented trajectories and interrupted trajectories to obtain accurate tracking results.
[0011] Specifically, Step2 is as follows:
[0012] Step2.1: Extract features from the current frame, the previous frame, and the current-frame heatmap of the tracking data and perform addition fusion.
[0013] Step2.2: Input C_256 and C_128 of the fused features into the prior denoising module.
[0014] Step2.3: Remove redundant noise from the fused features through unbiased feature extraction and feature recombination.
[0015] Specifically, Step4 is as follows:
[0016] Step4.1: Calculate the measurement pre-fit residual.
[0017] Step4.2: Obtain the smoothed adaptive observation noise matrix SG_R by combining the Gaussian function with the confidence of the detection result k , where the Gaussian radius σ is 3.
[0018] Step4.3: Calculate the pre-fit residual covariance.
[0019] Step4.4: Calculate the Kalman gain matrix based on the adaptive observation noise matrix smoothed by the Gaussian function to ensure the stability of the Kalman filter gain.
[0020] Step4.5: Update the system estimate.
[0021] Step4.6: Calculate the updated estimated covariance matrix.
[0022] Compared with the prior art, the present invention first introduces a prior denoising module to remove redundant noise in heatmap fusion; combines adaptive observation noise for Kalman filtering to smooth sudden change noise and stably obtain target motion information; designs a gradient-accelerated trajectory reconnection module to adaptively reconnect fragmented trajectories through the gradient-accelerated decision tree algorithm.
[0023] Based on the problem of redundant semantic information caused by early feature fusion in the anchor-free multi-object tracking algorithm of the present invention; the problem that traditional methods can only calculate motion information based on Kalman filtering with constant observation noise and cannot smoothly combine tracking system information to further obtain motion information; the problem that trajectory reconnection lacks consideration of the trend of trajectory ground truth and the reconnection result has a large deviation.
[0024] The beneficial effects of the present invention are as follows: The present invention is a multi-object tracking method for controlling and tracking noise at each stage. Compared with the prior art, the present method solves the following three problems: the problem of semantic information redundancy caused by early feature fusion of the anchor-free multi-object tracking algorithm; the problem that the traditional method can only calculate motion information based on the Kalman filter with constant observation noise and cannot smoothly combine the tracking system information to further obtain motion information; the problem that the trajectory reconnection lacks consideration of the trend of the trajectory ground truth and the reconnection result has a large deviation. The present method not only improves the tracking accuracy in the scenarios of small object tracking and complex environment tracking, but also enhances the acquisition of motion information in data association and alleviates the problem of trajectory drift. Description of the Drawings
[0025] Figure 1 is the flowchart of the present invention.
[0026] Figure 2 is the overall network diagram of multi-object tracking of the present invention.
[0027] Figure 3 is the principle illustration diagram of prior denoising of heatmap features designed by the present invention.
[0028] Figure 4 is the principle illustration diagram of smooth gain Kalman filter designed by the present invention.
[0029] Figure 5 is the principle illustration diagram of gradient-accelerated trajectory reconnection of the present invention. Detailed Embodiments
[0030] The present invention will be further described below in conjunction with the drawings and detailed embodiments.
[0031] Embodiment 1: As Figure 1 shown, a noise control multi-object tracking method has the following specific steps:
[0032] Step1: Read the camera video information.
[0033] Step2: Extract and additively fuse the features of the current frame, the previous frame, and the current frame heatmap of the tracking data, and remove the redundant noise in the fusion by introducing a prior denoising module.
[0034] Step2.1: Extract and additively fuse the features of the current frame, the previous frame, and the current frame heatmap of the tracking data;
[0035] Perform preliminary feature extraction on the current frame, the previous frame, and the current frame heatmap through convolution, regularization, and activation functions, and the three share weights when extracting features. After the three are additively fused, the size of the fused features is: 544×960×16.
[0036] Step2.2: Input \(C_{256}\) and \(C_{128}\) of the fused features into the prior denoising module;
[0037] As Figure 2 shown, further feature extraction is performed on the fused features of \(544\times960\times16\) to obtain features with different receptive fields from 16 to 512 channels. To achieve a trade-off between accuracy and speed, \(C_{256}\) and \(C_{128}\) of the fused features are selected and input into the prior denoising module.
[0038] Step2.3: Remove the redundant noise of the fused features through unbiased feature extraction and feature recombination;
[0039] As Figure 3 shown, the prior denoising module consists of four scales. Each scale has an identity skip connection between the 2x2 stride convolution transformation and the 2x2 transposed convolution upsampling operation. The number of channels in each layer from the first scale to the fourth scale is 64, 128, 256, and 512 respectively. In addition, to ensure the generalization of the model, no bias is used in the stride convolution and transposed convolution layers of the model.
[0040] Step3: Perform feature extraction on the fusion result to obtain the heatmap fusion features with different receptive fields, calculate the gradient of the center point of the heatmap in the fusion result, and obtain the target center point for target detection;
[0041] Step4: As Figure 4 shown, perform smooth gain Kalman filtering on the detection result, obtain the smooth adaptive observation noise matrix by combining the Gaussian function with the confidence of the detection result, further obtain the smooth gain, and obtain the motion information of the target;
[0042] Step4.1: Calculate the measurement pre-fit residual;
[0043]
[0044] where is the pre-fit residual, \(z\) k is the original fit residual, \(H\) k is the observation model, predicted estimated state.
[0045] Step4.2: Obtain the smooth adaptive observation noise matrix \(SG_R\) by combining the Gaussian function with the confidence of the detection result k ;
[0046]
[0047] where \(c\) kThe detection model measures the confidence level. The Gaussian radius σ is taken as 1 as in the standard Gaussian distribution, and the expectation μ is 1, which represents the probability that the confidence score deviates from the expectation. An adaptive observation noise covariance is introduced.
[0048] Step4.3: Calculate the pre-fitted residual covariance;
[0049]
[0050] Among them, P k|k-1 is the predicted estimation covariance.
[0051] Step4.4: Calculate the Kalman gain matrix based on the adaptive observation noise matrix smoothed by the Gaussian function to ensure the stability of the Kalman filter gain;
[0052]
[0053] Step4.5: Update the system estimated value;
[0054]
[0055] Step4.6: Calculate the updated estimated covariance matrix.
[0056] P k|k =(I-K k H k )P k|k-1 (6)
[0057] Step5: Combine the motion information to perform data association on the inter-frame targets to obtain the target motion trajectory. Design a gradient-accelerated trajectory reconnection module to reconnect the fragmented trajectories and interrupted trajectories to obtain accurate tracking results.
[0058] As Figure 5 shown, affected by the frame rate and the object motion direction, the true value of the target trajectory is a directed progressive broken line. Based on this, the gradient boosting trajectory reconnection module of the present invention adaptively approximates the true trajectory of the residual reduction direction through a gradient boosting decision tree, achieving a good compromise between accuracy and efficiency.
[0059] The gradient boosting trajectory reconnection model of the present invention for the i-th trajectory is as follows:
[0060] p t =GB (i) (t)+ξ (7)
[0061] Among them, p t is the position coordinate variable of the t-th frame, ξ~N(0,σ 2) is Gaussian noise. Before performing gradient boosting trajectory reconnection, linear interpolation is first performed on the interrupted trajectory, and the interpolated trajectory where l is the trajectory length, which ensures the efficiency of the module. GB (i) (t) is the gradient boosting decision tree regression, and the trajectory is input as shown in Equation (6) to obtain the reconnected trajectory with iterative recursive reduction of residuals.
[0062]
[0063]
[0064] where γ i is the fitted decision tree value, R i is the end domain, and L is the bias loss function. GB (i) (t) uses the additive model and the forward stepwise algorithm to achieve the iterative optimization process towards the true value, obtains the continuous trajectory after reconnection, and ensures the consistency of the trend between the reconnected trajectory and the true trajectory.
[0065] The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those of ordinary skill in the art.
Claims
1. A multi-object tracking method for noise control, characterized in that: Step1: Obtain video data through a camera; Step2: Extract features and perform addition fusion on the current frame, previous frame, and current frame heatmap of the tracking data, and remove redundant noise in the fusion by introducing a prior denoising module; The prior denoising module consists of four scales. Each scale has an identity skip connection between the 2x2 stride convolution transformation and the 2x2 transposed convolution upsampling operation. The number of channels in each layer from the first scale to the fourth scale is 64, 128, 256, and 512 respectively. Biases are not used in the stride convolution and transposed convolution layers in the model; Step3: Extract features from the fusion result to obtain heatmap fusion features with different receptive fields, calculate the gradient of the center point of the heatmap in the fusion result, and obtain the target center point for target detection; Step4: Perform smooth gain Kalman filtering on the detection result, obtain a smooth adaptive observation noise matrix by combining the Gaussian function with the confidence of the detection result, and obtain the motion information of the target; Step5: Combine the motion information to perform data association on the inter-frame targets to obtain the target motion trajectory; Step6: Design a gradient acceleration trajectory reconnection module to reconnect the fragmented trajectories and interrupted trajectories to obtain an accurate tracking result; The gradient boosting trajectory reconnection model for the i-th trajectory is as follows: p t = GB (i) (t) + ξ (7) where p t is the position coordinate variable of the t-th frame, ξ ∼ N(0, σ 2 ), and before performing gradient-boosted trajectory reconnection, linear interpolation is first performed on the interrupted trajectory, and the interpolated trajectory where l is the trajectory length, and GB (i) (t) is gradient-boosted decision tree regression.
2. The multi-object tracking method for noise control according to claim 1, wherein The specific content of Step2 is as follows: Step2.1: Extract features and perform addition fusion on the current frame, previous frame, and current frame heatmap of the tracking data; Step2.2: Input C_256 and C_128 of the fusion features into the prior denoising module; Step2.3: Remove redundant noise in the fusion features through unbiased feature extraction and feature recombination.
3. The noise control multi-object tracking method according to claim 1, characterized in that The specific content of Step4 is as follows: Step4.1: Calculate the measurement pre-fit residual; Step4.2: Obtain the smoothed adaptive observation noise matrix SG_R by combining the Gaussian function with the confidence of the detection result k , where the Gaussian radius σ is 3; Step4.3: Calculate the pre-fit residual covariance; Step4.4: Calculate the Kalman gain matrix based on the adaptive observation noise matrix smoothed by the Gaussian function; Step4.5: Update the system estimate; Step4.6: Calculate the updated estimated covariance matrix.