Multi-view target tracking algorithm based on region overlapping degree
Through the multi-view target tracking algorithm based on region overlap, the problem of inaccurate information aggregation when the number of cameras increases is solved. Through feature dimensionality reduction and timing information fusion, combined with the hollow convolution network, the multi-view target is improved The accuracy and consistency of tracking solves the problems of misjudgment and missed detection in traditional methods.
Patent Information
- Application Number
- CN202411063642.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-26
AI Technical Summary
When the number of cameras increases, the multi-view information aggregation is inaccurate, resulting in insufficient tracking performance, especially in crowded crowded scenarios.
Using a multi-view target tracking algorithm based on area overlap, by obtaining the time scale alignment video of the multi-camera monitoring system, extracting image features and performing perspective transformation to project into the bird's-eye direction, calculating the area overlap mask, and using timing information to fusion network and holes Convolutional decoding network realizes pedestrian occupation and offset prediction and improves tracking accuracy.
Effectively remove redundancy and noise from feature quantities, enhance model generalization performance, reduce computing resource consumption, alleviate the missed detection of information of occluded objects in single-view angles, improve the accuracy and consistency of multi-view angle information aggregation, and enhance tracking accuracy.
Smart Images

Figure CN119027464A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video analysis, and in particular to a multi-view target tracking algorithm based on region overlap. Background Art
[0002] Intelligent analysis of videos has become a hot topic in research, with a wide range of application scenarios. Using image processing algorithms and deep learning to preprocess and analyze video data can parse more useful information from the video, and it has a wide range of applications in various fields, such as security systems, smart transportation, medical diagnosis, unmanned supermarkets, etc. Multi-view target tracking refers to the task of tracking the target object from multiple perspectives. Traditional target tracking methods usually only consider target tracking from a single perspective, and the target is easily occluded, resulting in tracking failure. However, in real scenarios, the occluded target in one view may be clearly visible in another view, so that more accurate target tracking results can be obtained by utilizing information from multiple cameras. Compared with monocular camera systems, multi-camera systems can obtain more abundant and diverse information, and their application scenarios are also more extensive, and the needs in various fields are urgent.
[0003] The current mainstream multi-view target tracking algorithm is based on the multi-view aggregation method. The global tracking algorithm realizes multi-view tracking. It extracts features from each view image, projects the feature view to the bird's-eye view direction based on the internal and external parameters of the camera, aggregates multi-view information, and outputs a global pedestrian occupancy map. It performs global tracking of pedestrians in the multi-camera system based on the coordinates and volume of pedestrians in the previous and next frames in the bird's-eye view direction.
[0004] The traditional multi-view aggregation method is based on conditional random fields for reasoning. By associating multiple cameras, a feature view set is generated, and pedestrians in each view are detected using anchor points. The target probability distribution map produced is evaluated based on the anchor point detection method. The performance of this method is limited by the accuracy of the detection anchor frame. At the same time, the application of conditional random fields requires the introduction of additional feature processing and operations. The current mainstream multi-view information aggregation method adopts a full convolution structure. Experiments show that the convolution kernel can imitate some behaviors and characteristics of conditional random fields. Therefore, a large number of convolution structures can replace conditional random fields. However, due to the translation invariance of convolution, there are certain limitations in performance. The tracking algorithm is mainly based on the global pedestrian probability distribution map to calculate the corresponding cost matrix based on the pedestrian coordinates and volume to achieve tracking. However, due to the occlusion of crowded people, pedestrians occupy densely in the bird's-eye view direction, which is easy to cause misjudgment and missed detection, and the tracking performance is limited. For example, the existing Chinese patent technology CN103020989A discloses a multi-view target tracking method based on online scene feature clustering, which uses an online clustering method to calculate the foreground likelihood map of each view, project it into the reference ground plane, and perform real-time tracking based on a 3D particle filter algorithm. This invention solves the problem that traditional background modeling cannot truly simulate scene changes, but when the number of cameras in a multi-camera system increases, it cannot effectively aggregate multi-view information, resulting in poor performance. Summary of the invention
[0005] The purpose of the present invention is to provide a multi-view target tracking algorithm based on region overlap, so as to solve the problem that the current mainstream multi-view tracking algorithm has inaccurate multi-view information aggregation when the number of cameras increases.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] A multi-view target tracking algorithm based on region overlap, specifically comprising the following steps:
[0008] S1, obtain time-scale aligned multi-channel videos from a multi-camera monitoring system;
[0009] S2, extract the current frame and the previous frame image for each video, input them into the encoder for feature extraction, and obtain the feature vectors of the previous and next frames;
[0010] S3, based on the internal and external parameters of the camera, use perspective transformation to project the feature quantities of each fused image to the bird's-eye view (Bev) direction to obtain the Bev image feature quantity;
[0011] S4, splicing the image feature quantities of each channel's previous and next frames in the channel dimension, and inputting them into a pre-established temporal information fusion network to obtain a fused image feature quantity;
[0012] S5. Based on the coverage area of the image feature quantities of each camera in the Bev direction, the regional overlap is calculated to obtain low, medium and high overlap masks.
[0013] As a preferred solution of the present invention: the high overlap mask is to divide the Bev image feature into Feature low 、Feature mid and Feature high , the Feature low 、Feature mid and Feature high They represent the regional characteristics of low, medium and high overlap of camera coverage area in the Bev direction respectively.
[0014] As a preferred solution of the present invention: the regional features with low, medium and high overlap of the camera coverage area in the Bev direction are used to input into the corresponding low, medium and high decoders, restore the feature dimensions, sum the output features, and obtain the Bev high-dimensional feature quantity.
[0015] As a preferred solution of the present invention: the Bev high-dimensional feature quantity is used to be respectively applied to the pedestrian occupancy prediction decoupling head, the offset prediction decoupling head, the ID prediction decoupling head and the previous frame prediction decoupling head to obtain the pedestrian occupancy feature quantity, the offset prediction feature quantity, the ID prediction feature quantity and the previous frame prediction feature quantity.
[0016] As a preferred solution of the present invention: the pedestrian occupancy feature is used to decode and obtain the coordinates of the pedestrian in the Bev direction, the offset prediction feature is used to correct the pedestrian coordinates, and the ID prediction feature and the previous frame prediction feature are used to obtain the predicted pedestrian's ID number and previous frame coordinates.
[0017] As a preferred solution of the present invention, the coordinates of the pedestrian in the Bev direction, the predicted ID number of the pedestrian and the previous frame coordinates are used to update the tracker, specifically including the following steps:
[0018] S6.1. Match the ID prediction result with the trajectory set that has been discarded in the previous period, and update the trajectory set to be matched;
[0019] S6.2, calculate the distance cost matrix between the previous frame coordinates and the current trajectory pool, filter by threshold, and update the set of trajectories to be matched;
[0020] S6.3, use the Kalman filter algorithm to estimate the new confidence and variance of each ID, calculate the cost matrix according to the cosine distance calculation formula, set the threshold to obtain the rough matching and unmatched trajectories;
[0021] S6.4. Update the tracking trajectory set and discarded trajectory set according to the IOU and center coordinate cost matrix to complete the current frame tracking.
[0022] Compared with the prior art, the present invention has the following beneficial effects:
[0023] 1. The present invention performs feature dimensionality reduction processing on image features, which can effectively remove the redundancy and noise of feature quantities, avoid dimensionality disaster, enhance model generalization performance, and reduce computing resource consumption;
[0024] 2. The present invention projects the feature quantities of each viewing angle image to the Bev direction for information aggregation, which can enhance visual consistency, effectively alleviate the situation where single-view objects are blocked and information is missed, and enhance the generalization performance of model prediction;
[0025] 3. By extracting features from the current frame and the previous frame, and applying the time series information fusion module, the temporal changes of the object can be effectively captured by fusing the image information of the previous moment, and the motion compensation of the object at the current moment can be performed to reduce the position drift and improve the accuracy of the previous frame prediction and matching tracking;
[0026] 4. Use the regional overlap mask to divide the Bev feature quantity, introduce different degrees of error to the regions with different camera numbers, and correspond to different decoding processes, effectively aggregate multi-view information, reduce the conflict between feature quantities, and improve the accuracy and consistency of the overall information;
[0027] 5. Using dilated convolution as the main convolution operation of the decoding network, compared with the traditional convolution with fixed convolution kernel size, it can capture broader semantic information by adjusting the dilation factor, and introduce multiple dilated convolution layers in the decoding network. By setting different dilation factors, multi-scale feature information can be effectively extracted to enhance the perception ability of the model;
[0028] 6. The present invention introduces a previous frame prediction module to predict the Bev coordinates of the object at the previous moment, and performs cascade matching with the current tracker trajectory set based on the prediction result to improve tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 This is a flow chart of a multi-view tracking algorithm based on region overlap according to an embodiment of the present invention;
[0030] Figure 2 The front and back frame images of the multi-channel cameras in the embodiment of the present invention;
[0031] Figure 3 It is a visualization image of the coded feature quantity of an embodiment of the present invention;
[0032] Figure 4 This is a visualization image of the Bev feature quantity of an embodiment of the present invention;
[0033] Figure 5This is a visualization diagram of the Bev pedestrian prediction result and the previous frame prediction result in an embodiment of the present invention;
[0034] Figure 6 This is a network structure diagram of a feature dimension reduction encoder according to an embodiment of the present invention;
[0035] Figure 7 It is a timing information fusion module diagram of an embodiment of the present invention;
[0036] Figure 8 A flowchart of regional overlap decoding according to an embodiment of the present invention;
[0037] Fig. 9 It is a 0-1 mask image of each camera feature quantity projected to the Bev direction in an embodiment of the present invention;
[0038] Fig.10 is a region overlap mask map according to an embodiment of the present invention;
[0039] Fig.11 It is a mask map of the overlapping degree of the divided areas according to an embodiment of the present invention;
[0040] Fig.12 A decoder network structure diagram of an embodiment of the present invention;
[0041] Fig.13 This is a previous frame prediction flow chart of an embodiment of the present invention. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] See also Figure 1 , the present invention provides a technical solution:
[0044] A multi-view target tracking algorithm based on region overlap, specifically comprising the following steps:
[0045] S1, obtain time-scale aligned multi-channel videos from a multi-camera monitoring system;
[0046] S2, extract the current frame and the previous frame image for each video, input them into the encoder for feature extraction, and obtain the feature vectors of the previous and next frames;
[0047] S3, based on the internal and external parameters of the camera, use perspective transformation to project the feature quantities of each fused image to the bird's-eye view (Bev) direction to obtain the Bev image feature quantity;
[0048] S4, splicing the image feature quantities of each channel's previous and next frames in the channel dimension, and inputting them into a pre-established temporal information fusion network to obtain a fused image feature quantity;
[0049] S5. Based on the coverage area of the image feature quantities of each camera in the Bev direction, the regional overlap is calculated to obtain low, medium and high overlap masks.
[0050] Furthermore, the high overlap mask is to divide the Bev image feature into Feature low 、Feature mid and Feature high , the Feature low 、Feature mid and Feature high They represent the regional characteristics of low, medium and high overlap of camera coverage area in the Bev direction respectively.
[0051] Furthermore, the regional features with low, medium and high overlap of the camera coverage area in the Bev direction are used to input into the corresponding low, medium and high decoders, restore the feature dimensions, sum the output features, and obtain the Bev high-dimensional feature quantity.
[0052] Furthermore, the Bev high-dimensional feature quantity is used to be respectively applied to the pedestrian occupancy prediction decoupling head, the offset prediction decoupling head, the ID prediction decoupling head and the previous frame prediction decoupling head to obtain the pedestrian occupancy feature quantity, the offset prediction feature quantity, the ID prediction feature quantity and the previous frame prediction feature quantity.
[0053] Furthermore, the pedestrian occupancy feature is used to decode and obtain the coordinates of the pedestrian in the Bev direction, the offset prediction feature is used to correct the pedestrian coordinates, and the ID prediction feature and the previous frame prediction feature are used to obtain the predicted pedestrian's ID number and previous frame coordinates.
[0054] Further, the coordinates of the pedestrian in the Bev direction, the predicted ID number of the pedestrian and the coordinates of the previous frame are used to update the tracker, which specifically includes the following steps:
[0055] S6.1. Match the ID prediction result with the trajectory set that has been discarded in the previous period, and update the trajectory set to be matched;
[0056] S6.2, calculate the distance cost matrix between the previous frame coordinates and the current trajectory pool, filter by threshold, and update the set of trajectories to be matched;
[0057] S6.3, use the Kalman filter algorithm to estimate the new confidence and variance of each ID, calculate the cost matrix according to the cosine distance calculation formula, set the threshold to obtain the rough matching and unmatched trajectories;
[0058] S6.4. Update the tracking trajectory set and discarded trajectory set according to the IOU and center coordinate cost matrix to complete the current frame tracking.
[0059] The present invention is further described in detail below by way of specific implementation cases, so that the present invention can be more easily understood. Some operations related to the present invention may not be mentioned or described in detail in this specification, in order to avoid the creative solution proposed by the present invention being ignored. For those skilled in the art, they can understand the relevant operations based on the solution design mentioned in the present invention and the general technical skills they master.
[0060] Please refer to Figure 1-Figure 12 ,This figure shows a multi-view target tracking algorithm based on region overlap, which includes the following steps:
[0061] Step 100: Obtain images at time T and time T-1 from multiple downstream cameras, such as Figure 2 As shown;
[0062] Step 200: Apply the encoder to perform feature dimension reduction and remove redundancy and noise, such as Figure 3 As shown;
[0063] Step 300: Perspective transformation projects the image feature quantity to the Bev direction to enhance visual consistency, such as Figure 4 As shown;
[0064] Step 400: applying a temporal information fusion module, cascading context information, and fusing Bev feature quantities;
[0065] Step 500: Apply low, medium and high area overlap masks to divide Bev feature quantities;
[0066] Step 600: Apply a decoder to restore feature dimensions and obtain output feature quantities;
[0067] Step 700: Apply the pedestrian occupancy decoupling head, the offset prediction decoupling head, the ID prediction decoupling head, and the previous frame prediction decoupling head to obtain the pedestrian coordinates and ID of the current frame, as well as the pedestrian coordinates of the previous moment, such as Figure 5 As shown;
[0068] Step 800: Perform Bev global tracking based on the prediction results and update the tracker. In the tracking stage, the current track set and the track set to be matched are optimized by the tracking algorithm of the fourth cascade matching according to the ID number, the previous frame coordinates, the IOU, and the center cost matrix to obtain the final tracking result;
[0069] Please refer to Figure 6 , the figure shows the encoder network structure. In this embodiment, the feature dimension reduction process is as follows:
[0070] Taking the image image at a certain viewing angle and time T as an example, the image processing process for other viewing angles and other times is the same. First, the input image is input into the hierarchical feature extraction module to adjust the number of channels, and then the feature vector is input into the downsampling convolution module to extract low-dimensional features. The low-dimensional feature information is fused at multiple scales, and the coded feature quantity is obtained using the convolution feature extraction module.
[0071] The hierarchical feature extraction module is composed of convolutional layers and maximum pooling layers, and the downsampling convolution module is composed of convolutional layers with different parameter settings. The downsampling convolution module is applied to obtain low-dimensional features x1, x2, and x3. The x3 feature vector is upsampled and adjusted to be equal to the x2 feature vector. The x3 and x2 feature vectors are concatenated in the channel dimension. The same operation is performed to concatenate x1 and x2. After passing through the convolution feature extraction module, the features are extracted and the number of channels is adjusted to obtain the output encoded feature vector.
[0072] The convolution layer uses the local receptive field of the convolution kernel to locally connect and share local weights on the input data, effectively capturing the translation and local spatial features of the image. The maximum pooling layer downsamples the feature space dimensions, selects the maximum value in the pooling area as the output, retains the main feature information, effectively reduces the size and dimension of the data, and improves the efficiency of the model.
[0073] The encoded feature quantity is based on the internal and external parameters of the camera to construct the homography matrix of the camera perspective transformation to the bird's-eye view (Bev). The specific formula is: Feature view_i =M i *Feature view_i , where M i =K i *[R i |T i ]、Feature viem_i represents the feature vector of camera i in the original direction, M i Represents the perspective transformation matrix of camera i from the original perspective to the Bev perspective, K i represents the internal parameter matrix of camera i, R i represents the rotation vector of the external parameters of camera i, T i Represents the translation vector of the external parameters of camera i. According to the perspective transformation matrix M, each view encoding feature vector is projected to the Bev direction, and the temporal information fusion module is used to fuse the Bev feature vectors of the previous and next frames.
[0074] Please refer to Figure 7 , which is a flow chart of the temporal information fusion module. In this embodiment, the features of the previous and next frame images are fused by the temporal information fusion module to generate new feature quantities.
[0075] Among them, Xt-1 represents the Bev feature at time t-1, and Xt represents the Bev feature at time t. First, Xt-1 and Xt are concatenated in the channel dimension, and then the features are extracted through a 3x3 convolution kernel and the number of channels of the concatenated feature vector is adjusted to 128. Then, instance normalization (InstanceNorm) is used to standardize the input feature, and the Relu activation function is used for nonlinear transformation. Finally, 1x1 convolution is used to extract features to obtain the fused Bev feature.
[0076] The main function of instance normalization is to normalize the mean and variance of each channel of the feature quantity, thereby improving the convergence speed and training stability of the network. Moreover, because it works on each channel separately, it can effectively deal with the scale offset problem between features of different channels, making it easier for the network to learn the features of different channels.
[0077] In this embodiment, the process of decoding feature quantity based on region overlap is as follows: Figure 8 As shown, the calculation process of the area overlap mask is as follows:
[0078] First, calculate the 0-1 mask of the feature quantity of each camera in the Bev direction, such as Fig. 9 As shown, all 0-1 masks are then summed to obtain the region overlap mask, such as Fig.10 As shown, the low, medium and high area overlap masks are divided based on the true value of the area overlap mask, as shown in Fig.11 As shown, the Bev fusion feature is finally multiplied by the low, medium and high area overlap masks to divide the Bev feature, where the calculation formula is as follows:
[0079]
[0080] Among them, Feature bev Represents the Bev directional feature after fusion, Mask low-high Represents low, medium and high area overlap decoding, The 0-1 mask representing the feature quantity of each camera in the Bev direction, and value represents the true value of the area overlapping feature quantity.
[0081] Get the Bev feature after division low 、Feature mid 、Feature high The regional overlap decoding module is applied to restore the dimensions of each feature quantity, and the restored feature quantities are added as the output feature quantity Output Feature. Different decoupling heads are applied to the output feature quantity to realize the prediction of pedestrian coordinates in the Bev direction, offset prediction, ID prediction and previous frame prediction.
[0082] In this embodiment, the decoder Decoder network structure is as follows Fig.12 As shown in the figure, the network structure of the decoder with low, medium and high area overlap is consistent, and the weight parameters are optimized independently. The processing flow is to input the divided Bev feature into the downsampling module for feature dimension reduction, including a 3x3 convolution with a step size of 2 plus a Relu activation function, and then input the feature into the World Conv module for processing, that is: first, it undergoes a convolution operation with a step size of 1 and a convolution kernel size of 3x3, then the Relu activation function is used, and then it undergoes two 3x3 size and dilation factors of 2 and 4 respectively. The dilation factor is 2 and 4 respectively, and finally it passes through the Upsample module. After the feature dimension is restored by Bilinear bilinear interpolation, a 1x1 convolution is used for feature extraction, and the Relu activation function is used for nonlinear transformation.
[0083] refer to Fig.13 ,The figure shows the process of the previous frame prediction optimization tracking algorithm. The specific steps are as follows:
[0084] The output features are applied to the previous frame prediction decoupling head to obtain the pedestrian coordinate set at the previous moment, and the trajectory set in the current tracker and the discarded trajectory set are cascade matched to calculate the distance cost matrix respectively. The calculation formula is as follows:
[0085] Distance cost Matrix A(i,j)=||Tracklet(i)-Pos(j)||2
[0086] Distance cost Matrix B(i,j)=||TrackletDrop(i)-Pos(j)||2
[0087] Among them, Tracklet represents the current tracker coordinate set, TrackletDrop represents the discarded tracker coordinate set, and Pos represents the previous frame prediction coordinate set;
[0088] Then, the coordinates in the two sets are calculated pairwise to obtain the binorm as the value in the cost matrix. The satisfying point set is obtained based on the threshold = 0.5, and the point with the smallest distance cost is returned as the matching point pair. The ID number of the current matching coordinate point is set, and the matching trajectory set is updated.
[0089] To verify the effectiveness of the present invention, the multi-target detection accuracy (MODA), multi-target detection precision (MODP), multi-target tracking accuracy (MOTA), multi-target tracking precision (MOTP) and model parameter size (ModelSize) are used to evaluate the performance of the present invention.
[0090] The experiment uses the public multi-view pedestrian tracking dataset Wlidtrack, and the baseline uses the EarlyBird multi-target tracking algorithm to predict the coordinates of pedestrians in the Bev direction as sample points.
[0091] When measuring the detection performance of the present invention, according to the prediction result of the current frame (x i ,y i ) and each sample (x i ,y i ) is compared with the set distance threshold = 0.5. If there is a real sample coordinate less than α, it is recorded as a true positive (TP). If there is no real sample that meets the distance threshold requirement, the predicted coordinate is marked as a false positive (FP). If there is no possible predicted sample point around the real sample point, it is recorded as a false negative (FN). The predicted samples of the remaining Bev direction map grid points are negative examples, and the sample points where the real samples are negative examples are recorded as true negative examples (TN).
[0092] When measuring the tracking performance of the present invention, the total number of correctly associated targets in all frames is counted, that is, the ID number of the track match is consistent with the real ID number. If the match is successful but the ID number is inconsistent, it is recorded as an incorrect association.
[0093] After the model training is completed, the size of the weight parameter file is used as an indicator of the model parameter quantity. A smaller model parameter quantity means less computing resources consumed during model inference and faster inference speed.
[0094] In order to better evaluate the present invention, an ablation experiment is set up to compare the model performance indicators of the temporal information fusion module, the previous frame prediction module, and the regional overlap decoding module on the basis of the Baseline, as shown in the following table:
[0095] MODA MODP MOTA MOTP Model Size Baseline 89.71 82.33 88.34 11.04 99.38 + Timing Information 91.07 82.29 89.39 11.95 119.31 +Previous frame prediction 89.81 82.49 88.66 11.15 108.82 +Region Overlap Decoding 91.18 80.52 88.97 13.77 63.70 +ALL 90.86 81.20 90.34 12.55 67.31
[0096] It can be concluded from the above table that the multi-view target tracking algorithm based on regional overlap of the present invention has an increase in the corresponding algorithm performance indicators when the model parameters are significantly reduced, and the multi-target detection accuracy (MODA), multi-target detection precision (MODP), multi-target tracking accuracy (MOTA), and multi-target tracking precision (MOTP) are 90.86%, 81.20%, 90.34%, and 12.55%, respectively. It is proved that the method of the present invention can more effectively aggregate multi-view information and improve the performance of the multi-view target tracking algorithm in the multi-view target tracking task while consuming less computing resources.
[0097] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A multi-view target tracking algorithm based on region overlap, It is characterized in that The specific steps include: S1, obtain time-scale aligned multi-channel videos from a multi-camera monitoring system; S2, extract the current frame and the previous frame image for each video, input them into the encoder for feature extraction, and obtain the feature vectors of the previous and next frames; S3, based on the internal and external parameters of the camera, use perspective transformation to project the feature quantities of each fused image to the bird's-eye view (Bev) direction to obtain the Bev image feature quantity; S4, splicing the image feature quantities of each channel's previous and next frames in the channel dimension, and inputting them into a pre-established temporal information fusion network to obtain a fused image feature quantity; S5. Based on the coverage area of the image feature quantities of each camera in the Bev direction, the regional overlap is calculated to obtain low, medium and high overlap masks.
2. According to the multi-view target tracking algorithm based on region overlap as described in claim 1, Features: The high overlap mask is to divide the Bev image feature into Feature low 、Feature mid and Feature high , the Feature low 、Feature mid and Feature high They represent the regional characteristics of low, medium and high overlap of camera coverage area in the Bev direction respectively.
3. According to the multi-view target tracking algorithm based on area overlap as described in claim 2, Features: The regional features with low, medium and high overlap of the camera coverage area in the Bev direction are used to input into the corresponding low, medium and high decoders, restore the feature dimensions, sum the output features, and obtain the Bev high-dimensional feature quantity.
4. According to the multi-view target tracking algorithm based on area overlap as described in claim 3, Features: The Bev high-dimensional feature quantity is used to be respectively applied to the pedestrian occupancy prediction decoupling head, the offset prediction decoupling head, the ID prediction decoupling head and the previous frame prediction decoupling head to obtain the pedestrian occupancy feature quantity, the offset prediction feature quantity, the ID prediction feature quantity and the previous frame prediction feature quantity.
5. According to the multi-view target tracking algorithm based on area overlap as described in claim 4, Features: The pedestrian occupancy feature is used to decode and obtain the coordinates of the pedestrian in the Bev direction, the offset prediction feature is used to correct the pedestrian coordinates, and the ID prediction feature and the previous frame prediction feature are used to obtain the predicted pedestrian's ID number and previous frame coordinates.
6. The multi-view target tracking algorithm based on region overlap according to claim 5, Features: The coordinates of the pedestrian in the Bev direction, the predicted pedestrian ID number and the previous frame coordinates are used to update the tracker, specifically including the following steps: S6.
1. Match the ID prediction result with the trajectory set that has been discarded in the previous period, and update the trajectory set to be matched; S6.2, calculate the distance cost matrix between the previous frame coordinates and the current trajectory pool, filter by threshold, and update the set of trajectories to be matched; S6.3, use the Kalman filter algorithm to estimate the new confidence and variance of each ID, calculate the cost matrix according to the cosine distance calculation formula, set the threshold to obtain the rough matching and unmatched trajectories; S6.
4. Update the tracking trajectory set and discarded trajectory set according to the IOU and center coordinate cost matrix to complete the current frame tracking.
Citation Information
Patent Citations
Multi-view target tracking method based on on-line scene feature clustering
CN103020989A
Cited By
Target trajectory generation method based on multi-channel security video
CN119832024A
Multi-view pedestrian detection method based on multi-stage deformable disentanglement
WO2026199105A1
