Multi-target tracking method and system for multi-view scene

Abstract

The application provides a multi-target tracking method and system for a multi-view scene, relates to the technical field of computer vision, and comprises the following steps: collecting pedestrian image data through a multi-view device, performing target detection to obtain a position set in a unified coordinate system, calculating a preliminary association cost based on spatial position deviation and motion speed consistency, screening to obtain a confidence candidate association set and calculating Mahalanobis distance to form a cost matrix, executing an optimal matching algorithm to obtain an association result, and realizing continuous tracking of pedestrian targets. The application overcomes the single-view blind area problem and improves target tracking precision and stability.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a multi-target tracking method and system for multi-view scenarios. Background Technology

[0002] Multi-target tracking technology is an important research direction in the field of computer vision, mainly used for the continuous identification and tracking of multiple moving targets in video sequences. It has wide application value in scenarios such as security monitoring, intelligent transportation, and pedestrian flow analysis. Traditional multi-target tracking methods are mainly based on single-view processing. With technological advancements, multi-view multi-target tracking technology has gradually gained attention, improving the robustness and accuracy of the tracking system by integrating viewpoint information captured by multiple cameras.

[0003] The appearance features of targets vary significantly in multi-view scenarios. Relying solely on appearance features for association can easily lead to matching errors, especially in densely populated scenes where pedestrian targets with similar appearances are prone to mismatches and ID switching issues. Existing technologies face challenges in unifying multi-view coordinates and data fusion. Data collected by different cameras needs to be converted to the same coordinate system, and errors in this process accumulate and affect subsequent tracking performance, especially when camera calibration is inaccurate. Most existing algorithms lack effective motion consistency evaluation mechanisms and cannot fully utilize the continuous features of pedestrian targets in the spatiotemporal domain, resulting in decreased tracking performance in complex scenarios such as target intersections and occlusions, making it difficult to maintain stable long-term tracking results. Summary of the Invention

[0004] The embodiments of the present invention provide a multi-target tracking method and system for multi-view scenarios, which can solve the problems in the prior art.

[0005] A first aspect of the present invention provides a multi-target tracking method for multi-view scenes, comprising:

[0006] Acquire pedestrian image data from multiple perspective acquisition devices at the same time;

[0007] Multi-view target detection is performed on the pedestrian image data to obtain the set of pedestrian targets' positions and detection confidence in a unified coordinate system at the current moment;

[0008] In the current location set, for each pedestrian target location, the spatial position deviation and motion speed consistency with each trajectory in the trajectory set of the previous time are calculated to obtain the preliminary association cost. Based on the preliminary association cost, a confidence candidate association set is obtained. The Mahalanobis distance of each pair in the confidence candidate association set is calculated to form the confidence cost matrix.

[0009] The optimal matching algorithm is executed on the confidence cost matrix to obtain the confidence association result. The identity identifier of the pedestrian target position that is successfully matched in the confidence association result is inherited by the corresponding trajectory. The state vector of the corresponding trajectory is updated. The current position and timestamp are appended to the trajectory history of the corresponding trajectory to form the continuous spatiotemporal motion trajectory of the pedestrian target.

[0010] Motion prediction is performed on each of the continuous spatiotemporal motion trajectories to obtain the predicted state vector and state distribution covariance matrix for the next moment. The predicted state vector and the state distribution covariance matrix are used as the trajectory set input for the next moment to realize continuous tracking of multiple pedestrian targets across time in multi-view scenarios.

[0011] Multi-view target detection is performed on the pedestrian image data to obtain the set of pedestrian target positions in a unified coordinate system at the current moment and the detection confidence, including:

[0012] Perform pedestrian detection operation on each viewpoint image in the image data to obtain pedestrian target detection boxes in the coordinate system of each viewpoint image, and calculate the detection score based on the category prediction probability of the pedestrian target detection boxes;

[0013] Based on the camera intrinsic and extrinsic parameter matrices of multiple viewpoint devices, coordinate transformation is performed on the pedestrian target detection boxes in the image coordinate systems of each viewpoint. The pedestrian target detection boxes are projected from the image coordinate systems of each viewpoint to a common reference plane of a unified coordinate system, obtaining the candidate positions of each pedestrian target in the unified coordinate system. Cross-viewpoint fusion processing is performed on the candidate positions. By calculating the spatial distance between the candidate positions obtained from different viewpoint projections, candidate positions with a spatial distance less than a preset fusion threshold are assigned to the same pedestrian target. Multiple candidate positions assigned to the same pedestrian target are fused to obtain the fused position.

[0014] The fused position is taken as an element in the set of pedestrian target positions in a unified coordinate system at the current moment. The maximum value of the detection scores of each view participating in the fusion is taken as the detection confidence corresponding to the fused position.

[0015] For each viewpoint image in the image data, a pedestrian detection operation is performed to obtain pedestrian target detection boxes in the coordinate system of each viewpoint image. A detection score is then calculated based on the category prediction probability of the pedestrian target detection boxes, including:

[0016] For each viewpoint image in the image data, feature extraction and target localization are performed;

[0017] Multiple prior boxes with different scales and aspect ratios are preset at each location of the feature map obtained by feature extraction. The bounding box offset relative to the prior boxes is obtained by regression prediction. The coordinates of the prior boxes are adjusted using the bounding box offset to obtain the boundary position of the candidate region. The overlapping candidate regions are filtered and deduplicated by non-maximum suppression algorithm. The boundary coordinates of the retained candidate regions are converted into rectangular boxes in the image coordinate system of the viewpoint to form the pedestrian target detection box corresponding to the candidate region.

[0018] By performing binary classification prediction on each pedestrian target detection box, the probability value of the pedestrian target detection box belonging to the pedestrian category is output, and the category prediction probability is obtained; by calculating the degree of overlap or the uncertainty of the boundary deviation between the detection box boundary and the real target boundary, the localization quality assessment value is obtained.

[0019] The predicted probability of the category is combined with the localization quality assessment value to obtain the detection score of the pedestrian target detection box.

[0020] For each pedestrian target location, the spatial position deviation and velocity consistency with each trajectory in the trajectory set of the previous moment are calculated to obtain a preliminary association cost. Based on the preliminary association cost, a candidate association set with confidence scores is obtained, including:

[0021] Obtain the predicted position and predicted velocity of each trajectory in the trajectory set of the previous moment;

[0022] Calculate the Euclidean distance between the current pedestrian target position and the predicted position of each trajectory at the previous time to obtain the measure of spatial position deviation; in the trajectory set at the previous time, select the trajectory with the closest spatial distance to the pedestrian target position as the candidate matching position; calculate the current movement speed based on the current pedestrian target position and the candidate matching position; calculate the velocity vector angle and velocity amplitude difference between the current movement speed and the predicted speed of each trajectory at the previous time, and combine them to obtain the measure of movement speed consistency.

[0023] The initial correlation cost is obtained by combining the metric of the spatial position deviation with the metric of the consistency of the motion speed.

[0024] For each pedestrian target location in the current location set, retain the pedestrian target location-trajectory pairs with a preliminary association cost less than a preset cost threshold, and form the confidence candidate association set by retaining all the pedestrian target location-trajectory pairs.

[0025] Execute the optimal matching algorithm on the confidence cost matrix to obtain the confidence association result. Inherit the identity identifier of the corresponding trajectory from the successfully matched pedestrian target positions in the confidence association result, and update the state vector of the corresponding trajectory, including:

[0026] By iteratively finding the row and column allocation scheme that minimizes the sum of costs for all matching pairs in the confidence cost matrix, a one-to-one matching relationship between the pedestrian target location and the trajectory is obtained, and the one-to-one matching relationship is used as the confidence association result.

[0027] Extract the pairing relationship between the successfully matched pedestrian target location and the corresponding trajectory from the confidence association results. For each successfully matched pedestrian target location, inherit the identity identifier of the corresponding trajectory.

[0028] For each successfully matched trajectory, the state vector of the trajectory at the previous time step is obtained, and the position of the successfully matched pedestrian target is taken as the observation position at the current time step. The position update amount is calculated based on the observation position at the current time step and the position component in the state vector at the previous time step. The current observation velocity is calculated based on the observation position at the current time step and the historical position of the corresponding trajectory at the previous time step. The current observation velocity is fused with the velocity component in the state vector at the previous time step to obtain the velocity update amount.

[0029] The position update amount and the velocity update amount are combined to form a state update vector, and the state vector of the previous moment is fused with the state update vector to obtain the updated state vector.

[0030] Motion prediction is performed on each of the continuous spatiotemporal motion trajectories to obtain the predicted state vector and state distribution covariance matrix for the next moment, including:

[0031] For each of the continuous spatiotemporal motion trajectories, extract the state vector of the corresponding trajectory at the current moment;

[0032] Extract the position sequence and timestamp sequence from the trajectory history of the continuous spatiotemporal motion trajectory. Based on the most recent consecutive position points and corresponding timestamps in the position sequence, calculate the historical acceleration sequence by the ratio of the velocity difference between adjacent moments to the time interval.

[0033] Based on the historical acceleration sequence, the acceleration statistical features are calculated. Based on the position and velocity components in the current state vector, the mean of the acceleration statistical features is used as the predicted acceleration. The predicted position and predicted velocity at the next moment are calculated and combined to form the predicted state vector at the next moment.

[0034] Based on the variance of the acceleration statistical characteristics and the observation error statistics of the trajectory at historical moments, the process noise covariance and the observation noise covariance are calculated. Based on the process noise covariance and the observation noise covariance, the uncertainty distribution of the predicted state vector at the next moment is calculated. The uncertainty distribution is represented in the form of the state distribution covariance matrix.

[0035] A second aspect of the present invention provides a multi-target tracking system for multi-view scenarios, comprising:

[0036] The first unit is used to acquire pedestrian image data collected by multiple perspective acquisition devices at the same time;

[0037] The second unit is used to perform multi-view target detection on the pedestrian image data to obtain the set of positions of the pedestrian targets in a unified coordinate system at the current moment and the detection confidence.

[0038] The third unit is used to calculate the spatial position deviation and motion speed consistency between each pedestrian target position and each trajectory in the trajectory set of the previous time in the current position set, to obtain the preliminary association cost, to filter the candidate association set based on the preliminary association cost, to calculate the Mahalanobis distance of each pair in the candidate association set, and to form the confidence cost matrix.

[0039] The fourth unit is used to perform the optimal matching algorithm on the confidence cost matrix to obtain the confidence association result. The location of the pedestrian target that is successfully matched in the confidence association result inherits the identity identifier of the corresponding trajectory, updates the state vector of the corresponding trajectory, and appends the current position and timestamp to the trajectory history of the corresponding trajectory to form the continuous spatiotemporal motion trajectory of the pedestrian target.

[0040] The fifth unit is used to predict the motion of each of the continuous spatiotemporal motion trajectories, obtain the predicted state vector and the state distribution covariance matrix for the next moment, and use the predicted state vector and the state distribution covariance matrix as the trajectory set input for the next moment to realize continuous tracking of multiple pedestrian targets across time in multi-view scenarios.

[0041] A third aspect of the present invention,

[0042] An electronic device is provided, comprising:

[0043] processor;

[0044] Memory used to store processor-executable instructions;

[0045] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0046] Fourth aspect of the present invention,

[0047] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0048] The beneficial effects of this application are as follows:

[0049] The multi-target tracking method proposed in this invention for multi-view scenarios can effectively integrate pedestrian image data from multiple view acquisition devices at the same time. By performing target position analysis in a unified coordinate system, it significantly improves the accuracy and robustness of target detection and reduces occlusion and missed detection problems caused by a single view.

[0050] This method calculates the spatial positional deviation and motion speed consistency between the pedestrian target's location and its historical trajectory, and constructs a confidence cost matrix using Mahalanobis distance. This achieves efficient and accurate target association, effectively solving the problems of identity switching and target loss in multi-target tracking, and ensuring the continuity and consistency of target identity during the tracking process.

[0051] Based on the motion prediction mechanism, this method can accurately predict the next position of pedestrian targets. By iteratively updating the predicted state vector and the state distribution covariance matrix, it achieves stable and continuous tracking of multiple pedestrian targets in multi-view scenarios, and is suitable for practical application scenarios such as crowd monitoring and behavior analysis in complex environments. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a multi-target tracking method for multi-view scenarios according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the target matching and trajectory status update process. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0056] Figure 1This is a flowchart illustrating a multi-target tracking method for multi-view scenarios according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0057] Acquire pedestrian image data from multiple perspective acquisition devices at the same time;

[0058] Multi-view target detection is performed on the pedestrian image data to obtain the set of pedestrian targets' positions and detection confidence in a unified coordinate system at the current moment;

[0059] In the current location set, for each pedestrian target location, the spatial position deviation and motion speed consistency with each trajectory in the trajectory set of the previous time are calculated to obtain the preliminary association cost. Based on the preliminary association cost, a confidence candidate association set is obtained. The Mahalanobis distance of each pair in the confidence candidate association set is calculated to form the confidence cost matrix.

[0060] The optimal matching algorithm is executed on the confidence cost matrix to obtain the confidence association result. The identity identifier of the pedestrian target position that is successfully matched in the confidence association result is inherited by the corresponding trajectory. The state vector of the corresponding trajectory is updated. The current position and timestamp are appended to the trajectory history of the corresponding trajectory to form the continuous spatiotemporal motion trajectory of the pedestrian target.

[0061] Motion prediction is performed on each of the continuous spatiotemporal motion trajectories to obtain the predicted state vector and state distribution covariance matrix for the next moment. The predicted state vector and the state distribution covariance matrix are used as the trajectory set input for the next moment to realize continuous tracking of multiple pedestrian targets across time in multi-view scenarios.

[0062] In one optional implementation, multi-view target detection is performed on the pedestrian image data to obtain the current location set of the pedestrian target in a unified coordinate system and the detection confidence score, including:

[0063] Perform pedestrian detection operation on each viewpoint image in the image data to obtain pedestrian target detection boxes in the coordinate system of each viewpoint image, and calculate the detection score based on the category prediction probability of the pedestrian target detection boxes;

[0064] Based on the camera intrinsic and extrinsic parameter matrices of multiple viewpoint devices, coordinate transformation is performed on the pedestrian target detection boxes in the image coordinate systems of each viewpoint. The pedestrian target detection boxes are projected from the image coordinate systems of each viewpoint to a common reference plane of a unified coordinate system, obtaining the candidate positions of each pedestrian target in the unified coordinate system. Cross-viewpoint fusion processing is performed on the candidate positions. By calculating the spatial distance between the candidate positions obtained from different viewpoint projections, candidate positions with a spatial distance less than a preset fusion threshold are assigned to the same pedestrian target. Multiple candidate positions assigned to the same pedestrian target are fused to obtain the fused position.

[0065] The fused position is taken as an element in the set of pedestrian target positions in a unified coordinate system at the current moment. The maximum value of the detection scores of each view participating in the fusion is taken as the detection confidence corresponding to the fused position.

[0066] Pedestrian detection is performed separately for each viewpoint image using a convolutional neural network model, such as the improved YOLOv5 network. This network accurately identifies pedestrian targets in the image through multi-scale feature extraction and bounding box regression. The detection score is calculated by multiplying the localization quality assessment value by the category prediction probability. For example, if the localization quality assessment value of a detection box is 0.87 and the pedestrian category prediction probability is 0.95, the final detection score is 0.87 × 0.95 = 0.8265. In multi-view scenarios, images are typically acquired from at least three cameras at different angles. Each viewpoint generates multiple pedestrian detection results, each containing bounding box coordinates and a detection score.

[0067] After obtaining pedestrian detection results from various viewpoints, these results need to be transformed from their respective image coordinate systems to a unified world coordinate system. This process requires the use of the camera's intrinsic and extrinsic parameter matrices. The camera's intrinsic parameter matrix contains parameters such as focal length and principal point coordinates. For example, in the intrinsic parameter matrix of a certain camera, the focal length fx = 1200 pixels, fy = 1200 pixels, and the principal point coordinates cx = 960 pixels and cy = 540 pixels. The camera's extrinsic parameter matrix describes the camera's position and orientation in the world coordinate system and includes rotation matrices and translation vectors.

[0068] The coordinate transformation consists of two steps: first, converting the coordinates of the midpoint of the bottom edge of the detection box in the image coordinate system to the camera coordinate system; and second, using camera extrinsic parameters to transform the point in the camera coordinate system to the world coordinate system. To simplify calculations, it is usually assumed that all pedestrians are standing on the same plane, such as the ground, which can be considered a common reference plane. For the aforementioned detection box [325, 470, 405, 695], its midpoint is (365, 695). Assuming the ground is a plane with z=0, the position of this point in the world coordinate system can be obtained through back projection, such as (2.5 meters, 1.8 meters, 0 meters).

[0069] After pedestrian detection results from different viewpoints are transformed into the world coordinate system, cross-view fusion processing is required to calculate the spatial distance between candidate positions projected from different viewpoints. For example, if a pedestrian detected by viewpoint A has a position of (2.5 m, 1.8 m, 0 m) in the world coordinate system, and a pedestrian detected by viewpoint B has a position of (2.6 m, 1.75 m, 0 m), the Euclidean distance between the two points is 0.158 m. If the fusion threshold is set to 0.5 m, then these two candidate positions should be considered as the same pedestrian target.

[0070] For multiple candidate locations belonging to the same pedestrian, a weighted average method is used for fusion. The weights can be set according to the detection scores of each viewpoint; the higher the detection score, the greater the weight. For example, if the detection score of viewpoint A is 0.8265 and the detection score of viewpoint B is 0.7532, the fused location coordinates are (2.5×0.8265+2.6×0.7532) / (0.8265+0.7532)=2.547 meters and (1.8×0.8265+1.75×0.7532) / (0.8265+0.7532)=1.776 meters, and the final fused location is (2.547 meters, 1.776 meters, 0 meters).

[0071] The fused location is an element in the set of pedestrian target positions in a unified coordinate system at the current moment. For each fused location, its detection confidence is the maximum value of the detection scores of all participating viewpoints. In the example above, the detection score of viewpoint A is 0.8265, and the detection score of viewpoint B is 0.7532. Therefore, the detection confidence corresponding to this fused location is 0.8265.

[0072] The above processing steps effectively integrate pedestrian detection results from multiple perspectives, reducing occlusion and false detection issues inherent in single-view detection and improving the accuracy and robustness of pedestrian location detection. This method is particularly suitable for complex scenarios with high pedestrian traffic and frequent occlusion, such as pedestrian monitoring and analysis in public places like commercial areas and transportation hubs.

[0073] In one optional implementation, a pedestrian detection operation is performed on each viewpoint image in the image data to obtain pedestrian target detection boxes in the coordinate system of each viewpoint image, and a detection score is calculated based on the category prediction probability of the pedestrian target detection boxes, including:

[0074] For each viewpoint image in the image data, feature extraction and target localization are performed;

[0075] Multiple prior boxes with different scales and aspect ratios are preset at each location of the feature map obtained by feature extraction. The bounding box offset relative to the prior boxes is obtained by regression prediction. The coordinates of the prior boxes are adjusted using the bounding box offset to obtain the boundary position of the candidate region. The overlapping candidate regions are filtered and deduplicated by non-maximum suppression algorithm. The boundary coordinates of the retained candidate regions are converted into rectangular boxes in the image coordinate system of the viewpoint to form the pedestrian target detection box corresponding to the candidate region.

[0076] By performing binary classification prediction on each pedestrian target detection box, the probability value of the pedestrian target detection box belonging to the pedestrian category is output, and the category prediction probability is obtained; by calculating the degree of overlap or the uncertainty of the boundary deviation between the detection box boundary and the real target boundary, the localization quality assessment value is obtained.

[0077] The predicted probability of the category is combined with the localization quality assessment value to obtain the detection score of the pedestrian target detection box.

[0078] For each viewpoint image, feature extraction and target localization are performed. Feature extraction is achieved through a convolutional neural network, which consists of multiple convolutional layers, pooling layers, and activation functions. Specifically, a 5-layer convolutional network structure is used, with each layer having a 3×3 kernel size, a stride of 1, and padding of 1 to maintain the feature map size. The first convolutional layer has 3 input channels (corresponding to the RGB channels) and 64 output channels; the second convolutional layer has 64 input channels and 128 output channels; the third convolutional layer has 128 input channels and 256 output channels; the fourth convolutional layer has 256 input channels and 512 output channels; and the fifth convolutional layer has 512 input channels and 512 output channels. Each convolutional layer is followed by a ReLU activation function, and max pooling layers with a 2×2 kernel size and a stride of 2 are added after the first, second, and fourth layers. After the above network processing, the original 640×480 pixel input image is converted into feature maps of multiple sizes, with the smallest feature map size being 20×15 and the number of channels being 512.

[0079] Multiple prior bounding boxes of different scales and aspect ratios are pre-defined at each location of the feature map obtained from feature extraction. Specifically, six prior bounding boxes of different sizes and aspect ratios are set at each pixel location in the feature map: 32×64 pixels with an aspect ratio of 1:2; 64×128 pixels with an aspect ratio of 1:2; 128×256 pixels with an aspect ratio of 1:2; 32×32 pixels with an aspect ratio of 1:1; 64×64 pixels with an aspect ratio of 1:1; and 128×128 pixels with an aspect ratio of 1:1. These prior bounding boxes cover pedestrian targets of different scales and proportions, adapting to pedestrian detection needs at different distances and poses.

[0080] The bounding box offsets relative to the prior boxes are obtained through regression prediction. The regression prediction module consists of two fully connected layers. The first layer takes the feature vector (512-dimensional) at the corresponding position on the feature map as input and outputs a 256-dimensional intermediate feature. The second layer takes the 256-dimensional intermediate feature as input and outputs the offsets of all prior boxes at that position. Each prior box requires four offset values ​​to be predicted, corresponding to the adjustment of the coordinates of the top-left and bottom-right corners of the bounding box. The offsets are typically in the range of [-1, 1], representing the proportional adjustment relative to the width and height of the prior box. For example, if the width of the prior box is 64 pixels and the offset is 0.1, then the actual adjusted pixel value is 6.4 pixels.

[0081] The bounding box offset is used to adjust the coordinates of the prior box to obtain the boundary position of the candidate region. Assuming the coordinates of the top-left corner of the prior box are (x1, y1), the bottom-right corner are (x2, y2), and the predicted offset is (dx1, dy1, dx2, dy2), and the width of the prior box is w = x2 - x1, and the height is h = y2 - y1, then the adjusted bounding box coordinates are: top-left corner (x1 + dx1 × w, y1 + dy1 × h), bottom-right corner (x2 + dx2 × w, y2 + dy2 × h). In this way, the network can finely adjust the position of the prior box based on the image content to more accurately locate pedestrian targets.

[0082] The algorithm uses non-maximum suppression to filter and remove duplicate candidate regions, sorting all candidate regions in descending order of predicted category probability. Then, starting with the candidate region with the highest probability, the intersection-union ratio (IOU) with all remaining candidate regions is calculated. If the IOU is greater than a preset threshold (e.g., 0.5), candidate regions with lower probabilities are deleted. This process is repeated until all candidate regions have been processed. This effectively removes redundant detection boxes and retains detection results containing pedestrian targets.

[0083] The boundary coordinates of the retained candidate regions are converted into rectangular bounding boxes in the image coordinate system of this viewpoint, forming pedestrian target detection boxes. This conversion process needs to consider the size ratio between the feature map and the original image. Assuming the feature map size is 20×15 and the original image size is 640×480, the horizontal scaling ratio is 32, and the vertical scaling ratio is also 32. Therefore, the original image coordinates corresponding to the coordinates (fx, fy) on the feature map are (fx×32, fy×32).

[0084] By performing binary classification prediction on each pedestrian target detection box, the probability value of the pedestrian target detection box belonging to the pedestrian's category is output. The classification module also consists of two fully connected layers. The first layer takes a 512-dimensional feature vector as input and outputs a 256-dimensional intermediate feature. The second layer takes a 256-dimensional intermediate feature as input and outputs a 2-dimensional vector, which represents the probability of the background and the pedestrian respectively. After normalization, the probability value of the detection box belonging to the pedestrian's category is obtained, with a value range of [0, 1].

[0085] The localization quality assessment value is obtained by calculating the degree of overlap or the uncertainty of the boundary deviation between the detection box boundary and the ground truth bounding box. During the training phase, the intersection-union ratio (IUR) of the detection box and the ground truth bounding box can be calculated using labeled data as a measure of localization quality. During the inference phase, the localization quality can be estimated by the confidence score of the bounding box regression. The localization quality assessment value is usually in the range of [0, 1], with a larger value indicating higher localization quality.

[0086] The detection score of a pedestrian target detection box is obtained by combining the category prediction probability and the localization quality assessment value. A simple multiplication method can be used: Detection Score = Category Predicted Probability × Localization Quality Assessment Value. For example, if the category prediction probability of a detection box is 0.95 and the localization quality assessment value is 0.85, the final detection score is 0.8075. Alternatively, a weighted average can be used: Detection Score = w1 × Category Predicted Probability + w2 × Localization Quality Assessment Value, where w1 and w2 are weighting coefficients, and w1 + w2 = 1. For example, if w1 = 0.7 and w2 = 0.3, the detection score is 0.7 × 0.95 + 0.3 × 0.85 = 0.92. This method considers both classification accuracy and localization precision, resulting in a more comprehensive detection score that reflects the reliability of the detection results.

[0087] In one optional implementation, for each pedestrian target location, the spatial position deviation and motion speed consistency with each trajectory in the trajectory set of the previous moment are calculated to obtain a preliminary association cost. Based on the preliminary association cost, a confidence candidate association set is obtained by screening, including:

[0088] Obtain the predicted position and predicted velocity of each trajectory in the trajectory set of the previous moment;

[0089] Calculate the Euclidean distance between the current pedestrian target position and the predicted position of each trajectory at the previous time to obtain the measure of spatial position deviation; in the trajectory set at the previous time, select the trajectory with the closest spatial distance to the pedestrian target position as the candidate matching position; calculate the current movement speed based on the current pedestrian target position and the candidate matching position; calculate the velocity vector angle and velocity amplitude difference between the current movement speed and the predicted speed of each trajectory at the previous time, and combine them to obtain the measure of movement speed consistency.

[0090] The initial correlation cost is obtained by combining the metric of the spatial position deviation with the metric of the consistency of the motion speed.

[0091] For each pedestrian target location in the current location set, retain the pedestrian target location-trajectory pairs with a preliminary association cost less than a preset cost threshold, and form the confidence candidate association set by retaining all the pedestrian target location-trajectory pairs.

[0092] To obtain the predicted position and velocity of each trajectory in the trajectory set from the previous time step, assuming that N trajectories have been maintained at the previous time step t-1, the predicted position of the i-th trajectory is represented as (x_pred_i, y_pred_i), and the predicted velocity is represented as (vx_pred_i, vy_pred_i). This prediction information can be obtained through a Kalman filter or other prediction models. For example, for trajectory 1, its predicted position is (125.3, 78.6), and its predicted velocity is (2.1, 1.5); for trajectory 2, its predicted position is (156.8, 92.4), and its predicted velocity is (-1.8, 0.7).

[0093] At the current time t, M pedestrian target positions are detected. The j-th pedestrian target position is denoted as (x_cur_j, y_cur_j). The Euclidean distance between each pedestrian target position at the current time and the predicted position of each trajectory at the previous time is calculated to obtain a spatial position deviation metric. The Euclidean distance can be calculated by calculating the straight-line distance between two points. For pedestrian target position j and trajectory i, the spatial position deviation metric dist_pos_ij is the Euclidean distance between the coordinates of the two points. For example, if the currently detected pedestrian position is (128.5, 80.2), and the Euclidean distance between it and the predicted position of trajectory 1 (125.3, 78.6) is 4.2, the spatial position deviation metric is 4.2.

[0094] In the trajectory set from the previous moment, the trajectory with the closest spatial distance to the current pedestrian target position is selected as the candidate matching position. Assuming that for pedestrian target j, the found candidate matching trajectory is trajectory k, and its predicted position is (x_pred_k, y_pred_k), the current motion velocity is calculated based on the current pedestrian target position and the candidate matching position. The current motion velocity can be obtained by calculating the position difference and considering the time interval, i.e., vx_cur_j = (x_cur_j - x_pred_k) / dt, vy_cur_j = (y_cur_j - y_pred_k) / dt, where dt is the time interval, typically the time difference between adjacent frames. For example, if dt is 0.1 seconds, the pedestrian target position is (128.5, 80.2), and the candidate matching trajectory prediction position is (125.3, 78.6), then the current speed is calculated as ((128.5-125.3) / 0.1, (80.2-78.6) / 0.1) = (32, 16).

[0095] The angle between the velocity vectors and the difference in velocity amplitude between the current velocity and the predicted velocity of each trajectory at the previous moment are calculated, and combined to obtain a measure of velocity consistency. The angle between the velocity vectors can be obtained by calculating the dot product of the two velocity vectors, dividing by the product of their magnitudes, and then taking the inverse cosine. The difference in velocity amplitude can be obtained by calculating the absolute value of the difference in the magnitudes of the two velocity vectors. For pedestrian target j and trajectory i, the angle between the velocity vectors is angle_ij, and the difference in velocity amplitude is mag_diff_ij. For example, if the current velocity is (32, 16) and the predicted velocity of trajectory 1 is (2.1, 1.5), then the angle between the velocity vectors is approximately 15 degrees, and the difference in velocity amplitude is approximately 33.5.

[0096] The metric for motion speed consistency, dist_vel_ij, can be obtained by weighted combination of the angle between velocity vectors and the difference in velocity magnitude, i.e., dist_vel_ij = w1 × angle_ij + w2 × mag_diff_ij, where w1 and w2 are weight parameters. For example, if w1 = 0.7 and w2 = 0.3, then the metric for motion speed consistency is 0.7 × 15 + 0.3 × 33.5 = 20.55.

[0097] The initial association cost is obtained by combining the measure of spatial position deviation with the measure of motion velocity consistency. The combination method can be a weighted summation, i.e., cost_ij = alpha × dist_pos_ij + beta × dist_vel_ij, where alpha and beta are weight parameters. For example, if alpha = 0.6 and beta = 0.4, then the initial association cost is 0.6 × 4.2 + 0.4 × 20.55 = 10.74.

[0098] For each pedestrian target location in the current location set, pedestrian target location-trajectory pairs with an initial association cost less than a preset cost threshold are retained. For example, if the preset cost threshold is 15, the pairing of pedestrian target location j and trajectory i in the above example will be retained (because 10.74 < 15). All retained pedestrian target location-trajectory pairs constitute a confidence candidate association set. These pairs represent a location-trajectory correspondence that is considered to be the same pedestrian target at the current time with a high confidence level.

[0099] The above processing effectively associates the currently detected pedestrian location with existing trajectories, while considering the consistency of spatial location and motion characteristics, providing a reliable association basis for subsequent pedestrian trajectory tracking. In practical applications, various threshold and weight parameters can be adjusted according to the specific scene characteristics to obtain optimal tracking performance.

[0100] In one optional implementation, an optimal matching algorithm is executed on the confidence cost matrix to obtain a confidence association result. The identity identifier of the pedestrian target location that is successfully matched in the confidence association result is inherited by the corresponding trajectory. The state vector of the corresponding trajectory is updated by:

[0101] By iteratively finding the row and column allocation scheme that minimizes the sum of costs for all matching pairs in the confidence cost matrix, a one-to-one matching relationship between the pedestrian target location and the trajectory is obtained, and the one-to-one matching relationship is used as the confidence association result.

[0102] Extract the pairing relationship between the successfully matched pedestrian target location and the corresponding trajectory from the confidence association results. For each successfully matched pedestrian target location, inherit the identity identifier of the corresponding trajectory.

[0103] For each successfully matched trajectory, the state vector of the trajectory at the previous time step is obtained, and the position of the successfully matched pedestrian target is taken as the observation position at the current time step. The position update amount is calculated based on the observation position at the current time step and the position component in the state vector at the previous time step. The current observation velocity is calculated based on the observation position at the current time step and the historical position of the corresponding trajectory at the previous time step. The current observation velocity is fused with the velocity component in the state vector at the previous time step to obtain the velocity update amount.

[0104] The position update amount and the velocity update amount are combined to form a state update vector, and the state vector of the previous moment is fused with the state update vector to obtain the updated state vector.

[0105] like Figure 2 As shown, the method includes:

[0106] The confidence cost matrix is ​​processed using an optimal matching algorithm. Each element in this matrix represents the matching cost between a specific pedestrian target location and a specific trajectory. The Hungarian algorithm is used as the optimal matching algorithm, which can find the global optimal solution in many-to-many matching problems. In practical applications, the confidence cost matrix has a dimension of M×N, where M represents the number of pedestrian targets detected in the current frame, and N represents the number of existing trajectories. For example, assuming that 5 pedestrian target locations are detected in the current frame and there are 4 existing trajectories, the confidence cost matrix has a dimension of 5×4. The specific cost values ​​are: [[0.15, 0.82, 0.75, 0.63], [0.72, 0.21, 0.56, 0.74], [0.68, 0.75, 0.12, 0.45], [0.32, 0.85, 0.39, 0.18], [0.52, 0.43, 0.95, 0.34]].

[0107] Through iterative calculations, the Hungarian algorithm finds the row and column allocation scheme that minimizes the total cost of matching. The algorithm first performs row and column subtraction operations on the matrix, subtracting the minimum value of each row and column to include as many zero elements as possible. Then, it performs row and column covering operations, covering all zero elements in the matrix with the minimum number of lines. If the number of covered lines equals the matrix dimension, the optimal solution is found; otherwise, the uncovered elements are adjusted, and the above steps are repeated until the optimal solution is found.

[0108] For the example above, the matching relationships obtained after the algorithm is executed are as follows: pedestrian target 1 matches trajectory 1, pedestrian target 2 matches trajectory 2, pedestrian target 3 matches trajectory 3, pedestrian target 4 matches trajectory 4, and pedestrian target 5 fails to match. This matching relationship ensures that the overall cost is minimized, i.e., 0.15 + 0.21 + 0.12 + 0.18 = 0.66.

[0109] After obtaining the confidence level association results, the pairing relationship between successfully matched pedestrian target locations and their corresponding trajectories is extracted. For each successfully matched pedestrian target location, the identity identifier of the corresponding trajectory is inherited by that pedestrian target location. For example, if pedestrian target 1 matches the trajectory with trajectory ID T_001, then the identity identifier of pedestrian target 1 is set to T_001. This ensures the consistency of target identity across frames, enabling continuous tracking.

[0110] For each successfully matched trajectory, its state vector needs to be updated. The state vector typically contains two components: position and velocity. The state vector of the trajectory at the previous time step is obtained. For example, the state vector of trajectory T_001 at the previous time step t-1 is [x_{t-1}, y_{t-1}, v_x_{t-1}, v_y_{t-1}], where x_{t-1} and y_{t-1} represent the position coordinates, and v_x_{t-1} and v_y_{t-1} represent the velocity components in the x and y directions, respectively.

[0111] The location of the successfully matched pedestrian target is taken as the observation location at the current time t, denoted as [x_obs, y_obs]. The position update is calculated based on the observation location at the current time and the position components in the state vector at the previous time. The position update can be simply represented as the difference between the observation location and the predicted location, i.e., [x_obs - x_{t-1}, y_obs - y_{t-1}].

[0112] Calculate the current observation velocity. This can be achieved by comparing the current observation position with the historical position of the trajectory. Assuming the position of trajectory T_001 in the previous k frames is recorded as [x_{tk}, y_{tk}], the current observation velocity can be calculated as [(x_obs - x_{tk}) / (k×dt), (y_obs - y_{tk}) / (k×dt)], where dt represents the time interval between two adjacent frames, usually set to 1 / frame rate. For example, if the frame rate is 30fps, then dt = 1 / 30 seconds.

[0113] The velocity update is obtained by fusing the current observed velocity with the velocity component in the state vector of the previous time step. The velocity fusion can be performed using a weighted average method, i.e., new velocity = α × observed velocity + (1-α) × historical velocity, where α is a weighting coefficient that can be dynamically adjusted according to the reliability of the observation, and its value is usually in the range of [0.3, 0.7]. For example, if the observation is relatively reliable, α can be taken as 0.6, otherwise α can be taken as 0.4.

[0114] The position update and velocity update are combined to form the state update vector [Δx, Δy, Δv_x, Δv_y]. The state vector from the previous time step is then fused with the state update vector to obtain the updated state vector [x_t, y_t, v_x_t, v_y_t]. The fusion process can employ the Kalman filter method, considering the uncertainties of the predicted and observed states to provide the optimal estimate. Specifically, the weights β_p for position updates and β_v for velocity updates can be set. The updated state vector is calculated as [x_{t-1} + β_p×Δx, y_{t-1} + β_p×Δy, v_x_{t-1} + β_v×Δv_x, v_y_{t-1} + β_v×Δv_y], where the values ​​of β_p and β_v typically range from [0.5, 0.9], reflecting the confidence level in the new observation.

[0115] Through the above steps, the optimal matching of the confidence cost matrix, the inheritance of pedestrian target identity, and the updating of trajectory state vector are completed, providing reliable identity consistency and motion state estimation for pedestrian tracking tasks.

[0116] In one optional implementation, motion prediction is performed on each of the continuous spatiotemporal motion trajectories to obtain the predicted state vector and state distribution covariance matrix for the next moment, including:

[0117] For each of the continuous spatiotemporal motion trajectories, extract the state vector of the corresponding trajectory at the current moment;

[0118] Extract the position sequence and timestamp sequence from the trajectory history of the continuous spatiotemporal motion trajectory. Based on the most recent consecutive position points and corresponding timestamps in the position sequence, calculate the historical acceleration sequence by the ratio of the velocity difference between adjacent moments to the time interval.

[0119] Based on the historical acceleration sequence, the acceleration statistical features are calculated. Based on the position and velocity components in the current state vector, the mean of the acceleration statistical features is used as the predicted acceleration. The predicted position and predicted velocity at the next moment are calculated and combined to form the predicted state vector at the next moment.

[0120] Based on the variance of the acceleration statistical characteristics and the observation error statistics of the trajectory at historical moments, the process noise covariance and the observation noise covariance are calculated. Based on the process noise covariance and the observation noise covariance, the uncertainty distribution of the predicted state vector at the next moment is calculated. The uncertainty distribution is represented in the form of the state distribution covariance matrix.

[0121] For each continuous spatiotemporal motion trajectory in the road network, the state vector St of that trajectory at the current time t is extracted from the database. The state vector St typically contains position components (xt, yt) and velocity components (vxt, vyt), where (xt, yt) represents the two-dimensional coordinates of the trajectory object at the current time t, and (vxt, vyt) represents the velocity components of the object in the x and y directions. For example, the state vector of a vehicle at time t=120 seconds can be represented as St=[100.5 m, 200.3 m, 15.2 m / s, 5.8 m / s], indicating that the vehicle is located at coordinates (100.5, 200.3), with a velocity of 15.2 m / s in the x direction and 5.8 m / s in the y direction.

[0122] To calculate the acceleration information required for prediction, the most recent n position points and their corresponding timestamps are extracted from the historical trajectory data. Specifically, for the position sequence {(xt-n△t, yt-n△t), (xt-(n-1)△t, yt-(n-1)△t), ..., (xt, yt)} and the corresponding timestamp sequence {tn△t, t-(n-1)△t, ..., t} within the time window [tn△t, t-(n-1)△t, ..., t], the historical acceleration sequence is obtained by calculating the ratio of the velocity difference between adjacent moments to the time interval. For example, the acceleration between times ti△t and t-(i-1)△t can be calculated as: ax(ti△t)=(vx(t-(i-1)△t)-vx(ti△t)) / △t and ay(ti△t)=(vy(t-(i-1)△t)-vy(ti△t)) / △t, where △t is the sampling time interval, typically 0.1 seconds or 0.5 seconds. In this way, historical acceleration sequences {ax(tn△t), ax(t-(n-1)△t), ..., ax(t-△t)} and {ay(tn△t), ay(t-(n-1)△t), ..., ay(t-△t)} are generated.

[0123] Based on the obtained historical acceleration sequence, the statistical characteristics of the acceleration are calculated, including the mean acceleration μa and the variance σa. 2 The mean acceleration μa is obtained by averaging the elements in the historical acceleration sequence, and the variance σa is... 2 This is obtained by calculating the sum of the squares of the differences between each acceleration value and the mean, divided by the sequence length. For example, for the x-direction acceleration sequence {0.5 m / s²}... 2 0.8 m / s 2 0.3 m / s 2 0.6 m / s 2 0.4 m / s 2 Its mean μax is 0.52 m / s. 2 variance σax2 It is 0.0296 meters. 2 / Second 4 .

[0124] Based on the position (xt, yt) and velocity (vxt, vyt) in the current state vector, and combining the mean μa from the acceleration statistics as the predicted acceleration, the predicted position and velocity at the next time step t+Δt are calculated. The predicted position is calculated as: x(t+Δt)predicted = xt + vxt × Δt + 0.5 × μax × Δt 2 The predicted value of y(t+Δt) is: yt+vyt×Δt+0.5×μay×Δt 2 The predicted velocity is calculated as follows: vx(t+Δt) prediction = vxt + μax × Δt and vy(t+Δt) prediction = vyt + μay × Δt. The predicted position and predicted velocity are combined to form the predicted state vector for the next moment, S(t+Δt) prediction = [x(t+Δt) prediction, y(t+Δt) prediction, vx(t+Δt) prediction, vy(t+Δt) prediction]. For example, if the current state is St = [100.5 m, 200.3 m, 15.2 m / s, 5.8 m / s], and the average acceleration μax = 0.5 m / s². 2 μay = 0.2 m / s 2 If the time interval Δt = 0.5 seconds, then the predicted state vector S(t + Δt) prediction = [108.275 m, 203.175 m, 15.45 m / s, 5.9 m / s].

[0125] To describe the uncertainty of the predicted state, the state distribution covariance matrix is ​​calculated. The variance σa is based on the statistical characteristics of acceleration. 2 and historical observation error statistics σo 2 Calculate the process noise covariance Q and the observation noise covariance R. The process noise covariance Q reflects the uncertainty in the state transition process, and its value is related to the acceleration variance σa. 2 The acceleration variance σax is related to the time interval Δt and the system dynamics model. For example, for a two-dimensional constant acceleration model, when Δt = 0.5 seconds, the acceleration variance σax is... 2 =0.03 meters 2 / Second 4 ,σay 2 =0.02 meters 2 / Second 4 When the process noise covariance matrix Q is obtained, a 4×4 process noise covariance matrix can be obtained, where the diagonal elements represent the predicted variances of position and velocity in the x and y directions.

[0126] The observation noise covariance R is obtained by analyzing the errors in historical observation data. For example, if the standard deviation of the historical position observation error is 0.5 meters and the standard deviation of the velocity observation error is 0.2 meters per second, a 4×4 observation noise covariance matrix R can be constructed, with diagonal elements of 0.25 meters. 2 0.25 meters 2 0.04 meters 2 / Second 2 0.04 meters 2 / Second 2 .

[0127] Based on the process noise covariance Q and the observation noise covariance R, the uncertainty distribution of the predicted state vector at the next time step is calculated through the state prediction equation, i.e., the state distribution covariance matrix P(t+Δt). This matrix describes the variance of each component of the predicted state and the covariance between them, reflecting the reliability of the prediction result. For the above example, if the current state covariance matrix P(t) is a diagonal matrix diag([0.2m) 2 0.2 meters 2 0.1 meters 2 / Second 2 0.1 meters 2 / Second 2 If the predicted state covariance matrix P(t+Δt) is a non-diagonal 4×4 matrix, then the diagonal elements are 0.31m and 0.31m respectively. 2 0.29 meters 2 0.14 meters 2 / Second 2 0.12 meters 2 / Second 2 The off-diagonal elements reflect the correlation between position and velocity predictions.

[0128] Through the above steps, motion prediction was successfully performed on each continuous spatiotemporal trajectory, and the predicted state vector and state distribution covariance matrix for the next moment were obtained, providing a reliable basis for subsequent trajectory planning and collision warning.

[0129] This invention provides a multi-target tracking system for multi-view scenarios, the system comprising:

[0130] The first unit is used to acquire pedestrian image data collected by multiple perspective acquisition devices at the same time;

[0131] The second unit is used to perform multi-view target detection on the pedestrian image data to obtain the set of positions of the pedestrian targets in a unified coordinate system at the current moment and the detection confidence.

[0132] The third unit is used to calculate the spatial position deviation and motion speed consistency between each pedestrian target position and each trajectory in the trajectory set of the previous time in the current position set, to obtain the preliminary association cost, to filter the candidate association set based on the preliminary association cost, to calculate the Mahalanobis distance of each pair in the candidate association set, and to form the confidence cost matrix.

[0133] The fourth unit is used to perform the optimal matching algorithm on the confidence cost matrix to obtain the confidence association result. The location of the pedestrian target that is successfully matched in the confidence association result inherits the identity identifier of the corresponding trajectory, updates the state vector of the corresponding trajectory, and appends the current position and timestamp to the trajectory history of the corresponding trajectory to form the continuous spatiotemporal motion trajectory of the pedestrian target.

[0134] The fifth unit is used to predict the motion of each of the continuous spatiotemporal motion trajectories, obtain the predicted state vector and the state distribution covariance matrix for the next moment, and use the predicted state vector and the state distribution covariance matrix as the trajectory set input for the next moment to realize continuous tracking of multiple pedestrian targets across time in multi-view scenarios.

[0135] A third aspect of the present invention provides an electronic device, comprising:

[0136] processor;

[0137] Memory used to store processor-executable instructions;

[0138] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0139] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0140] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-target tracking method for multi-view scenarios, characterized in that, include: Acquire pedestrian image data from multiple perspective acquisition devices at the same time; Multi-view target detection is performed on the pedestrian image data to obtain the set of pedestrian targets' positions and detection confidence in a unified coordinate system at the current moment; In the current location set, for each pedestrian target location, the spatial position deviation and motion speed consistency with each trajectory in the trajectory set of the previous time are calculated to obtain a preliminary association cost. Based on the preliminary association cost, a candidate association set with confidence is obtained, including: Obtain the predicted position and predicted velocity of each trajectory in the trajectory set of the previous moment; Calculate the Euclidean distance between the current pedestrian target position and the predicted position of each trajectory at the previous time to obtain a measure of the spatial position deviation; In the trajectory set of the previous moment, the trajectory with the closest spatial distance to the pedestrian's target position is selected as the candidate matching position. The current movement speed is calculated based on the pedestrian's target position at the current moment and the candidate matching position. The velocity vector angle and velocity amplitude difference between the current movement speed and the predicted speed of each trajectory at the previous moment are calculated and combined to obtain the metric value of the consistency of the movement speed. The initial correlation cost is obtained by combining the metric of the spatial position deviation with the metric of the consistency of the motion speed. For each pedestrian target location in the current location set, retain the pedestrian target location-trajectory pairs with a preliminary association cost less than a preset cost threshold. Construct the confidence candidate association set by combining all retained pedestrian target location-trajectory pairs. Calculate the Mahalanobis distance of each pair in the confidence candidate association set to form a confidence cost matrix. The optimal matching algorithm is executed on the confidence cost matrix to obtain the confidence association result. The identity identifier of the pedestrian target position that is successfully matched in the confidence association result is inherited by the corresponding trajectory. The state vector of the corresponding trajectory is updated. The current position and timestamp are appended to the trajectory history of the corresponding trajectory to form the continuous spatiotemporal motion trajectory of the pedestrian target. Motion prediction is performed on each of the continuous spatiotemporal motion trajectories to obtain the predicted state vector and state distribution covariance matrix for the next moment. The predicted state vector and the state distribution covariance matrix are used as the trajectory set input for the next moment to realize continuous tracking of multiple pedestrian targets across time in multi-view scenarios.

2. The method according to claim 1, characterized in that, Multi-view target detection is performed on the pedestrian image data to obtain the set of pedestrian target positions in a unified coordinate system at the current moment and the detection confidence, including: Perform pedestrian detection operation on each viewpoint image in the image data to obtain pedestrian target detection boxes in the coordinate system of each viewpoint image, and calculate the detection score based on the category prediction probability of the pedestrian target detection boxes; Based on the camera intrinsic and extrinsic parameter matrices of multiple viewpoint devices, coordinate transformation is performed on the pedestrian target detection boxes in the image coordinate systems of each viewpoint, and the pedestrian target detection boxes are projected from the image coordinate systems of each viewpoint to the common reference plane of the unified coordinate system to obtain the candidate positions of each pedestrian target in the unified coordinate system. The candidate positions are subjected to cross-view fusion processing. By calculating the spatial distance between the candidate positions obtained from different view projections, the candidate positions with a spatial distance less than a preset fusion threshold are assigned to the same pedestrian target. Multiple candidate positions assigned to the same pedestrian target are fused to obtain the fused position. The fused position is taken as an element in the set of pedestrian target positions in a unified coordinate system at the current moment. The maximum value of the detection scores of each view participating in the fusion is taken as the detection confidence corresponding to the fused position.

3. The method according to claim 2, characterized in that, For each viewpoint image in the image data, a pedestrian detection operation is performed to obtain pedestrian target detection boxes in the coordinate system of each viewpoint image. A detection score is then calculated based on the category prediction probability of the pedestrian target detection boxes, including: For each viewpoint image in the image data, feature extraction and target localization are performed; Multiple prior boxes with different scales and aspect ratios are preset at each location of the feature map obtained by feature extraction. The offset of the bounding box relative to the prior box is obtained by regression prediction. The coordinates of the prior box are adjusted using the bounding box offset to obtain the boundary position of the candidate region. The overlapping candidate regions are filtered and deduplicated using a non-maximum suppression algorithm. The boundary coordinates of the remaining candidate regions are converted into rectangular boxes in the image coordinate system of that viewpoint, forming the pedestrian target detection box corresponding to the candidate region. By performing binary classification prediction on each pedestrian target detection box, the probability value of the pedestrian target detection box belonging to the pedestrian category is output, thus obtaining the category prediction probability; The positioning quality assessment value is obtained by calculating the degree of overlap or the uncertainty of the boundary deviation between the detection box boundary and the actual target boundary. The predicted probability of the category is combined with the localization quality assessment value to obtain the detection score of the pedestrian target detection box.

4. The method according to claim 1, characterized in that, Execute the optimal matching algorithm on the confidence cost matrix to obtain the confidence association result. Inherit the identity identifier of the corresponding trajectory from the successfully matched pedestrian target positions in the confidence association result, and update the state vector of the corresponding trajectory, including: By iteratively finding the row and column allocation scheme that minimizes the sum of costs for all matching pairs in the confidence cost matrix, a one-to-one matching relationship between the pedestrian target location and the trajectory is obtained, and the one-to-one matching relationship is used as the confidence association result. Extract the pairing relationship between the successfully matched pedestrian target location and the corresponding trajectory from the confidence association results. For each successfully matched pedestrian target location, inherit the identity identifier of the corresponding trajectory. For each successfully matched trajectory, obtain the state vector of that trajectory at the previous time step, and use the location of the successfully matched pedestrian target as the observation location at the current time step; The position update amount is calculated based on the current observation position and the position component in the state vector of the previous time. The current observation velocity is calculated based on the current observation position and the historical position of the corresponding trajectory of the previous time. The current observation velocity is fused with the velocity component in the state vector of the previous time to obtain the velocity update amount. The position update amount and the velocity update amount are combined to form a state update vector, and the state vector of the previous moment is fused with the state update vector to obtain the updated state vector.

5. The method according to claim 1, characterized in that, Motion prediction is performed on each of the continuous spatiotemporal motion trajectories to obtain the predicted state vector and state distribution covariance matrix for the next moment, including: For each of the continuous spatiotemporal motion trajectories, extract the state vector of the corresponding trajectory at the current moment; Extract the position sequence and timestamp sequence from the trajectory history of the continuous spatiotemporal motion trajectory. Based on the most recent consecutive position points and corresponding timestamps in the position sequence, calculate the historical acceleration sequence by the ratio of the velocity difference between adjacent moments to the time interval. Based on the historical acceleration sequence, the acceleration statistical features are calculated. Based on the position and velocity components in the current state vector, the mean of the acceleration statistical features is used as the predicted acceleration. The predicted position and predicted velocity at the next moment are calculated and combined to form the predicted state vector at the next moment. Based on the variance of the acceleration statistical characteristics and the observation error statistics of the trajectory at historical moments, the process noise covariance and the observation noise covariance are calculated. Based on the process noise covariance and the observation noise covariance, the uncertainty distribution of the predicted state vector at the next moment is calculated. The uncertainty distribution is represented in the form of the state distribution covariance matrix.

6. A multi-target tracking system for multi-view scenarios, used to implement the method as described in any one of claims 1-5, characterized in that, include: The first unit is used to acquire pedestrian image data collected by multiple perspective acquisition devices at the same time; The second unit is used to perform multi-view target detection on the pedestrian image data to obtain the set of positions of the pedestrian targets in a unified coordinate system at the current moment and the detection confidence. The third unit is used to calculate the spatial position deviation and motion speed consistency between each pedestrian target position and each trajectory in the trajectory set of the previous time in the current position set, to obtain the preliminary association cost, to filter the candidate association set based on the preliminary association cost, to calculate the Mahalanobis distance of each pair in the candidate association set, and to form the confidence cost matrix. The fourth unit is used to perform the optimal matching algorithm on the confidence cost matrix to obtain the confidence association result. The location of the pedestrian target that is successfully matched in the confidence association result inherits the identity identifier of the corresponding trajectory, updates the state vector of the corresponding trajectory, and appends the current position and timestamp to the trajectory history of the corresponding trajectory to form the continuous spatiotemporal motion trajectory of the pedestrian target. The fifth unit is used to predict the motion of each of the continuous spatiotemporal motion trajectories, obtain the predicted state vector and the state distribution covariance matrix for the next moment, and use the predicted state vector and the state distribution covariance matrix as the trajectory set input for the next moment to realize continuous tracking of multiple pedestrian targets across time in multi-view scenarios.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 5.

Images