Image Multi-Target Vehicle Tracking and Trajectory Verification Method with a Car-Following Model

Abstract

The present invention provides an image multi-target vehicle tracking and trajectory verification method with a car-following model, including: S1, obtaining continuous visible light image sequence data in a traffic scene, as well as image tracking area section information and target detection box positions; S2, performing background compensation on the image sequence, and converting the target detection box data from the image coordinate system to the top-down Frenet coordinate system; S3, initializing the parameters of the target tracking algorithm, designing a short-term fan-shaped displacement domain to estimate the target movement range, and preliminarily screening candidate tracking boxes; S4, designing a shape and position discrimination weighted scoring system, updating the predicted tracking box position according to the target type, and calculating the kinematic score of the candidate tracking box; S5, designing a multi-target trajectory verification method for the traffic scene and correcting the tracking results. The present invention improves the correct association rate and trajectory accuracy of the multi-target tracking algorithm, enhances the interpretability of the multi-target association results, and is beneficial to improving the target perception ability of the intelligent transportation system.

Description

Technical Field

[0001] The present invention relates to multi-target tracking technology and traffic information perception method, and particularly relates to an image multi-target vehicle tracking and trajectory verification method equipped with a car-following model. Background Art

[0002] With the rapid development of social economy, the vehicle occupancy rate has gradually increased, and road congestion and driving safety have become key issues; in the intelligent transportation system, vehicle target detection and tracking based on image data obtained by visible light sensors is the current mainstream traffic information perception method; by using the real-time and accurate traffic flow statistics results to establish a perfect intelligent transportation analysis and decision-making system, the utilization rate of the current road can be accurately evaluated, and then effective traffic dispatching can be guided to minimize the occurrence of congestion and traffic accidents to the greatest extent;

[0003] With the development of basic traffic flow theory, many scholars have established many typical car-following models based on engineering, psychology and data-driven methods, such as Gipps model, Intelligent Driver Model (IDM) and Full Velocity Difference Model (FVD), etc., and some other improved models have been proposed based on these models; accurate car-following models can reflect the real motion of vehicles in traffic flow, provide microscopic kinematic constraints for target detection and tracking, and increase the accuracy of vehicle target trajectories in road traffic scenarios;

[0004] Multi-target tracking based on vehicles is a feasible and effective traffic information perception technology; in existing research, Chinese patent CN202110061609.X discloses a traffic flow statistical algorithm based on vehicle detection and multi-target tracking, which uses Kalman filter and Hungarian algorithm to achieve multi-target associated tracking, and Chinese patent CN201910550286.3 discloses a traffic vehicle information acquisition method based on Mask R-CNN, in which SORT method is used to track vehicles within ROI; however, existing tracking algorithms mostly adopt data association based on global detection results and perform target tracking from a macroscopic perspective according to the distribution characteristics of detection frames. This idea has a fast calculation speed and a high tracking success rate, but does not consider microscopic vehicle trajectory characteristics, and there are a large number of noises and false detections in the tracking results, and cumbersome manual review and verification work is required to improve the tracking accuracy;

[0005] Trajectory analysis based on microscopic traffic flow models is a feasible and effective method for traffic flow theory analysis. In existing research, Chinese Patent CN202011030216.4 uses neural networks to optimize the parameters of the car-following model and reconstructs vehicle driving trajectories with the optimized car-following model. Chinese Patent CN201810628447.1 uses a microscopic vehicle car-following model to predict vehicle trajectories and optimizes the driving trajectory interval according to the prediction results. However, existing microscopic traffic flow models are mostly used for the prediction and optimization of formed vehicle trajectories, and few studies apply microscopic traffic flow models to object tracking, participate in trajectory reconstruction and stitching, and optimize the accuracy of generated trajectories. Summary of the Invention

[0006] To overcome the deficiencies of the prior art, the present invention proposes an image multi-object vehicle tracking and trajectory verification method with a car-following model. This method first performs background compensation on the image sequence, converts the target position information from the image coordinate system to the Frenet coordinate system, then designs a short-term fan-shaped displacement domain to estimate the target movement range and preliminarily screen candidate tracking frames. Next, a shape and position discrimination weighted scoring system is designed to calculate the short-term association results of the targets. Finally, a multi-object trajectory verification method for traffic scenes is designed, and the car-following model is introduced to correct the vehicle trajectories, improving the trajectory accuracy and usability.

[0007] To solve the above technical problems, the present invention adopts the following technical solutions:

[0008] An image multi-object vehicle tracking and trajectory verification method with a car-following model, comprising the following steps:

[0009] S1. Obtain continuous visible light image sequence data in a traffic scene, as well as information on the road section of the image tracking area and the positions of target detection frames.

[0010] S2. Perform background compensation on the image sequence, fix the pixel space information of the image sequence, and implement coordinate conversion to convert the target detection frame data from the image coordinate system to the top-down Frenet coordinate system.

[0011] S3. Initialize the parameters of the target tracking algorithm, design a short-term fan-shaped displacement domain to estimate the target movement range, and preliminarily screen candidate tracking frames.

[0012] S4. Design a shape and position discrimination weighted scoring system, update the predicted tracking frame positions according to the target types, and calculate the kinematic scores of the candidate tracking frames; obtain the target similarity according to the cosine distance of the feature points of the tracking frames, calculate the shape scores of the tracking frames, and estimate the inter-frame association results according to the comprehensive shape and position discrimination scores.

[0013] S5. Design a multi-objective trajectory verification method for traffic scenarios, extract complete trajectories within the scenario, eliminate invalid trajectories, collect fragmented trajectories and substitute them into the car-following model according to two-end spatio-temporal information, complete the missing trajectories, search for incorrect tracking trajectories by combining traffic parameters and trajectory time-varying characteristics, and correct the tracking results.

[0014] Further, in S1,

[0015] The acquisition angle of the image sequence is top-down or oblique;

[0016] The acquisition device is stationary;

[0017] When acquiring at a top-down angle, the distance between the acquisition device and the ground is greater than 6 meters;

[0018] When acquiring at an oblique angle, the distance between the acquisition device and the ground is greater than 100 meters;

[0019] The position of the target detection box is obtained through common target detection algorithms, and the target accuracy rate is set to be above 70%.

[0020] Further, the target accuracy rate is the ratio of the number of positively detected targets to the total number of targets.

[0021] Further, in S2, the horizontal axis of the coordinate system is the lane line, and the vertical axis is perpendicular to the lane line. The calculation method for converting the image coordinate system to the Frenet coordinate system is as follows:

[0022]

[0023]

[0024] Among them, {x(t), f(x(t)), (t = 0, 1, 2,...)} is the vehicle position at time t; {s(t), g(s(t)), (t = 0, 1, 2,...)} is the projection point of the current vehicle position on the closest lane center curve; Δq(t) is the vehicle movement along the lane; m(q(t)) is the vehicle movement perpendicular to the lane.

[0025] Further, in the initialization of the tracking algorithm parameters in S3, the specific steps for setting the initial screening tracking box in the short-term fan-shaped displacement domain are as follows:

[0026] S301. Calculate the search half-angle θ of the fan-shaped displacement domain:

[0027]

[0028] Among them, θ 0 represents the physical limit deviation angle of the real target, V 0 represents the physical limit displacement of the real target, represents the average intersection over union of the detection box and the real position of the target, h0 Denote the height of the target detection box;

[0029] S302. Calculate the search radius V of the sector displacement domain:

[0030]

[0031] where w 0 is the width of the target detection box, and DT is the maximum displacement coefficient, which is determined comprehensively according to the target type and physical limit;

[0032] S303. Screen candidate tracking boxes within the sector area with a search radius of V and a search half-angle of θ, and retain the position and size information of the tracking boxes within the area.

[0033] Furthermore, in the shape and position discrimination scoring system of S4, the specific steps for calculating the kinematic score are as follows:

[0034] S401. Calculate the predicted tracking box position

[0035]

[0036] where a 1 to a 4 are fitting parameters, S sh is the scale transformation coefficient, is the detection box of the existing trajectory;

[0037] S402. Update the predicted tracking box position according to the tracking target type. For pedestrian targets, the updated predicted tracking box is:

[0038]

[0039] For vehicle and non-motor vehicle targets, the updated predicted tracking box is:

[0040]

[0041] S403. Use GIOU to calculate the kinematic score

[0042]

[0043] where C is the minimum circumscribed box of A and B.

[0044] Furthermore, in the shape and position discrimination scoring system of S4, the specific steps for calculating the shape score are as follows:

[0045] S411. Use the FAST operator to extract the target edge within the detection box. For a certain value l in the image pFor the pixel point p, draw a discrete circle with a radius of 3 centered on the pixel point p, number the pixel points on the circle, check the pixel points numbered 1, 5, 9, and 13, set the pixel value threshold t, if at least three of the four points meet:

[0046] l i >l p +t

[0047] Or:

[0048] l i <l p -t

[0049] Then the pixel point p is an edge corner point, where i ∈ {1, 5, 9, 13};

[0050] S412. For the set of corner points P i calculated in the detection box and the tracking box and P j , design a clustering method / + to merge the corner points, extract the contour key points among them. If there are other corner points within the discrete circle with a radius of 3 of a certain corner point, then form these corner points into a corner point cluster PM until no other corner points can be added to the corner point cluster. Let the number of circles required to connect these corner points be K, then the total number N of corner points in the merged cluster is:

[0051]

[0052] The merging rule for the corner points within the cluster is that the sum of the total Euclidean distances between the corner points is the largest, that is:

[0053]

[0054] where PM′ is the merged point cluster, PM q is another point cluster with N corner points, ||a, b|| 2 is the second norm of a and b, representing the Euclidean distance;

[0055] S413. For the point cluster PM′ i after merging the detection boxes, it is necessary to multiply the feature points by the scale transformation coefficient S sh to obtain the expected feature point cluster to overcome the scale inconsistency problem between the detection box and the tracking box. Specifically, for a point PM′ i,n within the point cluster with the centroid Q:

[0056]

[0057] S414. Calculate the shape score through cosine similarity

[0058]

[0059] where x and y are the relative coordinates of the points in the point cluster with the upper left corner point of the corresponding detection / tracking box as the origin; the cosine similarity between two points takes values in [0, 2];

[0060] S415. Update the scale transformation coefficient S according to the distance of the feature point cluster from the corresponding centroid sh ; Let the point cluster PM' of the detection box i and the point cluster PM' of the tracking box j The corresponding centroids are Q i and Q j , then:

[0061]

[0062] When a certain target is associated for the first time, there is no S obtained from the previous association sh Participate in the calculation, then set S sh = 1, that is, the default scale remains unchanged.

[0063] Further, in the multi-target trajectory verification method for traffic scenarios of S5, the specific steps of the trajectory correction algorithm based on the car-following model are as follows:

[0064] S501. Screen the missing trajectories that can be spliced according to the effective frame interval:

[0065] 0 < f < f u

[0066] 0 < Δx < V max *F

[0067] Δy < w h

[0068] Among them, (Δx, Δy) is the distance between two points at the break of the two selected broken trajectories, F represents the number of frames of broken missing; the two broken trajectories that meet this requirement are used as alternative broken trajectories for splicing;

[0069] S502. Substitute into the stimulus-response car-following model to fit the vehicle trajectory along the lane line direction:

[0070]

[0071] Among them, x n (t) is the coordinate in the vehicle forward direction, n is the serial number of the trajectory point to be completed, t is the time, T is the total duration of the trajectory to be completed, λ is determined by the target vehicle attributes and does not change with time, and can be calibrated by the spatio-temporal motion information of the head and tail of the trajectory to be matched and the position and speed information of the corresponding leading vehicle;

[0072] S503. Splice the target trajectory perpendicular to the lane line direction and the tracking box size according to the following rules:

[0073] y f = f 3 (x f )

[0074]

[0075]

[0076] where f represents the number of frames being spliced (f < F), and (x f , y f ) represents the coordinates of the completion point; (x e , y e , l e , w e ) represents the coordinate position, length, and width of the end point of the previous track segment, and (x s , y s , l s , w s ) represents the coordinate position, length, and width of the start point of the subsequent track segment.

[0077] The beneficial effects of the present invention include:

[0078] 1. The present invention introduces a microscopic traffic flow following model into the multi-object tracking algorithm. For common road traffic scenarios, in the Frenet coordinate system, a stimulus-response model is used to correct the microscopic trajectories of target vehicles, improving the accuracy of the multi-object tracking algorithm and the rationality of the generated trajectories, providing a new idea for vehicle-based multi-object tracking technology in traffic scenarios;

[0079] 2. The present invention sets up a shape and position discrimination weighted scoring system. According to the contour and kinematic laws of vehicle targets, a shape score and a kinematic score are designed to comprehensively correlate the target detection results in the image sequence, improving the correct association rate of the multi-object tracking algorithm and enhancing the interpretability of the multi-object association results;

[0080] 3. The present invention designs a trajectory classification and processing flow from the perspective of trajectory integrity verification to screen complete trajectories, eliminate invalid trajectories, and splice missing trajectories, increasing the number of complete trajectories and improving the utilization rate of target detection results, creating conditions for trajectory accuracy verification based on the microscopic traffic flow model. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 is a schematic diagram of the Frenet coordinate transformation of the present invention;

[0082] Figure 2 is a flowchart of the method of the present invention;

[0083] Figure 3 is a schematic diagram of the short-time fan-shaped displacement domain of the present invention;

[0084] Figure 4This is a schematic diagram of the projection transformation of the present invention. Detailed implementation manners

[0085] The following details the implementation manners of the present invention. Examples of the implementation manners are shown in the accompanying drawings. The implementation manners described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0086] As Figures 1-4 shown, an image multi-target vehicle tracking and trajectory verification method with a car-following model specifically comprises the following steps:

[0087] Step 1: Obtain continuous visible light image sequence data in a traffic scene, as well as information on the road section of the image tracking area and the position of the target detection box.

[0088] Step 2: Perform background compensation on the image sequence, fix the pixel space information of the image sequence, and implement coordinate transformation to convert the target detection box data from the image coordinate system to the top-down Frenet coordinate system:

[0089] 2.1 Perform background compensation on the image sequence and fix the pixel space information of the image sequence. The specific steps are as follows:

[0090] 2.1.1 For a given source image P, first perform horizontal and vertical differences on each pixel point to obtain difference images P x , P y , and obtain matrix M through Gaussian smoothing filtering:

[0091]

[0092] where (x r , y r ) is the sliding window pixel point after the original pixel (x, y) is translated by u and v pixels, and w u,v is the sliding window weight;

[0093] 2.1.2 Calculate determinants λ 1 , λ 2 from matrix M, and determine whether the pixel point is a corner point according to the pixel score c(x, y):

[0094]

[0095] 2.1.3 Screen the feature points according to the Euclidean distance, merge the feature points with relatively close distances, reduce the calculation amount of subsequent steps, and retain a total of N p feature points in each source image and record them as key points for generating the perspective matrix, where N p ≥4 is required because at least 4 pairs of matching points are needed to calculate the perspective matrix;

[0096] 2.1.4. Use the KNN algorithm to associate and match points. This algorithm is simple to implement and has a relatively fast speed. The specific steps are as follows:

[0097] (1) For a certain key point in the previous frame image Calculate the Euclidean distance between this point and all key points in the subsequent frame image ;

[0098] (2) Sort the Euclidean distances of the key point pairs to obtain the top K key point pairs with the smallest distances, denoted as

[0099] (3) Obtain all the corresponding of the key points in the front and rear frame images, and select the association result with the smallest total distance of all key point matches as the association output of the KNN algorithm;

[0100] (4) Calculate the slope k of the line connecting the matching point pair i i , find the median slope k m , set the slope threshold k th , and filter out the incorrect matching point pairs where |k i - k m | > k th ;

[0101] 2.1.5. Use the correct matching point pairs to generate a perspective transformation matrix, perform perspective transformation on the front and rear frame images, and obtain an image sequence after background compensation;

[0102] 2.2. Implement coordinate transformation. The specific steps are as follows:

[0103] 2.2.1. Implement image coordinate perspective conversion to convert the original image coordinates to top-down perspective image coordinates. The specific steps are as follows:

[0104] (1) Obtain the position information of the lane edge lines. Next, select two lines perpendicular to each lane in the earth coordinate system and record their positions. These two sets of information will be sent to the projection model to calculate the projection mapping matrix in this scene; The matrix is a transformation matrix calculated based on the fact that the imaging point on the projection plane, the actual plane target point, and the projection origin are always collinear, as Figure 4 shown;

[0105] (2) Perform projection transformation according to the perspective model, which is equivalent to superimposing a linear transformation and a translation transformation on specific coordinate positions. The specific perspective model is as follows:

[0106] [x′, y′, w′] = [u, v, w] × M

[0107]

[0108]

[0109]

[0110] Among them, u, v represent the original image coordinates, x, y are the image coordinates after perspective transformation, w, w′ represent the scale parameters, and M is the perspective transformation matrix;

[0111] 2.2.2, implement the conversion from the image's top-view coordinate system to the geodetic Frenet coordinate system. The specific steps are as follows:

[0112] (1) Establish the Frenet coordinate system and establish the transformation relationship according to the target image position:

[0113]

[0114]

[0115] Where {x(t), f(x(t)), (t=0,1,2,…)} is the vehicle position at time t; {s(t), g(s(t)), (t=0,1,2,…)} is the projection point of the current vehicle position on the closest lane center curve; Δq(t) is the vehicle motion along the lane; m(q(t)) is the vehicle motion perpendicular to the lane;

[0116] (2) Select a reference object of known length in the image, calculate the ratio coefficient between the geodetic distance and the image distance, and multiply the ratio coefficient of the mapping ratio matrix at its target position to complete the conversion between the image scale and the geodetic scale;

[0117] Step 3: Initialize the target tracking algorithm parameters, design a short-time fan-shaped displacement domain to estimate the target movement range, and preliminarily screen the candidate tracking frames. The specific steps are as follows:

[0118] 3.1, Initialize the multi-target tracking algorithm parameters Score TH 、N th ,θ,DT,t and S sh , where the first associated target S sh Set to 1 to create files for all targets in the starting frame;

[0119] 3.2, calculate the fan-shaped displacement domain search half angle θ:

[0120]

[0121] where θ 0 Indicates the actual target physical limit deflection angle, V 0 Indicates the real target physical limit displacement, represents the average intersection-over-union ratio between the detection box and the target’s true position, h 0 Indicates the height of the target detection box;

[0122] 3.3. Calculate the search radius V of the sector displacement domain:

[0123]

[0124] where w 0 is the width of the target detection box, and DT is the maximum displacement coefficient, which is determined comprehensively according to the target type and the physical limit;

[0125] 3.4. Screen candidate tracking boxes within the sector area with search radius V and search semi-angle θ, and retain the position and size information of the tracking boxes within the area;

[0126] Step 4: Design a geometric discriminant weighted scoring system, and estimate the inter-frame association result according to the comprehensive geometric discriminant score. The specific steps are as follows:

[0127] 4.1. Calculate the kinematic score of the candidate tracking box. The specific steps are as follows:

[0128] 4.1.1. Calculate the predicted tracking box position

[0129]

[0130] where a 1 to a 4 are fitting parameters, S sh is the scale transformation coefficient, is the detection box of the existing trajectory;

[0131] 4.1.2. Update the predicted tracking box position according to the tracking target type. For pedestrian targets, the updated predicted tracking box is:

[0132]

[0133] For vehicle and non-motor vehicle targets, the updated predicted tracking box is:

[0134]

[0135] 4.1.3. Use GIOU to calculate the kinematic score

[0136]

[0137] where C is the minimum bounding box of A and B. Compared with the intersection over union IOU, GIOU can measure the distance between A and B without overlapping parts, and has a wider applicability;

[0138] 4.2. Calculate the shape score of the candidate tracking box. The specific steps are as follows:

[0139] 4.2.1. Use the FAST operator to extract the edges of the objects within the detection box. For a pixel point p with a value of l in the image, draw a discrete circle with a radius of 3 centered at this pixel point, number the pixel points on the circle, and check the pixel points numbered 1, 5, 9, and 13. Set the pixel value threshold t. If at least three of the four points satisfy: p or:

[0140] l i >l p +t

[0141] or:

[0142] l i <l p -t

[0143] Then the pixel point p is an edge corner point, where i ∈ {1, 5, 9, 13};

[0144] 4.2.2. For the set of corner points P i and P j calculated in the detection box and the tracking box, use the distance criterion to merge the corner points and extract the key contour points among them. If there are other corner points within the discrete circle with a radius of 3 of a certain corner point, then form these corner points into a corner point cluster PM until no other corner points can be added to the corner point cluster. Let the number of circles required to connect these corner points be K, then the total number N of corner points within the merged cluster is:

[0145]

[0146] The merging rule for the corner points within the cluster is that the sum of the total Euclidean distances between the corner points is the largest, that is:

[0147]

[0148] where PM′ is the merged point cluster, PM q is another point cluster with N corner points, ||a, b|| 2 is the two-norm of a and b, representing the Euclidean distance;

[0149] 4.2.3. For the point cluster PM′ i after merging the detection boxes, it is necessary to multiply the feature points by the scale transformation coefficient S sh to obtain the expected feature point cluster to overcome the problem of scale inconsistency between the detection box and the tracking box. Specifically, for a point PM′ i,n within the point cluster with the center of gravity Q:

[0150]

[0151] 4.2.4. Calculate the shape score through cosine similarity

[0152]

[0153] Where x and y are the relative coordinates of the points in the point cluster with the upper left corner point of the corresponding detection / tracking box as the origin; the cosine similarity between two points ranges from [0, 2]. The smaller the cosine similarity value, the closer the distance between the two points. For the target, it means the more similar the target in the detection box and the tracking box.

[0154] 4.2.5, update the scale transformation coefficient S through the distance between the feature point cluster and the corresponding centroid sh ; Let the point cluster PM' of the detection box i and the point cluster PM' of the tracking box j have corresponding centroids Q i and Q j , respectively. Then:

[0155]

[0156] When a target is associated for the first time, there is no S obtained from the previous association sh to participate in the calculation. Then set S sh = 1, that is, the default scale remains unchanged;

[0157] 4.3, calculate the comprehensive score of shape and position discrimination:

[0158]

[0159] Among them represents the kinematic score, represents the shape score, and λ 1 and λ 2 are normalized weight coefficients for adjusting the importance of the two scores. When there are fewer scene targets and the targets are more scattered, the detection results are more accurate and the target movement is less interfered. λ 1 should be increased to increase the kinematic discrimination weight; when there are more scene targets and the target distribution is more dense, the target movement is more affected by occlusion, missed detection, etc. λ 2 should be increased to increase the shape discrimination weight and use the target features for association; Score ij ranges from [0, 1]. The higher the score, the greater the probability that a certain candidate tracking box is the correct solution; considering the influence of misdetected targets, set the threshold Score TH . If there is a tracking box with a score greater than Score TH , select the one with the maximum score as the association result. If there is no tracking box exceeding the threshold, the current association fails, and the predicted tracking box is used as the association result to participate in the next association;

[0160] 4.4, Update the target and trajectory status. If a target fails to be associated for 5 consecutive times, it is determined as a false detection, and all information of this target is deleted; if a target is successfully associated 5 times in total, the association result of this target is filed as a trajectory; if a certain trajectory fails to be associated for 5 consecutive times, it is determined that this trajectory ends and does not participate in subsequent associations.

[0161] Step 5: Implement trajectory verification and completion. The specific steps are as follows:

[0162] 5.1, According to the start and end positions of the trajectory, determine whether there is a complete overlap with other trajectories, and divide them into complete trajectories, false detection trajectories, and fragmented trajectories; no processing is done for complete trajectories; false detection trajectories are eliminated according to the following rules;

[0163]

[0164] where (x 1 , y 1 ) and (x end , y end ) represent the start and end positions of a trajectory, and p represents the frame length of the trajectory duration;

[0165] 5.2, Screen the spliceable fragmented trajectories according to the effective frame interval:

[0166] 0 < f < f u

[0167] 0 < Δx < V max *F

[0168] Δy < w h

[0169] where (Δx, Δy) is the distance between two points at the fragmentation of the two selected fragmented trajectories, and F represents the number of missing frames of fragmentation; the two fragmented trajectories that meet this requirement are used as alternative fragmented trajectories for splicing;

[0170] 5.3, Substitute into the stimulus-response car-following model to fit the vehicle trajectory along the lane line direction:

[0171]

[0172] where x n (t) is the coordinate in the vehicle's forward direction, n is the sequence number of the trajectory points to be completed, t is the time, T is the total duration of the trajectory to be completed, and λ is determined by the driver traffic characteristics, traffic volume characteristics, road attributes, etc. of the target vehicle and does not change with time, and can be calibrated through the spatio-temporal motion information at the head and tail of the trajectory to be matched and the position and speed information of the corresponding leading vehicle;

[0173] 5.4, Splice the trajectory perpendicular to the lane line direction of the target and the tracking box size according to the following rules:

[0174] y f = f 3 (x f )

[0175]

[0176]

[0177] where f represents the number of frames being spliced (f < F), (x f , y f ) represents the coordinates of the completion point; (x e , y e , l e , w e ) represents the coordinate position, length, and width of the end point of the previous track segment, and (x s , y s , l s , w s ) represents the coordinate position, length, and width of the start point of the subsequent track segment;

[0178] 5.5. Check whether there are subsequent sampled frames. If not, end this tracking algorithm and output all tracks. If so, update each parameter and continue the association calculation starting from 4.1.

[0179] The above embodiments are only used to illustrate the technical idea of the present invention, and the protection scope of the present invention cannot be limited thereby. Any modification made on the basis of the technical solution according to the technical idea proposed by the present invention falls within the protection scope of the present invention.

Claims

1. A method for image multi-target vehicle tracking and trajectory verification using a car-following model. It is characterized in that The steps include: S1, obtaining continuous visible light image sequence data in traffic scenes, road section information in the image tracking area, and the position of the target detection frame; S2, perform background compensation on the image sequence, fix the pixel space information of the image sequence, implement coordinate conversion, and convert the target detection frame data from the image coordinate system to the top-view Frenet coordinate system; S3, initialize the target tracking algorithm parameters, design a short-time fan-shaped displacement domain to estimate the target movement range, preliminarily screen the candidate tracking frame, and determine the fan-shaped displacement domain area according to the physical motion limit of the target; S4. Set up a shape and position discrimination weighted scoring system, update the predicted tracking frame position according to the target type, and calculate the kinematic score of the candidate tracking frame; Set the clustering method to filter the contour feature points in the candidate tracking frame, obtain the target similarity based on the cosine distance of the tracking frame feature points, and calculate the shape score of the tracking frame; estimate the inter-frame association result based on the comprehensive shape and position discrimination score. If there is a candidate tracking frame that meets the conditions, the association is successful and a trajectory is generated. If not, update the algorithm parameters and start searching for a matching tracking frame again from the fan displacement domain estimation in S3; S5. Set up a multi-target trajectory verification method for traffic scenes. First, design the end-matching trajectory classification rules, extract the complete trajectory in the scene, eliminate invalid trajectories, and collect broken trajectories; perform an extended domain search on both ends of the broken trajectory, bring the trajectory endpoints that meet the requirements into S3 for endpoint data association, and obtain the matched broken trajectory; bring the complete trajectory and the matched broken trajectory into the stimulus-response following model, complete the missing trajectory, and constrain the target traffic parameters to be within a reasonable range; combine the traffic parameters and the time-varying characteristics of the trajectory to search for the wrong tracking trajectory, and correct the tracking result.

2. The image multi-target vehicle tracking and trajectory verification method equipped with a car-following model according to claim 1, It is characterized in that In S1, The image sequence is collected at a top-down or oblique angle; The acquisition equipment is stationary; When collecting data from a bird's-eye view, the distance between the collection device and the ground is greater than 6 meters; When collecting data at an oblique angle, the distance between the collection device and the ground is greater than 100 meters; The target detection box position is obtained through the common target detection algorithm, and the target accuracy is set to above 70%.

3. The image multi-target vehicle tracking and trajectory verification method equipped with a car-following model according to claim 2, It is characterized in that The target accuracy rate is the ratio of the number of positively detected targets to the total number of targets.

4. The image multi-target vehicle tracking and trajectory verification method equipped with a car-following model according to claim 1, It is characterized in that The coordinate system in S2 has a horizontal axis that is the lane line and a vertical axis that is perpendicular to the lane line. The calculation method for converting the image coordinate system to the Frenet coordinate system is as follows: Where {x(t), f(x(t)), (t=0,1,2,…)} is the vehicle position at time t; {s(t), g(s(t)), (t=0,1,2,…)} is the projection point of the current vehicle position on the closest lane center curve; Δq(t) is the vehicle motion along the lane; m(q(t)) is the vehicle motion perpendicular to the lane.

5. A method for multi-target vehicle tracking and trajectory verification of images equipped with a car-following model according to claim 1, characterized in that, in the initialization of the tracking algorithm parameters in S3, the specific steps for setting the initial screening tracking box in the short-term fan-shaped displacement domain are as follows: S301. Calculate the search half-angle θ of the fan-shaped displacement domain: Among them, θ 0 represents the true target physical limit deflection angle, V 0 represents the true target physical limit displacement, represents the average intersection over union of the detection box and the true position of the target, h 0 represents the height of the target detection box; S302. Calculate the search radius V of the fan-shaped displacement domain: where w 0 is the width of the target detection box, and DT is the maximum displacement coefficient, which is determined comprehensively according to the target type and the physical limit; S303. Screen candidate tracking boxes within the fan-shaped area with a search radius V and a search half-angle θ, and retain the position and size information of the tracking boxes within the area.

6. A method for multi-target vehicle tracking and trajectory verification of images equipped with a car-following model according to claim 1, characterized in that, in the kinematic discriminant scoring system of S4, the specific steps for calculating the kinematic score are as follows: S401. Calculate the position of the predicted tracking box Among them, a 1 to a 4 are fitting parameters, S sh is the scale transformation coefficient, is the detection box of the existing trajectory; S402. Update the position of the predicted tracking box according to the type of the tracking target. For a pedestrian target, the updated predicted tracking box is as follows: For vehicle and non-motor vehicle targets, the updated predicted tracking box is: S403. Calculate the kinematic score using GIOU where C is the minimum bounding box of A and B.

7. A method for multi-target vehicle tracking and trajectory verification of images equipped with a car-following model according to claim 1, characterized in that, in the shape discriminant scoring system of S4, the specific steps for calculating the shape score are as follows: S411. Extract the target edge within the detection box using the FAST operator. For a pixel point p with a value of l in the image, draw a discrete circle with a radius of 3 centered on the pixel point p, number the pixel points on the circle, check the pixel points numbered 1, 5, 9, and 13, and set the pixel value threshold t. If at least three of the four points meet the following conditions: p For a pixel point p with a value of l in the image, draw a discrete circle with a radius of 3 centered on the pixel point p, number the pixel points on the circle, check the pixel points numbered 1, 5, 9, and 13, and set the pixel value threshold t. If at least three of the four points meet the following conditions: l i >l p +t Or: l i <l p -t Then the pixel point p is an edge corner point, where i ∈ {1, 5, 9, 13}; S412. For the set of corner points P calculated in the detection box and the tracking box i and P j , design a cluster clustering method to merge the corner points, extract the contour key points among them. If there are other corner points within a discrete circle with a radius of 3 for a certain corner point, then these corner points are grouped into a corner point cluster PM until no other corner points can be added to the corner point cluster. Let the number of circles required to connect these corner points be K, then the total number N of corner points in the merged cluster is: The merging rule for the in-cluster corner points is that the sum of the total Euclidean distances between the corner points is the largest, that is: Among them, PM′ is the merged point cluster, and PM q is another point cluster with N corner points, ||a, b|| 2 is the two-norm of a and b, representing the Euclidean distance; S413. For the point cluster PM' after the detection box merging i , it is necessary to multiply the feature points by the scale transformation coefficient S sh to obtain the expected feature point cluster to overcome the scale inconsistency problem between the detection box and the tracking box. Specifically, for a point PM' within the point cluster with the center of gravity Q i,n : S414. Calculate the shape score by cosine similarity where x and y are the relative coordinates of the points in the point cluster with the upper left corner point of the corresponding detection / tracking box as the origin; the cosine similarity between two points takes values in [0, 2]; S415. Update the scale transformation coefficient S according to the distance between the feature point cluster and the corresponding centroid sh ; Let the detected bounding box point cluster be PM′ i and the tracked bounding box point cluster be PM′ j with corresponding centroids Q i and Q j , respectively. Then: When a certain target is associated for the first time, there is no S obtained from the previous association sh to participate in the calculation, then set S sh = 1, that is, the default scale remains unchanged.

8. A method for multi-target vehicle tracking and trajectory verification of images equipped with a car-following model according to claim 1, characterized in that, in the multi-target trajectory verification method of S5, the specific steps of the trajectory correction algorithm based on the car-following model are as follows: S501. Screen the missing trajectories that can be spliced according to the effective frame interval: 0 < f < f u 0 < Δx < V max *F Δy < w h where (Δx, Δy) is the distance between the two points at the break of the two selected broken trajectories, and F represents the number of frames of broken missing; the two broken trajectories that meet this requirement are used as alternative broken trajectories for splicing; S502. Substitute into the stimulus-response car-following model to fit the vehicle trajectory along the lane line direction: where x n (t) is the coordinate in the vehicle's forward direction, n is the sequence number of the trajectory point to be completed, t is time, T is the total duration of the trajectory to be completed, λ is determined by the target vehicle attributes and does not change with time, and can be calibrated through the spatio-temporal motion information at the beginning and end of the trajectory to be matched and the position and speed information of the corresponding leading vehicle; S503. Splice the target trajectory perpendicular to the lane line direction and the tracking box size according to the following rules: y f = f 3 (x f ) Among them, f represents the number of frames being spliced (f < F), (x f , y f ) represents the coordinates of the completion point; (x e , y e , l e , w e ) represents the coordinate position and length and width of the end point of the previous track segment, (x s , y s , l s , w s ) represents the coordinate position and length and width of the start point of the subsequent track segment.

Images