Target tracking post-processing method based on vehicle physical kinematics model and vehicle

Abstract

The present invention discloses a target tracking post-processing method and a vehicle based on a vehicle physical kinematic model, the method comprising: obtaining data information of a vehicle target; eliminating outliers in the data information; selecting a target vehicle for tracking; continuously and stably tracking the selected target vehicle; calculating parameters of the tracked target vehicle; judging whether the tracked target vehicle disappears, if so, ending the tracking, otherwise continuing the tracking. The present invention can correct certain errors and inaccurate results of detection according to the physical characteristics of the moving vehicle, handle missed detection and false detection, make the target tracking continuous and stable, and at the same time combine the kinematic characteristics based on the target vehicle movement with the vehicle movement information and detection information, and use pure visual scheme information to track vehicle-related targets, thereby reducing the inspection and data collection costs.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a target tracking post-processing method based on a vehicle physical kinematics model and a vehicle. Background Art

[0002] Target detection and tracking is an important research direction in the field of autonomous driving. The accuracy of target detection and tracking is closely related to the safety of autonomous driving. Ensuring the accuracy of target tracking can greatly avoid the possibility of collision between the autonomous driving vehicle and the target vehicle.

[0003] There are many relevant technical solutions for target tracking at this stage. For example, the invention patent application with publication number CN111932580A discloses a road 3D vehicle tracking method and system based on Kalman filtering and Hungarian algorithm. This method uses Kalman filtering for prediction and Hungarian algorithm for optimal matching to predict the tracking trajectory of the target. However, this method does not take into account the processing of errors that do not satisfy the Gaussian distribution due to other factors such as ranging, or the processing of tracking anomalies caused by abnormalities in the detection network; for example, the invention patent application with publication number CN112487919A discloses a 3D target detection and tracking method based on camera and lidar. This method uses lidar and camera to detect the target and matches and tracks the borders of the previous and next frames. This method requires the use of lidar to obtain the true value and does not fully utilize the target's motion and visual spatial information, thereby greatly increasing the detection and data acquisition costs. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a target tracking post-processing method and vehicle based on the physical kinematic model of the vehicle, which can correct certain errors and inaccurate results of detection according to the physical characteristics of the moving vehicle, deal with missed detections and false detections, make target tracking continuous and stable, and at the same time combine the kinematic characteristics of the target vehicle's motion with the vehicle's motion information and detection information, and use pure visual solution information to track vehicle-related targets, thereby reducing the inspection and data acquisition costs.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] The target tracking post-processing method based on the vehicle physical kinematic model includes the following steps:

[0007] Step 1) Obtain data information of vehicle target;

[0008] Step 2) Eliminate outlier points in the data information of step 1);

[0009] Step 3) Select the target vehicle to track;

[0010] Step 4) continuously and stably tracking the target vehicle selected in step 3);

[0011] Step 5) Calculating parameters of the target vehicle tracked in step 4);

[0012] Step 6) Determine whether the target vehicle has disappeared, if yes, execute step 7), if no, return to execute step 4);

[0013] Step 7) End.

[0014] Preferably, in step 1), the vehicle target is captured by a camera, and the data information of the vehicle target obtained includes: pixel coordinates of the vehicle target detection frame, target type, target number, and the number of the camera that captured the vehicle target.

[0015] Preferably, in step 3), it is determined whether the motion of the vehicle target satisfies the kinematic threshold value. If so, the vehicle target is tracked as the selected target vehicle and step 4) is executed. Otherwise, step 7) is executed.

[0016] Preferably, in step 3), the position information of two adjacent frames of the vehicle target is compared to determine whether the position relationship of the two adjacent frames of the vehicle target meets the kinematic threshold, and the determination is performed for three frames for each vehicle target.

[0017] Preferably, in step 4), the selected target vehicle is stably tracked, the physical parameters of the target vehicle are calculated using the tracking trajectory array of the target vehicle, and the physical parameters are reversely associated with the tracking trajectory array to achieve closed loop, mutual verification and filtering of the tracking trajectory array and the physical parameters.

[0018] Preferably, in step 4), the same target vehicle is further tracked for a second time by using a kinematic threshold value between two frames of the target vehicle and an intersection area of ​​detection frames between two frames of the target vehicle.

[0019] Preferably, in step 5), the target vehicle's parameter calculation includes speed and acceleration, and the calculation is combined with the state characteristic information of the target vehicle detection frame to solve the slight shaking of the target vehicle detection frame during the calculation process, and combined with the target vehicle's motion trend information to solve the ranging error.

[0020] Preferably, in step 6), when the target vehicle disappears for three frames, it is determined that the tracked target vehicle disappears and step 7) is executed. If the target vehicle disappears and is successfully re-matched within three frames, it is determined that the tracked target vehicle has not disappeared, and the process returns to step 4).

[0021] The present invention also provides a car, which adopts the above-mentioned target tracking post-processing method based on the vehicle physical kinematic model.

[0022] Compared with the prior art, the present invention collects data information of vehicle targets when tracking vehicle targets, then judges the collected data information, eliminates wild value points in the data information that obviously deviate from the true value, and then selects the target vehicle for tracking based on the eliminated data, so that certain errors and inaccurate results in detection can be corrected; at the same time, after the target vehicle is selected, the target vehicle is continuously tracked, and the parameters of the target vehicle are calculated according to the tracked data. At the same time, it is continuously judged whether the target vehicle disappears during the tracking process, so that continuous and stable tracking of the target vehicle can be achieved. At the same time, the kinematic characteristics based on the target vehicle's movement are combined with the vehicle's movement information and detection information, and the vehicle-related targets are tracked using pure visual solution information, thereby reducing the inspection and data collection costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the target tracking post-processing method based on the vehicle physical kinematic model of the present invention;

[0024] Figure 2 It is a schematic diagram of the tracking process principle of the target tracking post-processing method based on the vehicle physical kinematic model of the present invention;

[0025] Figure 3 It is a tracking process timing diagram of the target tracking post-processing method based on the vehicle physical kinematic model of the present invention;

[0026] Figure 4 Effect diagram after being processed by the target tracking post-processing method based on the vehicle physical kinematics model of the present invention;

[0027] Figure 5 This is an overall effect diagram after being processed using the target tracking post-processing method based on the vehicle physical kinematic model of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0029] In the following description, D represents distance, n represents the number of frames, v represents speed, and t represents the time difference between two frames.

[0030] In the attached Figure 2 In the left figure, the rectangle represents the vehicle, X represents the longitudinal distance, Y represents the lateral distance, and the coordinate system is formed by the rear axle of the vehicle. The triangles represent the target vehicles, v1 and v2 represent the speed of the target vehicles, t represents the time difference between two frames, and a and b represent the motion thresholds. Figure 2 In the right figure, the rectangle represents the detection box of the target vehicle, and the intersection part represents the intersection-over-union ratio.

[0031] Attached Figure 3 In it, it is assumed that the tracking target is A, v represents the speed at each time, and t represents the time difference between two frames.

[0032] The logic of the present invention is as follows: when the target is moving, the position of the target can be found at different times. In the absence of various interferences, the target positions at different times in the past period of time are connected to form the target's track. To a certain extent, the vehicle's short-term driving trajectory in the future can be predicted by the past track, and the predicted track can be used to verify the obtained track. However, due to the influence of process errors such as ranging detection at different times, a simple connection will obviously result in an unreal track, making the target appear to be doing Brownian motion. Therefore, it is necessary to process the target's track so that they form a continuous track, and to obtain information such as the target's speed and acceleration based on the track, so as to complete the task of tracking the vehicle target. It should be noted that the target tracking processing method of the present invention is not only applicable to vehicle tracking, but also to pedestrian tracking. In this specific embodiment, the tracking of a vehicle is used as an example for specific explanation:

[0033] As attached Figure 1 To the attached Figure 5 As shown, the target tracking post-processing method based on the vehicle physical kinematic model includes the following steps:

[0034] Step 1) Obtain the data information of the vehicle target, use the camera to collect the vehicle target, and obtain the input target frame of the vehicle target from the deep learning network of the detection. The data information of the vehicle target includes: the pixel coordinates of the upper left corner and the lower right corner of the vehicle target detection frame, the target type, the target number, and the number of the camera that collects the vehicle target. After obtaining the vehicle target, calculate the grounding point of the vehicle target, call the coordinate conversion function, and convert the obtained pixel coordinates into coordinates in the world coordinate system (the horizontal and vertical distances from the midpoint of the rear axle of the vehicle).

[0035] Step 2) Eliminate the outlier points in the data information of step 1). When the coordinate conversion in step 1) is completed, the distance between the vehicle target and the origin (the midpoint of the rear axle of the vehicle) is obtained, and the approximate position of the vehicle target in a certain frame can be preliminarily determined. However, due to the error of coordinate conversion and possible problems in detection, 5% of the data points often deviate seriously from the true value. Therefore, in this step, we eliminate seriously deviated data points through the position of the camera and the visible distance. For example, the X-direction coordinate of the front-view camera cannot be negative, the Y-direction coordinate of the left front camera target cannot be negative, etc. The farthest visible distance of the camera is 50m, so there will be no data beyond 100m (the maximum distance range is shown in the attached figure). Figure 2 (as shown in c in the middle left), the vehicle target data that is seriously deviated is directly discarded, and the data that meets the range enters step 3).

[0036] Step 3) Select the target vehicle for tracking and determine whether the vehicle target meets the kinematic threshold: compare the position information of the vehicle target in two adjacent frames to determine whether the position relationship of the vehicle target in two adjacent frames meets the kinematic threshold, and make a three-frame judgment for each vehicle target. If yes, the vehicle target is tracked as the selected target vehicle and execute step 4), otherwise execute step 7).

[0037] Specifically, in the reasonable vehicle target data remaining after the outlier point removal process, the vehicle target (ID) position information of the previous frame is stored in a container. When the vehicle target of the next frame is input, the same ID target stored in the container is compared to see whether it meets the kinematic threshold value, and three frames are verified for each vehicle target. For example, in most cases, the physical limit speed of a vehicle is assumed to be k meters per second. If the time difference between the previous and next frames is measured to be about t seconds at most, the maximum horizontal and vertical movement position of the vehicle in the previous and next frames will not exceed 2k*t, that is, the position difference between the previous and next frames of the same target cannot exceed the threshold value of 2k*t (that is, D n -D n-1 < 2k*t), if the vehicle target meets the selection condition, it will enter step 4 as the target vehicle), if the vehicle target does not meet the selection condition, the target data will be discarded and the attributes of the tracked vehicle target will not be output.

[0038] Step 4) Continuously and stably track the target vehicle selected in step 3): stably track the selected target vehicle, use the tracking trajectory array of the target vehicle to calculate the target vehicle's target speed, acceleration, length, width, heading angle and other physical parameters, and reversely associate the physical parameters with the tracking trajectory array to achieve a closed loop, mutual verification and filtering of the tracking trajectory array and the physical parameters; when there are many vehicles and the distance is close, the effect of comparing and tracking using only the attributes of the target vehicle is not good. At this time, the same target vehicle is also tracked for a second time through the threshold value between the two frames of the target vehicle and the intersection area of ​​the detection frame between the two frames of the target vehicle.

[0039] Specifically, for a target vehicle that has been stably tracked after three frames of confirmation, when the target vehicle data of each frame is input, all targets will be traversed for each target vehicle stored in the previous frame (because the target ID input by 2D detection may jump and misalign, as shown in the attached Figure 4 As shown in the figure, for the target vehicle stored in the previous frame and determined to be tracked, the target in the current frame satisfies the kinematic threshold value related to the relative speed of the target vehicle calculated in step 5) in the previous frame (the threshold value calculated by speed in the present invention is twice the product of the relative speed between the target vehicle and the vehicle and the time between the two frames, that is, Dn < 2 * v n-1 * t n , as attached Figure 2a and b in the figure), and for targets that meet the threshold value at the same time, the intersection area of ​​the target detection frames of the two frames before and after the 2D input target vehicle is judged. If the intersection and union ratio (IOU) of the two frames before and after is greater than a certain threshold value Z (as shown in the attached figure), Figure 2 As shown), the present invention is set to (95%), then it is considered to be the same target. If the input detection ID of the previous and next frames of a target vehicle is the same, it meets the set kinematic threshold value, and the intersection and union ratio meets the threshold condition, the target is tracked, and the attributes of the tracked target are output after this step.

[0040] Step 5) Calculate the parameters of the target vehicle tracked in step 4), including speed and acceleration, and combine the state characteristic information of the target vehicle detection frame during the calculation to solve the slight shaking of the target vehicle detection frame during the calculation process, and combine the motion trend information of the target vehicle to solve the ranging error.

[0041] Specifically, the speed of the target vehicle is calculated based on the stable tracking position. The speed of the target vehicle is obtained by dividing the position of the previous and next two frames by the interval between the two frames. If the speed does not change suddenly, the speed is stored in the cache array. If there is a sudden change, the speed is discarded and replaced with the speed of the previous moment. When five frames are stored, the average value is calculated, as shown in the attached figure. Figure 3 As shown. At the same time, when a new speed enters the last speed in the cache array, delete the first speed element in the cache array. If the target vehicle does not have a sudden change in speed, but the speed trend changes (from accelerating to decelerating), store the speed of the new trend in another three-element array. If the target speed is a decelerating trend in all three frames, output this speed. If there is only one or two frames of sudden changes, abandon the speed of these two frames. If the position of the target vehicle has not changed significantly, based on the calculation and judgment of the pixel values ​​of the target 2D frame for several consecutive frames, the target frame has only been shaking, then the relative speed of the target is 0, and the calculated speed value is input into step 4) for calculation;

[0042] Step 6) Determine whether the target vehicle has disappeared. If the target vehicle disappears, continue tracking for three frames to prevent the target vehicle from being temporarily missed or partially blocked. If the target vehicle appears again after three frames, continue tracking. Otherwise, store it in the cache matrix. If the target vehicle disappears completely after a period of time, stop tracking the target completely: as shown in the attached figure. Figure 3 As shown, when the target vehicle disappears for three frames, it is determined that the tracked target vehicle disappears and executes step 7); if the target vehicle disappears and is successfully re-matched within three frames, it is determined that the tracked target vehicle has not disappeared, and returns to execute step 4);

[0043] Specifically, if the intersection and union ratio of the three elements in step 4 (IDs of the previous and next frames, kinematic threshold, and intersection and union ratio) meets the requirements and the previous and next frames meet the kinematic threshold or have the same ID, the target vehicle will continue to be tracked and the target vehicle will be assigned the ID of the previous frame. If any two of the three conditions are not met or all three are not met, the target ID and parameters will be maintained for three frames, and the information of the target vehicle stored during stable tracking will be continuously compared within three frames. If a match can be successfully made within three frames, return to step 4), otherwise, the target will be abandoned and the target vehicle will disappear.

[0044] Step 7) End.

[0045] In addition, the present invention also provides a car, which adopts the above-mentioned target tracking post-processing method based on the vehicle physical kinematic model.

[0046] Compared with the prior art, the present invention collects data information of vehicle targets, then judges the collected data information, eliminates wild value points in the data information that obviously deviate from the true value, and then selects the target vehicle for tracking based on the eliminated data, so that certain errors and inaccurate results in detection can be corrected; at the same time, after the target vehicle is selected, the target vehicle is continuously tracked, and the parameters of the target vehicle are calculated according to the tracked data, and at the same time, it is continuously judged whether the target vehicle disappears during the tracking process, so that continuous and stable tracking of the target vehicle can be achieved, and at the same time, the kinematic characteristics based on the target vehicle movement are combined with the vehicle movement information and the detection information, and the vehicle-related targets are tracked using pure visual solution information (such as the attached Figure 4 and attached Figure 5 ), reducing inspection and data collection costs.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A target tracking post-processing method based on a vehicle physical kinematic model, characterized in that: The following steps are involved: Step 1) Obtain data information of vehicle target; Step 2) Eliminate outlier points in the data information of step 1); Step 3) Select the target vehicle to track; Determine whether the motion of the vehicle target meets the kinematic threshold value, compare the position information of the vehicle target in two adjacent frames, determine whether the position relationship of the vehicle target in two adjacent frames meets the kinematic threshold value, and make a three-frame judgment for each vehicle target. If yes, the vehicle target is tracked as the selected target vehicle and execute step 4), otherwise execute step 7); Step 4) continuously and stably tracking the target vehicle selected in step 3); The selected target vehicle is stably tracked, the physical parameters of the target vehicle are calculated using the tracking trajectory array of the target vehicle, and the physical parameters are reversely associated with the tracking trajectory array to achieve closed loop, mutual verification and filtering of the tracking trajectory array and the physical parameters, and the same target vehicle is tracked twice through the kinematic threshold value between two frames of the target vehicle and the intersection area of ​​the detection frame between two frames of the target vehicle; Step 5) Calculating parameters of the target vehicle tracked in step 4); Step 6) Determine whether the target vehicle has disappeared, if yes, execute step 7), if no, return to execute step 4); Step 7) End.

2. The target tracking post-processing method based on the vehicle physical kinematic model according to claim 1 is characterized in that: In step 1), the vehicle target is captured by a camera, and the data information of the vehicle target includes: the pixel coordinates of the vehicle target detection frame, the target type, the target number, and the number of the camera that captured the vehicle target.

3. The target tracking post-processing method based on vehicle physical kinematic model according to claim 1 is characterized in that: In step 5), the target vehicle's parameter calculation includes speed and acceleration, and the calculation is combined with the state characteristic information of the target vehicle detection frame to solve the slight shaking of the target vehicle detection frame during the calculation process, and combined with the target vehicle's motion trend information to solve the ranging error.

4. The target tracking post-processing method based on vehicle physical kinematic model according to claim 1 is characterized in that: In step 6), when the target vehicle disappears for three frames, it is determined that the tracked target vehicle disappears and step 7) is executed. If the target vehicle disappears and is successfully re-matched within three frames, it is determined that the tracked target vehicle has not disappeared, and the process returns to step 4).

5. A car, characterized in that: A target tracking post-processing method based on a vehicle physical kinematic model as described in any one of claims 1 to 4 is adopted.

Images