A learning-based acceleration smoothing method
By employing a learning-based acceleration smoothing method in autonomous vehicles, and utilizing the Frenet coordinate system and temporal convolutional networks to predict smooth velocity sequences, the problem of acceleration change detection delay in autonomous vehicles is solved, enabling more accurate acceleration prediction and safety decision-making.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU ZHIJIA SCI & TECH CO LTD
- Filing Date
- 2023-05-22
- Publication Date
- 2026-06-05
Smart Images
Figure CN116750009B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and in particular to a learning-based acceleration smoothing method. Background Technology
[0002] With the development of autonomous driving technology, in order to safely and efficiently navigate complex traffic scenarios and make optimal decisions, autonomous vehicles need the ability to predict the future intentions and trajectories of surrounding vehicles. The history and current state of surrounding vehicles have a significant impact on the behavior planning of autonomous vehicles. Currently, the speed information of surrounding vehicles is mostly obtained from Doppler radar, which contains considerable equipment noise and environmental noise. The noise of acceleration calculated from such speed data is even an order of magnitude higher than the original speed noise, making it impossible for autonomous vehicles to determine whether a specific nearby vehicle is accelerating or decelerating. Among the various tasks performed by autonomous vehicles, following the vehicle in front and saving fuel are two important tasks, and accurate and smooth acceleration information of the vehicle in front can help autonomous vehicles better accomplish these tasks.
[0003] Among existing methods, real-time filtering and smoothing methods based on vehicle motion models are the most widely used, such as the Constant Turn Rate and Acceleration (CTRA) model. However, all such real-time filtering methods suffer from a problem: the smoother the result, the higher the delay in changes in speed and acceleration. A common scenario is that a vehicle in front suddenly brakes during a period of uniform acceleration; real-time filtering methods may require one second or longer to detect the change in the sign of acceleration. This poses a significant threat to the safety of autonomous driving. Summary of the Invention
[0004] In view of this, the present invention provides a learning-based acceleration smoothing method to solve the technical problem of excessively high detection delay of velocity and acceleration changes in the prior art.
[0005] This invention provides a learning-based acceleration smoothing method, comprising:
[0006] S1. Select the Frenet road coordinate system and collect tracking trajectory data of vehicles within a set range near the target vehicle;
[0007] S2. Extract driving features based on the tracking trajectory data of vehicles within a set range near the target vehicle;
[0008] S3. Based on the driving characteristics, for the target vehicle in the set frame, input the feature matrix of the first set number of frames in the past set frame into the temporal convolutional network, train it, and output the time series of past speed, current speed and future speed to obtain the predicted smooth speed series.
[0009] S4. The frames containing future velocities in the predicted smoothed velocity sequence are post-processed to obtain the acceleration of the current frame.
[0010] Furthermore, in S1, the tracking trajectory data of vehicles within a set range near the target vehicle includes 39 features: target vehicle state features, road state features, and surrounding vehicle interaction features.
[0011] In step S1, the tracking trajectory data of vehicles within a defined range near the target vehicle includes: target vehicle state characteristics, road state characteristics, and surrounding vehicle interaction characteristics.
[0012] The target vehicle state characteristics include: the distance of the target vehicle relative to the left and right boundaries of the lane, the s-direction velocity of the target vehicle relative to the lane, the l-direction velocity of the target vehicle relative to the lane, the magnitude of the steering angle of the target vehicle in the lane, and the difference between the vehicle's orientation direction and the tangential direction angle at the projection point of the vehicle in the lane.
[0013] The road condition characteristics include: the number of vehicles within 100 meters in front of and behind the target vehicle in the current lane and the two lanes to the left and right;
[0014] The surrounding vehicle interaction features are: the relative distance between the vehicle and the target vehicle in the s direction, the s-direction velocity of the vehicle relative to the target vehicle, and the l-direction velocity of the vehicle relative to the target vehicle.
[0015] In S2,
[0016] The driving data includes the driving speeds of vehicles within a defined range near the target vehicle.
[0017] Furthermore, in S3,
[0018] The past speed includes a smoothed speed of a first predetermined number of frames past.
[0019] The future speed includes a first set number of smooth speeds for the future frame.
[0020] Further, S3 includes:
[0021] S31. Based on the driving characteristics, input the feature matrix of the target vehicle of the set frame, which is the first set number of frames past the set frame, into the temporal convolutional network.
[0022] S32. Using the smoothed speed as the true value of the output value, the feature matrix of the first set number of frames past the set frame is trained to obtain the predicted smoothed speed output time series of past speed, current speed and future speed, and obtain the predicted smoothed speed sequence, wherein the smoothed speed includes the prediction results of the speed of a certain frame in different frames.
[0023] Furthermore, in S31, the size of the feature matrix is a first set number of frames passed x 39.
[0024] In S32, the smoothing speed is achieved offline using Savitzky-Golay Smoothing.
[0025] Furthermore, in step S32, the speed fusion is performed using a weighted average method, and the calculation of the weight for each frame includes the following steps:
[0026] a. Based on the standard deviation, the confidence level is described, and the difference between the predicted smoothed velocity and the true value used as annotation is calculated using statistical methods to obtain the standard deviation sequence;
[0027] b. Based on the standard deviation sequence, output the corresponding weight series according to the error propagation formula.
[0028] Furthermore, the corresponding weight sequence includes the following expression:
[0029] W = [1 / σ1] 2 ,1 / σ2 2 ,...,1 / σ m+1 2 ,...,1 / σ 2*m 2 ,1 / σ 2*m+1 2 ]
[0030] =[w1,w2,...,w m+1 ,...,w 2*m ,w 2*m+1 ]
[0031] Further, S4 includes:
[0032] S41. For the frames containing future velocities in the predicted smoothed velocity sequence, smooth the frame by a set window size;
[0033] S42. Based on the smoothing result of setting the window size, obtain the acceleration of the smoothed velocity.
[0034] Furthermore, the size of the window is twice the number of frames contained in the future speed plus one.
[0035] Furthermore, the acceleration of the smoothed velocity includes the following expression:
[0036] acc i =(v i+1 s -v i s ) / (t i+1 s -t i s )
[0037] Among them, acc i This indicates the acceleration of the current frame.
[0038] The advantages of this invention compared to the prior art are:
[0039] 1. This invention uses the Frenet coordinate system, which is more suitable for autonomous vehicles to make decisions on acceleration or lane changes based on roads.
[0040] 2. This invention uses a time series of output smooth speed to avoid the lack of connection between the output acceleration between frames, which would prevent the smoothing effect from being achieved.
[0041] 3. This invention uses offline smoothing to take into account the future trend of speed changes, so that there is no delay in real-time filtering when speed changes, and the results are more accurate.
[0042] 4. Using velocity as the output value in this invention provides greater flexibility in post-processing compared to using acceleration directly as the output value. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in this invention, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0044] Figure 1 This is a flowchart of a learning-based acceleration smoothing method provided in an embodiment of the present invention;
[0045] Figure 2 This is a schematic diagram of a learning-based acceleration smoothing method provided in an embodiment of the present invention;
[0046] Figure 3 This is a schematic diagram of the Frenet coordinate system provided in an embodiment of the present invention;
[0047] Figure 4This is a schematic diagram illustrating the selection of vehicles near the target vehicle provided in an embodiment of the present invention;
[0048] Figure 5 This is a schematic diagram comparing the effects of smoothed and unsmoothed speeds provided in an embodiment of the present invention;
[0049] Figure 6 This is a schematic diagram of the network structure of TCN provided in this embodiment of the invention. Detailed Implementation
[0050] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.
[0051] The following will describe in detail, with reference to the accompanying drawings, a learning-based acceleration smoothing method of the present invention.
[0052] Figure 1 This is a flowchart of a learning-based acceleration smoothing method provided in an embodiment of the present invention.
[0053] Figure 2 This is a schematic diagram of a learning-based acceleration smoothing method provided in an embodiment of the present invention.
[0054] like Figure 1 As shown, the acceleration smoothing method for this noise data includes:
[0055] S1. Select the Frenet road coordinate system and collect tracking trajectory data of vehicles within a set range near the target vehicle;
[0056] In step S1, the tracking trajectory data of vehicles within a set range near the target vehicle includes 39 features: target vehicle state features, road state features, and surrounding vehicle interaction features.
[0057] The target vehicle state characteristics include: the distance of the target vehicle relative to the left and right boundaries of the lane, the s-direction velocity of the target vehicle relative to the lane, the l-direction velocity of the target vehicle relative to the lane, the magnitude of the steering angle of the target vehicle in the lane, and the difference between the vehicle's orientation direction and the tangential direction angle at the projection point of the vehicle in the lane.
[0058] The road condition features include: the number of vehicles within 100 meters in front of and behind the target vehicle in the lane where the target vehicle is located and in the two lanes to the left and right, excluding the target vehicle;
[0059] The surrounding vehicle interaction features are: the relative distance between the vehicle and the target vehicle in the s direction, the s-direction velocity of the vehicle relative to the target vehicle, and the l-direction velocity of the vehicle relative to the target vehicle.
[0060] The vehicle tracking trajectory data is generated by the autonomous driving perception system during driving road tests. It includes low-precision maps, vehicle physical information such as vehicle type, length, width and height, vehicle position based on Frenet road coordinate system, lane, and speed. It is generated by fusion of perception data from cameras, LiDAR, millimeter-wave radar and other sensors to track the vehicle.
[0061] Figure 3 This is a schematic diagram of the Frenet coordinate system provided in an embodiment of the present invention.
[0062] The Frenet coordinate system uses the road's centerline as a reference line, defining the coordinate system using the tangent and normal directions of that reference line. The tangential distance S and lateral distance L of the vehicle relative to the road's centerline represent the vehicle's coordinates in the Frenet coordinate system.
[0063] The target vehicle's state characteristics:
[0064]
[0065] The road condition characteristics are as follows:
[0066]
[0067] The surrounding vehicle interaction features:
[0068] In the target vehicle's lane and the two lanes to its left and right, the surrounding vehicles closest to the target vehicle will directly affect the target vehicle's speed and acceleration changes. In existing algorithms, to keep the feature matrix length constant, only the surrounding vehicles within 100 meters in front of and behind the target vehicle in the three nearest lanes are considered.
[0069] Figure 4 This is a schematic diagram illustrating the selection of vehicles near the target vehicle provided in an embodiment of the present invention.
[0070] like Figure 4 As shown, there are eight vehicles around the target vehicle. There are three vehicles in lane 1, but only vehicle 2 will be considered because it is the only vehicle within 100 meters in front of and behind the target vehicle. For lane 2, vehicles 5 and 6 will be considered, even though vehicle 4 is closer to the target vehicle than vehicle 6, because in a lane we consider one vehicle in front of and behind the target vehicle as a reference point. For lane 3, vehicles 7 and 8 will be considered.
[0071] At this stage, modeling the vehicle interactions of the six closest vehicles to the target vehicle is sufficient. Five features are needed for each of the two closest vehicles in front and behind the target vehicle in that lane:
[0072]
[0073] There are a total of 39 features.
[0074] S2. Extract driving features based on the tracking trajectory data of vehicles within a set range near the target vehicle;
[0075] In S2,
[0076] The driving data includes the driving speeds of vehicles within a defined range near the target vehicle.
[0077] S3. Based on the driving characteristics, the target vehicle of the set frame is input into the feature matrix of the first set number of frames past the set frame to the temporal convolutional network. After training, the network outputs the time series of past speed, current speed and future speed to obtain the predicted smooth speed series.
[0078] In S3,
[0079] The past speed includes a smoothed speed of a first predetermined number of frames past.
[0080] The future speed includes a first set number of smooth speeds for the future frame.
[0081] S3 includes:
[0082] S31. Based on the driving characteristics, input the feature matrix of the target vehicle of the set frame, which is the first set number of frames past the set frame, into the temporal convolutional network.
[0083] S32. Using the smoothed speed as the true value of the output value, the feature matrix of the first set number of frames past the set frame is trained to obtain the predicted smoothed speed output time series of past speed, current speed and future speed, and obtain the predicted smoothed speed sequence, wherein the smoothed speed includes the prediction results of the speed of a certain frame in different frames.
[0084] Predictions are made based on each frame of the past speed and each frame of the current speed to obtain a predicted sequence of past speed, current speed and future speed, and the smoothed predicted speed is obtained.
[0085] The smoothed speed has undergone a noise reduction process to eliminate noise from the target vehicle during its operation, such as equipment noise, so as not to interfere with the driving data of vehicles near the target vehicle.
[0086] Learning-based methods require data labeling and training. In this invention, the model trained by this method does not directly output the acceleration of the target vehicle in the current frame, but rather outputs a smoothed time series of speeds. The reasons include:
[0087] (1) Directly output the acceleration of the target vehicle in the current frame. In this way, the output acceleration between frames is not related and cannot achieve the purpose of smoothing. (2) Use the original speed value after offline smoothing as the true value input to the network for training. It should be noted that this smoothing is fundamentally different from the real-time filtering mentioned earlier. Offline smoothing takes into account the future trend of speed change, so there will be no delay in speed change compared to real-time filtering. (3) Speed as the output value is more flexible in post-processing than acceleration directly as the output value.
[0088] Figure 5 This is a schematic diagram comparing the effects of smoothed and unsmoothed speeds provided in an embodiment of the present invention.
[0089] In S31, the size of the feature matrix is the first set number of frames passed x 39.
[0090] In S32, the smoothing speed is achieved offline using Savitzky-Golay Smoothing.
[0091] like Figure 5 As shown, from top to bottom, the first jagged line at the top represents the original velocity. The smoother line intersecting this first jagged line represents the velocity smoothed offline using Savitzky-Golay Smoothing (smoothing window size 21, polynomial degree 2). The first jagged line at the bottom represents the acceleration calculated from the original data difference, and the smoother line intersecting this first jagged line represents the acceleration calculated from the smoothed velocity difference. It can be seen that the blue line effectively captures the trend of velocity change. The ideal acceleration (acceleration calculated with future velocity information) should look like this.
[0092] This invention trains a neural network using the speed of the purple line as the true output value. For the target vehicle in the nth frame, the network input is the feature matrix of the past 10 frames. The specific matrix size is 10×39. The output is a smoothed speed time series of length 2×m+1, that is, the output is the smoothed speed of the next m frames and the smoothed speed of the past m frames. For example, in this method, m is 10.
[0093] In this invention, speed fusion adopts TCN, short for Temporal Neural Network, a type of temporal convolutional network as the backbone network. The use of TCN in this invention is based on its parallelism and stable gradient.
[0094] Figure 6 This is a schematic diagram of the network structure of TCN provided in this embodiment of the invention.
[0095] For the i-th frame, extract the feature matrix from the i-10th frame to the i-th frame. Input the matrix into a temporal convolutional neural network such as TCN and output the smooth speed from the i-th frame to the i+mth frame.
[0096] In step S32, the speed fusion is performed using a weighted average method, and the calculation of the weight for each frame includes the following steps:
[0097] a. Based on the standard deviation, the confidence level is described, and the difference between the predicted smoothed velocity and the true value used as annotation is calculated using statistical methods to obtain the standard deviation sequence;
[0098] Each frame outputs 2×m+1 frames of smoothed velocity predictions. The first to the mth positions of the output sequence represent past smoothed velocities, the (m+1)th position represents the current frame's smoothed velocity, and the (m+2)th to the 2×m+1th positions represent future smoothed velocities. Clearly, the past and current frame smoothed velocity predictions have corresponding original velocities in the feature matrix, while the future velocities are purely predictions based on the feature matrix, resulting in a significant difference in confidence levels. To ensure smooth continuity between acceleration frames generated by the fused velocities, it's necessary to consider how to fuse the smoothed velocity prediction sequence of each frame with past predictions. This method uses a weighted average method, with the weight for each frame calculated as follows.
[0099] This invention uses standard deviation to describe confidence level, and uses statistical methods to calculate the difference between the predicted smoothing rate and the true value used as annotation. The corresponding standard deviation sequence is as follows:
[0100] [σ1,σ2,...,σ m+1 ,...,σ 2*m ,σ 2*m+1 ]
[0101] For each bit of the sequence:
[0102]
[0103] v k For the i-th model prediction value, Let j be the labeled value (true value) of the j-th model, and n be the number of statistical data used.
[0104] b. Based on the standard deviation sequence, output the corresponding weight series according to the error propagation formula;
[0105] The corresponding weight sequence includes the following expression:
[0106] W = [1 / σ1] 2 ,1 / σ2 2 ,...,1 / σ m+1 2 ,...,1 / σ 2*m 2 ,1 / σ 2*m+1 2 ]
[0107] =[w1,w2,...,w m+1 ,...,w 2*m ,w 2*m+1 ]
[0108] Where W represents the corresponding weight sequence, and w_1, w_2, ..., w_(m+1), ..., w_(2*m), w_(2*m+1) represent the corresponding weights respectively.
[0109] The weight sequence is a fixed value.
[0110] Assume the smoothed velocity sequence after fusion of the (i-1)th frame is as follows:
[0111] V smooth =[v1 s v2 s , ..., v i-2 s v i-1 s , ..., v i+m-2 s v i+m-1 s ]
[0112] Its corresponding weight is
[0113] W smooth =[w1 s w2 s , ..., w i-2 s w i-1 s , ..., w i+m-2 s w i+m-1 s ]
[0114] After predicting the smooth velocity for the i-th frame, we need to update V. smooth The predicted value for the i-th frame is:
[0115] V i =[v i-m v i-m+1 , ..., v i-1 v i , ..., vi+m-1 v i+m ]
[0116] The corresponding weight is W.
[0117] Then W smooth Corresponding item update: w i-m+k s =w i-m+k s +w k+1 k = 0, 1, ..., 2m-1
[0118] Finally, v i+m and w 2*m+1 Push directly into V smooth and W smooth
[0119] Perform the same operation on each frame of the velocity prediction sequence to obtain the smoothed predicted velocity.
[0120] S4. The frames containing future velocities in the predicted smoothed velocity sequence are post-processed to obtain the acceleration of the current frame.
[0121] S4 includes:
[0122] S41. For the frames containing future velocities in the predicted smoothed velocity sequence, smooth the frame by a set window size;
[0123] S42. Based on the smoothing result of setting the window size, obtain the acceleration of the smoothed velocity.
[0124] The set window size is twice the number of frames contained in the future speed plus one.
[0125] For example, for the i-th frame,
[0126] V smooth =[v1 s v2 s , ..., v i-2 s v i-1 s , ..., v i+m-2 s v i+m-1 s This sequence contains m frames of future velocity information. Applying Savitzky-Golay Smoothing smoothing with a window size of 2*m+1 to this velocity sequence, the acceleration of the current frame can be calculated using the following formula:
[0127] acc i =(v i+1s -v i s ) / (t i+1 s -t i s )
[0128] The acceleration of the smoothed velocity includes the following expression:
[0129] acc i =(v i+1 s -v i s ) / (t i+1 s -t i s )
[0130] Among them, acc i This indicates the acceleration of the current frame.
[0131] This invention uses the Frenet coordinate system, which is more suitable for autonomous vehicles to make decisions on acceleration or lane changes based on roads; it uses a time series of output smooth speed to avoid the lack of connection between output acceleration between frames, which would prevent the smoothing effect; this invention considers the future trend of speed changes, so that there is no delay in real-time filtering of speed changes, resulting in more accurate results; using speed as the output value is more flexible in post-processing than using acceleration directly as the output value.
[0132] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0133] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0134] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A learning-based acceleration smoothing method, characterized in that, include: S1. Select the Frenet road coordinate system and collect tracking trajectory data of vehicles within a set range near the target vehicle; S2. Extract driving features based on the tracking trajectory data of vehicles within a set range near the target vehicle; S3. Based on the driving characteristics, for the target vehicle of the set frame, input the feature matrix of the first set number of frames past the set frame into the temporal convolutional network; After training, the system outputs time series of past, current, and future velocities to obtain a predicted, smoothed velocity sequence. S3 includes: S31. Based on the driving characteristics, input the feature matrix of the target vehicle of the set frame, which is the first set number of frames past the set frame, into the temporal convolutional network. S32. Using the smoothed speed as the true value of the output value, the feature matrix of the first set number of frames past the set frame is trained to obtain the predicted smoothed speed output time series of past speed, current speed and future speed, and obtain the predicted smoothed speed sequence, wherein the smoothed speed includes the prediction results of the speed of a certain frame in different frames. In step S32, the speed fusion is performed using a weighted average method, and the calculation of the weight for each frame includes the following steps: a. Based on the standard deviation, the confidence level is described, and the difference between the predicted smoothed velocity and the true value used as annotation is calculated using statistical methods to obtain the standard deviation sequence; b. Based on the standard deviation sequence, output the corresponding weight sequence according to the error propagation formula; The corresponding weight sequence includes the following expression: ； S4. The frames containing future velocities in the predicted smoothed velocity sequence are post-processed to obtain the acceleration of the current frame.
2. The acceleration smoothing method according to claim 1, characterized in that, In step S1, the tracking trajectory data of vehicles within a set range near the target vehicle includes 39 features: target vehicle state features, road state features, and surrounding vehicle interaction features. In step S1, the tracking trajectory data of vehicles within a defined range near the target vehicle includes: target vehicle state characteristics, road state characteristics, and surrounding vehicle interaction characteristics. The target vehicle state characteristics include: the distance of the target vehicle relative to the left and right boundaries of the lane, the s-direction velocity of the target vehicle relative to the lane, the l-direction velocity of the target vehicle relative to the lane, the magnitude of the steering angle of the target vehicle in the lane, and the difference between the vehicle's orientation direction and the tangential direction angle at the projection point of the vehicle in the lane. The road condition characteristics include: the number of vehicles within 100 meters in front of and behind the target vehicle in the current lane and the two lanes to the left and right; The surrounding vehicle interaction features are: the relative distance between the vehicle and the target vehicle in the s direction, the s-direction velocity of the vehicle relative to the target vehicle, and the l-direction velocity of the vehicle relative to the target vehicle. In S2, The driving data includes the driving speeds of vehicles within a defined range near the target vehicle.
3. The acceleration smoothing method according to claim 1, characterized in that, In S3, The past speed includes a smoothed speed of a first predetermined number of frames past. The future speed includes a first set number of smooth speeds for the future frame.
4. The acceleration smoothing method according to claim 1, characterized in that, In S31, the size of the feature matrix is the first set number of frames passed x 39. In S32, the smoothing speed is achieved offline using Savitzky-Golay Smoothing.
5. The acceleration smoothing method according to claim 1, characterized in that, S4 includes: S41. For the frames containing future velocities in the predicted smoothed velocity sequence, smooth the frame by a set window size; S42. Based on the smoothing result of setting the window size, obtain the acceleration of the smoothed velocity.
6. The acceleration smoothing method according to claim 5, characterized in that, The window size is twice the number of frames contained in the future speed plus one.
7. The acceleration smoothing method according to claim 6, characterized in that, The acceleration of the smoothed velocity includes the following expression: Among them, acc i This indicates the acceleration of the current frame.