A scene consistency vehicle trajectory prediction method based on road constraints

Abstract

The application discloses a kind of scene consistency vehicle trajectory prediction methods based on road constraint, comprising the following steps: obtaining the history state of target vehicle and vehicle around target vehicle and local high-precision map;Predict the future trajectory of target vehicle based on target vehicle history state, surrounding vehicle history state and local high-precision map and traffic rules;Obtain the lane center line that can be driven around target vehicle and equidistant discretization processing, simultaneously judge whether the minimum distance of the endpoint of predicted trajectory to drivable lane center line exceeds the set threshold;The position point of lane center line that minimum distance exceeds threshold and the first n trajectory points of predicted trajectory are used as control point, and the predicted trajectory that meets scene consistency is obtained by using m order Bezier curve fitting.The application uses the prior knowledge that vehicle should comply with traffic rules and drive in the vicinity of drivable lane center line, and the predicted vehicle trajectory is constrained, which can ensure that the predicted trajectory meets traffic rules and scene consistency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned vehicle trajectory prediction, in particular to a scene consistency vehicle trajectory prediction method based on road constraints. BACKGROUND

[0002] In recent years, artificial intelligence (AI) has made unprecedented development, especially in the field of unmanned vehicles which has great social and commercial value. Unmanned vehicle technology contains multiple technical links, especially the technology of predicting the future trajectory of surrounding vehicles based on perception information and map information, which is crucial for the safe, comfortable and efficient operation of unmanned vehicles.

[0003] In real traffic environment, the motion trajectory of a vehicle is not completely random, but is limited by traffic rules. However, current trajectory prediction methods based on deep learning do not explicitly consider prior traffic rules, resulting in predicted trajectories that violate traffic rules, collide with each other, and other inconsistencies with the scene. Therefore, it is an urgent problem to explicitly incorporate prior traffic rules into trajectory prediction methods to achieve high accuracy, low collision rate, and traffic rule-compliant vehicle trajectory prediction. In addition, existing trajectory prediction methods usually only emphasize precision and efficiency, often ignoring the ultimate goal of trajectory prediction, which is to make downstream tasks work better. For example, although the predicted trajectory of the target vehicle is very close to the real situation, the trajectory appears in an unfeasible area such as outside the road boundary, causing an embarrassing situation where downstream decision and planning tasks cannot be used. Therefore, it is a problem worth studying how to use road structure to explicitly post-process and constrain the predicted trajectory to meet the requirements of scene consistency while ensuring prediction accuracy. SUMMARY

[0004] To solve the above problems existing in the prior art, the present application proposes a scene consistency vehicle trajectory prediction method with prior knowledge guidance input and road structure constraint output, which can help unmanned vehicles make safer and more robust decisions.

[0005] To achieve the above purpose, the technical solution of the present application is as follows: a scene consistency vehicle trajectory prediction method based on road constraints, comprising the following steps:

[0006] A, obtaining vehicle historical state and local high-precision map information

[0007] A1, obtaining the historical state information of the target vehicle and the historical state information of the vehicles around the target vehicle using the environment perception system and the positioning system of the unmanned vehicle ​The superscript "target" indicates the target vehicle, the superscript "neighbors" indicates surrounding vehicles, and the subscript "t" = -T. h -T h +1, ..., 0 represent different historical moments, p t v represents the position at time t. t h represents the velocity at time t. t Indicates the orientation at time t, l t w represents the length dimension at time t. t This represents the width dimension at time t.

[0008] A2. Using the current location of the target vehicle described in step A1, obtain a high-precision local map of its surroundings: Map = {Crosswalk} info Lane info}, where Crosswalk info Lane indicates pedestrian crossing information. info This indicates lane centerline information.

[0009] B. Predicting the future trajectory of the target vehicle

[0010] B1. Encode the target vehicle's historical state obtained in step A1 using a Long Short-Term Memory Neural Network. Historical status of vehicles surrounding the target vehicle Obtain the historical state feature vector of the target vehicle Historical state feature vectors of vehicles surrounding the target vehicle In addition, by using the local high-precision map Map around the target vehicle obtained in the multilayer perceptron encoding step A2, the local high-precision map feature vector V is obtained. Map :

[0011]

[0012]

[0013] V Map =MLP(Map; W Map )

[0014] In the formula, LSTM(·) is a Long Short-Term Memory neural network, and W target It is to encode the historical state of the target vehicle. The weight parameters W of a long short-term memory neural network neighbors It is the historical state of the surrounding vehicles. The weight parameters of the Long Short-Term Memory Neural Network, MLP(·) is a multilayer perceptron, W Map These are the weight parameters of the multilayer perceptron.

[0015] B2, obtain a set of executable maneuvers M according to traffic rules set and a set of traffic light states T set and encode M using a multi-layer perceptron set and T set to obtain embedded features R incorporating prior traffic rules e :

[0016] R e = Concat(MLP(M set ; W M ), MLP(T set ; W T ))

[0017] where Concat(·) is a feature concatenation operation, W M are weight parameters of a multi-layer perceptron encoding the set of executable maneuvers M set , and W T are weight parameters of a multi-layer perceptron encoding the set of traffic light states T set .

[0018] B3, capture interaction information I between the target vehicle and surrounding vehicles of the target vehicle using a multi-head attention mechanism M and capture interaction information I between the target vehicle and the surrounding traffic environment using a cross-attention mechanism C . Finally, use a multi-layer perceptron to obtain the predicted trajectory points of the target vehicle for the next T f time steps

[0019]

[0020]

[0021]

[0022]

[0023]

[0024] where is a historical state feature vector of the target vehicle, is a historical state feature vector of the target vehicle incorporating prior traffic rules, MultiheadAtt(·) is a multi-head attention mechanism, W multihead are weight parameters of the multi-head attention mechanism, CrossAtt(·) is a cross-attention mechanism, W cross are weight parameters of the cross-attention mechanism, F t is a fused feature of the target vehicle, and W decoderare weight parameters of a multilayer perceptron.

[0025] C, determining whether the closest distance from the end point of the predicted trajectory to the drivable lane centerline exceeds a set threshold value

[0026] C1, obtaining the drivable lane centerline in the local range around the target vehicle based on traffic rules and road topological relations After equal-interval discretization processing, it is an equal-interval lane centerline

[0027]

[0028] In the formula, Discrete(·) is an equal-interval discretization function, and δ is an equal-interval interval.

[0029] C2, according to the end point of the predicted trajectory of the target vehicle Calculate the Euclidean distance between the end point and the drivable lane centerline after equal-interval processing , and take the minimum value dmin of the distance:

[0030]

[0031] In the formula, dist(·) is a function of calculating the Euclidean distance between any two points, and min(·) is a minimum value function.

[0032] C3, determining whether the minimum distance d min in step C2 is greater than a set threshold value ε. If it is greater than the set threshold value ε, it is considered that the predicted trajectory of the target vehicle exceeds the drivable road boundary, and step D is turned to; otherwise, step A is turned to.

[0033] D, constraint post-processing of the predicted trajectory of the target vehicle

[0034] D1, obtaining the point Point Lane with the minimum distance from the end point of the predicted trajectory to the drivable lane centerline, and taking the point as a constraint end point.

[0035] D2, determining the first n trajectory points on the predicted trajectory of the target vehicle as constraint base points Point Traj using the current speed of the target vehicle:

[0036]

[0037]

[0038] In the formula, is the current speed of the target vehicle, the unit of speed is kilometer / hour, floor(·) is the down rounding function, max(·) is the maximum value function, constant a is the speed factor, and constant b is the constraint factor.

[0039] D3, the constraint end point Point Lane and the constraint base point Point Traj are taken together as the control points of the Bezier curve, and the m-order Bezier curve B(x) is used to fit the trajectory after the constraint

[0040]

[0041]

[0042] In the formula, m is the order of the Bezier curve, the value of m is the same as n in step D2, x is the independent variable of the Bezier curve, and the increment of x is

[0043] Compared with the prior art, the present application has the following beneficial effects:

[0044] 1. Based on the prior knowledge that the vehicle motion trajectory is limited by traffic rules, the present application can realize high accuracy, low collision rate and vehicle trajectory prediction conforming to traffic rules by explicitly incorporating the prior traffic rules and traffic light state information in the vehicle trajectory prediction method.

[0045] 2. From the perspective of making the downstream task work better, the present application uses road structure and traffic rules to explicitly post-process and constrain the predicted vehicle trajectory, which ensures the prediction accuracy and meets the scene consistency of the prediction result. DETAILED DESCRIPTION

[0046] Figure 1 is the flowchart of the present application.

[0047] Figure 2 is the structure diagram of the vehicle trajectory prediction model.

[0048] Figure 3 is the schematic diagram of obtaining the local high-precision map around the target vehicle.

[0049] Figure 4 is the schematic diagram of the lane center line equidistant discretization of the local high-precision map around the target vehicle.

[0050] Figure 5 is the schematic diagram of obtaining the constraint end point of the target vehicle prediction trajectory.

[0051] Figure 6is a schematic diagram of target vehicle predicted trajectory constraint processing. DETAILED DESCRIPTION

[0052] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. As shown in a multi-modal trajectory prediction method by paying attention to the scene and state, comprising the following steps: Figure 1

[0053] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. As shown in a multi-modal trajectory prediction method by paying attention to the scene and state, comprising the following steps: Figure 1

[0054] A, obtaining vehicle historical state and local high-precision map information

[0055] A1, using the environment perception system and positioning system of the unmanned vehicle to obtain the historical state information of the target vehicle and the historical state information of the vehicles around the target vehicle Wherein the superscript target represents the target vehicle, the superscript neighbors represents the surrounding vehicles, the subscript t=-T h , -T h +1,..., 0 represents different historical time, p t represents the position at time t, v t represents the speed at time t, h t represents the orientation at time t, l t represents the length dimension at time t, w t represents the width dimension at time t. In the embodiment of the present application, T h =20, the interval between adjacent time points is 0.1.

[0056] A2, using the current position of the target vehicle in step A1 to obtain the high-precision map Map={Crosswalk info , Lane info} in the surrounding local area, wherein Crosswalk info represents pedestrian crossing information, and Lane info represents lane centerline information. In the embodiment of the present application, the high-precision map within a range of 80 meters in front of the target vehicle, 40 meters behind the target vehicle, 60 meters to the left of the target vehicle and 60 meters to the right of the target vehicle is obtained, as shown in Figure 3

[0057] B, predicting the future trajectory of the target vehicle

[0058] B1, using a long short-term memory neural network to encode the historical state of the target vehicle and the historical state of the vehicles around the target vehicle obtained in step A1 ​​​​​get the target vehicle history state feature vector and the target vehicle surrounding vehicle history state feature vector As shown in formula (1) and formula (2) respectively. Figure 2 In addition, the target vehicle surrounding local high-definition map Map obtained by using the multilayer perceptron encoding step A2, get the local high-definition map feature vector V Map :

[0059]

[0060]

[0061] V Map = MLP (Map; W Map )

[0062] In the formula, LSTM(·) is a long short-term memory neural network, W target is the weight parameter of the long short-term memory neural network encoding the target vehicle history state , W neighbors is the weight parameter of the long short-term memory neural network encoding the history state of the surrounding vehicle , MLP(·) is a multilayer perceptron, and W Map is the weight parameter of the multilayer perceptron.

[0063] B2, according to the traffic rules, get the target vehicle current position executable maneuver set M set (for example, left turn, right turn, cruise and overtaking, etc.) and the traffic light state set T set (for example, red light, green light and yellow light, etc.), and use the multilayer perceptron to encode M set and T set , get the embedded feature R e which integrates prior traffic rules, as shown in formula (3): Figure 2

[0064] R e = Concat(MLP(M set ; W M ), MLP(T set ; W T ))

[0065] In the formula, Concat(·) is a feature splicing operation, W M is the weight parameter of the multilayer perceptron encoding the executable maneuver set M set , W T is the weight parameter of the multilayer perceptron encoding the traffic light state set T set .

[0066] ​B3, capture the interaction information I of the target vehicle and the vehicles around the target vehicle by using the multi-head attention mechanism M , and capture the interaction information I of the target vehicle and the surrounding traffic environment by using the cross-attention mechanism C . Finally, the multi-layer perceptron is used to obtain the predicted trajectory points of the target vehicle at T f future time points As shown in Figure 2 :

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] In the formula, is a target vehicle historical state feature vector, is a target vehicle historical state feature vector fused with prior traffic rules, MultiheadAtt(·) is a multi-head attention mechanism, W multihead is a weight parameter of the multi-head attention mechanism, CrossAtt(·) is a cross-attention mechanism, W cross is a weight parameter of the cross-attention mechanism, F t is a fusion feature of the target vehicle, W decoder is a weight parameter of the multi-layer perceptron. In the embodiment of the present application, T f = 30, and the interval between adjacent time points is 0.1.

[0073] C, determine whether the nearest distance from the end point of the predicted trajectory to the drivable lane center line exceeds a set threshold C1, obtain the drivable lane center line in the local range around the target vehicle based on traffic rules and road topological relations After equal-interval discretization processing, the equal-interval lane center line is obtained As shown in Figure 4 :

[0074]

[0075] In the formula, Discrete(·) is an equal-interval discretization function, and δ = 1 is an equal-interval interval.

[0076] C2, calculate the Euclidean distance between the end point of the predicted trajectory of the target vehicle and the equal-interval processed drivable lane center line , and take the minimum value d of the distance ​min :

[0077]

[0078] In the formula, dist(·) is the function for finding the Euclidean distance between any two points, and min(·) is the function for finding the minimum value.

[0079] C3. Determine the minimum distance d in step C2. min Is it greater than the set threshold ε = 2? If it is greater than the set threshold ε, the predicted trajectory of the target vehicle is considered to be outside the drivable road boundary, and proceed to step D; otherwise, proceed to step A.

[0080] D. Post-processing of constrained predicted trajectory of target vehicle

[0081] D1. Obtain the centerline point of the drivable lane with the smallest distance from the predicted trajectory endpoint. Lane And use that point as the constraint endpoint, such as Figure 5 As shown.

[0082] D2. Using the target vehicle's current speed, determine the first n trajectory points on the predicted trajectory of the target vehicle, and use them as constraint base points Point. Traj :

[0083]

[0084]

[0085] In the formula, The target vehicle's current speed is expressed in kilometers per hour. floor(·) is the floor function, max(·) is the maximum value function, the constant α = 10 is the speed factor, and the constant β = 12 is the constraint factor.

[0086] D3. Set the constraint endpoint Point from step D1... Lane and the constraint base point Point in step D2 Traj Together, they serve as control points for the Bézier curve, and the constrained trajectory is obtained by fitting the m-order Bézier curve B(x). like Figure 6 As shown.

[0087]

[0088]

[0089] In the formula, m is the order of the Bézier curve, and the value of m is the same as that of n in step D2. x is the independent variable of the Bézier curve, and the increment of x is...

[0090] The above describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only to illustrate the principles of the present application, and various changes and improvements can be made without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A method for road constraint based scene-consistent vehicle trajectory prediction, the method comprising: Comprising the following steps: A. Obtain vehicle historical state and local high-definition map information A1. Obtain historical state information of target vehicle and surrounding vehicles of target vehicle using environment perception system and localization system of unmanned vehicle where superscript target denotes target vehicle, superscript neighbors denotes surrounding vehicles, subscript t = -T h h +1, …, 0 denotes different historical time, p t denotes position at t, v t denotes speed at t, h t denotes heading at t, l t denotes length dimension at t, w t denotes width dimension at t;​​ A2, obtain the high-precision map Map = {Crosswalk info ,Lane info} of the local area around the target vehicle at the current time based on the current position of the target vehicle in step A1, wherein Crosswalk info represents the information of the crosswalk, and Lane info represents the information of the lane center line; B. Predict the future trajectory of the target vehicle B1. Using a long short-term memory neural network to encode the target vehicle historical state obtained in step A1 and the historical states of the surrounding vehicles of the target vehicle to obtain a target vehicle historical state feature vector and the historical states of the surrounding vehicles of the target vehicle In addition, using a multilayer perceptron to encode the target vehicle surrounding local high-definition map Map obtained in step A2, a local high-definition map feature vector V is obtained Map : V Map = MLP(Map; W Map ) where LSTM(·) is a long short-term memory neural network, W target is a weight parameter of a long short-term memory neural network encoding the history state of the target vehicle, W neighbors is a weight parameter of a long short-term memory neural network encoding the history state of the surrounding vehicles, MLP(·) is a multi-layer perceptron, W Map is a weight parameter of the multi-layer perceptron; B2. Obtain a set of executable maneuvers M for the target vehicle at the current location according to traffic rules set and a set of traffic light states T set and encode M using a multi-layer perceptron set and T set to obtain embedded features R that incorporate prior traffic rules e : R e = Concat(MLP(M set ; W M ), MLP(T set ; W T )) where Concat(·) is a feature concatenation operation, W M is a weight parameter of a multi-layer perceptron encoding the set of executable maneuvers M set , W T is a weight parameter of a multi-layer perceptron encoding the set of traffic light states T set , and W B3, capture the interaction information I of the target vehicle and the vehicles around the target vehicle by using the multi-head attention mechanism M , capture the interaction information I of the target vehicle and the surrounding traffic environment by using the cross attention mechanism C , and finally use the multi-layer perceptron to obtain the predicted trajectory points of the target vehicle at the future T f time points wherein, is a target vehicle history state feature vector, is a target vehicle history state feature vector fused with prior traffic rules, MultiheadAtt(·) is a multi-head attention mechanism, W multihead is a weight parameter of the multi-head attention mechanism, CrossAtt(·) is a cross-attention mechanism, W cross is a weight parameter of the cross-attention mechanism, F t is a fused feature of the target vehicle, W decoder is a weight parameter of the multi-layer perceptron; C. Determine whether the nearest distance from the endpoint of the predicted trajectory to the center line of the drivable lane exceeds the set threshold C1, obtaining drivable lane centerlines in a local range around the target vehicle based on traffic rules and road topology After equidistance discretization processing, equidistance lane centerlines In the formula, Discrete(·) is an equidistant discretization function, and δ is an equidistant interval; C2. Endpoint of the predicted trajectory of the target vehicle The Euclidean distance between the endpoint and the equidistant processed drivable lane centerline is calculated and the minimum value d of the distance is taken out min : In the formula, dist(·) is a function of calculating the Euclidean distance between any two points, and min(·) is a minimum value function; C3. Determine the minimum distance d in step C2. min Is it greater than the set threshold ε? If it is greater than the set threshold ε, the predicted trajectory of the target vehicle is considered to be outside the drivable road boundary, and proceed to step D; otherwise, proceed to step A. D. Constraint post-processing of the predicted trajectory of the target vehicle D1, obtain the feasible lane center line point Point with the minimum distance to the prediction trajectory endpoint Lane and take the point as the constraint endpoint; D2, determine the first n trajectory points on the trajectory predicted by the target vehicle as constraint base points Point Traj : In the formula, is the target vehicle current time speed, the unit of speed is kilometer / hour, floor(·) is the down rounding function, max(·) is the maximum value function, constant α is the speed factor, and constant β is the constraint factor; D3, the constrained end point Point Lane and the constrained base point Point Traj together as control points of a Bezier curve, using an mth order Bezier curve B(x) to fit the constrained trajectory In the formula, m is the order of the Bezier curve, m has the same value as n in step D2, x is the independent variable of the Bezier curve, and the increment of x is

Images