Long-time-domain driving risk identification method and system based on multi-modal trajectory prediction

Abstract

The invention discloses a long-time-domain driving risk identification method and system based on multi-modal trajectory prediction. The method comprises the steps that spatial-temporal feature representations of a target vehicle and adjacent vehicles of the target vehicle are obtained through unified vectorization processing; inputting a hierarchical interaction feature extraction network to obtain global hierarchical multi-scale interaction features; inputting the joint model, obtaining multi-modal driving intention distribution of the target vehicle, and selecting high-probability candidate intention target points from the multi-modal driving intention distribution; calculating acceleration and front wheel steering angle distribution parameters of the target vehicle at each moment in the future, inputting the acceleration and front wheel steering angle distribution parameters into the vehicle kinematic model to obtain future trajectory coordinates of the target vehicle, and generating a high-quality multi-modal prediction trajectory set; and calculating a comprehensive driving risk field, identifying driving risk distribution in a future time domain according to the risk field, and planning a proper vehicle path. According to the method, the comprehensive driving risk field is constructed based on multi-modal trajectory prediction, and the uncertainty and potential risk distribution of the driving environment in a future prediction time domain are quantitatively identified.

Description

Technical Field

[0001] The present invention relates to the field of intelligent vehicle trajectory prediction and safe driving technology, and in particular to a long-term driving risk identification method and system based on multi-modal trajectory prediction. Background Art

[0002] In an open and uncertain driving environment, the complex and diverse traffic participants (such as pedestrians, motor vehicles and non-motor vehicles), the dynamically changing behaviors of traffic participants, and the game interaction relationships between traffic participants make vehicles face many risks in a dynamic traffic environment. These factors seriously threaten the driving safety of vehicles, and accurately identifying the dynamic driving risks brought about by them has become a key issue that needs to be urgently solved by the autonomous driving system. By deeply analyzing the dynamic factors in the traffic environment and accurately predicting the future trajectories of surrounding traffic participants, the accuracy of driving risk assessment can be effectively improved, thereby providing a strong guarantee for the safe driving of vehicles.

[0003] However, the existing driving risk assessment methods are firstly insufficient in generalization ability. The assessment methods based on risk indicators are usually highly coupled with specific scenarios and are difficult to adapt to complex and changing traffic environments. Secondly, the existing methods only focus on the current environmental situation or only use kinematic parameter indicators for simple predictions. There are obvious deficiencies in the understanding of the dynamic driving environment, especially the understanding of the changes in dynamic vehicle motion trends. In addition, the existing methods do not consider the uncertainty of their own intentions, static road environment, and the complex interactions between multiple participants on their motion trends, and cannot accurately identify the resulting dynamic driving risks.

[0004] For the above reasons, it is difficult for existing methods to accurately model and predict these complex relationships through simplified mechanisms or rule-based methods. Therefore, it is urgent to design a method that can deeply understand the dynamic driving environment, accurately predict the trajectories of traffic participants, and effectively assess driving risks. Summary of the invention

[0005] The present invention proposes a long-term driving risk identification method and system based on multi-modal trajectory prediction, which are used to accurately predict the trajectories of traffic participants and effectively evaluate driving risks.

[0006] The technical solution adopted by the present invention is:

[0007] A long-term driving risk identification method based on multimodal trajectory prediction is provided, the method comprising:

[0008] S1. Obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the historical trajectory after vectorization processing;

[0009] S2. Input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interactive feature extraction network, decompose it into two levels: local and global. In the local level, extract the multi-vehicle spatiotemporal interactive features and vehicle-road interactive features, embed the geometric relationship vectors between the local levels to achieve local and global information fusion, and output the global level multi-scale interactive features.

[0010] S3. Obtain the discrete road structure points and their continuous offsets of the target vehicle based on the global hierarchical multi-scale features, input them into the joint model, obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which high-probability candidate intention target points are selected;

[0011] S4. Calculate the acceleration and front wheel turning angle distribution parameters of the target vehicle at each future moment based on the global multi-scale interaction features and high-probability candidate intention target points, input the vehicle kinematic model to obtain the future trajectory coordinates of the target vehicle, and generate a high-quality multi-modal prediction trajectory set;

[0012] S5. Calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the static road structure information after vector processing, and identify the driving risk distribution in the future time domain based on the risk field to plan the appropriate vehicle path.

[0013] According to the above scheme, the method for extracting the spatiotemporal interaction features of multiple vehicles in step S2 is: based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels, local and global, and the spatiotemporal feature representations of the target vehicle and adjacent vehicles at a certain moment in the local level are respectively mapped into query vectors, key vectors and value vectors, and input into the scaled dot product attention module to obtain the features of the central vehicle and the adjacent vehicles, and the multi-head attention mechanism is used to fuse the features of the central vehicle and the adjacent vehicles to obtain the multi-vehicle spatiotemporal interaction features between the vehicles at that moment in the local level.

[0014] According to the above scheme, the method for extracting the vehicle-road interaction features in step S2 is: based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels, local and global, and feature mapping is performed on the vehicle interaction relationship features at the local level and the vectorized expression of the local road segment, and the corresponding query vector, key vector and value vector in the attention mechanism are obtained, and the scaled dot product attention module is input to obtain the features of the vehicle and the road, and the multi-head attention mechanism is used to fuse the features of the vehicle and the road, and the vehicle-road interaction features of the central vehicle and the local road segment are output.

[0015] According to the above scheme, the multimodal driving intention distribution of the target vehicle in step S3 is specifically achieved in the following manner:

[0016] Select a number of discrete road structure points along the lane centerline, add continuous offsets relative to the discrete road structure points, and obtain a fine-tuned fitting curve;

[0017] According to the initial probability distribution of discrete road structure points in the fitting curve and the probability distribution of the corresponding offset of each structure point, the initial vehicle driving intention distribution is solved;

[0018] Given the true destination trajectory information, the road structure point closest to the true destination trajectory is marked as a positive sample, and the other road structure points are marked as negative samples. The one-hot encoded true value distribution is constructed, and the joint model is trained to obtain the final discrete road structure point probability distribution and its continuous offset probability distribution, and the multimodal driving intention distribution of the target vehicle is solved.

[0019] According to the above scheme, the high-quality multimodal prediction trajectory set in step S4 is obtained in the following way:

[0020] A multimodal predicted trajectory set is obtained according to the future trajectory coordinates of the target vehicle;

[0021] Design a scoring function for the predicted trajectory and score each predicted trajectory;

[0022] The predicted trajectories are sorted according to the trajectory scores, and the predicted trajectories with scores higher than the preset value are retained. Then, trajectories with a difference from the selected trajectories greater than a certain value are selected from the remaining trajectories. The cycle is iterated a certain number of times to finally obtain a multimodal predicted trajectory set that covers a variety of potential future driving trends and satisfies the road structure and vehicle dynamics constraints.

[0023] According to the above plan, the comprehensive driving risk field specifically includes the static obstacle risk field, the road risk field and the multimodal predicted dynamic obstacle risk field. The static obstacle risk field, the road risk field and the multimodal predicted dynamic obstacle risk field are weighted and fused according to the proportional coefficient to form a comprehensive risk field.

[0024] According to the above scheme, the static obstacle risk field is specifically calculated by the static obstacle coordinates, the vector distance between the vehicle and the static obstacle, and the risk distribution factor of the risk field along the coordinate axis direction.

[0025] According to the above scheme, the road risk field is specifically calculated by the lane line coordinates, the relative position of the vehicle and the lane line, the total number of lane dividing lines, the lane line risk distribution factor, and the lane dividing line and lane boundary line risk field strength scaling factor.

[0026] According to the above scheme, the multimodal predicted dynamic obstacle risk field is obtained by weighted superposition of the single modal risk fields of each predicted trajectory and weighted correction by introducing the time attenuation factor.

[0027] A long-term driving risk identification system based on multi-modal trajectory prediction is provided, including:

[0028] A vectorization processing module is used to obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the historical trajectory after vectorization processing;

[0029] The global-level multi-scale interaction feature acquisition module is used to input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, decompose it into two levels: local and global, extract multi-vehicle spatiotemporal interaction features and vehicle-road interaction features at the local level, embed the geometric relationship vectors between local levels to achieve local and global information fusion, and output global-level multi-scale interaction features;

[0030] The high-probability candidate intention target point acquisition module is used to obtain the discrete road structure points and their continuous offsets of the target vehicle based on the global hierarchical multi-scale features, input the joint model, obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which the high-probability candidate intention target points are selected;

[0031] The high-quality multimodal prediction trajectory acquisition module is used to calculate the acceleration and front wheel turning angle distribution parameters of the target vehicle at each future moment based on the global hierarchical multi-scale interaction features and high-probability candidate intention target points, input the vehicle kinematic model to obtain the future trajectory coordinates of the target vehicle, and generate a high-quality multimodal prediction trajectory set;

[0032] The comprehensive driving risk field calculation and application identification module is used to calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the static road structure information after vector processing, and identify the driving risk distribution in the future time domain based on the risk field to plan the appropriate vehicle path.

[0033] The beneficial effects produced by the present invention are:

[0034] The present invention calculates a comprehensive driving risk field based on multimodal trajectory prediction, integrates the predicted multimodal trajectory with multi-source scene information such as static obstacles and road structure, and expresses the potential impact of dynamic and static elements in a field-based manner, providing a quantifiable risk field for different spatiotemporal positions, and realizing quantitative identification of the uncertainty and potential risk distribution of the driving environment in the future prediction time domain, thereby improving the risk identification accuracy and interpretability of safety decisions of the autonomous driving system in a complex open environment, and introducing a vehicle kinematic model to iteratively solve the future trajectory coordinates of the target vehicle, thereby generating a high-quality multimodal prediction trajectory set that meets the constraints of road structure and vehicle dynamics, achieving accurate and reasonable long-term trajectory prediction covering a variety of potential future driving trends, and significantly improving the accuracy and adaptability of trajectory prediction.

[0035] Furthermore, the present invention can accurately and efficiently express the historical trajectory, static road structure and other information of the target vehicle and its surrounding vehicles by vectorizing and expressing the historical trajectory, static road structure and other information of the target vehicle and its surrounding vehicles with translation and rotation invariance.

[0036] Furthermore, the present invention divides the complex heterogeneous interaction relationship into two levels, local and global, through a hierarchical feature extraction method based on Transformer, and performs hierarchical feature extraction and fusion respectively, which effectively reduces the calculation complexity of the model and can efficiently capture the multi-level comprehensive interaction characteristics of the driving scene.

[0037] Furthermore, the present invention introduces a time attenuation factor to perform weighted correction on the risk distribution in the distant future, thereby accurately and reasonably reflecting the spatiotemporal evolution of the potential risks of dynamic obstacles in the prediction time domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of a method flow of a long-term driving risk identification method based on multi-modal trajectory prediction according to an embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of the structure of a hierarchical interactive feature extraction network model based on Transformer according to an embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of a specific structure of vehicle driving intention prediction according to an embodiment of the present invention;

[0041] Figure 4 is a schematic diagram of a multi-modal trajectory prediction method based on vehicle driving intention and vehicle kinematic constraints according to an embodiment of the present invention;

[0042] Figure 5 It is a logic block diagram of a long-term driving risk identification method according to an embodiment of the present invention;

[0043] Figure 6 It is a long-term driving risk identification effect diagram based on multimodal trajectory prediction according to an embodiment of the present invention; wherein (a) represents the long-term driving risk identification effect diagram of scene one; (b) represents the long-term driving risk identification effect diagram of scene two; (c) represents the long-term driving risk identification effect diagram of scene three; (d) represents the long-term driving risk identification effect diagram of scene four. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0045] Example 1

[0046] This embodiment provides a long-term driving risk identification method based on multi-modal trajectory prediction. Figure 1 As shown, the method includes:

[0047] S1. Obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the historical trajectory after vectorization processing;

[0048] S2. Input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interactive feature extraction network, decompose it into two levels: local and global. In the local level, extract the multi-vehicle spatiotemporal interactive features and vehicle-road interactive features, embed the geometric relationship vectors between the local levels to achieve local and global information fusion, and output the global level multi-scale interactive features.

[0049] S3. Obtain the discrete road structure points and their continuous offsets of the target vehicle based on the global hierarchical multi-scale features, input them into the joint model, obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which high-probability candidate intention target points are selected;

[0050] S4. Calculate the acceleration and front wheel turning angle distribution parameters of the target vehicle at each future moment based on the global multi-scale interaction features and high-probability candidate intention target points, input the vehicle kinematic model to obtain the future trajectory coordinates of the target vehicle, and generate a high-quality multi-modal prediction trajectory set;

[0051] S5. Calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the static road structure information after vector processing, and identify the driving risk distribution in the future time domain based on the risk field to plan the appropriate vehicle path.

[0052] Specifically, in step S2, the method for extracting the spatiotemporal interaction features of multiple vehicles is as follows: based on the Transformer, the spatiotemporal feature representations of the target vehicle and its adjacent vehicles are decomposed into two levels, local and global, and the spatiotemporal feature representations of the target vehicle and adjacent vehicles at a certain moment in the local level are respectively mapped into a query vector, a key vector, and a value vector, and input into a scaled dot product attention module to obtain the features of the central vehicle and the adjacent vehicles, and the multi-head attention mechanism is used to fuse the features of the central vehicle and the adjacent vehicles to obtain the multi-vehicle spatiotemporal interaction features between the vehicles at that moment in the local level.

[0053] Specifically, in step S2, the method for extracting vehicle-road interaction features is as follows: based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels, local and global, and feature mapping is performed on the vehicle interaction relationship features at the local level and the vectorized expression of the local road segment, and the corresponding query vector, key vector and value vector in the attention mechanism are obtained, and the scaled dot product attention module is input to obtain the features of the vehicle and the road, and the multi-head attention mechanism is used to fuse the features of the vehicle and the road, and the vehicle-road interaction features of the central vehicle and the local road segment are output.

[0054] Specifically, in step S3, the multimodal driving intention distribution of the target vehicle is implemented in the following manner:

[0055] Select a number of discrete road structure points along the lane centerline, add continuous offsets relative to the discrete road structure points, and obtain a fine-tuned fitting curve;

[0056] According to the initial probability distribution of discrete road structure points in the fitting curve and the probability distribution of the corresponding offset of each structure point, the initial vehicle driving intention distribution is solved;

[0057] Given the true destination trajectory information, the road structure point closest to the true destination trajectory is marked as a positive sample, and the other road structure points are marked as negative samples. The one-hot encoded true value distribution is constructed, and the joint model is trained to obtain the final discrete road structure point probability distribution and its continuous offset probability distribution, and the multimodal driving intention distribution of the target vehicle is solved.

[0058] Specifically, in step S4, the high-quality multimodal prediction trajectory set is obtained by:

[0059] A multimodal predicted trajectory set is obtained according to the future trajectory coordinates of the target vehicle;

[0060] Design a scoring function for the predicted trajectory and score each predicted trajectory;

[0061] The predicted trajectories are sorted according to the trajectory scores, and the predicted trajectories with scores higher than the preset value are retained. Then, trajectories with a difference from the selected trajectories greater than a certain value are selected from the remaining trajectories. The cycle is iterated a certain number of times to finally obtain a multimodal predicted trajectory set that covers a variety of potential future driving trends and satisfies the road structure and vehicle dynamics constraints.

[0062] Specifically, in step S5, the comprehensive driving risk field specifically includes a static obstacle risk field, a road risk field and a multimodal predicted dynamic obstacle risk field. The static obstacle risk field, the road risk field and the multimodal predicted dynamic obstacle risk field are weighted and fused according to a proportional coefficient to form a comprehensive risk field.

[0063] The static obstacle risk field is specifically calculated by the static obstacle coordinates, the vector distance between the vehicle and the static obstacle, and the risk distribution factor of the risk field along the coordinate axis.

[0064] The road risk field is specifically calculated by the lane line coordinates, the relative position of the vehicle and the lane line, the total number of lane dividing lines, the lane line risk distribution factor, and the lane dividing line and lane boundary line risk field strength scaling factor.

[0065] The multimodal prediction dynamic obstacle risk field is obtained by weighted superposition of the single modal risk fields of each predicted trajectory and weighted correction by introducing the time attenuation factor.

[0066] In addition, the present embodiment further provides a long-term driving risk identification system based on multimodal trajectory prediction, which is used to implement the long-term driving risk identification method based on multimodal trajectory prediction of the present embodiment.

[0067] The system includes:

[0068] A vectorization processing module is used to obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the historical trajectory after vectorization processing;

[0069] The global-level multi-scale interaction feature acquisition module is used to input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, decompose it into two levels: local and global, extract multi-vehicle spatiotemporal interaction features and vehicle-road interaction features at the local level, embed the geometric relationship vectors between local levels to achieve local and global information fusion, and output global-level multi-scale interaction features;

[0070] The high-probability candidate intention target point acquisition module is used to obtain the discrete road structure points and their continuous offsets of the target vehicle based on the global hierarchical multi-scale features, input the joint model, obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which the high-probability candidate intention target points are selected;

[0071] The high-quality multimodal prediction trajectory acquisition module is used to calculate the acceleration and front wheel turning angle distribution parameters of the target vehicle at each future moment based on the global hierarchical multi-scale interaction features and high-probability candidate intention target points, input the vehicle kinematic model to obtain the future trajectory coordinates of the target vehicle, and generate a high-quality multimodal prediction trajectory set;

[0072] The comprehensive driving risk field calculation and application identification module is used to calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the static road structure information after vector processing, and identify the driving risk distribution in the future time domain based on the risk field to plan the appropriate vehicle path.

[0073] The present embodiment provides a long-term driving risk identification method and system based on multimodal trajectory prediction. By calculating a comprehensive driving risk field based on multimodal trajectory prediction, the predicted multimodal trajectory is integrated with multi-source scene information such as static obstacles and road structures, and the potential impact of dynamic and static elements is expressed in a field-based manner. A quantifiable risk field is provided for different time and space positions, and quantitative identification of the uncertainty and potential risk distribution of the driving environment in the future prediction time domain is achieved. The risk identification accuracy and interpretability of safety decisions of the autonomous driving system in a complex open environment are improved. A vehicle kinematic model is introduced to iteratively solve the future trajectory coordinates of the target vehicle, thereby generating a high-quality multimodal prediction trajectory set that meets the road structure and vehicle dynamics constraints, achieving accurate and reasonable long-term trajectory prediction that covers a variety of potential future driving trends, and significantly improving the accuracy and adaptability of trajectory prediction.

[0074] Furthermore, this embodiment can accurately and efficiently express the historical trajectory, static road structure and other information of the target vehicle and its surrounding vehicles by vectorizing and expressing the historical trajectory, static road structure and other information of the target vehicle and its surrounding vehicles with translation and rotation invariance.

[0075] Furthermore, this embodiment uses a Transformer-based hierarchical feature extraction method to divide complex heterogeneous interaction relationships into two levels, local and global, and perform hierarchical feature extraction and fusion respectively, which effectively reduces the calculation complexity of the model and can efficiently capture the multi-level comprehensive interaction features of the driving scene.

[0076] Furthermore, this embodiment introduces a time attenuation factor to perform a weighted correction on the risk distribution in the distant future, thereby accurately and reasonably reflecting the spatiotemporal evolution of the potential risks of dynamic obstacles in the prediction time domain.

[0077] Example 2

[0078] The principles and ideas of this embodiment are basically the same as those of the first embodiment, except that this embodiment further supplements the steps of the long-term driving risk identification method based on multimodal trajectory prediction in the first embodiment, so as to provide comprehensive and explainable technical support for the safety decision-making of the autonomous driving system in a complex dynamic environment, and improve the accuracy and adaptability of trajectory prediction. The specific steps are as follows:

[0079] In step S1, the historical trajectories and static road structure information of the target vehicle and its adjacent vehicles are uniformly vectorized, the trajectory vector of the target vehicle at the last moment is selected as the reference vector, the historical trajectory and relative position are rotationally aligned, and the spatiotemporal interaction feature representation of the target vehicle and its adjacent vehicles with translation and rotation invariance is obtained.

[0080] In step S2, Figure 2 As shown in the figure, this embodiment constructs a hierarchical interaction feature extraction network based on Transformer and trains it: decomposes the complex spatiotemporal interaction relationship into two levels: local and global. In the local scene, the spatial Transformer and temporal Transformer modules and the multi-head attention mechanism are used to extract the spatiotemporal interaction features of multiple vehicles and the vehicle-road interaction features, and the attention mechanism is used to embed the geometric relationship vectors between local regions to achieve the fusion of local and global information, and obtain the global level multi-scale interaction features. For the target center vehicle i and the adjacent vehicle j in the local area, the feature vector of vehicle i is Mapped to query vector query, feature vector of adjacent vehicle j Mapping to key vector Sum value vector The mapping process is as follows:

[0081]

[0082] In the formula, is a learnable linear projection matrix, d k is the dimension of the vector after mapping.

[0083] Secondly, the query vector Key Vector Sum value vector As the input of the scaled dot product attention module, the spatial interaction relationship between vehicles at time t is modeled:

[0084]

[0085] In the formula, [·] represents the common feature concatenation operation in deep learning; N i is the set of adjacent vehicle sequences of the central target vehicle i, W gate With W self is a learnable matrix.

[0086] The multi-head attention mechanism is used to fuse the features of the central vehicle i and the adjacent vehicle j to obtain the spatial interaction features between vehicles in the local area. The formula is as follows:

[0087]

[0088] In the formula, [·] is the Concat operation, is a learnable linear projection matrix, is the feature vector of the spatial interaction relationship between vehicles in the local area at time t, which is aggregated around the central vehicle i.

[0089] Construct a time series Transformer module along the time dimension to obtain the spatial interaction features at each moment Add time position coding Forming a time series feature sequence Then, the self-attention mechanism is used to model the temporal dependency of multi-vehicle interaction features and obtain the interaction relationship features of dynamic traffic participants. The specific calculation formula is as follows:

[0090]

[0091] The interactive relationship characteristics of dynamic traffic participants in local areas are analyzed separately and the vectorized expression z of the local road segment ε iε Perform feature mapping to obtain the query vector query, key vector key and value vector value in the attention mechanism. The calculation formula of the mapping process is as follows:

[0092]

[0093] The query vector q i , key vector k iε Sum value vector v iε Will be used as input to the scaled dot product attention module:

[0094]

[0095] Where, L i is the set of all adjacent lane segments of vehicle i in the local area center, W gate With Wself is a learnable matrix.

[0096] A multi-head attention mechanism is used to fuse the features of dynamic traffic participants (vehicles) and static roads, and the vehicle-road interaction features of all local road segments of the central vehicle i are output.

[0097] Taking the target vehicle i as the center, calculate the geometric relationship between the central vehicle i in the local area and the central vehicle j in other local areas, that is, the relative position representation of the trajectories of the two at the current moment The angle between the direction θ j -θ i , and its calculation formula is as follows:

[0098]

[0099] The geometric relationship between local regions is embedded into the local interaction feature vector centered on vehicle i. The calculation formula is as follows:

[0100]

[0101] In the formula, and are the local interaction feature vectors corresponding to vehicle i and vehicle j, is the learnable projection matrix.

[0102] Furthermore, the above-mentioned local scene scaling dot product attention mechanism and multi-head attention mechanism are also used to output the multi-scale feature vector of the global interaction relationship centered on vehicle i

[0103] The spatiotemporal feature representation of the target vehicle and its adjacent vehicles is input into a trained hierarchical interaction feature extraction network and decomposed into two levels, local and global. Multi-vehicle spatiotemporal interaction features and vehicle-road interaction features are extracted at the local level, and the geometric relationship vectors between local levels are embedded to achieve local and global information fusion, and multi-scale interaction features at the global level are output.

[0104] In step S3, based on the obtained global scene multi-scale features, the target vehicle's future driving intention is visualized as a combination of discrete road structure points and their continuous offsets. The specific structural diagram of vehicle driving intention prediction is shown in Figure 3 As shown. Several discrete road structure points (x n ,y n ), add a continuous offset (Δx) relative to the discrete road structure point. n ,Δy n ) is fine-tuned to achieve more precise intent prediction. The calculation formula is as follows:

[0105]

[0106] According to the initial discrete road structure point probability distribution π(τ n |c) and the probability distribution g of the corresponding offset of each structural point x (Δx n ), g y (Δy n ), solve the preliminary vehicle driving intention distribution p(τ n |c), the calculation formula is as follows:

[0107]

[0108] In the formula, the exponential function is used to make the probability of network generation always positive, and to ensure that π(τ n |c) is in (0,1), and the probability sum of N road structure points is 1; the horizontal and vertical offsets obey the normal distribution g(·)~N(μ,σ 2 ), and μ = ψ(c), σ 2 =1; f(·) and ψ(·) are both learnable two-layer MLP networks.

[0109] Given the real destination trajectory information, the road structure point closest to the real destination trajectory is marked as a positive sample, and the other road structure points are marked as negative samples. The one-hot encoded true value distribution is constructed and input into the joint model training to obtain the final discrete road structure point probability distribution and its continuous offset probability distribution. The multimodal driving intention distribution of the target vehicle is solved and the high probability candidate intention target point is selected from it. The loss function of the one-hot encoded true value distribution is as follows:

[0110] L1=L cls (π,u)+L offset (ψ x ,ψ y ,Δx u ,Δy u )

[0111] Where, L cls is the classification loss between the predicted distribution π and the true value distribution u; L offset is the predicted intention target point τ n Deviation (ψ x ,ψ y ) and τ u The relative offset of the actual trajectory end point (Δx u ,Δy u ) is the regression loss between .

[0112] In step S4, a schematic diagram of a multi-modal trajectory prediction method based on vehicle driving intention and vehicle kinematic constraints is shown in FIG. Figure 4 The first K high-probability intention target points are selected in descending order of probability. Given the intention target points and the global interaction feature vector, the MLP network is used to calculate and predict the acceleration and front wheel angle distribution parameters of the target vehicle at each future moment, and then input into the vehicle kinematic model to iteratively solve the trajectory coordinates at each future moment. The calculation formula is as follows:

[0113]

[0114] In the formula, p(a t ,θ t ∣τ,c)~N(μ,σ 2 ), and μ = ψ(τ,c), σ 2 = 1, ψ(·) is a learnable two-layer MLP network, G(·) is the vehicle kinematic model, which is used to use the current acceleration a t , heading angle θ t , front wheel steering angle δt and position (x t ,y t ) to find the position at the next moment.

[0115] The solution process of G(·) is as follows:

[0116]

[0117] In the formula, v t is the speed of the vehicle at the current time t, dt is the time step, L is the vehicle wheelbase, x t+1 ,y t+1 、v t+1 and θ t+1 They are the position, velocity and heading angle corresponding to the time t+1 respectively.

[0118] The scoring function of the predicted trajectory is designed according to the distance between the predicted trajectory and the true trajectory, and each predicted trajectory is scored.

[0119] Specifically, the scoring function is based on the predicted trajectory s F With the real trajectory The Euclidean distance between the two sets of true value scores is formulated, and its loss function is:

[0120]

[0121] The mathematical expression of the scoring function for the predicted trajectory is:

[0122]

[0123] In the formula, g(·) is a two-layer MLP network, ψ(s F |c)∈(0,1).

[0124] The loss function of the prediction model at this step is the score ψ(s F |c) and the corresponding truth score The cross entropy loss is calculated as follows:

[0125]

[0126] The score calculation formula for the predicted trajectory is as follows:

[0127]

[0128] In the formula, α is the sensitivity value for adjusting the distance metric, is the maximum value of the distance between the predicted trajectory point and the actual trajectory point at each moment. The calculation formula is as follows:

[0129]

[0130] In step S5, the comprehensive driving risk field is finally calculated based on the risk field theory, and the potential impact of dynamic and static elements is expressed in a field-based manner to provide a quantifiable risk field for different time and space locations.

[0131] Among them, the comprehensive driving risk field specifically includes static obstacle risk field, road risk field and multimodal predicted dynamic obstacle risk field. The static obstacle risk field, road risk field and multimodal predicted dynamic obstacle risk field are weighted and fused according to the proportional coefficient to form a comprehensive risk field.

[0132] Specifically, the static obstacle risk field is constructed using the risk distribution factor based on the location of the static obstacle as the field source center and the relative distance and direction of any point relative to the obstacle. The calculation formula can be expressed as:

[0133]

[0134] In the formula, Φ(x,y) is the risk distribution centered on the static obstacle. is the coordinate of obstacle i, is the vector distance from any point (x, y) to the obstacle, is the unit distance;

[0135] Among them, the calculation formula of its risk distribution Φ(x,y) is as follows:

[0136]

[0137] In the formula, σ x , σ y is the risk distribution factor of the risk field along the coordinate axis.

[0138] Specifically, the road risk field is constructed by taking lane boundary lines and lane dividing lines as reference objects, and the risk distribution factors along the lane direction and lateral direction are constructed. The calculation formula can be expressed as:

[0139]

[0140] In the formula, represents the ordinate of lane line k, is the vector from any point to lane line k, N mark is the total number of lane dividing lines, σ is the risk distribution factor of lane lines, and α and β are the risk field strength scaling factors of lane dividing lines and lane boundary lines, respectively.

[0141] Specifically, the multimodal dynamic obstacle prediction risk field is obtained by weighted superposition of single modal risk fields according to the occurrence probability of each multimodal prediction trajectory. In order to balance the uncertainty of long-term prediction, a time attenuation factor is introduced into the superimposed risk field, and a weighted correction is made to the risk distribution in the distant future, thereby accurately and reasonably reflecting the spatiotemporal evolution of the potential risks of dynamic obstacles in the prediction time domain.

[0142] Among them, the future coordinate points of each multimodal prediction trajectory are regarded as “potential static field sources”, and the single modal prediction risk field of each multimodal prediction trajectory is obtained. The calculation formula can be expressed as:

[0143]

[0144] In the formula, t is the current moment, t+k is the kth moment in the future, and T f To predict the total time step, (x, y) is the specific position at the corresponding time. Represents the predicted position of the moving obstacle at the future time t+k.

[0145] The time attenuation factor calculation formula is as follows:

[0146]

[0147] Furthermore, the multimodal dynamic obstacle prediction risk field is solved by weighted superposition of the single-mode prediction risk field of each multimodal prediction trajectory and its single-mode risk field of occurrence probability:

[0148] E m (x,y,t)=E st (x,y,t)·G(t+k)

[0149] The static obstacle risk field, road risk field and multi-modal predicted dynamic obstacle risk field are weighted and fused through proportional coefficients to form a comprehensive risk field, which fully presents the uncertainty and risk distribution of the driving environment in the future prediction time domain to plan a suitable vehicle path. The effect diagram is shown below: Figure 6 The calculation formula can be expressed as:

[0150] E syn (x,y,t)=A o E o (x,y,t)+A b E b (x,y,t)+A m E m (x,y,t)

[0151] In the formula, E syn A is the comprehensive risk field. o , A b , A m They are the proportional coefficients of static obstacle risk field, road risk field and multimodal prediction dynamic obstacle risk field.

[0152] The logic block diagram of the long-term driving risk identification method in this embodiment is as follows: Figure 5 As shown, the implementation effect is as follows Figure 6 As shown. It can be seen that in this embodiment, by constructing a comprehensive driving risk field based on multimodal trajectory prediction, the predicted multimodal trajectory is integrated with multi-source scene information such as static obstacles and road structures, so as to achieve quantitative identification of the uncertainty and potential risk distribution of the driving environment in the future prediction time domain, and provide a solution to the driving risk assessment problems caused by the complex and diverse traffic participants, dynamic and changeable vehicle movement trends, and complex interactions between multiple factors in an open and uncertain driving environment, as well as the shortcomings of existing methods in terms of generalization ability, understanding of future movement trends, and uncertainty prediction.

[0153] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0154] The order of execution of each step in the above embodiment does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0155] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A long-term driving risk identification method based on multimodal trajectory prediction, characterized in that: The method comprises: S1. Obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the historical trajectory after vectorization processing; S2. Input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interactive feature extraction network, decompose it into two levels: local and global. In the local level, extract the multi-vehicle spatiotemporal interactive features and vehicle-road interactive features, embed the geometric relationship vectors between the local levels to achieve local and global information fusion, and output the global level multi-scale interactive features. S3. Obtain the discrete road structure points and their continuous offsets of the target vehicle based on the global hierarchical multi-scale features, input them into the joint model, obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which high-probability candidate intention target points are selected; S4. Calculate the acceleration and front wheel turning angle distribution parameters of the target vehicle at each future moment based on the global multi-scale interaction features and high-probability candidate intention target points, input the vehicle kinematic model to obtain the future trajectory coordinates of the target vehicle, and generate a high-quality multi-modal prediction trajectory set; S5. Calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the static road structure information after vector processing, and identify the driving risk distribution in the future time domain based on the risk field to plan the appropriate vehicle path.

2. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 1 is characterized in that: The method for extracting the spatiotemporal interaction features of multiple vehicles in step S2 is as follows: based on Transformer, the spatiotemporal feature representations of the target vehicle and its adjacent vehicles are decomposed into two levels, local and global, and the spatiotemporal feature representations of the target vehicle and adjacent vehicles at a certain moment in the local level are respectively mapped into a query vector, a key vector and a value vector, and input into a scaled dot product attention module to obtain the features of the central vehicle and the adjacent vehicles, and the multi-head attention mechanism is used to fuse the features of the central vehicle and the adjacent vehicles to obtain the multi-vehicle spatiotemporal interaction features between the vehicles at that moment in the local level.

3. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 1 is characterized in that: The method for extracting the vehicle-road interaction features in step S2 is as follows: based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels, local and global, and feature mapping is performed on the vehicle interaction relationship features at the local level and the vectorized expression of the local road segment, and the corresponding query vector, key vector and value vector in the attention mechanism are obtained, and the scaled dot product attention module is input to obtain the features of the vehicle and the road, and the multi-head attention mechanism is used to fuse the features of the vehicle and the road, and the vehicle-road interaction features of the central vehicle and the local road segment are output.

4. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 1 is characterized in that: The multimodal driving intention distribution of the target vehicle in step S3 is specifically achieved in the following manner: Select a number of discrete road structure points along the lane centerline, add continuous offsets relative to the discrete road structure points to obtain a fine-tuned fitting curve; According to the probability distribution of the preliminary discrete road structure points in the fitting curve and the probability distribution of the corresponding offset of each structure point, the preliminary vehicle driving intention distribution is solved; Given the true destination trajectory information, the road structure point closest to the true destination trajectory is marked as a positive sample, and the other road structure points are marked as negative samples. The one-hot encoded true value distribution is constructed, and the joint model is trained to obtain the final discrete road structure point probability distribution and its continuous offset probability distribution, and the multimodal driving intention distribution of the target vehicle is solved.

5. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 1 is characterized in that: The high-quality multimodal prediction trajectory set in step S4 is obtained specifically in the following way: A multimodal predicted trajectory set is obtained according to the future trajectory coordinates of the target vehicle; Design a scoring function for the predicted trajectory and score each predicted trajectory; The predicted trajectories are sorted according to the trajectory scores, and the predicted trajectories with scores higher than the preset value are retained. Then, trajectories with a difference from the selected trajectories greater than a certain value are selected from the remaining trajectories. The cycle is iterated a certain number of times to finally obtain a multimodal predicted trajectory set that covers a variety of potential future driving trends and satisfies the road structure and vehicle dynamics constraints.

6. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 1 is characterized in that: The comprehensive driving risk field specifically includes the static obstacle risk field, the road risk field and the multimodal predicted dynamic obstacle risk field. The static obstacle risk field, the road risk field and the multimodal predicted dynamic obstacle risk field are weighted and fused according to the proportional coefficient to form a comprehensive risk field.

7. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 6 is characterized in that: The static obstacle risk field is calculated by the static obstacle coordinates, the vector distance between the vehicle and the static obstacle, and the risk distribution factor of the risk field along the coordinate axis.

8. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 6 is characterized in that: The road risk field is specifically calculated by the lane line coordinates, the relative position of the vehicle and the lane line, the total number of lane dividing lines, the lane line risk distribution factor, and the lane dividing line and lane boundary line risk field strength scaling factor.

9. The long-term driving risk identification method based on multi-modal trajectory prediction according to claim 6 is characterized in that: The multimodal prediction dynamic obstacle risk field is obtained by weighted superposition of the single modal risk fields of each predicted trajectory and weighted correction by introducing the time attenuation factor.

10. A long-term driving risk identification system based on multi-modal trajectory prediction, characterized in that: include: A vectorization processing module is used to obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the historical trajectory after vectorization processing; The global-level multi-scale interaction feature acquisition module is used to input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, decompose it into two levels: local and global, extract multi-vehicle spatiotemporal interaction features and vehicle-road interaction features at the local level, embed the geometric relationship vectors between local levels to achieve local and global information fusion, and output global-level multi-scale interaction features; The high-probability candidate intention target point acquisition module is used to obtain the discrete road structure points and their continuous offsets of the target vehicle based on the global hierarchical multi-scale features, input the joint model, obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which the high-probability candidate intention target points are selected; The high-quality multimodal prediction trajectory acquisition module is used to calculate the acceleration and front wheel turning angle distribution parameters of the target vehicle at each future moment based on the global hierarchical multi-scale interaction features and high-probability candidate intention target points, input the vehicle kinematic model to obtain the future trajectory coordinates of the target vehicle, and generate a high-quality multimodal prediction trajectory set; The comprehensive driving risk field calculation and application identification module is used to calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the static road structure information after vector processing, and identify the driving risk distribution in the future time domain based on the risk field to plan the appropriate vehicle path.

Images