Long-term driving risk identification method and system based on multimodal trajectory prediction

Abstract

This invention discloses a long-term driving risk identification method and system based on multimodal trajectory prediction. The method includes: obtaining spatiotemporal feature representations of the target vehicle and its adjacent vehicles through unified vectorization processing; inputting the data into a hierarchical interactive feature extraction network to obtain global hierarchical multi-scale interactive features; inputting the data into a joint model to obtain the multimodal driving intention distribution of the target vehicle, and selecting high-probability candidate intention target points from it; calculating the acceleration and front wheel steering angle distribution parameters of the target vehicle at various future times, and inputting these parameters into a vehicle kinematics model to obtain the future trajectory coordinates of the target vehicle, generating a high-quality multimodal predicted trajectory set; calculating a comprehensive driving risk field, identifying the driving risk distribution in the future time domain based on this risk field, and planning a suitable vehicle path. This invention constructs a comprehensive driving risk field based on multimodal trajectory prediction, enabling quantitative identification of the uncertainty and potential risk distribution of the driving environment in the future prediction time domain.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle trajectory prediction and safe driving technology, and in particular to a long-term driving risk identification method and system based on multimodal trajectory prediction. Background Technology

[0002] In open and uncertain driving environments, the complex and diverse traffic participants (such as pedestrians, motor vehicles, and non-motorized vehicles), the dynamically changing behaviors of these participants, and the game-like interactions between them all present vehicles with numerous risks. These factors seriously threaten vehicle safety, and accurately identifying the resulting dynamic driving risks has become a critical issue that autonomous driving systems urgently need to address. By deeply analyzing the dynamic factors in the traffic environment and accurately predicting the future trajectories of surrounding traffic participants, the accuracy of driving risk assessment can be effectively improved, thereby providing strong protection for safe vehicle operation.

[0003] However, existing driving risk assessment methods firstly lack generalization ability. Risk indicator-based assessment methods are usually highly coupled with specific scenarios and are difficult to adapt to complex and ever-changing traffic environments. Secondly, existing methods only focus on the current environmental situation or simply use kinematic parameters for simple predictions, resulting in a significant lack of understanding of the dynamic driving environment, especially the dynamic changes in vehicle motion trends. In addition, existing methods do not consider the uncertain impact of the vehicle's own intentions, the static road environment, and the complex interactions between multiple participants on its motion trends, making it impossible to accurately identify the resulting dynamic driving risks.

[0004] For the reasons mentioned above, existing methods struggle to accurately model and predict these complex relationships using simplified mechanisms or rule-based approaches. Therefore, there is an urgent need to design a method capable of deeply understanding the dynamic driving environment, accurately predicting the trajectories of traffic participants, and effectively assessing driving risks. Summary of the Invention

[0005] This invention proposes a long-term driving risk identification method and system based on multimodal trajectory prediction, which can accurately predict the trajectory of traffic participants and effectively assess driving risks.

[0006] The technical solution adopted in this invention is:

[0007] A long-term driving risk identification method based on multimodal trajectory prediction is provided, the method comprising:

[0008] S1. Obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the obtained information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the vectorized historical trajectory.

[0009] S2. Input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, decompose it into two levels: local and global. Extract multi-vehicle spatiotemporal interaction features and vehicle-road interaction features in the local level, embed geometric relationship vectors between local levels to achieve information fusion between local and global levels, and output multi-scale interaction features in the global level.

[0010] S3. Based on global hierarchical multi-scale features, obtain the discrete road structure points and their continuous offsets of the target vehicle, input them into the joint model, obtain the probability distribution of discrete road structure points and the probability distribution of continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which high-probability candidate intention target points are selected.

[0011] S4. Calculate the acceleration and front wheel angle distribution parameters of the target vehicle at each future time based on the global hierarchical multi-scale interaction features and high-probability candidate intention target points. Input the parameters into the vehicle kinematics model to obtain the future trajectory coordinates of the target vehicle and generate a high-quality multimodal predicted trajectory set.

[0012] S5. Calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the vectorized static road structure information, and identify the driving risk distribution in the future time domain based on the risk field, and plan a suitable vehicle route.

[0013] According to the above scheme, the method for extracting the spatiotemporal interaction features of multiple vehicles in step S2 is as follows: Based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels: local and global. The spatiotemporal feature representation of the target vehicle and its adjacent vehicles at a certain moment in the local level is mapped into query vector, key vector and value vector respectively, and input into the scaling dot product attention module to obtain the features of the center vehicle and the adjacent vehicles. The features of the center vehicle and the adjacent vehicles are fused using a multi-head attention mechanism to obtain the spatiotemporal interaction features of multiple vehicles at that moment in the local level.

[0014] According to the above scheme, the method for extracting vehicle-road interaction features in step S2 is as follows: Based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels: local and global. Feature mapping is performed on the vehicle interaction relationship features at the local level and the vectorized representation of the local road segment. The corresponding query vector, key vector and value vector in the attention mechanism are obtained and input into the scaling dot product attention module to obtain the features of the vehicle and the road. The features of the vehicle and the road are fused using a multi-head attention mechanism to output the vehicle-road interaction features of the central vehicle and the local road segment.

[0015] According to the above scheme, the multimodal driving intention distribution of the target vehicle in step S3 is specifically achieved in the following way:

[0016] Select several discrete road structure points along the lane centerline, and add continuous offsets relative to their own positions to these discrete road structure points to obtain the fine-tuned fitting curve.

[0017] Based on the initial probability distribution of discrete road structure points in the fitted curve and the probability distribution of the corresponding offsets of each structure point, the initial distribution of vehicle driving intentions is solved.

[0018] Given the true endpoint trajectory information, the road structure point closest to the true endpoint trajectory is marked as a positive sample, and other road structure points are marked as negative samples. The ground truth distribution of one-hot encoding is constructed, and the joint model is trained to obtain the final discrete road structure point probability distribution and its continuous offset probability distribution. The multimodal driving intention distribution of the target vehicle is then obtained by solving the problem.

[0019] According to the above scheme, the high-quality multimodal prediction trajectory set in step S4 is obtained in the following way:

[0020] A multimodal predicted trajectory set is obtained based on the future trajectory coordinates of the target vehicle;

[0021] Design a scoring function for the predicted trajectory to score each predicted trajectory;

[0022] The predicted trajectories are sorted according to their trajectory scores. The predicted trajectories with scores higher than the preset value are retained. From the remaining trajectories, the trajectories with a difference greater than a certain value from the selected trajectories are selected. This process is repeated a certain number of times to finally obtain a multimodal predicted trajectory set that covers multiple potential future driving trends and meets the constraints of road structure and vehicle dynamics.

[0023] According to the above scheme, the comprehensive driving risk field specifically includes a static obstacle risk field, a road risk field, and a multimodal predicted dynamic obstacle risk field. The static obstacle risk field, the road risk field, and the multimodal predicted dynamic obstacle risk field are weighted and merged according to a proportional coefficient to form a comprehensive risk field.

[0024] According to the above scheme, the static obstacle risk field is specifically calculated from the coordinates of the static obstacle, the vector distance between the vehicle and the static obstacle, and the risk distribution factor along the coordinate axis.

[0025] According to the above scheme, the road risk field is specifically calculated from the lane line coordinates, the relative position of the vehicle and the lane line, the total number of lane dividing lines, the lane line risk distribution factor, and the lane dividing line and lane boundary line risk field intensity scaling factor.

[0026] According to the above scheme, the multimodal prediction dynamic obstacle risk field is specifically obtained by weighted superposition of the single-modal risk fields of each prediction trajectory, and weighted correction by introducing a time decay factor.

[0027] A long-term driving risk identification system based on multimodal trajectory prediction is provided, including:

[0028] The vectorization processing module is used to acquire the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the vectorized historical trajectory.

[0029] The global hierarchical multi-scale interaction feature acquisition module is used to input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, which is decomposed into local and global levels. In the local level, multi-vehicle spatiotemporal interaction features and vehicle-road interaction features are extracted, and geometric relationship vectors between local levels are embedded to achieve information fusion between local and global levels, and output global hierarchical multi-scale interaction features.

[0030] The high-probability candidate intention target point acquisition module is used to obtain the discrete road structure points and their continuous offsets of the target vehicle based on global hierarchical multi-scale features. The input is fed into the joint model to obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and to obtain the multimodal driving intention distribution of the target vehicle, from which high-probability candidate intention target points are selected.

[0031] The high-quality multimodal predicted trajectory acquisition module is used to calculate the acceleration and front wheel steering angle distribution parameters of the target vehicle at various future times based on global hierarchical multi-scale interaction features and high-probability candidate intention target points. The input vehicle kinematics model is used to obtain the future trajectory coordinates of the target vehicle and generate a high-quality multimodal predicted trajectory set.

[0032] The integrated driving risk field calculation and application identification module is used to calculate the integrated driving risk field based on the high-quality multimodal predicted trajectory and the vectorized static road structure information, and to identify the driving risk distribution in the future time domain based on the risk field, and to plan a suitable vehicle route.

[0033] The beneficial effects of this invention are:

[0034] This invention calculates a comprehensive driving risk field based on multimodal trajectory prediction, fusing predicted multimodal trajectories with multi-source scene information such as static obstacles and road structures. It provides a field-domain representation of the potential impact of dynamic and static elements, offering a quantifiable risk field for different spatiotemporal locations. This enables quantitative identification of the uncertainty and potential risk distribution of the driving environment within the future prediction time domain, improving the risk identification accuracy and interpretability of safety decisions in complex open environments for autonomous driving systems. Furthermore, by introducing a vehicle kinematics model, iteratively solving for the future trajectory coordinates of the target vehicle, it generates a high-quality multimodal predicted trajectory set that satisfies road structure and vehicle dynamics constraints. This achieves accurate and reasonable long-term trajectory prediction covering multiple potential future driving trends, significantly improving the accuracy and adaptability of trajectory prediction.

[0035] Furthermore, by performing vectorization processing and expression on the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles with translation and rotation invariance, the present invention can accurately and efficiently express the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles.

[0036] Furthermore, this invention uses a Transformer-based hierarchical feature extraction method to divide complex heterogeneous interaction relationships into two levels, local and global, for hierarchical feature extraction and fusion, which effectively reduces the computational complexity of the model and can efficiently capture the multi-level comprehensive interaction features of driving scenarios.

[0037] Furthermore, this invention introduces a time decay factor to weight and correct the risk distribution in the more distant future, thereby accurately and reasonably reflecting the spatiotemporal evolution of potential risks of dynamic obstacles in the prediction time domain. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the process flow of a long-term driving risk identification method based on multimodal trajectory prediction according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of a hierarchical interactive feature extraction network model based on Transformer according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the specific structure of vehicle driving intention prediction according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of a multimodal trajectory prediction method based on vehicle driving intention and vehicle kinematic constraints according to an embodiment of the present invention;

[0042] Figure 5 This is a logic block diagram of a long-term driving risk identification method according to an embodiment of the present invention;

[0043] Figure 6 This is a diagram showing the effect of long-term driving risk identification based on multimodal trajectory prediction according to an embodiment of the present invention; wherein (a) represents the effect of long-term driving risk identification in scenario one; (b) represents the effect of long-term driving risk identification in scenario two; (c) represents the effect of long-term driving risk identification in scenario three; and (d) represents the effect of long-term driving risk identification in scenario four. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0045] Example 1

[0046] This embodiment provides a long-term time-domain driving risk identification method based on multimodal trajectory prediction, such as... Figure 1 As shown, the method includes:

[0047] S1. Obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the obtained information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the vectorized historical trajectory.

[0048] S2. Input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, decompose it into two levels: local and global. Extract multi-vehicle spatiotemporal interaction features and vehicle-road interaction features in the local level, embed geometric relationship vectors between local levels to achieve information fusion between local and global levels, and output multi-scale interaction features in the global level.

[0049] S3. Based on global hierarchical multi-scale features, obtain the discrete road structure points and their continuous offsets of the target vehicle, input them into the joint model, obtain the probability distribution of discrete road structure points and the probability distribution of continuous offsets, and obtain the multimodal driving intention distribution of the target vehicle, from which high-probability candidate intention target points are selected.

[0050] S4. Calculate the acceleration and front wheel angle distribution parameters of the target vehicle at each future time based on the global hierarchical multi-scale interaction features and high-probability candidate intention target points. Input the parameters into the vehicle kinematics model to obtain the future trajectory coordinates of the target vehicle and generate a high-quality multimodal predicted trajectory set.

[0051] S5. Calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the vectorized static road structure information, and identify the driving risk distribution in the future time domain based on the risk field, and plan a suitable vehicle route.

[0052] Specifically, in step S2, the method for extracting multi-vehicle spatiotemporal interaction features is as follows: Based on Transformer, the spatiotemporal feature representations of the target vehicle and its adjacent vehicles are decomposed into two levels: local and global. The spatiotemporal feature representations of the target vehicle and its adjacent vehicles at a certain moment in the local level are mapped into query vector, key vector, and value vector, respectively, and input into the scaling dot product attention module to obtain the features of the center vehicle and its adjacent vehicles. The features of the center vehicle and its adjacent vehicles are fused using a multi-head attention mechanism to obtain the multi-vehicle spatiotemporal interaction features between vehicles at that moment in the local level.

[0053] Specifically, in step S2, the method for extracting vehicle-road interaction features is as follows: Based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels: local and global. Feature mapping is performed on the vehicle interaction relationship features at the local level and the vectorized representation of the local road segment. The corresponding query vector, key vector, and value vector in the attention mechanism are obtained and input into the scaling dot product attention module to obtain the features of the vehicle and the road. The features of the vehicle and the road are fused using a multi-head attention mechanism to output the vehicle-road interaction features of the central vehicle and the local road segment.

[0054] Specifically, in step S3, the multimodal driving intention distribution of the target vehicle is achieved in the following way:

[0055] Select several discrete road structure points along the lane centerline, and add continuous offsets relative to their own positions to these discrete road structure points to obtain the fine-tuned fitting curve.

[0056] Based on the initial probability distribution of discrete road structure points in the fitted curve and the probability distribution of the corresponding offsets of each structure point, the initial distribution of vehicle driving intentions is solved.

[0057] Given the true endpoint trajectory information, the road structure point closest to the true endpoint trajectory is marked as a positive sample, and other road structure points are marked as negative samples. The ground truth distribution of one-hot encoding is constructed, and the joint model is trained to obtain the final discrete road structure point probability distribution and its continuous offset probability distribution. The multimodal driving intention distribution of the target vehicle is then obtained by solving the problem.

[0058] Specifically, in step S4, the high-quality multimodal predicted trajectory set is obtained in the following way:

[0059] A multimodal predicted trajectory set is obtained based on the future trajectory coordinates of the target vehicle;

[0060] Design a scoring function for the predicted trajectory to score each predicted trajectory;

[0061] The predicted trajectories are sorted according to their trajectory scores. The predicted trajectories with scores higher than the preset value are retained. From the remaining trajectories, the trajectories with a difference greater than a certain value from the selected trajectories are selected. This process is repeated a certain number of times to finally obtain a multimodal predicted trajectory set that covers multiple potential future driving trends and meets the constraints of road structure and vehicle dynamics.

[0062] Specifically, in step S5, the comprehensive driving risk field includes a static obstacle risk field, a road risk field, and a multimodal predicted dynamic obstacle risk field. The static obstacle risk field, the road risk field, and the multimodal predicted dynamic obstacle risk field are weighted and fused together according to a proportional coefficient to form a comprehensive risk field.

[0063] Specifically, the static obstacle risk field is calculated from the coordinates of the static obstacle, the vector distance between the vehicle and the static obstacle, and the risk distribution factor along the coordinate axis.

[0064] The road risk field is specifically calculated from lane line coordinates, the relative position of vehicles and lane lines, the total number of lane dividing lines, lane line risk distribution factor, and the scaling factor of the risk field intensity of lane dividing lines and lane boundary lines.

[0065] The multimodal prediction dynamic obstacle risk field is specifically obtained by weighted superposition of the single-modal risk fields of each predicted trajectory, and weighted correction by introducing a time decay factor.

[0066] In addition, this embodiment also provides a long-term driving risk identification system based on multimodal trajectory prediction, which is used to implement the long-term driving risk identification method based on multimodal trajectory prediction in this embodiment.

[0067] The system includes:

[0068] The vectorization processing module is used to acquire the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the vectorized historical trajectory.

[0069] The global hierarchical multi-scale interaction feature acquisition module is used to input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, which is decomposed into local and global levels. In the local level, multi-vehicle spatiotemporal interaction features and vehicle-road interaction features are extracted, and geometric relationship vectors between local levels are embedded to achieve information fusion between local and global levels, and output global hierarchical multi-scale interaction features.

[0070] The high-probability candidate intention target point acquisition module is used to obtain the discrete road structure points and their continuous offsets of the target vehicle based on global hierarchical multi-scale features. The input is fed into the joint model to obtain the probability distribution of the discrete road structure points and the probability distribution of the continuous offsets, and to obtain the multimodal driving intention distribution of the target vehicle, from which high-probability candidate intention target points are selected.

[0071] The high-quality multimodal predicted trajectory acquisition module is used to calculate the acceleration and front wheel steering angle distribution parameters of the target vehicle at various future times based on global hierarchical multi-scale interaction features and high-probability candidate intention target points. The input vehicle kinematics model is used to obtain the future trajectory coordinates of the target vehicle and generate a high-quality multimodal predicted trajectory set.

[0072] The integrated driving risk field calculation and application identification module is used to calculate the integrated driving risk field based on the high-quality multimodal predicted trajectory and the vectorized static road structure information, and to identify the driving risk distribution in the future time domain based on the risk field, and to plan a suitable vehicle route.

[0073] This embodiment provides a long-term driving risk identification method and system based on multimodal trajectory prediction. By calculating a comprehensive driving risk field based on multimodal trajectory prediction, the predicted multimodal trajectory is fused with multi-source scene information such as static obstacles and road structure. The potential impact of dynamic and static elements is expressed in a field-domain manner, providing a quantifiable risk field for different spatiotemporal locations. This enables quantitative identification of the uncertainty and potential risk distribution of the driving environment in the future prediction time domain, improving the risk identification accuracy and interpretability of safety decisions of autonomous driving systems in complex open environments. Furthermore, by introducing a vehicle kinematics model, the future trajectory coordinates of the target vehicle are iteratively solved, thereby generating a high-quality multimodal predicted trajectory set that meets the constraints of road structure and vehicle dynamics. This achieves accurate and reasonable long-term trajectory prediction that covers multiple potential future driving trends, significantly improving the accuracy and adaptability of trajectory prediction.

[0074] Furthermore, this embodiment uses vectorization processing and expression with translation and rotation invariance to accurately and efficiently express the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles.

[0075] Furthermore, this embodiment uses a Transformer-based hierarchical feature extraction method to divide complex heterogeneous interaction relationships into two levels, local and global, and extracts and fuses hierarchical features separately. This effectively reduces the computational complexity of the model and can efficiently capture the multi-level comprehensive interaction features of driving scenarios.

[0076] Furthermore, this embodiment introduces a time decay factor to weight and correct the risk distribution in the more distant future, thereby accurately and reasonably reflecting the spatiotemporal evolution of potential risks of dynamic obstacles in the prediction time domain.

[0077] Example 2

[0078] This embodiment shares the same principles and ideas as Embodiment 1, but differs in that it further supplements the steps of the long-term driving risk identification method based on multimodal trajectory prediction in Embodiment 1. This provides comprehensive and interpretable technical support for the safety decisions of autonomous driving systems in complex dynamic environments, improving the accuracy and adaptability of trajectory prediction. The specific steps are as follows:

[0079] In step S1, the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles are uniformly vectorized. The trajectory vector of the target vehicle at the last moment is selected as the reference vector. The historical trajectory and relative position are rotated and aligned to obtain the spatiotemporal interaction feature representation of the target vehicle and its adjacent vehicles with translation and rotation invariance.

[0080] In step S2, such as Figure 2 As shown, this embodiment constructs and trains a Transformer-based hierarchical interaction feature extraction network: Complex spatiotemporal interaction relationships are decomposed into local and global levels. In the local scene, spatial and temporal Transformer modules, along with a multi-head attention mechanism, are used to extract multi-vehicle spatiotemporal interaction features and vehicle-road interaction features. An attention mechanism is then applied to embed geometric relationship vectors between local regions to achieve the fusion of local and global information, obtaining global-level multi-scale interaction features. For the target center vehicle i and adjacent vehicle j within the local region, the feature vector of vehicle i is... Mapped to the query vector `query`, and the feature vectors of neighboring vehicle `j`. Mapped to key vector Sum value vector The mapping process is as follows:

[0081]

[0082] In the formula, It is a learnable linear projection matrix, d k This represents the dimension of the mapped vector.

[0083] Secondly, the obtained query vector key vector Sum value vector As input to the scaled dot product attention module, the spatial interaction relationships between vehicles at time t are modeled:

[0084]

[0085] In the formula, [·] represents the feature concatenation operation commonly used in deep learning; N i W is the set of adjacent vehicle sequences of the central target vehicle i. gate With W self It is a learnable matrix.

[0086] A multi-head attention mechanism is used to fuse the features of the central vehicle i and its neighboring vehicles j to obtain the spatial interaction features between vehicles in a local area, as shown in the following formula:

[0087]

[0088] In the formula, [·] represents the Concat operation. It is a learnable linear projection matrix. This is a feature vector representing the spatial interaction relationship between vehicles within a local region at time t, centered on the central vehicle i.

[0089] Construct a temporal Transformer module along the time dimension to obtain spatial interaction features at each time step. Add time and location encoding Forming time-series feature sequences Then, using a self-attention mechanism, the temporal dependencies of multi-vehicle interaction features are modeled to obtain the interaction relationship features of dynamic traffic participants. The specific calculation formula is as follows:

[0090]

[0091] The dynamic traffic participant interaction characteristics of local areas and the vectorized representation z of local road segment ε iε Feature mapping is performed to obtain the query vector (query), key vector (key), and value vector (value) in the attention mechanism. The mapping process is calculated using the following formula:

[0092]

[0093] The obtained query vector q i Key vector k iε Sum vector v iε This will be used as input to the scaled dot product attention module:

[0094]

[0095] In the formula, L i W is the set of all adjacent lane segments of vehicle i at the center of the local area. gate With Wself It is a learnable matrix.

[0096] A multi-head attention mechanism is employed to fuse features of dynamic traffic participants (vehicles) and static roads, outputting vehicle-road interaction features for all local road segments of the central vehicle i.

[0097] Taking the target vehicle i as the center, calculate the geometric relationship between the center vehicle i of this local region and the center vehicle j of other local regions, that is, the relative position of their current trajectories. and the angle θ between the directions j -θ i The calculation formula is as follows:

[0098]

[0099] The geometric relationships between local regions are embedded into the local interaction feature vector centered on vehicle i, and the calculation formula is as follows:

[0100]

[0101] In the formula, and These are the local interaction feature vectors corresponding to vehicle i and vehicle j, respectively. It is a learnable projection matrix.

[0102] Furthermore, using the same local scene scaling dot product attention mechanism and multi-head attention mechanism, we output a multi-scale feature vector of global interaction relationships centered on vehicle i.

[0103] The spatiotemporal feature representations of the target vehicle and its adjacent vehicles are input into a pre-trained hierarchical interaction feature extraction network, which is decomposed into local and global layers. Multi-vehicle spatiotemporal interaction features and vehicle-road interaction features are extracted in the local layer. Geometric relationship vectors between local layers are embedded to achieve information fusion between local and global layers, and the global layer multi-scale interaction features are output.

[0104] In step S3, based on the obtained global scene multi-scale features, the future driving intention of the target vehicle is visualized as a combination of discrete road structure points and their continuous offsets. A schematic diagram of the specific structure for predicting vehicle driving intention is shown below. Figure 3 As shown. Several discrete road structure points (x, x) are selected along the lane centerline. n ,y n Add a continuous offset (Δx) relative to its own position to the discrete road structure point. n ,Δy n Fine-tuning the fit is performed to achieve more precise intent prediction. The calculation formula is as follows:

[0105]

[0106] Based on the preliminary discrete road structure point probability distribution π(τ) in the fitted curve n |c) and the probability distribution of the offsets corresponding to each structural point g x (Δx n ), g y (Δy n Solve for the initial vehicle driving intention distribution p(τ). n |c), its calculation formula is as follows:

[0107]

[0108] In the formula, an exponential function is used to ensure that the probability of network generation is always positive, and to guarantee π(τ) n The value of |c) is in (0,1), and the sum of the probabilities of the N road structure points is 1; both the horizontal and vertical offsets follow a normal distribution g(·)~N(μ,σ). 2 ), and μ=ψ(c), σ 2 =1; f(·) and ψ(·) are both learnable two-layer MLP networks.

[0109] Given the true destination trajectory information, the road structure points closest to the true destination trajectory are marked as positive samples, and other road structure points are marked as negative samples. A one-hot encoded ground truth distribution is constructed and input into the joint model for training, yielding the final discrete road structure point probability distribution and its continuous offset probability distribution. The multimodal driving intention distribution of the target vehicle is then solved, and high-probability candidate intention target points are selected from this distribution. The loss function for the one-hot encoded ground truth distribution is as follows:

[0110] L1 = L cls (π,u)+L offset (ψ x ,ψ y ,Δx u ,Δy u )

[0111] In the formula, L cls The classification loss is calculated between the predicted distribution π and the true distribution u; L offset To predict the intended target point τ n Offset (ψ) x ,ψ y ) and τ u Relative offset to the true trajectory endpoint (Δx) u ,Δy u The regression loss between ).

[0112] In step S4, a schematic diagram of the multimodal trajectory prediction method based on vehicle driving intention and vehicle kinematic constraints is shown below. Figure 4 As shown, the top K high-probability intention target points are selected using a probability-descending order. Given the intention target points and the global interaction feature vector, an MLP network is used to calculate and predict the acceleration and front wheel steering angle distribution parameters of the target vehicle at future time points. These parameters are then input into the vehicle kinematics model to iteratively solve for the trajectory coordinates at each future time point. The calculation formula is as follows:

[0113]

[0114] In the formula, p(a t ,θ t |τ,c)~N(μ,σ 2 ), and μ=ψ(τ,c), σ 2 =1, ψ(·) is a learnable two-layer MLP network, and G(·) is the vehicle kinematics model used to utilize the current acceleration a. t Heading angle θ t Front wheel steering angle δt and position (x) t ,y t Find the position at the next moment.

[0115] The solution process for G(·) is as follows:

[0116]

[0117] In the formula, v t dt is the vehicle's speed at time t, L is the vehicle's wheelbase, and x is the vehicle's speed at time t. t+1 y t+1 v t+1 and θ t+1 These represent the position, velocity, and heading angle at time t+1, respectively.

[0118] A scoring function for the predicted trajectory is designed based on the distance between the predicted trajectory and the actual trajectory, and a score is given for each predicted trajectory.

[0119] Specifically, the scoring function is based on the predicted trajectory s F With the actual trajectory The Euclidean distance between them determines the truth score, and the loss function is:

[0120]

[0121] The mathematical expression for the scoring function of the predicted trajectory is:

[0122]

[0123] In the formula, g(·) is a two-layer MLP network, and ψ(s) is a two-layer MLP network. F |c)∈(0,1).

[0124] The loss function for this step of the prediction model is the score ψ(s) for each predicted trajectory. F |c) and the corresponding truth score The cross-entropy loss is calculated using the following formula:

[0125]

[0126] The formula for calculating the score of the predicted trajectory is as follows:

[0127]

[0128] In the formula, α is the sensitivity value for adjusting the distance metric. The maximum distance between the predicted trajectory point and the actual trajectory point at each time point is calculated using the following formula:

[0129]

[0130] In step S5, the comprehensive driving risk field is calculated based on the risk field theory, and the potential impact of dynamic and static elements is expressed in a field-specific manner, providing a quantifiable risk field for different spatiotemporal locations.

[0131] The comprehensive driving risk field specifically includes a static obstacle risk field, a road risk field, and a multimodal predicted dynamic obstacle risk field. The static obstacle risk field, the road risk field, and the multimodal predicted dynamic obstacle risk field are weighted and fused together according to a proportional coefficient to form the comprehensive risk field.

[0132] Specifically, the static obstacle risk field is constructed using the location of the static obstacle as the source center, based on the relative distance and direction of any point relative to the obstacle, and utilizing risk distribution factors. Its calculation formula can be expressed as:

[0133]

[0134] In the formula, Φ(x,y) represents the risk distribution centered on the static obstacle. Let i be the coordinates of obstacle i. Let (x, y) be the vector distance from any point (x, y) to the obstacle. Unit distance;

[0135] The formula for calculating its risk distribution Φ(x,y) is as follows:

[0136]

[0137] In the formula, σ x σ y This represents the risk distribution factor along the coordinate axis of the risk field.

[0138] Specifically, the road risk field is constructed using lane boundary lines and lane dividers as references, and risk distribution factors are generated along the lane direction and lateral direction. Its calculation formula can be expressed as:

[0139]

[0140] In the formula, This represents the ordinate of lane line k. Let N be a vector pointing from any point to lane line k. mark Let σ be the total number of lane lines, α be the lane line risk distribution factor, and α and β be the scaling factors of the risk field intensity of lane lines and lane boundary lines, respectively.

[0141] Specifically, the multimodal dynamic obstacle prediction risk field is obtained by weighting and superimposing the single-modal risk field based on the occurrence probability of each multimodal prediction trajectory. In order to balance the uncertainty of long-term prediction, a time decay factor is introduced into the superimposed risk field to make weighted corrections to the risk distribution in the more distant future, thereby accurately and reasonably reflecting the spatiotemporal evolution of potential risks of dynamic obstacles in the prediction time domain.

[0142] In this approach, the future coordinates of each multimodal prediction trajectory are considered as "potential static field sources," resulting in the single-mode prediction risk field for each multimodal prediction trajectory. The calculation formula for this field can be expressed as follows:

[0143]

[0144] In the formula, t is the current time, t+k is the kth future time, and T f To predict the total time step, (x, y) represents the specific location at the corresponding time. This represents the predicted position of the moving obstacle at time t+k in the future.

[0145] The formula for calculating the time decay factor is as follows:

[0146]

[0147] Furthermore, based on the weighted superposition of the single-modal prediction risk field of each multimodal prediction trajectory and its occurrence probability single-modal risk field, the multimodal dynamic obstacle prediction risk field is solved:

[0148] E m (x,y,t)=E st (x,y,t)·G(t+k)

[0149] The static obstacle risk field, road risk field, and multimodal predicted dynamic obstacle risk field are weighted and fused using proportional coefficients to form a comprehensive risk field. This comprehensive risk field fully presents the uncertainty and risk distribution of the driving environment in the future prediction time domain, enabling the planning of appropriate vehicle routes. The resulting diagram is shown below. Figure 6 As shown. Its calculation formula can be expressed as:

[0150] E syn (x,y,t)=A o E o (x,y,t)+A b E b (x,y,t)+A m E m (x,y,t)

[0151] Where, E syn For a comprehensive risk field, A o 、A b 、A m These are the proportional coefficients for the static obstacle risk field, the road risk field, and the multimodal prediction dynamic obstacle risk field, respectively.

[0152] The logic block diagram of the long-term driving risk identification method described in this embodiment is as follows: Figure 5 As shown, its implementation effect is as follows Figure 6 As shown in the figure, this embodiment constructs a comprehensive driving risk field based on multimodal trajectory prediction, and integrates the predicted multimodal trajectory with multi-source scene information such as static obstacles and road structures. This enables quantitative identification of the uncertainty and potential risk distribution of the driving environment in the future prediction time domain. It provides a solution to the challenges of driving risk assessment caused by the complex and diverse traffic participants, dynamic and changing vehicle movement trends, and complex interactions between multiple factors in open and uncertain driving environments, as well as the shortcomings of existing methods in terms of generalization ability, understanding of future movement trends, and uncertainty prediction.

[0153] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0154] The order of the steps in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0155] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A long-term time-domain driving risk identification method based on multimodal trajectory prediction, characterized in that, The method includes: S1. Obtain the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the obtained information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the vectorized historical trajectory. S2. Input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, decompose it into two levels: local and global. Extract multi-vehicle spatiotemporal interaction features and vehicle-road interaction features in the local level, embed geometric relationship vectors between local levels to achieve information fusion between local and global levels, and output multi-scale interaction features in the global level. S3. Based on global hierarchical multi-scale features, obtain the discrete road structure points and their continuous offsets of the target vehicle, input them into the joint model, obtain the probability distribution of discrete road structure points and the probability distribution of continuous offsets through the joint model, and calculate the multimodal driving intention distribution of the target vehicle based on the probability distribution of discrete road structure points and the probability distribution of continuous offsets, and select high-probability candidate intention target points from them. S4. Calculate the acceleration and front wheel angle distribution parameters of the target vehicle at each future time based on the global hierarchical multi-scale interaction features and high-probability candidate intention target points. Input the parameters into the vehicle kinematics model to obtain the future trajectory coordinates of the target vehicle and generate a high-quality multimodal predicted trajectory set. S5. Calculate the comprehensive driving risk field based on the high-quality multimodal predicted trajectory and the vectorized static road structure information, and identify the driving risk distribution in the future time domain based on the comprehensive driving risk field, and plan a suitable vehicle path.

2. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 1, characterized in that, The method for extracting multi-vehicle spatiotemporal interaction features in step S2 is as follows: Based on Transformer, the spatiotemporal feature representations of the target vehicle and its adjacent vehicles are decomposed into two levels: local and global. The spatiotemporal feature representations of the target vehicle and its adjacent vehicles at a certain moment in the local level are mapped into query vector, key vector, and value vector, respectively, and input into the scaling dot product attention module to obtain the features of the target vehicle and its adjacent vehicles. The features of the target vehicle and its adjacent vehicles are fused using a multi-head attention mechanism to obtain the multi-vehicle spatiotemporal interaction features between vehicles at that moment in the local level.

3. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 1, characterized in that, The method for extracting vehicle-road interaction features in step S2 is as follows: Based on Transformer, the spatiotemporal feature representation of the target vehicle and its adjacent vehicles is decomposed into two levels: local and global. Feature mapping is performed on the vehicle interaction relationship features at the local level and the vectorized representation of the local road segment. The corresponding query vector, key vector, and value vector in the attention mechanism are obtained and input into the scaling dot product attention module to obtain the features of the vehicle and the road. The features of the vehicle and the road are fused using a multi-head attention mechanism to output the vehicle-road interaction features of the target vehicle and the local road segment.

4. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 1, characterized in that, The multimodal driving intent distribution of the target vehicle in step S3 is specifically achieved in the following way: Select several discrete road structure points along the lane centerline, and add continuous offsets relative to their own positions to these discrete road structure points to obtain the fine-tuned fitting curve. Based on the initial probability distribution of discrete road structure points in the fitted curve and the probability distribution of the corresponding offsets of each structure point, the initial distribution of vehicle driving intentions is solved. Given the true endpoint trajectory information, the road structure point closest to the true endpoint trajectory is marked as a positive sample, and other road structure points are marked as negative samples. The ground truth distribution of one-hot encoding is constructed, and the joint model is trained to obtain the final discrete road structure point probability distribution and its continuous offset probability distribution. The multimodal driving intention distribution of the target vehicle is then obtained by solving the problem.

5. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 1, characterized in that, The high-quality multimodal predicted trajectory set in step S4 is obtained in the following way: A multimodal predicted trajectory set is obtained based on the future trajectory coordinates of the target vehicle; Design a scoring function for the predicted trajectory to score each predicted trajectory; The predicted trajectories are sorted according to their trajectory scores. The predicted trajectories with scores higher than the preset value are retained. From the remaining trajectories, the trajectories with a difference greater than a certain value from the selected trajectories are selected. This process is repeated a certain number of times to finally obtain a multimodal predicted trajectory set that covers multiple potential future driving trends and meets the constraints of road structure and vehicle dynamics.

6. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 1, characterized in that, The comprehensive driving risk field specifically includes the static obstacle risk field, the road risk field, and the multimodal predicted dynamic obstacle risk field. The static obstacle risk field, the road risk field, and the multimodal predicted dynamic obstacle risk field are weighted and merged according to a proportional coefficient to form the comprehensive risk field.

7. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 6, characterized in that, The static obstacle risk field is specifically calculated from the coordinates of the static obstacle, the vector distance between the vehicle and the static obstacle, and the risk distribution factor along the coordinate axis.

8. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 6, characterized in that, The road risk field is specifically calculated from lane line coordinates, the relative position of vehicles and lane lines, the total number of lane dividing lines, lane line risk distribution factor, and the scaling factor of the risk field intensity of lane dividing lines and lane boundary lines.

9. The long-term driving risk identification method based on multimodal trajectory prediction according to claim 6, characterized in that, The multimodal prediction dynamic obstacle risk field is specifically obtained by weighted superposition of the single-modal risk fields of each predicted trajectory, and weighted correction by introducing a time decay factor.

10. A long-term time-domain driving risk identification system based on multimodal trajectory prediction, characterized in that, include: The vectorization processing module is used to acquire the historical trajectory and static road structure information of the target vehicle and its adjacent vehicles, perform unified vectorization processing on the acquired information, and obtain the spatiotemporal feature representation of the target vehicle and its adjacent vehicles based on the vectorized historical trajectory. The global hierarchical multi-scale interaction feature acquisition module is used to input the spatiotemporal feature representation of the target vehicle and its adjacent vehicles into the hierarchical interaction feature extraction network, which is decomposed into local and global levels. In the local level, multi-vehicle spatiotemporal interaction features and vehicle-road interaction features are extracted, and geometric relationship vectors between local levels are embedded to achieve information fusion between local and global levels, and output global hierarchical multi-scale interaction features. The high-probability candidate intention target point acquisition module is used to obtain the discrete road structure points and their continuous offsets of the target vehicle based on global hierarchical multi-scale features. The input is fed into the joint model, and the probability distribution of discrete road structure points and continuous offsets is obtained through the joint model. Based on the probability distribution of discrete road structure points and continuous offsets, the multimodal driving intention distribution of the target vehicle is calculated, and high-probability candidate intention target points are selected from them. The high-quality multimodal predicted trajectory acquisition module is used to calculate the acceleration and front wheel steering angle distribution parameters of the target vehicle at various future times based on global hierarchical multi-scale interaction features and high-probability candidate intention target points. The input vehicle kinematics model is used to obtain the future trajectory coordinates of the target vehicle and generate a high-quality multimodal predicted trajectory set. The integrated driving risk field calculation and application identification module is used to calculate the integrated driving risk field based on the high-quality multimodal predicted trajectory and the vectorized static road structure information, and to identify the driving risk distribution in the future time domain based on the integrated driving risk field, and plan appropriate vehicle routes.

Images