Collision risk assessment method and system based on digital twin and trajectory prediction

Abstract

The invention discloses a collision risk assessment method and system based on digital twinning and trajectory prediction, and the method comprises the steps: constructing a vehicle-road cooperation environment through a distributed digital twinning system, guaranteeing that vehicle information is always in a latest state through a prediction and updating system of a model, and providing a reliable data source for trajectory prediction; based on the multi-dimensional space-time interaction characteristics of vectorization processing, a multi-agent multi-mode parallel trajectory prediction algorithm is provided, and the motion states of multiple vehicles at the future moment are obtained at the same time; a driving risk field is dynamically generated according to multi-dimensional kinematics characteristics of a vehicle, and multi-source collision risks in a traffic scene are modeled in a three-dimensional mode by combining road characteristics and static obstacle characteristics. According to the method, the optimization method is integrated, fine-grained modeling is carried out on the traffic scene by using the digital twinborn technology, multi-agent trajectory prediction is combined with the field theory, and the timeliness and accuracy of collision risk assessment in the complex traffic scene are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of collision risk assessment in traffic scenarios, and in particular to a collision risk assessment method and system based on digital twins and trajectory prediction. Background Art

[0002] In recent years, the rapid development of autonomous driving technology has attracted great attention worldwide. This technology has a good prospect for solving hot issues such as traffic safety, road congestion, and energy consumption. However, according to a study by the US Highway Safety Administration, more than 1,496 traffic accidents have been caused by autonomous vehicles since 2019. Therefore, improving the driving safety of autonomous vehicles is an urgent problem that needs to be solved.

[0003] Effectively evaluating the collision risks in the surrounding traffic environment is an effective means to improve driving safety. Existing risk assessment methods usually require the use of some clear indicators, but a single indicator is only suitable for solving risk assessment problems in specific scenarios, and its applicability is poor. Some researchers have proposed using risk field theory to solve risk assessment problems in traffic scenarios, but most existing risk field models only generate driving risk fields based on the instantaneous vehicle motion state, and do not involve the assessment of future risks. Their timeliness and accuracy are insufficient. A more comprehensive collision risk assessment method is needed to address the challenges existing in the above risk assessment methods. Summary of the invention

[0004] This application proposes a collision risk assessment method and system based on digital twins and trajectory prediction. A vehicle-road collaborative environment is constructed through a distributed digital twin system, and a multi-agent and multi-modal parallel trajectory prediction algorithm is used to obtain the future motion state of the vehicle. A driving risk field is dynamically generated based on the multi-dimensional kinematic characteristics of the vehicle, and multi-source collision risks in three-dimensional modeling traffic scenes are effectively solved. The problems of poor applicability of collision warning indicators in complex traffic scenes, insufficient timeliness of risk field models, and poor accuracy are effectively solved.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] A collision risk assessment method based on digital twin and trajectory prediction includes the following steps:

[0007] The vehicle-road collaborative environment is built through a three-layer synchronous digital twin system of cloud server-edge server-mobile edge node, providing a reliable data source for trajectory prediction;

[0008] Vectorize the multi-dimensional spatiotemporal interactive features of traffic scenes, and use a multi-agent multi-modal parallel trajectory prediction algorithm to simultaneously obtain the future motion status of multiple vehicles;

[0009] Dynamically generate a driving risk field based on the multi-dimensional kinematic characteristics of the vehicle, and combine it with the road characteristics and static obstacle risk field to three-dimensionally model the multi-source collision risk in the traffic scene;

[0010] Preferably, the digital twin technology for building a vehicle-road collaborative environment includes:

[0011] Based on the fine-grained digital twin copies of traffic scenarios, a three-layer synchronous digital twin system of cloud server-edge server-mobile edge node is constructed;

[0012] Based on the information collection of sensors and multi-modal data fusion, high-precision map information and historical vehicle movement status information within the jurisdiction are stored in the edge server;

[0013] Ensure that the vehicle information in the edge server is up to date based on the model’s predictions and update system;

[0014] Based on the quantified value of the collision risk between vehicles, the set of vehicles with collision risk in the scene is determined, and a collision warning status signal is sent to them.

[0015] Preferably, the multi-agent multi-modal parallel trajectory prediction algorithm based on vectorized encoding of multi-dimensional spatiotemporal interaction features includes:

[0016] Vectorize map information and vehicle historical motion trajectories to improve the computational efficiency of trajectory prediction;

[0017] Divide each vehicle into local areas and dynamically predict the future motion state of the vehicle based on the local multi-dimensional spatiotemporal interaction characteristics;

[0018] The locally encoded embedding vector is input into the global feature extraction module to dynamically obtain the global interaction features between vehicles;

[0019] Generate a multimodal future trajectory of the vehicle based on the local feature vector and the global feature vector;

[0020] Select the trajectory with the smallest error with the true value from the multi-modal future trajectory for optimization as the final result of vehicle trajectory prediction;

[0021] Preferably, the method for dynamically generating a driving risk field based on multi-dimensional kinematic characteristics of a vehicle includes:

[0022] Generate a road feature risk field and a static obstacle risk field based on the scene map information and obstacle information stored in the edge server;

[0023] Obtain the equivalent mass based on the mass, type and speed of the vehicle;

[0024] Obtain the corrected distance in digital space according to the steering angle, horizontal and vertical safety thresholds during vehicle movement;

[0025] Dynamically generate the vehicle's driving risk field based on the vehicle's multi-dimensional kinematic characteristics and time decay factor;

[0026] The road feature risk field, static obstacle risk field and vehicle risk field are integrated to determine the current quantitative risk index of each vehicle;

[0027] The present application also discloses a collision risk assessment system based on digital twins and trajectory prediction, including a prediction and update module of a distributed digital twin framework and model, a multi-agent multi-modal parallel trajectory prediction algorithm module, and a risk assessment module based on multi-dimensional kinematic features:

[0028] The distributed digital twin framework is used to collect and process dynamic information of traffic scenarios. The prediction and update modules of the model ensure that the vehicle information in the edge server is always up to date, providing a reliable data source for trajectory prediction and the generation of driving risk fields.

[0029] The multi-agent multi-modal parallel trajectory prediction algorithm module includes using a vectorized method to encode the multi-dimensional spatiotemporal interaction characteristics of the traffic scene, and obtaining the future motion state of the vehicle through a multi-agent multi-modal parallel trajectory prediction algorithm;

[0030] The risk assessment module based on multi-dimensional kinematic characteristics includes dynamically generating a driving risk field according to the multi-dimensional kinematic characteristics of the vehicle, combining road characteristics and static obstacle risk fields, and three-dimensionally modeling multi-source collision risks in traffic scenes.

[0031] The beneficial effects of this application are:

[0032] This application discloses a collision risk assessment method and system based on digital twin and trajectory prediction. It builds a vehicle-road collaborative environment through a distributed digital twin system, uses a multi-agent multi-modal parallel trajectory prediction algorithm to obtain the future motion state of the vehicle, dynamically generates a driving risk field based on the multi-dimensional kinematic characteristics of the vehicle, and three-dimensionally models the multi-source collision risk in the traffic scene, effectively solving the problems of poor applicability of collision warning indicators in complex traffic scenes, insufficient timeliness and poor accuracy of risk field models. Experiments have shown that compared with other traditional collision warning measures and existing driving risk field models, the applicability, timeliness and accuracy of the risk assessment method disclosed in this application have been greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solution of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 This is the overall architecture diagram of the digital twin system - trajectory prediction - dynamic prediction driving risk field of Example 1 of the present application;

[0035] Figure 2 This is a multi-level synchronous digital twin system architecture diagram of Example 1 of the present application;

[0036] Figure 3 This is a conceptual diagram of the model prediction and update module of the first embodiment of the present application;

[0037] Figure 4 This is a flow chart of a multi-agent multi-modal trajectory prediction algorithm according to Embodiment 1 of the present application;

[0038] Figure 5 This is a schematic diagram of a road characteristic risk field according to Embodiment 1 of the present application;

[0039] Figure 6 This is a schematic diagram of a static obstacle risk field according to Embodiment 1 of the present application;

[0040] Figure 7 This is a comparison diagram of the Euclidean distance and the corrected distance in Example 1 of the present application;

[0041] Figure 8 This is a schematic diagram of a vehicle risk field under different motion states in Example 1 of the present application;

[0042] Fig. 9 This is a schematic diagram of the overall risk assessment method of Example 1 of the present application; DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0045] Embodiment 1

[0046] like Figure 1As shown, this is the overall architecture diagram of the digital twin system-trajectory prediction-dynamic prediction driving risk field of the first embodiment of the present application, including the following parts:

[0047] Multi-level synchronous distributed digital twin system: On-board sensors and roadside cameras collect traffic information, which is stored in the edge server after data processing and data fusion; the prediction and update system of the model is run based on the collected information to dynamically deduce the vehicle's motion state at future moments; the final driving risk field is generated based on the predicted multi-dimensional kinematic characteristics of the vehicle, map characteristics, and static obstacle characteristics, and the collision risk index between vehicles is calculated; when it is detected that the collision index between vehicles reaches the danger threshold, the edge server sends a collision warning status signal to the set of vehicles at risk of collision.

[0048] Specifically, the vehicle motion state data is expressed as follows:

[0049]

[0050] in, represents the vehicle's location information set, including the horizontal and vertical coordinates in the past period of time and the current moment; v represents the vehicle's speed information, including the lateral speed v x and longitudinal velocity v y ; a represents the acceleration information of the vehicle, including the horizontal acceleration a x and vertical acceleration a y ; θ represents the vehicle's front steering angle information; m ij Indicates the mass of the vehicle; c ij Indicates the type of vehicle.

[0051] The surrounding environment data with collision risk with vehicle j is expressed as:

[0052]

[0053] Among them, S ij is a set whose elements represent the collision risk of other vehicles to the current vehicle. The larger the value, the greater the collision risk. is the current motion state set of other vehicles that have a collision risk with this vehicle; D ij It is the relative distance set between other vehicles with collision risk and the vehicle.

[0054] Regarding the working of edge server layer and cloud server layer, RSUs collect vehicle data within its coverage area in real time and upload it to the cloud periodically. Vehicles outside the RSU coverage can request twin models from the cloud to maintain digital twins.

[0055] Figure 3The model's prediction update process is demonstrated. When the predicted vehicle operating status is compared with the actual vehicle operating status and reaches a difference threshold related to the vehicle density and average speed in the area, the RSU will actively update the vehicle's digital twin model to ensure that the vehicle digital twin information stored in the digital twin system can accurately reflect the vehicle's actual operating status in real time.

[0056] Multi-agent multi-modal parallel trajectory prediction algorithm: vectorize multi-dimensional spatiotemporal interaction features; extract vehicle motion features through local encoders, input them and global interaction features into the multimodal future trajectory decoder, and obtain the future trajectories of multiple vehicles at the same time; screen the multimodal future trajectories, and use the trajectory with the smallest error with the true trajectory as the trajectory for generating the risk field.

[0057] Figure 4 The flowchart of the multi-agent trajectory prediction algorithm of Example 1 of the present application is shown.

[0058] Specifically, the multi-dimensional spatiotemporal interaction features include the vehicle historical motion trajectory features and map features. The historical motion trajectory of a single agent i is expressed as follows:

[0059]

[0060] in, is the position of agent i at time t, p x , p y denote the horizontal and vertical coordinates of the vehicle, respectively, and T is the total historical time step. The geometric attributes of the lane segment ξ in the map information are expressed as:

[0061]

[0062] in, and are the starting and ending coordinates of the lane segment.

[0063] After vectorization of the multi-dimensional spatiotemporal interaction features, we input the motion features of each agent and its neighboring agents into the local encoder. The local encoder extracts the interaction relationship between the central agent and the neighboring agents through the vehicle-vehicle interaction model, and inputs it into the time transformer after adding the position and spatial embeddings at the end of the input sequence. After the multi-head attention block and the multi-layer perceptron (MLP) block alternately act, the extracted local spatiotemporal feature embedding is input into the vehicle-lane interaction model to learn the future motion intention of the central agent. The embedding at this time contains rich information about the central agent in the local area. Subsequently, the embedding of each agent is input into the global interaction module to capture the long-range dependencies in the traffic scene and make up for the lack of interaction information between vehicles in the local area.

[0064] The future motion of the vehicle has multimodal characteristics. We input local embedding and global embedding into the multimodal future trajectory decoder to predict trajectories for multiple agents. We use the hybrid Laplace model to parameterize the distribution of future trajectories, and the final output is [F, N, H, 4], which represents the number of predicted trajectory modes, the number of agents, the predicted future time length, coordinates and their uncertainty, respectively. We use a multi-layer perceptron block and softmax function to output the probabilities of different modes, with a shape of [N, F]. We calculate the error between each trajectory of each agent and the true trajectory, and select the trajectory with the smallest error as the best predicted trajectory, which is used as the reference trajectory data for generating the driving risk field at future moments.

[0065] Dynamically predict driving risk field based on multi-dimensional kinematic characteristics of vehicles: Calculate multi-dimensional kinematic characteristics based on the future motion state of the vehicle obtained from trajectory prediction, and dynamically generate the vehicle's driving risk field; Generate road feature risk field and static obstacle risk field based on the map information and obstacle information within the range stored in the edge server; Fusion the vehicle risk field with the road feature risk field and the static obstacle risk field to generate the final risk field, calculate the risk level based on the quantitative risk index of the vehicle's location, and complete the risk assessment task.

[0066] Specifically, Figure 5 The schematic diagram of the road feature risk field is shown, including lane dividing lines, double yellow lines, and road boundaries. The generation function of the lane dividing line can be expressed as:

[0067]

[0068] Among them, N L Indicates the total number of lane dividing lines outside the double yellow line. D Represents the field strength coefficient for lane dividing lines. σ is used to determine how quickly the field strength increases or decreases as the vehicle approaches or moves away from the lane markings. is the distance vector from point A to the i-th lane dividing line or road boundary, y A is the vertical coordinate value of point A on the road, y (L，i) Represents the y-axis position coordinate of the i-th lane marking or road boundary.

[0069] The risk field strength function of the double yellow line used to separate oncoming vehicles in the center of a two-way lane is:

[0070]

[0071] α A Used to represent the field strength coefficient of the double yellow line. Generally speaking, α A >α DBecause when a vehicle is close to a double yellow line, it will be threatened by oncoming vehicles, and the risk level is higher than when it is close to the lane dividing line.

[0072] Field intensity function E of road boundary risk field B The formula is:

[0073]

[0074] α B Represents the field strength coefficient at the road boundary.

[0075] The total field strength function E of the road characteristic risk field L The expression is:

[0076] |E L |=|E D |+|E A |+|E B |

[0077] Figure 6 A schematic diagram of the static obstacle risk field is shown, and the calculation formula is:

[0078]

[0079] Among them, G o is the coefficient that maps the risk intensity of static obstacles, m o is the mass of the static obstacle, r is the distance from the point on the map to the static obstacle o, σ ox and σ oy is the safety distance coefficient, which is used to control the descending speed of the risk field in the x direction and y direction respectively. (x obs ,y obs ) are the location coordinates of the obstacle.

[0080] Figure 7 A comparison chart of Euclidean distance and corrected distance is shown. Other risk field models use Euclidean distance to calculate the risk field strength. Regardless of whether the vehicle and the moving obstacle are in the same lane, the distance impact on the vehicle at the same distance is the same, which is inconsistent with the collision risk of vehicles in reality. This application proposes a corrected distance to solve this problem. The formula is:

[0081]

[0082] Among them, σ vx and σ vy is a coefficient related to the safety distance, which is used to control the speed of change of the risk field of the vehicle in the horizontal and vertical directions. μ is an undetermined coefficient related to the speed. v is the speed of the vehicle. is the steering angle of the vehicle head, assuming that the counterclockwise direction is the positive direction, and (x′, y′) represents the corrected coordinates of the original coordinates (x, y).

[0083] The multi-dimensional kinematic characteristics of a vehicle include its equivalent mass, velocity, acceleration, and steering angle.

[0084] The equivalent mass expression is:

[0085] M i =m i ·c i ·(1.566×10 -14 ·v 6.687 +0.3345)

[0086] Among them, M i is the equivalent mass of the vehicle, m i is the actual mass of the vehicle, c i is the type of vehicle and v is the speed of the vehicle.

[0087] The vehicle risk field can be calculated by the formula:

[0088]

[0089] Among them, β v and G v is the coefficient that affects the risk size, M i represents the equivalent mass of the vehicle, a is the vehicle acceleration, θ is the angle between the point where the field strength is calculated and the center of mass of the vehicle and the direction of vehicle movement, k′ is the corrected distance between the point where the field strength is calculated and the center of mass of the vehicle, represents a unit vector.

[0090] The future motion state of the vehicle is calculated by the following formula:

[0091]

[0092] Where v(t) is the vehicle speed at time t, a(t) is the vehicle acceleration, is the steering angle of the vehicle, assuming that the counterclockwise direction is the positive direction. λ is the time interval of the predicted trajectory points, and atan2 is the four-quadrant inverse tangent function, whose output range is (-π, π], which can cover the angles within the plane range.

[0093] The formula for the time attenuation factor used to control the decrease of risk field intensity with time is:

[0094]

[0095] Among them, t represents the current time step, t halfRepresents the half-life of the risk field, which is a hyperparameter uniformly applied to vehicles to ensure that the risk fields generated by different vehicles follow the same decay rule.

[0096] Based on the driving risk field and time attenuation factor at each time step, we can finally obtain the dynamic predicted driving risk field E of the vehicle: P :

[0097]

[0098] Where γ(t) is the time decay factor mentioned in formula (38), and T represents the total predicted time.

[0099] The embodiments described above are only descriptions of the preferred methods of the present application, and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present application.

Claims

1. A collision risk assessment method and system based on digital twin and trajectory prediction, characterized in that: The steps include: Build a vehicle-road collaborative environment through a distributed digital twin system, use the model's prediction and update system to ensure that vehicle information is up to date, and provide a reliable data source for trajectory prediction tasks; Based on the multi-dimensional spatiotemporal interaction characteristics of vectorized processing, a multi-agent multi-modal parallel trajectory prediction algorithm is used to simultaneously obtain the future motion status of multiple vehicles; Dynamically generate a driving risk field based on the multi-dimensional kinematic characteristics of the vehicle, and three-dimensionally model the multi-source collision risk in the traffic scene; Integrate the above three parts into one system to complete the collision risk assessment task with high reference value in traffic scenarios.

2. The collision risk assessment method based on digital twin and trajectory prediction according to claim 1, characterized in that: Digital twin technologies for building a vehicle-road collaborative environment include: Based on the fine-grained digital twin copies of traffic scenarios, a three-layer synchronous digital twin system of cloud server-edge server-mobile edge node is constructed; Based on the information collection of sensors and multi-modal data fusion, high-precision map information and historical vehicle movement status information within the jurisdiction are stored in the edge server; Ensure that the vehicle information in the edge server is up to date based on the model’s predictions and update system; Based on the quantified value of the collision risk between vehicles, the set of vehicles with collision risk in the scene is determined, and a collision warning status signal is sent to them.

3. The collision risk assessment method based on digital twin and trajectory prediction according to claim 1, characterized in that: Methods for multi-agent trajectory prediction include: Vectorize map information and vehicle historical motion trajectories to improve the computational efficiency of trajectory prediction; Divide each vehicle into local areas and dynamically predict the future motion state of the vehicle based on the local multi-dimensional spatiotemporal interaction characteristics; The locally encoded embedding vector is input into the global feature extraction module to dynamically obtain the global interaction features between vehicles; Generate a multimodal future trajectory of the vehicle based on the local feature vector and the global feature vector; Among the multimodal future trajectories, the trajectory with the smallest error with the true value is selected for optimization as the final result of vehicle trajectory prediction.

4. The collision risk assessment method based on digital twin and trajectory prediction according to claim 1 is characterized in that: Risk assessment methods based on multi-dimensional kinematic characteristics of vehicles include: Generate a road feature risk field and a static obstacle risk field based on the scene map information and obstacle information stored in the edge server; Obtain the equivalent mass based on the mass, type and speed of the vehicle; Obtain the corrected distance in digital space according to the steering angle, horizontal and vertical safety thresholds during vehicle movement; Dynamically generate the vehicle's driving risk field based on the vehicle's multi-dimensional kinematic characteristics and time decay factor; The road feature risk field, static obstacle risk field and vehicle risk field are integrated to determine the current quantitative risk index of each vehicle.

5. A collision risk assessment system based on digital twin and trajectory prediction, characterized in that: A vehicle-road collaborative environment is built through a distributed digital twin system, and a multi-agent multi-modal trajectory prediction algorithm is used to obtain the vehicle's future motion state. The driving risk field is dynamically generated based on multi-dimensional kinematic characteristics, and multi-source collision risks are modeled in three dimensions. The various methods collaborate and interact with each other to jointly complete risk assessment tasks in complex traffic scenarios.

Images