Fine-grained multi-task driving risk prediction method fused with trajectory prediction auxiliary task

Abstract

The present application relates to the field of automatic driving and traffic safety, and particularly relates to a fine-grained multi-task driving risk prediction method fusing trajectory prediction auxiliary tasks, which comprises the following steps: step 1: automatically labeling risk labels based on trajectory data; step 2: main task design of the driving risk prediction model; step 3: auxiliary task design of the driving risk prediction model; step 4: construction and training of the driving risk prediction model. The experimental results based on the public NGSIM data set show that introducing trajectory prediction as an auxiliary task can significantly improve the accuracy and robustness of the driving risk prediction model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent transportation and vehicle-road cooperation, and particularly relates to a fine-grained multi-task driving risk prediction method fusing trajectory prediction auxiliary tasks. BACKGROUND

[0002] In the intelligent transportation system, traffic accident risk prediction, as an important part of active safety control, has been widely used in various intelligent driving and traffic management scenarios. Traditional research usually constructs risk variables by extracting traffic flow, average speed, lane changing frequency and other macro statistical features from traffic data, and uses various regression models or classification models to predict accident risk. Such methods have the advantages of easy data acquisition and low computational complexity, but due to the coarse granularity of the features used, it is difficult to accurately describe the interaction between vehicles and individual differences, and the ability to identify abnormal driving behavior is limited, resulting in insufficient risk prediction accuracy.

[0003] With the advancement of smart highway construction and the development of sensor technology and deep learning methods, it is possible to obtain high-frequency and high-precision vehicle trajectory data, and micro-risk prediction based on fine-grained driving behavior features has gradually become a research hotspot. Such methods can more comprehensively capture dynamic interaction information between vehicles, thereby providing a new technical path for high-precision prediction of driving risk. SUMMARY

[0004] In view of the problems of insufficient feature expression and single task modeling in the prior art, the present application proposes a fine-grained multi-task driving risk prediction method fusing trajectory prediction auxiliary tasks. The method includes a risk prediction main task and a trajectory prediction auxiliary task, and the networks of the two tasks share feature representations in the encoding layer to realize information collaboration and knowledge transfer between tasks. In view of the problem of inconsistent loss scales in multi-task learning, a dynamic weight averaging strategy is adopted to adaptively adjust the loss weights of each task to improve the training stability and prediction performance of the model. Experimental results based on the public NGSIM dataset show that the introduction of trajectory prediction as an auxiliary task can significantly improve the accuracy and robustness of the driving risk prediction model.

[0005] TECHNICAL SCHEME

[0006] The fine-grained multi-task driving risk prediction method fusing trajectory prediction auxiliary tasks comprises the following steps:

[0007] Step 1: Automatically labeling risk labels based on trajectory data;

[0008] Collect trajectory data, calculate the Modified Time to Collision (MTTC) risk index, identify risk scenarios according to the MTTC index, and label risk labels.

[0009] Step 2 Risk prediction main task design;

[0010] The risk prediction main task is used to predict whether there is a potential risk event after the current scene for a future duration of n seconds. The trajectory data is input, and first the spatiotemporal behavior representation of the target vehicle and its surrounding environment is extracted by the encoder module, and then the risk prediction main task network is used to predict the probability of risk of the input sample.

[0011] Step 3 Trajectory prediction auxiliary task design;

[0012] The trajectory prediction auxiliary task is an auxiliary task in the multi-task framework of the application, and shares the encoder structure with the risk prediction main task. The trajectory prediction auxiliary task network takes the vehicle feature representation output by the encoder as input, combines the trajectory center point position information, and predicts the future trajectory coordinate sequence. Meanwhile, the trajectory prediction auxiliary task participates in joint training in the total loss function with a dynamic weight.

[0013] Step 4 Construction and training of the driving risk prediction model;

[0014] The driving risk prediction model framework is constructed, including the risk judgment main task and the trajectory prediction auxiliary task. The model loss function considers the loss of the main and auxiliary tasks, and adopts a dynamic weight average (DWA) mechanism to adaptively adjust the loss weight of the main and auxiliary tasks. Iterative training is performed until the loss converges.

[0015] In the inference stage, the trained driving risk prediction model is used to predict the risk of the input trajectory data.

[0016] Further, step 1 is specifically as follows:

[0017] Step 1.1 Calculate the modified TTC risk index;

[0018] The modified time to collision (TTC) risk index is calculated as follows:

[0019] The time to collision (TTC) index is a commonly used risk index, which has been widely used in micro risk judgment tasks due to its simple calculation and clear physical meaning. Let the longitudinal position of the front vehicle be , the speed ; the following vehicle position , the speed (taking the positive direction along the driving direction, and ). Under the assumption that the speeds of the two vehicles remain constant, the longitudinal time to collision TTC is as follows:

[0020]

[0021] TTC describes the remaining time to rear-end collision between two vehicles under the condition of no braking or acceleration. The smaller the value, the higher the risk of traffic accidents.

[0022] TTC has a constant speed assumption. To overcome the shortcomings of this assumption, longitudinal acceleration correction is introduced into TTC to form a modified TTC (MTTC) index. Considering the constant acceleration of the following vehicle , the constant acceleration of the leading vehicle , and the relative displacement between the two vehicles satisfying:

[0023]

[0024] The smallest positive real root that reduces the longitudinal distance between the two vehicles to the set minimum safety distance is the MTTC:

[0025]

[0026] The advantage of MTTC is that it allows vehicles to move at constant acceleration, thereby distinguishing between typical traffic conflict situations such as "deceleration while following but still in a safe distance" and "acceleration leading to rear-end collision". When , it is determined as a potential conflict event. Where is a pre-set threshold value.

[0027] Step 1.2 automatically identifies the risk of the scene;

[0028] In actual vehicle operation scenarios, in order to take timely risk avoidance measures, a risk warning of m seconds in advance is usually required. That is, the trajectory data of the past n seconds is used to predict the accident risk value m seconds later. The past n seconds and the future m seconds form a scene sample with a duration of n+m seconds, and the MTTC index at the time of the future m seconds is calculated as a substitute index for the risk value.

[0029] Assuming the risk time is t, considering that trajectory data has noise, the instantaneous data at time t cannot be used to calculate the MTTC index, so the last L-frame MTTC set before time t is taken , if the set elements satisfy:

[0030]

[0031] then the scene is defined as a high-risk state, and is assigned a label , otherwise it is recorded as a low-risk, and the label is .

[0032] Process the collected trajectory data, identify the risk scene according to the MTTC index, and label the risk label. ​

[0033] Furthermore, step 2 is detailed below:

[0034] Step 2.1 Vehicle trajectory feature extraction;

[0035] The encoder module is designed to extract spatiotemporal behavioral representations of the target vehicle and its surrounding environment. The encoder includes a two-stage structure: a single-vehicle trajectory feature extraction network and an inter-vehicle interaction module.

[0036] The model input is all valid vehicles in the past Frame trajectory tensor ,in Indicates the batch size. For the number of frames in the historical trajectory, 2 represents the maximum number of vehicles per frame, and 2 represents the dimension of the trajectory data. The input is first multiplied by a mask tensor. The missing trajectories are masked to obtain the effective trajectory tensor. :

[0037]

[0038] Valid vehicle trajectory data for each sample scene in the batch Trajectory feature extraction is performed, where For the first The number of valid vehicles in each sample scenario.

[0039] The single-vehicle trajectory feature extraction network employs a three-layer stacked Res1D network to perform one-dimensional convolutional modeling on the trajectory of each vehicle, extracting local temporal features. The outputs from each scale are fused to a unified dimension via lateral connections, and then passed through a residual module to output vehicle feature codes.

[0040]

[0041] in, This refers to the specified vehicle representation dimension, i.e., the sample scene. The effective vehicle characterization combination is as follows:

[0042]

[0043] in, For single vehicles The temporal features obtained after encoding the historical trajectory sequence have the following center position: .

[0044] The vehicle-to-vehicle interaction module uses a two-layer attention mechanism to model the mutual influence between vehicles, performing spatial attention modeling on each vehicle and its neighboring vehicles. Using the aligned vehicle center position as the geometric reference, each pair of center positions is then sequentially... and target vehicle and surrounding vehicles (The surrounding vehicles refer to those that meet the distance constraints) The model aggregates information from vehicles. At each layer, it constructs a contextual representation based on the relative positions of vehicles, the query vector, and neighbor features, and updates the target vehicle features through residual connections and normalization.

[0045]

[0046] in, and These represent the first and second layer attention aggregation modules, respectively. Indicates to vehicles The temporal features obtained after encoding the historical trajectory sequence. Indicates vehicle The central location, , They represent the target vehicles respectively. Intermediate features and final interaction features after two-layer attention aggregation Indicates the target vehicle The set of features of all neighboring vehicles within the neighborhood.

[0047] Final output This indicates the target vehicle that incorporates information from multiple levels of neighbors. The features are represented and used as shared inputs for subsequent risk assessment and trajectory prediction tasks.

[0048] Step 2.2 Network structure design for the main task of risk prediction;

[0049] The main task of risk prediction is to predict whether potential risk events will occur in the current scenario after n seconds. This task not only makes judgments based on the feature representations of individual vehicles, but also incorporates contextual information between vehicles through a multi-head attention mechanism to capture potential collaborative risk relationships.

[0050] The input to the main risk prediction task network is the vehicle feature information obtained from the encoder, denoted as:

[0051]

[0052] in, This represents the number of valid target vehicles in the current input sample. For feature dimension, the first The characteristics of each target vehicle are represented as follows: The task first applies to each Self-attention updates are performed using the vehicle itself and other target vehicles, i.e.:

[0053]

[0054] in, This represents a standard multi-head attention mechanism. These are the Query, Key, and Value matrices for a multi-head attention mechanism, with both input and output dimensions being [missing information]. Subsequently, the updated representation was... Perform a two-layer fully connected mapping to obtain the logit value for risk prediction. As shown below:

[0055]

[0056] During the inference phase, to obtain the probability output that the sample is risky, a Sigmoid mapping is further performed:

[0057]

[0058] in, This represents the Sigmoid function. For the weights of the fully connected layer, This is a bias term.

[0059] In model design, the multi-head attention mechanism enables the model to perceive the global semantic relationships between target vehicles, thereby improving the modeling ability of collaborative risk behaviors in the scenario.

[0060] Step 2.3 Design of the loss function for the main task of risk prediction;

[0061] During training, risk prediction uses binary cross-entropy loss:

[0062]

[0063] in, It is the first Sample The true label is either 0 or 1. The true label is obtained in step 1, where 1 indicates a high-risk state and 0 indicates a low-risk state. It is the first Sample The "logit" value in the model output; yes The output of the sigmoid function represents the probability that the model predicts the sample to be at risk (label 1), calculated as follows:

[0064]

[0065] Furthermore, step 3 is detailed below:

[0066] Step 3.1 Network structure design for trajectory prediction auxiliary task;

[0067] The trajectory prediction auxiliary task is used to regress the trajectory position of each target vehicle within a future time window, thus constructing a multi-time-step position prediction task.

[0068] The trajectory prediction auxiliary task network takes the vehicle feature representation output by the encoder as input and combines it with the trajectory center point location information to predict the future trajectory coordinate sequence. To make the predicted trajectory have multimodal characteristics, K different predicted trajectories are generated simultaneously.

[0069] No. The characteristics of each target vehicle are represented as follows: The corresponding historical trajectory center point is Using K independent multilayer perceptrons (MLPs) to... Parallel modeling outputs a sequence of relative displacements for K predicted trajectories:

[0070]

[0071] in, This represents the predicted relative displacement sequence. It is then compared to the center point of the last historical frame. Add them together to get the predicted position for each trajectory:

[0072]

[0073] This structure avoids direct regression to absolute position, thus improving the stability of the prediction.

[0074] Step 3.2 Design of the loss function for the trajectory prediction auxiliary task;

[0075] During the training phase, the model first calculates all predicted trajectories. With the actual trajectory The average Euclidean distance is used to determine the optimal mode, and the trajectory with the smallest error is selected as the optimal mode, denoted as . The smoothed L1 loss is calculated only for the optimal trajectory as a regression loss term:

[0076]

[0077] in, This represents the number of target vehicles in the current training batch. To enhance the model's ability to discriminate multimodal outputs, the classification branch simultaneously scores the confidence of all candidate trajectories. Maximum margin classification loss is used. The model is encouraged to award higher scores to the optimal mode. The total loss for the final trajectory prediction task is defined as:

[0078]

[0079] in, This represents the loss weight balancing coefficient. The maximum margin classification loss is defined as follows:

[0080]

[0081] in, Indicates the number of vehicles. Indicates the number of modes. Indicates interval, This represents the score of the non-optimal mode trajectory of vehicle v. This represents the score of the optimal modal trajectory of vehicle m.

[0082] Furthermore, step 4 is detailed below:

[0083] Step 4.1 Overall structural design of the driving risk prediction model;

[0084] The overall framework of the driving risk prediction model adopts an end-to-end structure, including the main task of risk assessment, the auxiliary task of trajectory prediction, and a shared encoder. The input historical trajectory is first processed by the encoder to extract local temporal features, and then an attention mechanism is introduced to model the interaction between the target vehicle and surrounding vehicles to capture potential dynamic influence relationships. Based on the shared features output by the encoder, two decoding paths are established: the main task is binary classification risk assessment, which outputs the accident probability; the auxiliary task is trajectory point regression, which outputs the trajectory prediction sequence for several future time steps.

[0085] Step 4.2 Design of loss function for driving risk prediction model;

[0086] To improve training stability and multi-task collaboration efficiency, the model loss function considers the weighted sum of the main and auxiliary tasks, and adopts the Dynamic Weighted Average (DWA) algorithm to adaptively adjust the loss weights of the main and auxiliary tasks, thereby realizing a unified modeling process that combines feature sharing and target decoupling.

[0087]

[0088] in, , These are the loss functions for the primary and secondary tasks, respectively. and Calculated using the DWA algorithm.

[0089] Specifically, the DWA algorithm automatically adjusts the optimization intensity of each task. If the loss of a task decreases slowly, it indicates that the training is difficult, so its weight is increased to make the model pay more attention to the task; conversely, it indicates that the training is easy, so its weight is reduced accordingly.

[0090] The specific formula for weight update is as follows:

[0091]

[0092] in, Indicates task Loss of (main task or auxiliary task) in the (t-1)th iteration The loss at the (t-2)th time The ratio reflects the convergence speed of the task loss. Indicates to Softmax normalization is performed, and the loss weights assigned to task i in the t-th iteration are used. I represents the number of tasks, and T is the temperature coefficient.

[0093] Step 4.3 Training the driving risk prediction model;

[0094] The specific process of iterative training is as follows:

[0095] (1) Input is the trajectory data of the target vehicle and its neighboring vehicles within a straight distance of less than D meters for m seconds. Identify risk scenarios based on the MTTC index and label them with risk tags.

[0096] (2) Calculate the loss function of the main task. Calculate the loss function of the auxiliary task. The loss weights are dynamically adjusted using a dynamic weighted averaging algorithm. and Calculate the total loss of the model ;

[0097] (3) Save the magnitude of risk loss and trajectory loss in this batch to prepare for the loss weight calculation of the next batch;

[0098] (4) According to Backpropagation updates the driving risk prediction model parameters ;

[0099] (5) Repeat (1)-(4) until the loss converges, then return to the driving risk prediction model. .

[0100] Beneficial effects

[0101] By treating trajectory prediction as an auxiliary task and sharing an encoder with risk assessment, this invention leverages the inherent connection between the two tasks to achieve effective feature sharing and knowledge transfer. Furthermore, it injects physical consistency regularization into the model through auxiliary loss, ultimately improving the accuracy and robustness of risk prediction. The specific reasons for its performance advantages are analyzed below:

[0102] (1) Essential Relevance: Driving risk is essentially determined by the vehicle's future motion trend and its interaction with surrounding targets (e.g., physical quantities such as speed change, acceleration, relative displacement, and collision time). Trajectory prediction directly models these future motion states, and therefore has a close semantic relationship with risk assessment.

[0103] (2) Representation learning and knowledge transfer: The main task (risk classification) and the auxiliary task (trajectory generation) share the same encoder. During training, the trajectory prediction task forces the encoder to learn more complete vehicle dynamics and vehicle interaction features from the trajectory data. This multi-task sharing enables the encoder to transfer fine-grained features useful for motion evolution to risk discrimination, making it difficult to capture the temporal causal relationship in a single risk classification loss.

[0104] (3) Inductive bias and regularization effect: Trajectory prediction, as an auxiliary loss, introduces an inductive bias with physical consistency into the model, reducing overfitting and improving generalization ability. More accurate representation of future trajectories makes risk judgment based on physical quantities (such as TTC, relative speed, path intersection distance, etc.) more reliable, thereby improving the accuracy and stability of risk prediction. Attached Figure Description

[0105] Figure 1 Schematic diagram of the processing flow of the method of this invention;

[0106] Figure 2 Network structure diagram of the driving risk prediction model in this embodiment of the invention;

[0107] Figure 3 Network structure diagram of trajectory prediction auxiliary task in this embodiment of the invention;

[0108] Figure 4 ROC curve of the model in this embodiment of the invention. Detailed Implementation

[0109] The technical solution provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0110] This invention proposes a fine-grained multi-task vehicle risk prediction method that integrates trajectory prediction as an auxiliary task. The method includes a main risk prediction task and a trajectory prediction auxiliary task. The networks for both tasks share feature representations at the encoding layer, achieving information collaboration and knowledge transfer between tasks. To address the issue of inconsistent loss scales in multi-task learning, a dynamic weight averaging strategy is adopted to adaptively adjust the loss weights for each task, thereby improving the model's training stability and prediction performance. The network structure of the vehicle risk prediction model used in this invention is as follows: Figure 2 .

[0111] Example

[0112] A fine-grained multi-task driving risk prediction method integrating trajectory prediction assistance includes the following steps: (e.g.) Figure 1 )

[0113] Step 1: Automatically label risk data

[0114] When collecting trajectory data, the target vehicle is the center, and the movement trajectories of surrounding vehicles are also recorded. Trajectory data is collected on a vehicle-by-vehicle basis, recording its motion state over continuous time slices. Core fields typically include vehicle ID, vehicle size parameters, timestamp (or frame number), and two-dimensional plane coordinates. Using a specific risk prediction time point as a boundary, trajectories before that time point are defined as historical trajectories, while trajectories after that point (whether to be predicted or actually observed) are defined as future trajectories.

[0115] A frame represents a snapshot of a traffic scene at a specific timestamp, including trajectory data of all observed vehicles and related road environment information at that moment. Road environment information may include lane number, lane geometry, lane topology, lane type, various semantic information, relationships between vehicles, and key vehicle interaction features such as distance between vehicles, used to describe the local traffic structure and the interactions between vehicles. Because the number of participating vehicles in different time frames changes dynamically, the vehicle dimension needs to be padded with a uniform length during tensor quantization.

[0116] To support modeling of a dynamic number of vehicles, a binary mask tensor is introduced. The mask indicates whether a vehicle slot in a frame corresponds to a real vehicle trajectory. Model computation only performs feature extraction and network forward propagation at the position where Mask=1, thus effectively shielding invalid trajectories generated by padding while maintaining the uniformity of the tensor structure.

[0117] A sample, or scenario, is a spatiotemporal segment extracted from a continuous frame sequence using a sliding window. Spatially, it is centered on the target vehicle and includes multi-vehicle trajectory data and corresponding road environment information within the target vehicle's neighborhood. Temporally, it covers a traffic segment spanning m+n seconds before and after the risk prediction time point; that is, the historical scene m seconds before the prediction time and the future scene n seconds after. Each sample is associated with a driving risk label, indicating whether a potential collision risk will occur to the target vehicle n seconds later, thus serving as a training and performance evaluation tool for the driving risk prediction model.

[0118] Step 1.1 Calculate the modified TTC risk index

[0119] The Time to Collision (TTC) metric is a commonly used risk indicator. Due to its simple calculation and clear physical meaning, it has been widely applied in micro-risk assessment tasks. Let the longitudinal position of the preceding vehicle be... ,speed Following position ,speed (Take the positive direction along the driving direction, and) Under the assumption that the speeds of both vehicles remain constant, the longitudinal collision time (TTC) is as follows:

[0120]

[0121] Time To Tightness (TTC) describes the remaining time before a rear-end collision occurs between two vehicles under conditions of no braking or acceleration. A lower TTC value indicates a higher risk of traffic accident. TTC makes a constant-speed assumption; to overcome this limitation, a longitudinal acceleration correction can be introduced into TTC, forming the Modified TTC (MTTC) index. This considers constant acceleration while following the vehicle. The car in front has constant acceleration. The relative displacements of the two vehicles satisfy:

[0122]

[0123] Find a way to reduce the longitudinal distance between the two vehicles to the set minimum safe distance. The smallest positive real root This is MTTC:

[0124]

[0125] The advantage of MTTC (Mean Traffic Control) lies in its ability to allow vehicles to move under constant acceleration conditions, thus distinguishing between typical traffic conflict scenarios such as "decelerating while maintaining a safe distance" and "accelerating to the point of a rear-end collision." When this occurs, it is determined to be a potential conflict event.

[0126] In this embodiment, the trajectory acquisition frequency is 10Hz. MTTC has better robustness for trajectory data with sampling frequencies between 5 and 10Hz, effectively smoothing out errors caused by instantaneous velocity fluctuations. MTTC threshold for risk events. Set to 2 seconds.

[0127] Step 1.2 Automatically identify scenario risks

[0128] For a vehicle v, in order to take timely evasive action, a risk warning is typically issued m seconds in advance. This involves using trajectory data from the past n seconds to predict the accident risk value m seconds later. Here, the past n seconds and the future m seconds form a scenario sample with a duration of n+m seconds. The MTTC (Mean Time To Cost) index at the future m-second moment needs to be calculated as a proxy indicator for the risk value.

[0129] Assuming the risk time is t, and considering the noise in the trajectory data, the MTTC index cannot be calculated using the instantaneous data at time t. Therefore, the MTTC set of the last L frames before time t is used. If the elements of the set satisfy:

[0130]

[0131] This scenario is then defined as a high-risk state and labeled accordingly. Otherwise, it will be recorded as low risk and labeled as such. .

[0132] In this embodiment, m=3 and n=2, meaning that vehicle trajectory data from the past 3 seconds is used to predict whether a risk event will occur in the next 2 seconds. The complete scene sample length is 5 seconds, and L=3 is used to determine whether a risk event has occurred based on the last 3 frames.

[0133] Step 2: Design of the main task for risk prediction

[0134] Step 2.1 Design of Vehicle Trajectory Feature Extraction Module

[0135] The encoder module is designed to extract spatiotemporal behavioral representations of the target vehicle and its surrounding environment. This part includes a two-stage structure: a single-vehicle trajectory feature extraction network and an inter-vehicle interaction module.

[0136] The model input is all valid vehicles in the past Frame trajectory tensor ,in Indicates the batch size. For the number of frames in the historical trajectory, This represents the maximum number of vehicles per frame. The input is first multiplied by the mask tensor. The missing trajectories are masked to obtain the effective trajectory tensor. :

[0137]

[0138] Valid vehicle trajectory data for each sample scene in the batch Trajectory feature extraction is performed, where For the first The number of valid vehicles in each sample scenario.

[0139] A trajectory feature extraction network is designed, employing a three-layer stacked Res1D network to perform one-dimensional convolutional modeling on the trajectory of each vehicle, extracting local temporal features. The outputs from each scale are fused to a unified dimension via lateral connections, and then passed through a residual module to output vehicle feature codes.

[0140]

[0141] in, This refers to the specified vehicle representation dimension, i.e., the sample scene. The effective vehicle characterization combination is as follows:

[0142]

[0143] in, For single vehicles The temporal features obtained after encoding the historical trajectory sequence have the following center position: .

[0144] Next, a two-layer attention mechanism is designed to model the interactions between vehicles, performing spatial attention modeling on each vehicle and its neighboring vehicles. Using the aligned vehicle center position as the geometric reference, attention is sequentially applied to each pair of target vehicles. and surrounding vehicles (The surrounding vehicles refer to those that meet the distance constraints) The model aggregates information from vehicles. At each layer, it constructs a contextual representation based on the relative positions of vehicles, the query vector, and neighbor features, and updates the target vehicle features through residual connections and normalization.

[0145]

[0146] in, and These represent the first and second layer attention aggregation modules, respectively. Indicates to vehicles The temporal features obtained after encoding the historical trajectory sequence. Indicates vehicle The central location, , They represent the target vehicles respectively. Intermediate features and final interaction features after two-layer attention aggregation Indicates the target vehicle The set of features of all neighboring vehicles within the neighborhood.

[0147] Final output This indicates the target vehicle that incorporates information from multiple levels of neighbors. The features are represented and used as shared inputs for subsequent risk assessment and trajectory prediction tasks.

[0148] In this embodiment, C=128. =100.

[0149] Step 2.2 Network Structure Design for the Main Task of Risk Prediction

[0150] The main task is to predict whether potential risk events will occur in the current scene after n seconds. This task not only makes judgments based on the feature representations of individual vehicles, but also incorporates contextual information between vehicles through a multi-head attention mechanism to capture potential collaborative risk relationships.

[0151] The input to the main risk prediction task is the vehicle feature information (vehicle embedding information) obtained from the encoder, denoted as...

[0152]

[0153] in, This represents the number of valid target vehicles in the current input sample. For feature dimensions.

[0154] The task first applies to each Self-attention updates are performed using the vehicle itself and other target vehicles, i.e.:

[0155]

[0156] in, This represents a standard multi-head attention mechanism, where both input and output dimensions are... Subsequently, the updated representation was... Perform a two-layer fully connected mapping to obtain the logit value for risk prediction. As shown below:

[0157]

[0158] During the inference phase, to obtain the probability output, a Sigmoid mapping can be further performed:

[0159]

[0160] in, This represents the Sigmoid function. For the weights of the fully connected layer, This is a bias term.

[0161] In model design, the multi-head attention mechanism enables the model to perceive the global semantic relationships between target vehicles, thereby improving the modeling ability of collaborative risk behaviors in the scenario.

[0162] In this embodiment, n=2, and in the multi-head attention mechanism, the number of attention heads is set to 4.

[0163] Step 2.3 Loss Function Design for the Main Risk Prediction Task

[0164] During training, risk prediction uses binary cross-entropy loss:

[0165]

[0166] in, It is the first Sample The true label is either (low risk) or 1 (high risk). It is the first Sample The "logit" value in the model output; yes The output of the sigmoid function represents the probability that the model predicts the sample to be high-risk (label 1), calculated as follows:

[0167]

[0168] Step 3: Design of the trajectory prediction auxiliary task

[0169] The trajectory prediction task, as an auxiliary task in the multi-task framework, shares the encoder structure with the main task and participates in joint training with dynamic weights in the total loss function.

[0170] Step 3.1 Network Structure Design for Trajectory Prediction Auxiliary Task

[0171] The trajectory prediction auxiliary task regresses the trajectory position of each target vehicle within a future time window, constructing a multi-time-step position prediction task. This module takes the vehicle feature representation (vehicle embedding information) output by the encoder as input, combines it with the trajectory center point position information, and predicts the future trajectory coordinate sequence. The network structure design of the auxiliary task is as follows: Figure 3 As shown.

[0172] No. The characteristics of each target vehicle are represented as follows: The corresponding historical trajectory center point is To enable the predicted trajectory to have multimodal characteristics, K independent multilayer perceptrons (MLPs) were used. Parallel modeling outputs a sequence of relative displacements for K predicted trajectories:

[0173]

[0174] in, This represents the predicted relative displacement sequence. It is then compared to the center point of the last historical frame. Add them together to get the predicted position for each trajectory:

[0175]

[0176] This structure avoids direct regression to absolute position, thus improving the stability of the prediction.

[0177] In this embodiment, K=6, and 6 modes are selected.

[0178] Step 3.2 Loss Function Design for Trajectory Prediction Auxiliary Task

[0179] During the training phase, the model first calculates all predicted trajectories. With the actual trajectory The average Euclidean distance is used to determine the optimal mode, and the trajectory with the smallest error is selected as the optimal mode, denoted as . The smoothed L1 loss is calculated only for the optimal trajectory as a regression loss term:

[0180]

[0181] in, This represents the number of target vehicles in the current training batch. To enhance the model's ability to discriminate multimodal outputs, the classification branch simultaneously scores the confidence of all candidate trajectories. Maximum margin classification loss is used. The model is encouraged to award higher scores to the optimal mode. The total loss for the final trajectory prediction task is defined as:

[0182]

[0183] in, This represents the loss weight balancing coefficient. The maximum margin classification loss is defined as follows:

[0184]

[0185] in, Indicates the number of vehicles. Indicates the number of modes. Indicates interval, This represents the score of the non-optimal mode trajectory of vehicle v. This represents the score of the optimal modal trajectory of vehicle m.

[0186] In this embodiment , .

[0187] Step 4: Construction and Training of the Driving Risk Prediction Model

[0188] Step 4.1 Overall Model Structure Design

[0189] The overall framework of the driving risk prediction model adopts an end-to-end structure, including a main task of risk prediction, an auxiliary task of trajectory prediction, and a shared encoder. The input historical trajectory first extracts local temporal features through the encoder module, then introduces an attention mechanism to model the interaction between the target vehicle and surrounding vehicles, capturing potential dynamic influence relationships. Based on the shared features output by the encoder, two decoding paths are established: the main task is binary risk classification, outputting the accident probability; the auxiliary task is trajectory point regression, outputting a trajectory prediction sequence for several future time steps. The model structure is as follows: Figure 2 As shown.

[0190] Step 4.2 Loss Function Design

[0191] To improve training stability and multi-task collaboration efficiency, the model loss function consists of binary cross-entropy, margin contrastive loss, and regression loss. The Dynamic Weighted Average (DWA) mechanism is used to adaptively adjust the loss weights of the main and auxiliary tasks, realizing a unified modeling process that combines feature sharing and target decoupling.

[0192]

[0193] in, and Calculated using the DWA algorithm.

[0194] Step 4.3 Training the driving risk prediction model

[0195] The specific process of iterative training is as follows:

[0196] (1) Input is the trajectory data of the target vehicle and its neighboring vehicles within a straight distance of less than D meters for m seconds, and label the risk according to the MTTC index;

[0197] (2) Calculate the loss function of the main task. Calculate the loss function of the auxiliary task. The loss weights are dynamically adjusted using a dynamic weighted averaging algorithm. and Calculate the total loss of the model ;

[0198] (3) Save the magnitude of risk loss and trajectory loss in this batch to prepare for the loss weight calculation of the next batch;

[0199] (4) According to Backpropagation updates the driving risk prediction model parameters ;

[0200] (5) Repeat (1)-(4) until the loss converges, then return to the driving risk prediction model. .

[0201] The dynamic weight averaging algorithm automatically adjusts the optimization intensity of each task. If the loss of a task decreases slowly, it indicates high training difficulty, so its weight is increased to allow the model to focus more on that task; conversely, if the loss decreases slowly, it indicates easy training, so its weight is decreased accordingly. The specific formula for weight updates is shown below:

[0202]

[0203] in, Indicates task The ratio of the loss in the (t-1)th iteration to the loss in the (t-2)th iteration reflects the convergence speed of the task loss. Indicates to Softmax normalization is performed, and the loss weights assigned to task i in the t-th iteration are used. I represents the number of tasks, and T is the temperature coefficient.

[0204] In this embodiment, D=200, m=3. When calculating the dynamic weights, I is 2 and T is 5.

[0205] Step 5: Comparative Experiment and Advancement Analysis

[0206] Step 5.1 Verification of the effectiveness and necessity of the model design

[0207] Verification experiments of the method of this invention were conducted based on the NGSIM trajectory dataset. The NGSIM (Next Generation Simulation) trajectory data comes from the U.S. Federal Highway Administration and was collected from the US-101 freeway in Los Angeles. Multiple elevated cameras were set up on 5 main lanes and 1 auxiliary lane. The collected video was processed to obtain lane-level high-precision trajectory data of approximately 6,150 vehicles with a sampling frequency of 10Hz, covering a variety of typical traffic scenarios such as free flow, traffic merging, and congestion.

[0208] The 5-second scene samples were divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5. Performance metrics were calculated on the test set, including AUC (Area Under Curve), precision, recall, and F1 score. The ROC curve (Receiver Operating Characteristic Curve) of the model on the test set is shown below. Figure 4As shown, the horizontal and vertical axes represent the FPR (False Positive Rate) and TPR (True Positive Rate), respectively. It can be seen that the curve is close to the upper left corner, indicating that the model has a strong ability to predict risk. The corresponding AUC (Area Under the ROC Curve) value is 0.9305, significantly higher than the level of random guessing (AUC=0.5), indicating that the model effectively captures potential traffic risk characteristics from the trajectory data.

[0209] To verify the contribution of each key module in the proposed model to the overall performance, multiple ablation experiments were designed. From the perspectives of task structure, weight scheduling mechanism, and interaction modeling method, specific components were gradually removed or replaced, and the impact on model performance was observed. All experiments were conducted on the same dataset and training configuration, and the results are shown in Table 1.

[0210] Table 1. Comparison of risk assessment performance under different model configurations

[0211]

[0212] As can be seen, the multi-task learning model outperforms the single-task model overall, exhibiting more stable performance across all four metrics. This demonstrates that unifying risk assessment and trajectory prediction in modeling effectively leverages their synergistic effect, enhancing the model's ability to identify potential risk states while simultaneously perceiving vehicle movement trends. In contrast, the single-task model focuses solely on risk output, lacking supplementary modeling of scene dynamics through auxiliary tasks. This results in insufficient understanding of complex interactive scenarios and difficulty in establishing stable and generalizable risk assessment boundaries.

[0213] It's worth noting that removing DWA resulted in a relatively limited decrease in AUC and recall (AUC remained at 0.91, and recall was even slightly higher). This indicates that under the current data distribution and loss scale, the learning conflict between the two tasks is not significant, and even with fixed weights, a certain degree of multi-task collaborative training effect can be maintained. In more complex multi-task scenarios with greater task differences, dynamic weighting mechanisms may play a greater role.

[0214] Step 5.2 Baseline Method Selection

[0215] To verify the advancement of the driving risk prediction model proposed in this invention, a publicly available and representative baseline method was selected for comparison, specifically the vehicle group (VG)-based accident risk prediction framework proposed in the paper "Vehicle-Group-Based Crash Risk Prediction and Interpretation on Highways", with a prediction time of 1 second.

[0216] Step 5.3 Performance Indicator Comparison

[0217] Table 2 summarizes the performance of the present invention and baseline methods (including VGRL (Vehicle Group's Risk Level) and TRSC (Trend of Risk Scale Change) methods). The baseline method metrics are from the paper, while the metrics of the present invention are from experimental results.

[0218] Table 2 Performance comparison of different methods in risk assessment tasks

[0219]

[0220] As can be seen from the table, our method achieves the best performance with an AUC of 0.93. Although the results are comparable to those of the VGRL method at a prediction time of 1 second, our method requires twice the prediction time of VGRL (the longer the prediction time, the greater the prediction difficulty). In summary, the comparison of AUC metrics demonstrates the advantages of our method.

[0221] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. A fine-grained multi-task driving risk prediction method integrating trajectory prediction auxiliary tasks, characterized in that, Includes the following steps: Step 1: Automatically label risk data based on trajectory data; Collect trajectory data, calculate the corrected collision time MTTC risk index, identify risk scenarios based on the MTTC index, and label the risks. Step 2: Design of the main task for risk prediction; The main risk prediction task is used to predict whether there are potential risk events in the current scene after n seconds. The input trajectory data is first extracted by the encoder module to extract the spatiotemporal behavior representation of the target vehicle and its surrounding environment, and then the risk prediction main task network predicts the probability that the input sample has a risk. Step 3: Design of trajectory prediction auxiliary task; The trajectory prediction auxiliary task is an auxiliary task in the multi-task framework. It shares the encoder structure with the risk prediction main task. The trajectory prediction auxiliary task network takes the vehicle feature representation output by the encoder as input, combines the trajectory center point location information, predicts the future trajectory coordinate sequence, and participates in joint training with dynamic weights in the total loss function. Step 4: Construction and training of the driving risk prediction model; A driving risk prediction model framework is constructed, including the main task of risk assessment and the auxiliary task of trajectory prediction. The model loss function considers the losses of the main and auxiliary tasks, and a dynamic weight averaging mechanism is used to adaptively adjust the loss weights of the main and auxiliary tasks. Iterative training is carried out until the loss converges. In the inference phase, the trained driving risk prediction model is used to predict the risk of the input trajectory data.

2. The fine-grained multi-task vehicle risk prediction method for fusion trajectory prediction auxiliary task according to claim 1, characterized in that, Step 1 is as follows: Step 1.1 Calculate the revised TTC risk index; The corrected collision time risk index is calculated as follows: Let the longitudinal position of the front vehicle be... ,speed Following position ,speed Take the positive direction along the driving direction, and Under the assumption that the speeds of both vehicles remain constant, the longitudinal collision time (TTC) is as follows: A longitudinal acceleration correction is introduced into TTC to form the modified TTC index MTTC; constant acceleration while following the vehicle is considered. The car in front has constant acceleration. The relative displacements of the two vehicles satisfy: Find a way to reduce the longitudinal distance between the two vehicles to the set minimum safe distance. The smallest positive real root This is MTTC: when At that time, it was determined to be a potential conflict event, among which The preset threshold value; Step 1.2 Automatically identify scenario risks; Take the set of MTTCs for the last L frames before time t: If the elements of the set satisfy: This scenario is then defined as a high-risk state and labeled accordingly. Otherwise, it will be recorded as low risk and labeled as such. ; Process the collected trajectory data, identify risk scenarios based on the MTTC index, and label them with risk tags.

3. The fine-grained multi-task vehicle risk prediction method for fusion trajectory prediction auxiliary task according to claim 1, characterized in that, Step 2 is as follows: Step 2.1 Vehicle trajectory feature extraction; The encoder module is designed to extract the spatiotemporal behavioral representation of the target vehicle and its surrounding environment. The encoder includes a two-stage structure: a single-vehicle trajectory feature extraction network and an inter-vehicle interaction module. The model input is all valid vehicles in the past Frame trajectory tensor ,in Indicates the batch size. For the number of frames in the historical trajectory, The maximum number of vehicles per frame is 2, and the dimension of the trajectory data is 2; the input is first multiplied by the mask tensor. The missing trajectories are masked to obtain the effective trajectory tensor. : Valid vehicle trajectory data for each sample scene in the batch Trajectory feature extraction is performed, where For the first The number of valid vehicles in each sample scenario; The single-vehicle trajectory feature extraction network employs a three-layer stacked Res1D network to perform one-dimensional convolutional modeling on the trajectory of each vehicle, extracting local temporal features. The outputs from each scale are fused to a unified dimension via lateral connections, and then passed through a residual module to output vehicle feature codes, as follows: in, The specified vehicle representation dimension; i.e., the sample scenario. The effective vehicle characterization combination is as follows: in, For single vehicles The temporal features obtained after encoding the historical trajectory sequence have the following center position: ; The vehicle-to-vehicle interaction module uses a two-layer attention mechanism to model the mutual influence between vehicles, performing spatial attention modeling on each vehicle and its neighboring vehicles; using the aligned vehicle center position as the geometric reference, it sequentially performs attention modeling on each pair of center positions. and target vehicle and surrounding vehicles Information aggregation is performed; in each layer, the model constructs a contextual representation based on the relative positions of vehicles, query vectors, and neighbor features, and updates the target vehicle features through residual connections and normalization. in, and These represent the first and second layer attention aggregation modules, respectively. Indicates to vehicles The temporal features obtained after encoding the historical trajectory sequence. Indicates vehicle The central location, , They represent the target vehicles respectively. The intermediate features and final interaction features after two-layer attention aggregation Indicates the target vehicle The set of features of all neighboring vehicles within the neighborhood; Final output This indicates the target vehicle that incorporates information from multiple levels of neighbors. The feature representation serves as a shared input for subsequent risk assessment and trajectory prediction tasks; Step 2.2 Network structure design for the main task of risk prediction; The main task of risk prediction is to predict whether there are potential risk events in the current scenario after n seconds. This task not only makes judgments based on the feature representation of a single vehicle, but also introduces contextual information between vehicles through a multi-head attention mechanism to capture potential collaborative risk relationships. The input to the main risk prediction task network is the vehicle feature information obtained from the encoder, denoted as: in, This represents the number of valid target vehicles in the current input sample. For feature dimension, the first The characteristics of each target vehicle are represented as follows: The task first applies to each Self-attention updates are performed using the vehicle itself and other target vehicles, i.e.: in, This represents a standard multi-head attention mechanism. These are the Query, Key, and Value matrices for a multi-head attention mechanism, with both input and output dimensions being [missing information]. ; then the updated representation Perform a two-layer fully connected mapping to obtain the logit value for risk prediction. As shown below: During the inference phase, to obtain the probability output that the sample is risky, a Sigmoid mapping is further performed: in, This represents the Sigmoid function. For the weights of the fully connected layer, For bias terms; Step 2.3 Design of the loss function for the main task of risk prediction; During training, risk prediction uses binary cross-entropy loss: in, It is the first Sample The true label is either 0 or 1; the true label is obtained in step 1, where 1 indicates a high-risk state and 0 indicates a low-risk state. It is the first Sample The logit value in the model output; yes The output of the sigmoid function represents the probability that the model predicts the sample to be at risk, calculated as follows: 。 4. The fine-grained multi-task vehicle risk prediction method for fusion trajectory prediction auxiliary task according to claim 1, characterized in that, Step 3 is as follows: Step 3.1 Network structure design for trajectory prediction auxiliary task; The trajectory prediction auxiliary task is used to regress the trajectory position of each target vehicle within a future time window, thus constructing a multi-time-step position prediction task. The trajectory prediction auxiliary task network takes the vehicle feature representation output by the encoder as input and combines it with the trajectory center point location information to predict the future trajectory coordinate sequence; in order to make the predicted trajectory have multimodal characteristics, it generates K different predicted trajectories at the same time. No. The characteristics of each target vehicle are represented as follows: The corresponding historical trajectory center point is Using K independent multilayer perceptrons to Parallel modeling outputs a sequence of relative displacements for K predicted trajectories: in, Represent the predicted relative displacement sequence; compare it with the center point of the last historical frame. Add them together to get the predicted position for each trajectory: Step 3.2 Design of the loss function for the trajectory prediction auxiliary task; During the training phase, the model first calculates all predicted trajectories. With the actual trajectory The average Euclidean distance is used to determine the optimal mode, and the trajectory with the smallest error is selected as the optimal mode, denoted as . The smoothed L1 loss is calculated only for this trajectory as the regression loss term. in, This represents the number of target vehicles in the current training batch; to enhance the model's ability to discriminate multimodal outputs, the classification branch simultaneously scores the confidence of all candidate trajectories; maximum margin classification loss is used. The model is encouraged to give higher scores to the optimal mode; the total loss of the final trajectory prediction task is defined as: in, The weighting coefficients are the loss weighting coefficients; the maximum margin classification loss is defined as follows: in, Indicates the number of vehicles. Indicates the number of modes. Indicates interval, This represents the score of the non-optimal mode trajectory of vehicle v. This represents the score of the optimal modal trajectory of vehicle m.

5. The fine-grained multi-task vehicle risk prediction method for fusion trajectory prediction auxiliary task according to claim 1, characterized in that, Step 4 is as follows: Step 4.1 Overall structural design of the driving risk prediction model; The overall framework of the driving risk prediction model adopts an end-to-end structure, including the main task of risk assessment, the auxiliary task of trajectory prediction, and a shared encoder. The input historical trajectory first extracts local temporal features through the encoder, and then introduces an attention mechanism to model the interaction between the target vehicle and surrounding vehicles to capture potential dynamic influence relationships. On the shared features output by the encoder, two decoding paths are established: the main task is binary classification risk assessment, which outputs the accident probability. The auxiliary task is trajectory point regression, which outputs a trajectory prediction sequence for several future time steps; Step 4.2 Design of loss function for driving risk prediction model; To improve training stability and multi-task collaboration efficiency, the model loss function considers the weighted sum of the main and auxiliary tasks, and adopts the Dynamic Weighted Average (DWA) algorithm to adaptively adjust the loss weights of the main and auxiliary tasks, thereby realizing a unified modeling process of feature sharing and target decoupling. in, , These are the loss functions for the primary and secondary tasks, respectively. and Calculated by the DWA algorithm; Step 4.3 Training the driving risk prediction model; The specific process of iterative training is as follows: (1) Input is the trajectory data of the target vehicle and its neighboring vehicles within a straight line distance of less than D meters for m seconds. Identify risk scenarios based on the MTTC index and label them with risk tags. (2) Calculate the loss function of the main task. Calculate the loss function of the auxiliary task. The loss weights are dynamically adjusted using a dynamic weighted averaging algorithm. and Calculate the total loss of the model ; (3) Save the magnitude of risk loss and trajectory loss in this batch to prepare for the loss weight calculation of the next batch; (4) According to Backpropagation updates the driving risk prediction model parameters ; (5) Repeat (1)-(4) until the loss converges, then return to the driving risk prediction model. .

6. The fine-grained multi-task vehicle risk prediction method for fusion trajectory prediction auxiliary task according to claim 5, characterized in that, The specific formula for updating the loss weights of the primary and secondary tasks using the DWA algorithm is shown below: in, Indicates task Loss in the (t-1)th iteration The loss at the (t-2)th time The ratio reflects the convergence speed of the task loss; Indicates to Softmax normalization is performed, which is the loss weight assigned to task i in the t-th iteration; I represents the number of tasks, and T is the temperature coefficient.

Images