Vehicle lane changing risk assessment method and device, computer equipment and storage medium

Abstract

The application relates to a risk assessment method and device for lane changing of a vehicle, a computer device and a storage medium. The method comprises the following steps: acquiring sample data in a plurality of lane changing scenarios; constructing a training data set by using the sample data, and assigning and marking the sample data in the training data set; inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model; and inputting sample data corresponding to any lane changing scenario into the ResNet-LSTM model to output a risk assessment result. The application takes sample data in a lane changing scenario as input, does not need to process errors caused by methods such as environment perception and trajectory prediction, extracts spatial feature data based on a time dimension, more accurately identifies scene information, and more accurately assesses the risk probability of each lane changing scenario.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a method, apparatus, computer equipment, and storage medium for risk assessment of vehicle lane changes. Background Technology

[0002] With the widespread application of artificial intelligence in the field of autonomous driving, risk assessment of various driving scenarios can help autonomous driving systems predict and avoid dangerous situations.

[0003] Vehicles may encounter situations where other vehicles change lanes and cut in line while driving. According to current big data statistics on traffic accidents, whether on highways, elevated roads, or urban roads, accidents caused by vehicles changing lanes and cutting in line consistently account for a large proportion. Therefore, the application of artificial intelligence in risk assessment of vehicle lane-changing and cutting-in scenarios is particularly widespread.

[0004] Some existing technical solutions for risk assessment have the following shortcomings:

[0005] (1) In the trajectory and intent prediction-based scheme, deep learning is used to predict the lane changes and trajectories of surrounding vehicles, thereby determining whether there is a risk of collision with another vehicle changing lanes in the future; such as Figure 1 As shown, a deep learning algorithm is used to train and evaluate a behavior prediction model using the NGSIM dataset to classify driving intentions and obtain the trajectory probability distribution of autonomous vehicles. The intention model is constructed based on LSTM, with a time series length of 6, a hidden dimension of 128, a learning rate of 0.000125, and softmax cross-entropy as the training loss. Preprocessed vehicle driving data and vehicle driving intention data are input into the trajectory prediction model to obtain the predicted trajectory of the vehicle. The collision time (TTC), headway (TH), and enhanced collision time (ETTC) are calculated based on the predicted trajectory. However, this method relies excessively on the prediction results of vehicle intentions and trajectories, with too many intermediate steps. Furthermore, the current prediction of vehicle intentions and trajectories depends on the accuracy of environmental perception. Therefore, the final risk assessment result will be compounded by errors from previous steps, resulting in a reduced risk assessment effectiveness. Moreover, deep learning solutions are data-driven, and collecting and mining training data from various traffic scenarios in real-world traffic environments is a time-consuming and labor-intensive task that requires substantial financial support.

[0006] (2) Based on deep learning operators, features are extracted from video data of driving records. Through perceptual data encoding, risk assessment is performed to determine whether a collision occurs due to another vehicle changing lanes. This technique uses a single frame of the original RGB image as input, performs semantic segmentation using a Mask R-CNN model trained on the COCO dataset, and assigns high-resolution pixel values ​​to the masked portion, transforming the original RGB image into a masked image. Then, a CNN network is used to extract features from the masked image, and finally, fully connected layers and activation functions are used for risk assessment and classification. However, since the input of a single frame image lacks temporal information, while lane-changing scenarios have temporal contextual relationships, for example, the target vehicle's slow speed in the preceding frames may become the target vehicle's original motivation for choosing to change lanes, thus allowing the target vehicle to recognize the lane-changing behavior. Therefore, relying solely on a single frame image for scene recognition, the model cannot learn the relationship features between preceding and following frames, resulting in poor robustness. Summary of the Invention

[0007] Based on this, it is necessary to provide a risk assessment method, device, computer equipment, and storage medium for vehicle lane changing to address the above-mentioned technical problems. This method uses a sequence of lane change frame images in a lane change scenario as input, eliminating the need to consider error processing caused by algorithms such as environmental perception and trajectory prediction. Furthermore, based on spatial feature data in the time dimension, it can more accurately identify scene information, thereby outputting risk assessment results for each lane change scenario more accurately.

[0008] This invention provides a method for risk assessment of vehicle lane changes, the method comprising:

[0009] Obtain sample data for several lane-changing scenarios;

[0010] Construct a training dataset using sample data, and assign values ​​and labels to the sample data in the training dataset;

[0011] The training dataset is input into a pre-built ResNet-LSTM network to train the model and obtain the ResNet-LSTM model.

[0012] Input the sample data corresponding to any lane change scenario into the ResNet-LSTM model and output the risk assessment result.

[0013] In one implementation, the step of acquiring sample data for several lane-changing scenarios includes:

[0014] Based on the system and graphics card environment, a UE4 engine was built so that the CARLA application could be loaded using the UE4 engine;

[0015] Create the simulation environment required for lane change scenarios using the server-side application in CARLA.

[0016] The first vehicle and the second vehicle are determined by a client in the CARLA application, wherein the first vehicle is the main vehicle and the second vehicle is a vehicle changing lanes relative to the first vehicle in the adjacent lane.

[0017] By adjusting the lane change parameters of the Python API in the CARLA application, the lane change time, speed, and abruptness of the second vehicle are controlled to generate a sequence of lane change frame images of the second vehicle from the perspective of the first vehicle in different lane change scenarios, and the sequence of lane change frame images is used as sample data for each lane change scenario.

[0018] In one implementation, the step of constructing a training dataset using sample data and labeling the sample data in the training dataset as safe or hazardous includes:

[0019] Multiple hazard levels are pre-defined, and a hazard coefficient is assigned to each hazard level;

[0020] For sample data of the same lane-changing scenario, discrete random variables are used to repeatedly label the danger level of the current lane-changing scenario;

[0021] Based on the preset risk coefficients for each risk level, the average risk level coefficient of the current lane change scenario is calculated, and the average risk coefficient is used as the final risk coefficient.

[0022] Based on the median and final hazard coefficients of multiple hazard coefficients, the assigned values ​​for the current sample data to be labeled are obtained through threshold binarization.

[0023] In one implementation, the step of inputting the training dataset into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model includes:

[0024] Spatial features are extracted from the sample data using the ResNet structure in the ResNet-LSTM network to obtain a set of spatial feature sequences.

[0025] By using the LSTM structure in the ResNet-LSTM network, temporal features are extracted from spatial feature sequences to obtain spatial feature data based on the time dimension.

[0026] Based on the assigned values ​​of the sample data, the Softmax classifier is used to obtain the probability values ​​of safety or danger, and then the cross-entropy loss function is used to calculate the loss between the probability values ​​and the assigned values.

[0027] In one implementation, the step of extracting spatial features from sample data using the ResNet structure in a ResNet-LSTM network to obtain a set of spatial feature sequences includes:

[0028] Obtain the corresponding lane change frame image sequence for each lane change scenario in the training dataset;

[0029] Using the pre-defined ResBlock unit structure in the ResNet architecture, spatial features are extracted from each lane change frame image in the lane change frame image sequence based on a predetermined feature dimension, in order to obtain a spatial feature sequence with a predetermined feature dimension.

[0030] In one implementation, the step of extracting temporal features from spatial feature sequences using the LSTM structure in a ResNet-LSTM network to obtain time-dimensional spatial feature data includes:

[0031] Obtain the spatial feature sequence, where the spatial feature sequence input at each time step in the LSTM structure corresponds to the output of the ResNet structure;

[0032] Using the spatial feature sequence at each time step, the learning parameters of the hidden layer, input gate, output gate, forget gate, long memory, and short memory in the LSTM cell architecture are iteratively updated, as well as the cell state of the LSTM cell architecture is updated.

[0033] Based on the updated learning parameters and cell state at each time step, temporal features are extracted from the spatial feature sequence to obtain spatial feature data based on the time dimension.

[0034] In one implementation, the steps of obtaining a safe or dangerous probability value using a Softmax classifier based on the assigned values ​​of the sample data, and then calculating the loss between the probability value and the assigned values ​​using a cross-entropy loss function, include:

[0035] Acquire spatial feature data based on the time dimension;

[0036] Based on the assigned values ​​of the sample data, the probability values ​​of safety or danger are calculated using the Softmax classifier.

[0037] The cross-entropy loss function is used to calculate the loss between the probability value and the assigned value, where,

[0038]

[0039] Where loss represents the loss, y i ′ This represents the assignment of the i-th sample data, y i ′ =0 indicates safety, y i ′ =1 indicates danger.

[0040] This invention provides a risk assessment device for vehicle lane changes, the device comprising:

[0041] The data acquisition module is used to acquire sample data under several lane change scenarios;

[0042] The data annotation module is used to construct a training dataset using sample data and to assign labels to the sample data in the training dataset.

[0043] The model training module is used to input the training dataset into a pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model.

[0044] The risk assessment module is used to input sample data corresponding to any lane change scenario into the ResNet-LSTM model and output the risk assessment results.

[0045] This invention provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0046] Obtain sample data for several lane-changing scenarios;

[0047] Construct a training dataset using sample data, and assign values ​​and labels to the sample data in the training dataset;

[0048] The training dataset is input into a pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model;

[0049] Input the sample data corresponding to any lane change scenario into the ResNet-LSTM model and output the risk assessment result.

[0050] This invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the following steps:

[0051] Obtain sample data for several lane-changing scenarios;

[0052] Construct a training dataset using sample data, and assign values ​​and labels to the sample data in the training dataset;

[0053] The training dataset is input into a pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model;

[0054] Input the sample data corresponding to any lane change scenario into the ResNet-LSTM model and output the risk assessment result.

[0055] The aforementioned risk assessment methods, devices, computer equipment, and storage media for vehicle lane changes utilize a constructed simulation environment to acquire sample data from several lane-changing scenarios. This provides a sufficient amount of model training data without requiring the collection and mining of training data from real-world traffic scenarios, reducing manpower and material costs and simplifying the data collection process. The target model can be trained according to accuracy requirements. For example, CARLA+UE4, as a virtual data production tool, can generate an unlimited number of samples needed for model training, reducing the time and financial costs of sample collection. Furthermore, before model training, a training dataset is constructed using the acquired sample data, and the sample data in the training dataset is labeled and assigned values ​​for use in the Softmax classifier within the LSTM structure to calculate the probability values ​​of safety or danger, simplifying the scene recognition process. Furthermore, in the step of inputting the training dataset into the pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model, the sequence of lane change frame images in the lane change scenario can be used as input, without using the trajectory prediction results as input. This avoids the introduction of errors in the trajectory prediction algorithm, and directly using sensor data makes the model more independent and more applicable. Moreover, it can add time dimension information, and multi-frame input is more conducive to the model learning context information, which helps the model to understand the scene content more accurately and fully utilize the model's transferability and robustness. The virtual training dataset can theoretically be generated infinitely, which can easily meet the training requirements. Attached Figure Description

[0056] Figure 1 This is a structural block diagram of a trajectory and intent prediction scheme in the background technology.

[0057] Figure 2 This is a flowchart illustrating a risk assessment method for vehicle lane changing in one embodiment of this application;

[0058] Figure 3 In one embodiment of this application, CARLA is used to generate lane change frame images;

[0059] Figure 4 In one embodiment of this application, CARLA is used to generate lane change frame images;

[0060] Figure 5 This is a schematic diagram of the ResNet-LSTM model structure in one embodiment of this application;

[0061] Figure 6 This is a schematic diagram of the network structure for spatial feature extraction in one embodiment of this application;

[0062] Figure 7 This is a schematic diagram of the network structure for temporal feature extraction in one embodiment of this application;

[0063] Figure 8 This is a structural block diagram of a vehicle lane change risk assessment device in one embodiment of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] This application provides a risk assessment method for vehicle lane changes. Whether driving manually or autonomously, risk assessment of driving scenarios can effectively predict and avoid dangerous situations. The lane change risk assessment method implemented in this embodiment effectively eliminates reliance on the output of environmental perception algorithms and vehicle trajectory prediction algorithms, avoiding the inability of the model to extract temporal features due to a lack of contextual information, and avoiding the significant time and labor costs associated with data collection.

[0066] In one embodiment, such as Figure 2 As shown, a risk assessment method for vehicle lane changes is provided, which includes the following steps.

[0067] Step S100: Obtain sample data for several lane-changing scenarios.

[0068] Step S200: Construct a training dataset using the sample data, and assign values ​​and labels to the sample data in the training dataset.

[0069] Step S300: Input the training dataset into the pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model.

[0070] Step S400: Input the sample data corresponding to any lane change scenario into the ResNet-LSTM model and output the risk assessment result.

[0071] In one implementation, step S100, the step of acquiring sample data under several lane-changing scenarios, includes:

[0072] Step S110: Based on the operating system and graphics card environment, build the UE4 engine and load the CARLA application using the UE4 engine. In one implementation, an Ubuntu 20.04 system and an NVIDIA RTX 3090 graphics card environment are used to build the UE4 engine. UE4 stands for Unreal Engine 4, a design engine developed by Epic Games. The CARLA application is an open-source autonomous driving simulation simulator, also known as an autonomous driving simulation platform. It supports the generation of various standardized sensors, environments, dynamic and static NPCs for autonomous driving systems to control vehicles in a virtual environment for testing. It can create test environments that closely resemble reality for autonomous driving systems to control vehicles for development and testing. The CARLA application also provides digital assets such as city, vehicle, and sensor models. The CARLA application includes a server and a client. The server simulates the simulation environment, while the client receives data from the simulation and modifies the state of things in the simulation. The client transmits operation commands to the server via IP address and port to achieve modification.

[0073] Step S120: Create the simulation environment required for the lane change scenario through the server in the CARLA application.

[0074] Step S130: The first vehicle and the second vehicle are determined by the client in the CARLA application, wherein the first vehicle is the main vehicle and the second vehicle is the vehicle changing lanes relative to the first vehicle in the adjacent lane.

[0075] Step S140: By adjusting the lane change parameters of the Python API in the CARLA application, the lane change time, speed and abruptness of the second vehicle are controlled to generate a sequence of lane change frame images of the second vehicle from the perspective of the first vehicle in different lane change scenarios, and the sequence of lane change frame images is used as sample data for each lane change scenario.

[0076] In this implementation, the CARLA application's built-in Python API is used to obtain all coordinate points on the map where various actors can be generated. A coordinate point is randomly selected as the first vehicle generation point. Before determining the first vehicle generation point, it is determined whether the selected point meets the requirements for a lane-changing scenario. If not, a new first vehicle generation point is selected; alternatively, it is determined whether the lane containing the selected first vehicle generation point is a single lane. If so, a new first vehicle generation point is selected. After selecting a suitable first vehicle generation point, a second vehicle is selected from the adjacent lanes that is not far from the first vehicle in longitudinal distance. The parameters in the `Lanechange` function of the Python API are adjusted to control the speed and abruptness of the second vehicle's lane change, thereby generating a lane-changing scenario that poses a certain danger to the first vehicle. Figure 3-4 As shown, this is a lane change frame image generated using the CARLA application.

[0077] In one implementation, step S200, which involves constructing a training dataset using sample data and labeling the sample data in the training dataset as safe or hazardous, further includes:

[0078] Step S210: Pre-set multiple hazard levels and assign a hazard coefficient to each hazard level.

[0079] Step S220: For sample data of the same lane change scenario, use discrete random variables to repeatedly calibrate the danger level of the current lane change scenario.

[0080] Step S230: Based on the preset risk coefficients for each risk level, calculate the average risk coefficient of the current lane change scenario, and use the average risk coefficient as the final risk coefficient.

[0081] Step S240: Based on the intermediate values ​​of multiple risk coefficients and the final risk coefficient, the value to be labeled for the current sample data is obtained by binarization based on the threshold.

[0082] In one implementation, five danger levels are pre-defined, and for ease of calculation, danger coefficients are assigned to these five levels: -2, -1, 0, 1, and 2. The training dataset contains sample data for several lane-changing scenarios. For the same lane-changing scenario, the danger level of the current lane-changing scenario can be repeatedly labeled using discrete random variables. This can be achieved by selecting N (e.g., 20) drivers with varying experience levels to label the danger level of the current lane-changing scenario. Based on the pre-assigned danger coefficients for each danger level, the mean of the current danger coefficients is calculated. Since the assigned danger coefficients in this embodiment are -2, -1, 0, 1, and 2, the mean danger coefficients greater than level 0 (since 0 is the median value in this case) can be classified as dangerous and represented by "1," while the mean danger coefficients less than 0 (since 0 is the median value in this case) can be classified as safe and represented by "0." Therefore, for the sample data corresponding to the current lane-changing scenario, the assigned values ​​are calculated based on threshold binarization as follows:

[0083]

[0084] In one embodiment, step S300 involves inputting the training dataset into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model.

[0085] Reference Appendix Figure 5 As shown, the steps of inputting the training dataset into a pre-built ResNet-LSTM network for model training include:

[0086] Step S310: Using the ResNet structure in the ResNet-LSTM network, spatial features are extracted from the sample data to obtain at least one set of spatial feature sequences;

[0087] Step S320: Using the LSTM structure in the ResNet-LSTM network, temporal features are extracted from the spatial feature sequence to obtain spatial feature data based on the time dimension.

[0088] Step S330: Based on the assigned values ​​of the sample data, use the Softmax classifier to obtain the probability values ​​of safety or danger, and then use the cross-entropy loss function to calculate the loss between the probability values ​​and the assigned values.

[0089] Then, based on the probability value calculated by the cross-entropy loss function and the assigned loss, the parameters in the ResNet-LSTM network are continuously adjusted to achieve the purpose of model training.

[0090] Reference Appendix Figure 6As shown, in one embodiment, step S310, which involves extracting spatial features from the sample data using the ResNet structure in the ResNet-LSTM network to obtain a set of spatial feature sequences, further includes:

[0091] Obtain the corresponding lane change frame image sequence for each lane change scenario in the training dataset;

[0092] By utilizing the pre-defined ResBlock unit structure in the ResNet architecture, spatial features are extracted from each lane change frame image in the corresponding lane change frame image sequence based on a predetermined feature dimension, thereby obtaining a spatial feature sequence with a predetermined feature dimension.

[0093] The basic unit structure of the ResNet structure in this embodiment is the ResBlock structure. The ResNet structure represents a residual network structure, which can be understood as the preprocessing structure for lane change frame image sequences in the ResNet-LSTM network structure. Of course, other models trained on large classification datasets can also be used. The ResBlock structure in this embodiment includes a three-layer structure. The first layer uses a predetermined convolutional kernel (e.g., 1*1 convolution) to compress the number of channels in the lane change frame image sequence, obtaining a compressed feature map of the lane change frame image sequence. The second layer performs feature extraction on the compressed feature map. The input dimension of the ResBlock structure is set to batch_size*sequence_length*3*72*128. After spatial feature extraction, a spatial feature sequence with a spatial feature dimension of batch_size*sequence_length*2048*1*1 is obtained. The third layer again uses a predetermined convolutional kernel (1*1 convolution) to restore the number of channels in the spatial feature sequence.

[0094] In this embodiment, the step of extracting spatial features from sample data using a ResNet structure to obtain a set of spatial feature sequences includes, in sequence, an input layer, a convolutional layer, a pooling layer, a ResBlock structure, a pooling layer, an overfitting prevention layer, a first fully connected layer, a second fully connected layer, and an output layer. Further representation is as follows:

[0095] x j →C(64, 1, 1)→P→ResNet backbone→P→D→FC(200)→FC(50)→z j .

[0096] Where, x j = (x1, x2, ..., x T) represents the sequence of lane change frame images, C(64, 1, 1) represents the convolution process with a stride of 64 and a kernel size of (1, 1), P represents pooling, ResNet backbone represents spatial feature extraction using the ResBlock structure, D represents the dropout layer (to prevent overfitting), in one embodiment the dropout rate is set to 0.2, and FC represents the fully connected layer.

[0097] Reference Appendix Figure 7 As shown, in one embodiment, step S320, which involves extracting temporal features from the spatial feature sequence using the LSTM (Long Short Term Memory) structure in the ResNet-LSTM network to obtain spatial feature data based on the time dimension, further includes:

[0098] Obtain the spatial feature sequence, where the spatial feature sequence input at each time step in the LSTM structure corresponds to the output of the ResNet structure;

[0099] Using the spatial feature sequence at each time step, the learning parameters of the hidden layer, input gate, output gate, forget gate, long memory, and short memory in the LSTM cell architecture are iteratively updated, as well as the cell state of the LSTM cell architecture is updated.

[0100] Based on the updated learning parameters and cell state at each time step, temporal features are extracted from the spatial feature sequence to obtain spatial feature data based on the time dimension.

[0101] In this context, the spatial feature sequence input at each time step in the LSTM structure corresponds to the output of the ResNet structure, and can be represented as:

[0102] y = f LSTM (f ResNet50 (x1), ..., f ResNet50 (x j ), ..., f ResNet50 (x T ),).

[0103] The LSTM structure in this implementation includes a forget gate, an input gate, and an output gate, using the Sigmoid function as the activation function. When generating candidate memories, the hyperbolic tangent function tanh is used as the activation function. The Sigmoid output is between 0 and 1, conforming to the physical definition of gating. Furthermore, when the input is large or small, its output will be very close to 1 or 0, thus ensuring the gate is open or closed. The tanh function is used when generating candidate memories because its output is between -1 and 1, which matches the zero-centered feature distribution in most scenarios. In addition, the tanh function has a larger gradient than the Sigmoid function when the input is close to 0, usually leading to faster model convergence. Calculating the exponent of the Sigmoid function requires a certain amount of computation. A 0 / 1 gate can be used to make the gating output a discrete value of 0 or 1; that is, when the input is less than the threshold, the gating output is 0; when the input is greater than the threshold, the output is 1. This reduces the computational cost without significantly degrading performance.

[0104] In this embodiment, the LSTM (Long Short Term Memory) structure receives the spatial feature sequence z = (z1, z2, ..., z...). T ), LSTM updates the hidden layer parameters h = (h1, h2, ..., h T ),

[0105] At each time step, the hidden layer parameters are updated as follows:

[0106] g t =σ(W g z i，t +U g h t-1 +b g )

[0107] i t =σ(W i z i，t +U i h t-1 +b i )

[0108] o t =σ(W o z i，t +U o h t-1 +b o )

[0109] c t =g t .c t-1 +i t .tanh(W c z i，t +U c ht-1 +b c )

[0110] h t =o t .tanh(c t )

[0111] Where σ represents the sigmoid activation function, W, U, and b represent the parameters to be learned, and g t The activation function of the forget gate, o t This represents the activation function of the output gate, i. t Let c represent the activation function of the input gate. t Indicates the cell state.

[0112] In one implementation, step S330, which involves obtaining a safe or dangerous probability value using a Softmax classifier based on the assigned values ​​of the sample data, and then calculating the loss between the probability value and the assigned values ​​using a cross-entropy loss function, further includes:

[0113] Acquire spatial feature data based on the time dimension;

[0114] Based on the assigned values ​​of the sample data, the probability values ​​of safety or danger are calculated using the Softmax classifier.

[0115] The cross-entropy loss function is used to calculate the loss between the probability value and the assigned value, where,

[0116]

[0117] Where loss represents the loss, y i ' represents the assignment of the i-th sample data, y i ' = 0 indicates safety, y i =1 indicates danger.

[0118] To further explain, the computational process for temporal feature extraction and classification in this embodiment is as follows:

[0119] z j →Z→LSTM(q, 20)→Softmax(2)→y

[0120] Here, LSTM(q, h) represents the time step size of the LSTM structure as q, and h represents the dimension of the hidden layer. The Softmax activation function is used to calculate the probability value y of being safe or dangerous. i , 0≤y i If the value is ≤1, cross-entropy is chosen as the loss function to calculate the loss between the probability value and the assigned value.

[0121] In this implementation, the probability value and the assigned loss are calculated based on the cross-entropy loss function during training, as shown below:

[0122]

[0123] Where loss represents the loss, y i ' represents the assignment of the i-th sample, y i ' = 0 indicates safety, y i =1 indicates danger.

[0124] Furthermore, the ResNet-LSTM model obtained in this step takes sample data of lane change scenarios as input and the risk probability of such scenarios as output. However, during the training of the ResNet-LSTM network model, it is necessary to continuously adjust the various computational parameters using the probability values ​​calculated by the cross-entropy loss function and the assigned loss. To further clarify, the sample data is represented as a sequence of lane change frame images x. i = (x1, x2, ..., x T T represents the length of the time series. After being input into the ResNet-LSTM network, spatial features of each lane change frame image are extracted first to obtain a set of spatial feature sequences based on the lane change scenario. Then, temporal feature extraction is performed on this set of spatial feature sequences to determine the dynamic changes of spatial features in the spatial feature sequence based on the time dimension. This facilitates the correlation of any spatial feature with the upper and lower dimensions based on the time dimension. In this embodiment, the LSTM structure is selected to perform temporal extraction and classification. The trained ResNet-LSTM model can be understood as a preprocessing model and a deep learning model f,y i =(x i In this embodiment, Y can represent an encoded one-hot array.

[0125]

[0126] The aforementioned risk assessment method for vehicle lane changes utilizes a constructed simulation environment to acquire sample data from several lane-changing scenarios, thus providing sufficient model training data. Furthermore, it eliminates the need to collect and mine training data from real-world traffic scenarios, reducing manpower and material costs and simplifying the data collection process. The target model can be trained according to accuracy requirements. For example, CARLA+UE4, as a virtual data production tool, can generate an unlimited number of samples needed for model training, reducing the time and financial costs of sample collection.

[0127] The model training in this implementation is based on the PyTorch deep learning framework. The training optimizer is Adam, the learning rate is 0.0001, the decay exponent is 0.01, the batch size is 32, the epochs are 1000, and the training-to-validation ratio is 9:1. During training, the parameters of the pre-trained ResNet model are loaded. In actual use, 20 frames of images captured by a front-facing visible light sensor are read, the model is loaded, and the images are pushed in. The model performs a series of calculations, the same as the training process, and outputs a two-dimensional array: [safe_score, risk_score], where safe_score + risk_score = 1. This output corresponds to the risk assessment result for the lane change scenario and is directly used as the module's output for subsequent use.

[0128] Before training the model, a training dataset is constructed using the acquired sample data, and the sample data in the training dataset is labeled with values ​​so that the Softmax classifier in the LSTM structure can obtain safe or dangerous probability values, thus simplifying the scene recognition process.

[0129] In the step of inputting the training dataset into a pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model, the sequence of lane change frame images in the lane change scenario is used as input. The trajectory prediction result is not used as input, which avoids the introduction of errors in the trajectory prediction algorithm. Directly using sensor data makes the model more independent and has higher applicability. In addition, the addition of time dimension information and multi-frame input are more conducive to the model learning context information, which helps the model to understand the scene content more accurately and make full use of the model's transferability and robustness. The virtual training dataset can theoretically be generated infinitely, which can easily meet the training requirements.

[0130] In one embodiment, such as Figure 8 As shown, a risk assessment device for vehicle lane changes is provided, comprising: a data acquisition module 100, a data annotation module 200, a model training module 300, and a risk assessment module 400, wherein:

[0131] The data acquisition module 100 is used to acquire sample data under several lane change scenarios.

[0132] The data annotation module 200 is used to construct a training dataset using sample data and to assign labels to the sample data in the training dataset.

[0133] The model training module 300 is used to input the training dataset into the pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model.

[0134] The risk assessment module 400 is used to input sample data corresponding to any lane change scenario into the ResNet-LSTM model and output the risk assessment results.

[0135] Specific limitations regarding the risk assessment device for vehicle lane changes can be found in the limitations of the risk assessment method for vehicle lane changes mentioned above, and will not be repeated here. Each module in the aforementioned risk assessment device for vehicle lane changes can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0136] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores risk assessment data for vehicle lane changes. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a risk assessment method for vehicle lane changes.

[0137] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a risk assessment method for vehicle lane changes. The display screen may be a liquid crystal display (LCD) or an e-ink display. The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0138] Those skilled in the art will understand that the above description is only a partial structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. The specific computer device may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0139] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0140] Obtain sample data for several lane-changing scenarios;

[0141] A training dataset is constructed using sample data, and the sample data in the training dataset is labeled with values. The training dataset is then input into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model. Sample data corresponding to any lane change scenario is input into the ResNet-LSTM model, and a risk assessment result is output. In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: acquiring sample data under several lane change scenarios; constructing a training dataset using the sample data, and labeling the sample data in the training dataset as safe or dangerous; inputting the training dataset into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model; inputting sample data corresponding to any lane change scenario into the ResNet-LSTM model, and outputting a risk assessment result.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A risk assessment method for vehicle lane changes, characterized in that, The method includes: Obtain sample data for several lane-changing scenarios; A training dataset is constructed using the sample data, and the sample data in the training dataset are labeled with values. The training dataset is input into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model. The step of inputting the training dataset into the pre-built ResNet-LSTM network for model training to obtain the ResNet-LSTM model includes: extracting spatial features from the sample data using the ResNet structure in the ResNet-LSTM network to obtain at least one set of spatial feature sequences; extracting temporal features from the spatial feature sequences using the LSTM structure in the ResNet-LSTM network to obtain spatial feature data based on the time dimension; and, based on the assigned values ​​of the sample data, using a Softmax classifier to obtain a probability value of safety or danger, and then using a cross-entropy loss function to calculate the loss between the probability value and the assigned value. Input the sample data corresponding to any of the lane change scenarios into the ResNet-LSTM model and output the risk assessment result; The steps for obtaining sample data under several lane-changing scenarios include: controlling the lane-changing time, speed and abruptness of the second vehicle by adjusting the lane-changing parameters of the Python API in the CARLA application, so as to generate a sequence of lane-changing frame images of the second vehicle under different lane-changing scenarios from the perspective of the first vehicle, and using the sequence of lane-changing frame images as the sample data under each lane-changing scenario.

2. The method according to claim 1, characterized in that, The steps for obtaining sample data for several lane-changing scenarios also include: Based on the operating system and graphics card environment, a UE4 engine is built, and the CARLA application is loaded using the UE4 engine. The simulation environment required for the lane change scenario is created through the server-side of the CARLA application. The first vehicle and the second vehicle are determined by a client in the CARLA application, wherein the first vehicle is the main vehicle and the second vehicle is a vehicle changing lanes relative to the first vehicle in the adjacent lane.

3. The method according to claim 1, characterized in that, The steps of constructing a training dataset using the sample data and assigning labels to the sample data in the training dataset include: Multiple hazard levels are pre-defined, and a hazard coefficient is assigned to each hazard level; For sample data of the same lane change scenario, the danger level of the current lane change scenario is determined multiple times using discrete random variables; Based on the risk coefficient of each risk level, calculate the average risk coefficient of the current lane change scenario, and use the average risk coefficient as the final risk coefficient. Based on the intermediate values ​​of multiple risk coefficients and the final risk coefficient, the assigned value of the current sample data to be labeled is obtained by threshold binarization.

4. The method according to claim 1, characterized in that, The steps of extracting spatial features from the sample data using the ResNet structure in the ResNet-LSTM network to obtain a set of spatial feature sequences include: Obtain the corresponding lane change frame image sequence for each lane change scenario in the training dataset; Using the preset ResBlock unit structure in the ResNet structure, spatial features are extracted from each lane change frame image in the corresponding lane change frame image sequence according to a predetermined feature dimension, thereby obtaining a spatial feature sequence with a predetermined feature dimension.

5. The method according to claim 1, characterized in that, The steps of extracting temporal features from the spatial feature sequence using the LSTM structure in the ResNet-LSTM network to obtain spatial feature data based on the time dimension include: Obtain the spatial feature sequence, wherein the spatial feature sequence input at each time step in the LSTM structure corresponds to the output of the ResNet structure; Using the spatial feature sequence at each time step, the learning parameters of the hidden layer, input gate, output gate, forget gate, long-term memory, and short-term memory in the LSTM cell architecture are iteratively updated, as well as the cell state of the LSTM cell architecture is updated. Based on the updated learning parameters and cell state at each time step, temporal features are extracted from the spatial feature sequence to obtain spatial feature data based on the time dimension.

6. The method according to claim 1, characterized in that, Based on the assigned values ​​of the sample data, the steps of obtaining safe or dangerous probability values ​​using a Softmax classifier, and then calculating the loss between the probability values ​​and the assigned values ​​using the cross-entropy loss function include: Obtain the spatial feature data based on the time dimension; Based on the assigned values ​​of the sample data, the probability values ​​of safety or danger are calculated using the Softmax classifier. The cross-entropy loss function is used to calculate the loss between the probability value and the assigned value, where, Where loss represents the loss. This represents the assignment of the i-th sample data. =0 indicates safety. =1 indicates danger, N is the total number of sample data, and yi represents the predicted probability value that the i-th sample data belongs to the danger category.

7. A risk assessment device for vehicle lane changes, characterized in that, The device includes: The data acquisition module is used to control the lane change time, speed and abruptness of the second vehicle by adjusting the lane change parameters of the Python API in the CARLA application, so as to generate a sequence of lane change frame images of the second vehicle in different lane change scenarios from the perspective of the first vehicle, and use the lane change frame image sequence as sample data for each lane change scenario. The data annotation module is used to construct a training dataset using the sample data and to assign annotation values ​​to the sample data in the training dataset. The model training module is used to input the training dataset into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model. Specifically, the model training module uses the ResNet structure in the ResNet-LSTM network to extract spatial features from the sample data to obtain at least one set of spatial feature sequences; it then uses the LSTM structure in the ResNet-LSTM network to extract temporal features from the spatial feature sequences to obtain time-dimensional spatial feature data; based on the assigned values ​​of the sample data, a Softmax classifier is used to obtain the probability values ​​of safety or danger, and then the cross-entropy loss function is used to calculate the loss between the probability values ​​and the assigned values. The risk assessment module is used to input sample data corresponding to any of the lane change scenarios into the ResNet-LSTM model and output the risk assessment results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images