A vehicle control simulation method and device
By using feature extraction network and LSTM neural network to predict pedestrian trajectory in vehicle pedestrian interaction simulation scenarios, and combining DQN control decision network to generate vehicle control strategies, the problem of traditional methods being difficult to capture pedestrian dynamic behavior is solved, and autonomous vehicles are improved. Safety and intelligence in complex traffic environments.
Patent Information
- Application Number
- CN202410816720.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-06-24
AI Technical Summary
Traditional pedestrian trajectory prediction methods and deep learning algorithms are difficult to accurately capture the dynamic behavior of pedestrians and the complex interactions between vehicles and pedestrians, limiting the safety and intelligence of autonomous vehicles in complex traffic environments.
By constructing a simulation scenario for vehicles and pedestrian interaction, collecting picture data, using feature extraction networks and LSTM neural networks to predict pedestrian trajectory, and combining with the DQN control decision network to generate vehicle control strategies, achieving more accurate vehicle control.
It improves the accuracy and real-timeness of pedestrian trajectory prediction, improves the safety and intelligence of interaction between autonomous driving vehicles and pedestrians, and provides a better avoidance strategy.
Smart Images

Figure CN118395879B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving simulation technology, and more specifically, to a vehicle control simulation method and device. Background Art
[0002] With the rapid development of autonomous driving technology, how to ensure that autonomous vehicles can interact with pedestrians safely and efficiently in complex traffic environments has become an important research topic. Traditional pedestrian trajectory prediction methods, such as those based on physical models or simple machine learning models, often fail to accurately capture the dynamic behavior of pedestrians and the complex interactions between pedestrians and vehicles. Although traditional deep learning algorithms can learn the characteristics of dynamic scenes, they often cannot directly learn the decision action sequence in the corresponding scene, thus limiting the performance of autonomous vehicles in pedestrian avoidance. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a vehicle control simulation method and device, which solves the problem that simple LSTM modeling ignores the impact of pedestrian-vehicle interaction, avoids the defect that traditional deep learning algorithms cannot learn decision-making action sequences corresponding to scene features, and can not only improve the accuracy and real-time performance of pedestrian trajectory prediction, but also improve the safety and intelligence of autonomous driving vehicles interacting with pedestrians in complex traffic environments, providing strong support for the development and application of autonomous driving technology.
[0004] The first aspect of the present application provides a vehicle control simulation method, comprising:
[0005] Construct vehicle-pedestrian interaction simulation scenarios;
[0006] Collecting image data of vehicle-pedestrian interaction based on the vehicle-pedestrian interaction simulation scene;
[0007] Using a pre-built feature extraction network to extract features from the image data to obtain target feature data;
[0008] Using a pre-configured annotation tool to annotate the feature data to obtain annotated data;
[0009] Pedestrian motion prediction is performed based on the labeled data and the human-vehicle interaction LSTM neural network to obtain a pedestrian trajectory prediction result;
[0010] Generate a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network;
[0011] Vehicle control operations are performed on the test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy.
[0012] In the above implementation process, the method can perform a large number of driving avoidance simulations based on the vehicle-pedestrian interaction simulation scene, thereby obtaining a large amount of feature information and vehicle control strategies corresponding to the human-vehicle interaction scene, and then the LSTM model learns the pedestrian trajectory prediction method based on the human-vehicle interaction scene based on the feature information, and the reinforcement learning model learns the vehicle control strategy decision method based on the human-vehicle interaction scene based on the pedestrian trajectory prediction results. It can be seen that this method can make more accurate vehicle control strategies when considering the interaction between pedestrians and vehicles, thereby better assisting driving.
[0013] Furthermore, the construction of the vehicle-pedestrian interaction simulation scene includes:
[0014] Initialize the simulation world;
[0015] defining a vehicle blueprint and a pedestrian blueprint in the simulation world;
[0016] Setting driving data and a test vehicle in the simulation world based on the vehicle blueprint to obtain a first simulation world;
[0017] Setting pedestrian data in the first simulation world based on the pedestrian blueprint to obtain a second simulation world;
[0018] An RGB camera and a laser radar sensor are configured for the test vehicle in the second simulation world to obtain a vehicle-pedestrian interaction simulation scene.
[0019] Furthermore, the collecting of image data of vehicle-pedestrian interaction based on the vehicle-pedestrian interaction simulation scene includes:
[0020] Pre-build the DQN data collection network;
[0021] Image data under different road conditions and weather conditions are collected through the DQN data collection network, the RGB camera and the lidar sensor.
[0022] Furthermore, the extracting features of the image data using a pre-built feature extraction network to obtain target feature data includes:
[0023] Obtain a pre-built feature extraction network and an LSTM feature fusion network; wherein the feature extraction network includes a ResNet50 network and a ResNet18 network;
[0024] Perform feature extraction on the image data through the ResNet50 network to obtain a first image feature;
[0025] Performing lateral image feature extraction on the image data through the ResNet18 network to obtain a second image feature;
[0026] Performing feature joint processing on the first image feature and the second image feature to obtain joint feature data;
[0027] The joint feature data is subjected to feature fusion processing through the LSTM feature fusion network to obtain target feature data.
[0028] Furthermore, the pedestrian motion prediction is performed based on the labeled data and the human-vehicle interaction LSTM neural network to obtain the pedestrian trajectory prediction result, including:
[0029] Get the pre-built pedestrian trajectory generation neural network and human-vehicle interaction LSTM neural network;
[0030] Generate a neural network based on the labeled data and the pedestrian trajectory to obtain multiple groups of pedestrian states;
[0031] Determining a current pedestrian trajectory according to the multiple groups of pedestrian states;
[0032] Acquire human-vehicle interaction information according to the current pedestrian trajectory and the human-vehicle interaction LSTM neural network;
[0033] Pedestrian trajectory prediction is performed based on the human-vehicle interaction information and the current pedestrian trajectory to obtain a pedestrian trajectory prediction result.
[0034] Furthermore, the performing of vehicle control operations on the test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy includes:
[0035] Obtaining an underlying control algorithm and current vehicle parameters of a test vehicle in the vehicle-pedestrian interaction simulation scenario; the underlying control algorithm includes a PID algorithm and an LQR algorithm, and the PID algorithm includes an incremental PID algorithm;
[0036] Obtaining desired vehicle parameters according to the vehicle control decision strategy; wherein the desired vehicle parameters include at least a desired longitudinal speed, a desired lateral position, and a desired heading angle;
[0037] Calculating a control increment based on the desired vehicle parameters and the PID algorithm;
[0038] Calculating a lateral error and a heading angle error according to the desired vehicle parameters and the current vehicle parameters;
[0039] Calculate the target control amount according to the lateral error, the heading angle error, the control increment and the LQR algorithm;
[0040] The target control amount and the control increment are input to an actuator of the test vehicle, so that the actuator controls the test vehicle accordingly according to the target control amount and the control increment.
[0041] A second aspect of the present application provides a vehicle control simulation device, the vehicle control simulation device comprising:
[0042] A scene construction unit, used to construct a vehicle-pedestrian interaction simulation scene;
[0043] A collection unit, used for collecting image data of vehicle-pedestrian interaction based on the vehicle-pedestrian interaction simulation scene;
[0044] A feature extraction unit, used to extract features from the image data using a pre-built feature extraction network to obtain target feature data;
[0045] A labeling unit, used to label the feature data using a pre-configured labeling tool to obtain labeling data;
[0046] A pedestrian motion prediction unit, used to perform pedestrian motion prediction based on the labeled data and the human-vehicle interaction LSTM neural network to obtain a pedestrian trajectory prediction result;
[0047] A generating unit, used for generating a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network;
[0048] A simulation control unit is used to perform vehicle control operations on a test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy.
[0049] Furthermore, the scene construction unit includes:
[0050] Initialization subunit, used to initialize the simulation world;
[0051] A definition subunit, used to define a vehicle blueprint and a pedestrian blueprint in the simulation world;
[0052] A setting subunit, configured to set driving data and a test vehicle in the simulation world based on the vehicle blueprint to obtain a first simulation world;
[0053] The setting subunit is further used to set pedestrian data in the first simulation world based on the pedestrian blueprint to obtain a second simulation world;
[0054] The setting subunit is further used to configure an RGB camera and a lidar sensor for the test vehicle in the second simulation world to obtain a vehicle-pedestrian interaction simulation scene.
[0055] Furthermore, the acquisition unit comprises:
[0056] Build subunits to pre-build the DQN data collection network;
[0057] The collection subunit is used to collect image data under different road conditions and weather conditions through the DQN data collection network, the RGB camera and the lidar sensor.
[0058] Furthermore, the feature extraction unit comprises:
[0059] A first acquisition subunit is used to acquire a pre-built feature extraction network and an LSTM feature fusion network; wherein the feature extraction network includes a ResNet50 network and a ResNet18 network;
[0060] An extraction subunit, configured to perform feature extraction on the image data through the ResNet50 network to obtain a first image feature;
[0061] The extraction subunit is further used to extract lateral image features from the image data through the ResNet18 network to obtain a second image feature;
[0062] a combining subunit, configured to perform feature combining processing on the first image feature and the second image feature to obtain combined feature data;
[0063] The fusion subunit is used to perform feature fusion processing on the joint feature data through the LSTM feature fusion network to obtain target feature data.
[0064] Furthermore, the pedestrian motion prediction unit comprises:
[0065] The second acquisition subunit is used to acquire the pre-built pedestrian trajectory generation neural network and the human-vehicle interaction LSTM neural network;
[0066] A generating subunit, used for generating a neural network according to the labeled data and the pedestrian trajectory to obtain multiple groups of pedestrian states;
[0067] a determination subunit, configured to determine a current pedestrian trajectory according to the plurality of groups of pedestrian states;
[0068] A third acquisition subunit is used to acquire human-vehicle interaction information according to the current pedestrian trajectory and the human-vehicle interaction LSTM neural network;
[0069] The prediction subunit is used to predict the pedestrian trajectory according to the human-vehicle interaction information and the current pedestrian trajectory to obtain a pedestrian trajectory prediction result.
[0070] Furthermore, the simulation control unit comprises:
[0071] A fourth acquisition subunit is used to acquire the underlying control algorithm and the current vehicle parameters of the test vehicle in the vehicle-pedestrian interaction simulation scene; the underlying control algorithm includes a PID algorithm and an LQR algorithm, and the PID algorithm includes an incremental PID algorithm;
[0072] The fourth acquisition subunit is further used to acquire desired vehicle parameters according to the vehicle control decision strategy; wherein the desired vehicle parameters at least include a desired longitudinal speed, a desired lateral position, and a desired heading angle;
[0073] A calculation subunit, configured to calculate a control increment according to the desired vehicle parameters and the PID algorithm;
[0074] The calculation subunit is further used to calculate the lateral error and the heading angle error according to the expected vehicle parameters and the current vehicle parameters;
[0075] The calculation subunit is further used to calculate the target control amount according to the lateral error, the heading angle error, the control increment and the LQR algorithm;
[0076] The control subunit is used to input the target control amount and the control increment to the actuator of the test vehicle, so that the actuator controls the test vehicle accordingly according to the target control amount and the control increment.
[0077] A third aspect of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle control simulation method described in any one of the first aspects of the present application.
[0078] A fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the vehicle control simulation method described in any one of the first aspects of the present application is executed.
[0079] The beneficial effects of the present application are as follows: (1) the method and device can train the LSTM neural network based on the human-vehicle interaction information to improve the pedestrian trajectory prediction accuracy and achieve better dynamic prediction effect;
[0080] (2) The method and device can also combine the perception ability of deep learning with the decision-making ability of reinforcement learning through a deep reinforcement learning algorithm, thereby achieving direct control from raw input to output, thereby providing the vehicle with a better avoidance strategy. BRIEF DESCRIPTION OF THE DRAWINGS
[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0082] Figure 1 A flow chart of a vehicle control simulation method provided in an embodiment of the present application;
[0083] Figure 2 A flow chart of another vehicle control simulation method provided in an embodiment of the present application;
[0084] Figure 3 A schematic diagram of a process for building a vehicle-pedestrian interaction simulation scene in a CARLA simulator provided in an embodiment of the present application;
[0085] Figure 4 A schematic diagram of the principle of an LSTM feature fusion network provided in an embodiment of the present application;
[0086] Figure 5 An overall schematic diagram of a network consisting of a feature extraction network and an LSTM feature fusion network provided in an embodiment of the present application;
[0087] Figure 6 A schematic diagram of an example process of predicting pedestrian trajectories provided in an embodiment of the present application;
[0088] Figure 7 A schematic diagram of a deep reinforcement learning training process for vehicle-pedestrian interaction provided in an embodiment of the present application;
[0089] Figure 8 A schematic diagram of a vehicle state transfer process provided in an embodiment of the present application;
[0090] Fig. 9 A schematic diagram of the principle of a cascade PID speed control provided in an embodiment of the present application;
[0091] Fig.10 A schematic diagram of the structure of a vehicle control simulation device provided in an embodiment of the present application;
[0092] Fig.11 A schematic diagram of the structure of another vehicle control simulation device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0093] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0094] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0095] Example 1
[0096] Please see Figure 1 , Figure 1 A flow chart of a vehicle control simulation method provided in this embodiment. The vehicle control simulation method includes:
[0097] S101. Construct a vehicle-pedestrian interaction simulation scene.
[0098] In this embodiment, the method can use the simulation world of CARLA (i.e., the vehicle-pedestrian interaction simulation scene) to create vehicles and pedestrians, control the vehicles and pedestrians, build a vehicle-pedestrian interaction scene, and realize the simulation of the real traffic environment.
[0099] In this embodiment, in the vehicle-pedestrian interaction simulation scene, the pedestrian can set the starting point and end point of the movement, so that the pedestrian walks repeatedly between the starting point and the end point to simulate the scene of the pedestrian crossing the road.
[0100] S102: Collect image data of vehicle-pedestrian interaction based on a vehicle-pedestrian interaction simulation scenario.
[0101] In this embodiment, the method can collect image data using RGB cameras and lidar sensors using reinforcement learning algorithms (DQN, A3C) in a vehicle-pedestrian interaction simulation scenario. The image data can show different vehicle-pedestrian interaction conditions in complex traffic scenarios.
[0102] In this embodiment, DQN in the method is a deep Q network, A3C is an Actor-critic method, and data is collected mainly by iteratively running a reinforcement learning algorithm.
[0103] S103: Use a pre-built feature extraction network to extract features from the image data to obtain target feature data.
[0104] In this embodiment, the method can use a convolutional neural network to extract features from the collected image data.
[0105] S104: Use a pre-configured annotation tool to annotate the feature data to obtain annotated data.
[0106] In this embodiment, the method may use a labeling tool to label the acquired feature data (the feature data may be an image).
[0107] S105: Pedestrian motion prediction is performed based on the labeled data and the human-vehicle interaction LSTM neural network to obtain pedestrian trajectory prediction results.
[0108] In this embodiment, the method can input the labeled images into a long short-term memory network (LSTM) so that the LSTM can predict the movement trend and movement speed of dynamic pedestrians based on information over a period of time in the past.
[0109] In this embodiment, the method uses a convolutional neural network to extract features from image data and uses an LSTM network to predict trajectories. Unlike ordinary recurrent networks, the long short-term memory network (LSTM) can memorize its input. Among them, because LSTM adds the concepts of memory cell and input, forget, and output gates, memory cell is simply understood as the memory of the previous state in this state. LSTM operates on this state through these three gates, so it is very suitable for learning sequential data.
[0110] S106: Generate a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network.
[0111] In this embodiment, the method can use a deep reinforcement learning (DRL) algorithm to learn the vehicle's decision-making strategy based on the results of pedestrian trajectory prediction, and finally obtain a suitable vehicle control strategy.
[0112] S107. Performing vehicle control operations on the test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy.
[0113] In this embodiment, the method may use PID and LQR algorithms to perform lateral and longitudinal control operations on the vehicle.
[0114] In this embodiment, the method can learn good behaviors by exploring and receiving feedback from the environment, thereby formulating a reasonable pedestrian avoidance strategy and further handing it over to the control algorithm for execution. Among them, the method uses a proportional-integral-derivative controller (PID) for longitudinal control and a linear quadratic regulator (LQR) for lateral control.
[0115] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.
[0116] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.
[0117] It can be seen that the implementation of the vehicle control simulation method described in this embodiment can build a scene of interaction between cars and pedestrians in the simulation world constructed by CARLA, and use a reinforcement learning algorithm to collect data. Then, a convolutional neural network is used to extract features from the data, and input data is obtained through data preprocessing. The long short-term memory network (LSTM) is used to predict the movement trend and speed of pedestrians, and finally a deep reinforcement learning (DRL) algorithm is used to learn the vehicle's decision-making strategy, so that it can output a suitable vehicle control strategy.
[0118] Example 2
[0119] Please see Figure 2 , Figure 2 A flow chart of a vehicle control simulation method provided in this embodiment. The vehicle control simulation method includes:
[0120] S201, initializing the simulation world.
[0121] In this embodiment, this step is a step of initializing the CARLA virtual world.
[0122] In this embodiment, this step may specifically set the TCP port and the timeout response time, obtain the CARLA virtual world city map, and start the virtual engine.
[0123] S202. Define a vehicle blueprint and a pedestrian blueprint in the simulation world.
[0124] In this embodiment, the method can construct a Client, and obtain the blueprint library of the simulation world through the constructed Client to subsequently add vehicles and pedestrians, and at the same time set the CARLA server to run in synchronization mode.
[0125] S203: Setting driving data and a test vehicle in a simulation world based on the vehicle blueprint to obtain a first simulation world.
[0126] S204. Pedestrian data is set in the first simulation world based on the pedestrian blueprint to obtain a second simulation world.
[0127] In this embodiment, this step is intended to illustrate the process of setting the road conditions, which includes setting the required number of pedestrians and cars, and setting initial information such as speed and position.
[0128] In this embodiment, the method can configure the curve for pedestrians to walk in Unreal. The starting point of the curve defaults to the pedestrian's birth point. At the same time, the movement, posture, action, etc. of the pedestrian are controlled through API calls, and then the target point of the own vehicle is configured.
[0129] S205 , configuring an RGB camera and a lidar sensor for the test vehicle in the second simulation world to obtain a vehicle-pedestrian interaction simulation scene.
[0130] In this embodiment, this step can test the vehicle by adding sensors for capturing environmental information (fixing them on the car and setting properties such as location and capture frequency).
[0131] like Figure 3 As shown, Figure 3 A schematic diagram of a process for building a vehicle-pedestrian interaction simulation scene in the CARLA simulator is shown.
[0132] S206. Pre-build a DQN data collection network.
[0133] In this embodiment, the method uses the reinforcement learning algorithm Deep Q-Network to collect data, wherein the steps of DQN training include:
[0134] (1) Initialize the network and input the state , output The Q value of all actions;
[0135] (2) Use the strategy to select an action ,Bundle Input into the environment to obtain a new state and reward r;
[0136] (3) Calculate TD target: ;
[0137] (4) Calculate the loss function: ;
[0138] (5) Update the Q parameter so that As close as possible , can be treated as a regression problem and updated using gradient descent;
[0139] (6) The first 5 steps yield a quaternion ( , , , );
[0140] (7) Enter the new state and repeat the above steps.
[0141] S207, collect image data under different road conditions and weather conditions through the DQN data collection network, RGB camera and lidar sensor.
[0142] In this embodiment, after the above operations are completed, the method can add camera_actor.listen(lambda image:process_img(image)) after the camera is generated. This statement is used to listen to the camera data, and the data streams are processed and converted into images through the process_img() code written in the Python script. The RGB camera collects rich and differentiated image information under different road conditions and weather conditions and saves them in the format of saving images locally.
[0143] S208. Obtain a pre-built feature extraction network and an LSTM feature fusion network; wherein the feature extraction network includes a ResNet50 network and a ResNet18 network.
[0144] Please see Figure 4 , Figure 4 The schematic diagram of the LSTM feature fusion network is shown. Among them, σ is the forget gate, which determines what information to throw away from the cell state, that is, the information in the previous cell state is selectively forgotten. The forget gate implemented by the σ layer is the information in the previous cell state. And this step As input, then Each number in outputs a value between 0 and 1, recorded as , indicating how much information is retained, As shown below:
[0145] ;
[0146] Secondly, the sigmoid layer (input gate layer) decides what to store in the cell state, that is, it selectively records the new information into the cell state. Specifically, the probability of updating determined by the sigmoid layer (input gate layer) is expressed as , and then the tanh layer creates a candidate value vector , it will be added to the cell state. Among them,
[0147] ;
[0148] ;
[0149] .
[0150] Finally, the sigmoid layer (output layer gate) is used to determine the output state of the cell. Which parts of the cell state; then the cell state passes through the tanh layer (to make the value between -1 and 1), and then multiplies it with the output of the sigmoid layer to get the final output .in,
[0151] ;
[0152] .
[0153] Please see Figure 5 , Figure 5 The overall schematic diagram of the network consisting of the feature extraction network and the LSTM feature fusion network is shown, wherein the feature extraction network is a CNN network.
[0154] In this embodiment, the LSTM feature fusion network can be specifically divided into two parts, one part is the LSTM encoder, and the other part is the multi-layer perceptron MLP decoder.
[0155] In this embodiment, the method can construct the image information obtained in the CARLA virtual environment into a model training data set with a training set, a validation set and a test set; then, the LSTM-based pedestrian trajectory prediction network is trained through the model training data set, and this network iteratively trains the neural network model until the loss function value of the model is lower than the set threshold or reaches the set number of training times, obtains the trained model and maintains the model parameters, uses the validation set to verify and correct the obtained model, obtains the final target detection model, and finally uses the test set to test the model to obtain indicator information.
[0156] S209. Perform feature extraction on the image data through a ResNet50 network to obtain a first image feature.
[0157] In this embodiment, the method can use the ResNet50 network to extract image features, and the feature dimension is 512 dimensions.
[0158] S210. Perform lateral image feature extraction on the image data through a ResNet18 network to obtain a second image feature.
[0159] In this embodiment, the method can use the ResNet18 network to extract side image features.
[0160] S211, perform feature joint processing on the first image feature and the second image feature to obtain joint feature data.
[0161] S212. Perform feature fusion processing on the joint feature data through the LSTM feature fusion network to obtain target feature data.
[0162] For example, three frames of images are passed through the ResNet50 network to obtain three 512-dimensional feature vectors, and three frames of images are passed through the ResNet18 network to obtain three 512-dimensional feature vectors. The combined feature vectors are passed through six fully connected layers to obtain 16 64-dimensional feature vectors.
[0163] At this time, the 16 feature vectors obtained by the residual network are fused through an LSTM network with 256 hidden nodes, and the dimension of the obtained fused features is 512 dimensions.
[0164] Then, it passes through the MLP decoder and the fully connected layer to finally output a vector.
[0165] In this embodiment, the LSTM network has two improvements compared to the recurrent neural network. One is the new internal state, which can record historical information up to time t, and the other is the gate mechanism, which controls the proportion of information transmission.
[0166] S213: Use a pre-configured annotation tool to annotate the feature data to obtain annotated data.
[0167] S214. Obtain a pre-built pedestrian trajectory generation neural network and a human-vehicle interaction LSTM neural network.
[0168] S215. Generate a neural network based on the labeled data and pedestrian trajectories to obtain multiple groups of pedestrian states.
[0169] S216: Determine the current pedestrian trajectory according to the multiple groups of pedestrian states.
[0170] S217. Obtain human-vehicle interaction information according to the current pedestrian trajectory and the human-vehicle interaction LSTM neural network.
[0171] S218: predicting a pedestrian trajectory based on the human-vehicle interaction information and the current pedestrian trajectory to obtain a pedestrian trajectory prediction result.
[0172] Please see Figure 6 , Figure 6 A schematic diagram of an example process of predicting pedestrian trajectories is shown.
[0173] S219: Generate a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network.
[0174] Please see Figure 7 , Figure 7 A schematic diagram of the deep reinforcement learning training process for vehicle and pedestrian interaction is shown.
[0175] Please see Figure 8 , Figure 8 The figure shows the process flow of vehicle state transfer after the deep learning model training is completed. The deep learning model refers to the DQN control decision network.
[0176] In this embodiment, there are three situations in the vehicle driving process, namely, stopping to give way, driving normally and avoiding and detouring. Among them, the first situation is that the vehicle predicts that it will collide with pedestrians and stops to wait for passing; the second situation is that the vehicle predicts that there is no danger of collision and passes normally; the third situation is that pedestrians are about to pass through the vehicle driving area, and the vehicle chooses a route to avoid pedestrians to pass.
[0177] In this embodiment, the DQN control decision network is composed of two network structures with the same initial parameters, one is an estimation network, which is used to output an estimation value function; and the other is a target network, which is used to output a target value function.
[0178] In the implementation process, the method can obtain samples through the interaction between the robot and the environment ( , , , ), put all sample sets into the experience replay pool.
[0179] When the neural network is trained, batchsize samples are randomly extracted from the experience replay pool and input into the neural network. The nonlinear fitting ability of the neural network is used to fit a nonlinear function to express the Q value, and the e-greedy strategy is used to select the action of the intelligent agent.
[0180] After the agent performs the corresponding action, the environment will feedback a state and reward value. Finally, the training parameters of the network are obtained through the training and optimization of the neural network model, and relatively accurate action output is obtained.
[0181] ;
[0182] .
[0183] S220, obtaining the underlying control algorithm and the current vehicle parameters of the test vehicle in the vehicle-pedestrian interaction simulation scenario; the underlying control algorithm includes a PID algorithm and an LQR algorithm, and the PID algorithm includes an incremental PID algorithm.
[0184] In this embodiment, after the decision module of the vehicle is planned, the decision instruction needs to be executed. The underlying control adopts the PID+LQR algorithm, and the PID control uses the incremental PID. At the same time, in order to make the control more stable, the speed and acceleration cascade control method is adopted. Finally, the incremental PID cascade controls the longitudinal speed. Please refer to Fig. 9 , Fig. 9 The schematic diagram of the principle of cascade PID speed control is shown.
[0185] In this embodiment, the LQR algorithm is a linear quadratic optimizer. We calculate the lateral error between the vehicle position and the reference point and the heading angle error and other parameters, input them into the linear quadratic optimizer, and obtain the optimal control amount by minimizing the loss function. The given system is executed to achieve the purpose of minimizing the lateral error. The principle of the LQR algorithm is as follows:
[0186] ;
[0187] ;
[0188] .
[0189] S221. Obtaining expected vehicle parameters according to the vehicle control decision strategy; wherein the expected vehicle parameters include at least an expected longitudinal speed, an expected lateral position, and an expected heading angle.
[0190] S222: Calculate the control increment according to the desired vehicle parameters and the PID algorithm.
[0191] S223. Calculate the lateral error and the heading angle error according to the expected vehicle parameters and the current vehicle parameters.
[0192] S224, calculating the target control amount according to the lateral error, the heading angle error, the control increment and the LQR algorithm.
[0193] S225 , input the target control amount and the control increment to the actuator of the test vehicle, so that the actuator controls the test vehicle accordingly according to the target control amount and the control increment.
[0194] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.
[0195] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.
[0196] It can be seen that the implementation of the vehicle control simulation method described in this embodiment can build a scene of interaction between cars and pedestrians in the simulation world constructed by CARLA, and use a reinforcement learning algorithm to collect data. Then, a convolutional neural network is used to extract features from the data, and input data is obtained through data preprocessing. The long short-term memory network (LSTM) is used to predict the movement trend and speed of pedestrians, and finally a deep reinforcement learning (DRL) algorithm is used to learn the vehicle's decision-making strategy, so that it can output a suitable vehicle control strategy.
[0197] Example 3
[0198] Please see Fig.10 , Fig.10 FIG. 1 is a schematic diagram of the structure of a vehicle control simulation device provided in this embodiment. Fig.10 As shown, the vehicle control simulation device comprises:
[0199] A scene construction unit 310 is used to construct a vehicle-pedestrian interaction simulation scene;
[0200] A collection unit 320, used to collect image data of vehicle-pedestrian interaction based on a vehicle-pedestrian interaction simulation scene;
[0201] A feature extraction unit 330 is used to extract features from the image data using a pre-built feature extraction network to obtain target feature data;
[0202] A labeling unit 340 is used to label the feature data using a pre-configured labeling tool to obtain labeling data;
[0203] A pedestrian motion prediction unit 350 is used to perform pedestrian motion prediction based on the labeled data and the human-vehicle interaction LSTM neural network to obtain a pedestrian trajectory prediction result;
[0204] A generating unit 360, configured to generate a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network;
[0205] The simulation control unit 370 is used to perform vehicle control operations on the test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy.
[0206] In this embodiment, the explanation of the vehicle control simulation device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.
[0207] It can be seen that the vehicle control simulation device described in this embodiment can build a scene of interaction between cars and pedestrians in the simulation world constructed by CARLA, and use a reinforcement learning algorithm to collect data, then use a convolutional neural network to extract features from the data, and then obtain input data through data preprocessing, and use a long short-term memory network (LSTM) to predict the movement trend and speed of pedestrians, and finally use a deep reinforcement learning (DRL) algorithm to learn the vehicle's decision-making strategy, so that it can output a suitable vehicle control strategy.
[0208] Example 4
[0209] Please see Fig.11 , Fig.11 FIG. 1 is a schematic diagram of the structure of a vehicle control simulation device provided in this embodiment. Fig.11 As shown, the vehicle control simulation device comprises:
[0210] A scene construction unit 310 is used to construct a vehicle-pedestrian interaction simulation scene;
[0211] A collection unit 320, used to collect image data of vehicle-pedestrian interaction based on a vehicle-pedestrian interaction simulation scene;
[0212] A feature extraction unit 330 is used to extract features from the image data using a pre-built feature extraction network to obtain target feature data;
[0213] A labeling unit 340 is used to label the feature data using a pre-configured labeling tool to obtain labeling data;
[0214] A pedestrian motion prediction unit 350 is used to perform pedestrian motion prediction based on the labeled data and the human-vehicle interaction LSTM neural network to obtain a pedestrian trajectory prediction result;
[0215] A generating unit 360, configured to generate a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network;
[0216] The simulation control unit 370 is used to perform vehicle control operations on the test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy.
[0217] As an optional implementation, the scene construction unit 310 includes:
[0218] Initialization subunit 311, used to initialize the simulation world;
[0219] A definition subunit 312, used to define a vehicle blueprint and a pedestrian blueprint in the simulation world;
[0220] A setting subunit 313 is used to set driving data and test vehicles in the simulation world based on the vehicle blueprint to obtain a first simulation world;
[0221] The setting subunit 313 is further used to set pedestrian data in the first simulation world based on the pedestrian blueprint to obtain a second simulation world;
[0222] The setting subunit 313 is also used to configure an RGB camera and a laser radar sensor for the test vehicle in the second simulation world to obtain a vehicle-pedestrian interaction simulation scene.
[0223] As an optional implementation, the collection unit 320 includes:
[0224] A construction subunit 321, for pre-constructing a DQN data collection network;
[0225] The collection subunit 322 is used to collect image data under different road conditions and weather conditions through the DQN data collection network, RGB camera and lidar sensor.
[0226] As an optional implementation, the feature extraction unit 330 includes:
[0227] The first acquisition subunit 331 is used to acquire a pre-built feature extraction network and an LSTM feature fusion network; wherein the feature extraction network includes a ResNet50 network and a ResNet18 network;
[0228] An extraction subunit 332 is used to extract features from the image data through a ResNet50 network to obtain a first image feature;
[0229] The extraction subunit 332 is further used to extract lateral image features from the image data through a ResNet18 network to obtain a second image feature;
[0230] A combining subunit 333 is used to perform feature combining processing on the first image feature and the second image feature to obtain combined feature data;
[0231] The fusion subunit 334 is used to perform feature fusion processing on the joint feature data through the LSTM feature fusion network to obtain target feature data.
[0232] As an optional implementation manner, the pedestrian motion prediction unit 350 includes:
[0233] The second acquisition subunit 351 is used to acquire a pre-built pedestrian trajectory generation neural network and a human-vehicle interaction LSTM neural network;
[0234] A generating subunit 352, for generating a neural network according to the labeled data and the pedestrian trajectory to obtain multiple groups of pedestrian states;
[0235] A determination subunit 353, configured to determine a current pedestrian trajectory according to multiple groups of pedestrian states;
[0236] The third acquisition subunit 354 is used to acquire the human-vehicle interaction information according to the current pedestrian trajectory and the human-vehicle interaction LSTM neural network;
[0237] The prediction subunit 355 is used to predict the pedestrian trajectory according to the human-vehicle interaction information and the current pedestrian trajectory to obtain the pedestrian trajectory prediction result.
[0238] As an optional implementation, the simulation control unit 370 includes:
[0239] The fourth acquisition subunit 371 is used to acquire the underlying control algorithm and the current vehicle parameters of the test vehicle in the vehicle-pedestrian interaction simulation scene; the underlying control algorithm includes a PID algorithm and an LQR algorithm, and the PID algorithm includes an incremental PID algorithm;
[0240] The fourth acquisition subunit 371 is further used to acquire desired vehicle parameters according to the vehicle control decision strategy; wherein the desired vehicle parameters at least include a desired longitudinal speed, a desired lateral position, and a desired heading angle;
[0241] A calculation subunit 372, for calculating a control increment according to desired vehicle parameters and a PID algorithm;
[0242] The calculation subunit 372 is further used to calculate the lateral error and the heading angle error according to the expected vehicle parameters and the current vehicle parameters;
[0243] The calculation subunit 372 is further used to calculate the target control amount according to the lateral error, the heading angle error, the control increment and the LQR algorithm;
[0244] The control subunit 373 is used to input the target control amount and the control increment to the actuator of the test vehicle, so that the actuator controls the test vehicle accordingly according to the target control amount and the control increment.
[0245] In this embodiment, the explanation of the vehicle control simulation device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.
[0246] It can be seen that the vehicle control simulation device described in this embodiment can build a scene of interaction between cars and pedestrians in the simulation world constructed by CARLA, and use a reinforcement learning algorithm to collect data, then use a convolutional neural network to extract features from the data, and then obtain input data through data preprocessing, and use a long short-term memory network (LSTM) to predict the movement trend and speed of pedestrians, and finally use a deep reinforcement learning (DRL) algorithm to learn the vehicle's decision-making strategy, so that it can output a suitable vehicle control strategy.
[0247] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle control simulation method in Embodiment 1 or Embodiment 2 of the present application.
[0248] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the vehicle control simulation method in Embodiment 1 or Embodiment 2 of the present application is executed.
[0249] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0250] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0251] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
[0252] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0253] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0254] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
Claims
1. A vehicle control simulation method, It is characterized in that include: Construct vehicle-pedestrian interaction simulation scenarios; Collecting image data of vehicle-pedestrian interaction based on the vehicle-pedestrian interaction simulation scene; Using a pre-built feature extraction network to extract features from the image data to obtain target feature data; Using a pre-configured annotation tool to annotate the feature data to obtain annotated data; Pedestrian motion prediction is performed based on the labeled data and the human-vehicle interaction LSTM neural network to obtain a pedestrian trajectory prediction result; Generate a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network; Performing vehicle control operations on the test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy; Wherein, the collecting of image data of vehicle-pedestrian interaction based on the vehicle-pedestrian interaction simulation scene includes: Pre-build the DQN data collection network; Image data under different road conditions and weather conditions are collected through the DQN data collection network, the RGB camera and the lidar sensor configured on the test vehicle.
2. The vehicle control simulation method according to claim 1, It is characterized in that The constructing of the vehicle-pedestrian interaction simulation scene includes: Initialize the simulation world; defining a vehicle blueprint and a pedestrian blueprint in the simulation world; Setting driving data and a test vehicle in the simulation world based on the vehicle blueprint to obtain a first simulation world; Setting pedestrian data in the first simulation world based on the pedestrian blueprint to obtain a second simulation world; An RGB camera and a laser radar sensor are configured for the test vehicle in the second simulation world to obtain a vehicle-pedestrian interaction simulation scene.
3. The vehicle control simulation method according to claim 1, It is characterized in that The method of extracting features from the image data using a pre-built feature extraction network to obtain target feature data includes: Obtain a pre-built feature extraction network and an LSTM feature fusion network; wherein the feature extraction network includes a ResNet50 network and a ResNet18 network; Perform feature extraction on the image data through the ResNet50 network to obtain a first image feature; Performing lateral image feature extraction on the image data through the ResNet18 network to obtain a second image feature; Performing feature joint processing on the first image feature and the second image feature to obtain joint feature data; The joint feature data is subjected to feature fusion processing through the LSTM feature fusion network to obtain target feature data.
4. The vehicle control simulation method according to claim 1, It is characterized in that The pedestrian motion prediction is performed according to the labeled data and the human-vehicle interaction LSTM neural network to obtain the pedestrian trajectory prediction result, including: Get the pre-built pedestrian trajectory generation neural network and human-vehicle interaction LSTM neural network; Generate a neural network based on the labeled data and the pedestrian trajectory to obtain multiple groups of pedestrian states; Determining a current pedestrian trajectory according to the multiple groups of pedestrian states; Acquire human-vehicle interaction information according to the current pedestrian trajectory and the human-vehicle interaction LSTM neural network; Pedestrian trajectory prediction is performed based on the human-vehicle interaction information and the current pedestrian trajectory to obtain a pedestrian trajectory prediction result.
5. The vehicle control simulation method according to claim 1, It is characterized in that The performing a vehicle control operation on the test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy includes: Obtaining an underlying control algorithm and current vehicle parameters of a test vehicle in the vehicle-pedestrian interaction simulation scenario; the underlying control algorithm includes a PID algorithm and an LQR algorithm, and the PID algorithm includes an incremental PID algorithm; Obtaining desired vehicle parameters according to the vehicle control decision strategy; wherein the desired vehicle parameters include at least a desired longitudinal speed, a desired lateral position, and a desired heading angle; Calculating a control increment based on the desired vehicle parameters and the PID algorithm; Calculating a lateral error and a heading angle error according to the desired vehicle parameters and the current vehicle parameters; Calculate the target control amount according to the lateral error, the heading angle error, the control increment and the LQR algorithm; The target control amount and the control increment are input to an actuator of the test vehicle, so that the actuator controls the test vehicle accordingly according to the target control amount and the control increment.
6. A vehicle control simulation device, It is characterized in that The vehicle control simulation device comprises: A scene construction unit, used to construct a vehicle-pedestrian interaction simulation scene; A collection unit, used for collecting image data of vehicle-pedestrian interaction based on the vehicle-pedestrian interaction simulation scene; A feature extraction unit, used to extract features from the image data using a pre-built feature extraction network to obtain target feature data; A labeling unit, used to label the feature data using a pre-configured labeling tool to obtain labeling data; A pedestrian motion prediction unit, used to perform pedestrian motion prediction based on the labeled data and the human-vehicle interaction LSTM neural network to obtain a pedestrian trajectory prediction result; A generating unit, used for generating a vehicle control decision strategy based on the pedestrian trajectory prediction result and the DQN control decision network; A simulation control unit, configured to perform vehicle control operations on a test vehicle in the vehicle-pedestrian interaction simulation scenario based on the vehicle control decision strategy; Wherein, the acquisition unit comprises: Build subunits to pre-build the DQN data collection network; The collection subunit is used to collect image data under different road conditions and weather conditions through the DQN data collection network, the RGB camera and the lidar sensor configured on the test vehicle.
7. The vehicle control simulation device according to claim 6, It is characterized in that The scene construction unit comprises: Initialization subunit, used to initialize the simulation world; A definition subunit, used to define a vehicle blueprint and a pedestrian blueprint in the simulation world; A setting subunit, configured to set driving data and a test vehicle in the simulation world based on the vehicle blueprint to obtain a first simulation world; The setting subunit is further used to set pedestrian data in the first simulation world based on the pedestrian blueprint to obtain a second simulation world; The setting subunit is further used to configure an RGB camera and a lidar sensor for the test vehicle in the second simulation world to obtain a vehicle-pedestrian interaction simulation scene.
8. An electronic device, It is characterized in that The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the vehicle control simulation method according to any one of claims 1 to 5.
9. A readable storage medium, It is characterized in that The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the vehicle control simulation method according to any one of claims 1 to 5 is executed.
Citation Information
Patent Citations
Method and device for building simulation environment for automatic driving
CN116882122A
