Scene-driven and imitation learning based human-like driving behavior control method and system
By employing scenario-driven and imitation learning methods, we extract the evolutionary trend and interaction features of driving behavior, train a human-like control strategy network, and solve the problem of unreasonable decision-making in intelligent driving systems under complex scenarios. This achieves efficient and human-like driving behavior control, improving the safety and smoothness of autonomous vehicles.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH OF CHINA
- Filing Date
- 2026-04-29
- Publication Date
- 2026-07-14
AI Technical Summary
Existing intelligent driving behavior control systems struggle to achieve unified cognition in complex scenarios and lack effective learning of human driving patterns, resulting in insufficient interpretability of behavioral decisions and poor human-likeness.
A scene-driven and imitation learning approach is adopted. Evolutionary trend features of driving behavior are extracted through temporal neural networks. The interaction features of human-vehicle-environment are captured by combining structured coding and attention mechanisms and fused to form scene feature vectors. Deep imitation learning is then used to train a human-like control strategy network to output vehicle control actions.
It significantly enhances the autonomous decision-making capabilities of driverless vehicles in complex scenarios, achieving smooth and highly human-like driving behavior control, and improving driving safety and traffic efficiency.
Smart Images
Figure CN122101237B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent driving technology, and in particular to a human-like driving behavior control method and system based on scene-driven and imitation learning. Background Technology
[0002] With the development of autonomous driving technology, road traffic environments are becoming increasingly complex, and human-like driving behavior control is considered an effective way to improve driving safety. Current intelligent driving behavior control tasks typically include two levels: driving behavior cognition and vehicle control. Driving behavior cognition focuses on understanding driving intentions in typical driving scenarios such as following other vehicles, changing lanes, overtaking, merging, and cornering. Vehicle control focuses on translating the cognitive results into continuous and smooth control commands.
[0003] However, existing intelligent driving behavior control systems struggle to achieve a unified understanding of complex scenarios, and their control models lack effective learning of human driving patterns, resulting in insufficient interpretability and poor human-likeness in behavioral decisions. Existing technologies are mostly designed for single scenarios or specific tasks, lacking versatility and human-likeness, making it difficult to adapt to the diverse driving scenarios and complex behavioral control requirements.
[0004] Therefore, improving the rationality and human-likeness of driving behavior decisions in complex scenarios has become an urgent technical challenge. Summary of the Invention
[0005] The main objective of this invention is to provide a human-like driving behavior control method and system based on scene-driven and imitation learning, which aims to improve the rationality and human-likeness of driving behavior decisions in complex scenarios.
[0006] To achieve the above objectives, this invention proposes a human-like driving behavior control method based on scene-driven and imitation learning, comprising the following steps:
[0007] Step S1: Classify typical driving scenarios in the driving dataset, model historical driving behavior sequences using a temporal neural network, and extract the evolution trend features of driving behavior as the scenario changes.
[0008] Step S2: Structure the human-vehicle-environment elements in the driving scene, and perform interactive modeling of scene features and traffic participant features based on the attention mechanism to obtain interactive features;
[0009] Step S3: Integrate the evolutionary trend features and the interaction features to form a scene feature vector, which serves as a scene-driven behavioral cognitive representation;
[0010] Step S4: Input the real-time perceived scene feature vector into the humanoid control policy network that has been pre-trained through deep imitation learning, and output the vehicle control actions by the humanoid control policy network, which include longitudinal acceleration and steering angle.
[0011] Preferably, the driving scenarios include following scenarios, lane changing scenarios, overtaking scenarios, merging scenarios, and curve scenarios.
[0012] Preferably, in step S1, the temporal neural network employs a long short-term memory network with a length of [missing information]. Within the time window, the historical driving behavior sequence is represented as ;in, Indicates time The vehicle state / driving behavior vector, which is composed of vehicle speed and steering angle constitute.
[0013] Preferably, the long short-term memory network uses the historical driving behavior sequence. As input, the output is a prediction of future driving behavior. By minimizing the prediction result Compared to real driving behavior loss function between Training is performed, and the loss function is... Defined as:
[0014] .
[0015] Preferably, in step S2, the structured coding includes: acquiring the topological / geometric information of the road network, as well as the location and speed information of vehicle flow and pedestrians, and continuously encoding them as input to the multilayer perceptron.
[0016] Preferably, in step S3, the scene feature vector Represented as:
[0017]
[0018] In the formula, For the interaction features, The evolutionary trend features; the scene feature vector It is used to characterize the spatial structural relationships, lane topology, and dynamic interaction states of traffic participants.
[0019] Preferably, the observation space of the humanoid control strategy network during training includes environmental features represented by the scene feature vector, as well as the vehicle's position, speed, and acceleration; its action space includes acceleration for adjusting longitudinal behavior and steering angle for adjusting lateral behavior.
[0020] Preferably, the training process of the human-like control strategy network includes: taking driving scene features as input, outputting predicted control actions through forward propagation, comparing the predicted control actions with the actual operation behavior of a human driver in the same scene, minimizing the error between the two based on a batch supervised learning method, and iteratively optimizing the network parameters using a gradient descent method.
[0021] Preferably, in step S4, the vehicle perceives the current driving scene status information in real time during each control cycle. The status information includes scene structure features and vehicle operating status. The humanoid control strategy network outputs the vehicle control actions in real time based on the status information, which are then executed by the underlying control module.
[0022] This application also discloses a human-like driving behavior control system based on scene-driven and imitation learning, including:
[0023] Memory, used to store computer programs;
[0024] A processor for executing the computer program to implement the method as described in any one of the above.
[0025] The above technical solution has the following advantages:
[0026] This invention achieves a high-dimensional cognitive representation of complex driving environments by classifying typical driving scenarios in a driving dataset and extracting evolutionary trend features using temporal neural network modeling. Simultaneously, it combines structured coding and attention mechanisms to capture the dynamic interaction features between human, vehicle, and environmental elements. By fusing evolutionary trends and interaction features to form scene feature vectors and using deep imitation learning to train the control policy network, the longitudinal acceleration and lateral steering angle commands generated by autonomous vehicles can highly fit the control logic of human drivers. This method effectively solves the problem of lag or unreasonable decision-making in traditional control models in complex scenarios, significantly improving the traffic efficiency and driving safety of autonomous vehicles in different typical driving scenarios, and achieving smooth and highly human-like autonomous control. Attached Figure Description
[0027] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings, wherein:
[0028] Figure 1This is a flowchart illustrating the human-like driving behavior control method based on scene-driven and imitation learning provided in an embodiment of the present invention.
[0029] Figure 2 This is a block diagram illustrating the principle of scene feature construction and control strategy output in a humanoid driving behavior control system provided in an embodiment of the present invention. Detailed Implementation
[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0031] Example 1
[0032] like Figures 1 to 2 As shown, this embodiment provides a human-like driving behavior control method based on scene-driven and imitation learning. This method is executed by a human-like driving behavior control system, which can be integrated into the computing platform of an autonomous vehicle and implemented by the processor calling instructions from memory. The core idea of this method is to establish a deep mapping relationship between scene features and human driving behavior by recognizing typical driving scenarios and extracting behavioral features, thereby enabling the autonomous vehicle to exhibit handling characteristics that conform to human driving habits in complex traffic flows.
[0033] refer to Figure 1 The specific execution steps of this embodiment are as follows.
[0034] Step S1: Classify typical driving scenarios in the driving dataset, model historical driving behavior sequences using a temporal neural network, and extract the evolution trend features of driving behavior as the scenario changes.
[0035] In this step, the human-like driving behavior control system first preprocesses the massive amount of human driving data. The driving dataset covers a variety of typical driving scenarios, such as following other vehicles, lane changing, overtaking, merging, and cornering. The system analyzes the vehicle's trajectory, relative position, and velocity vectors to segment the dataset into different scene segments.
[0036] To capture the dynamic characteristics of human driving behavior, this embodiment employs a temporal neural network, specifically a Long Short-Term Memory (LSTM) network, to establish a temporal neural network model 210. In a length of... Within the time window, the historical driving behavior sequence is represented as In the formula, Indicates time The vehicle state / driving behavior vector, which is composed of vehicle speed and steering angle Composition, that is .
[0037] Temporal neural network model 210 uses historical driving behavior sequences As input, the network learns the nonlinear evolution of driving behavior over time through its internal forget gate, input gate, and output gate mechanisms. The network outputs a prediction of future driving behavior. During the training phase, the system minimizes the prediction results. Compared to real driving behavior loss function between To optimize network parameters. The loss function Defined as:
[0038]
[0039] In this way, the temporal neural network model 210 can extract the evolutionary trend features of behavior under different driving scenarios. This feature reflects the evolution of human drivers' intentions in specific environments, such as the gradual increase in steering angle in lane-changing scenarios or the fine-tuning of speed in following scenarios.
[0040] Step S2 involves structurally encoding the human-vehicle-environment elements in typical driving scenarios, and then using an attention mechanism to model the interaction between scene features and traffic participant features to obtain interactive features.
[0041] refer to Figure 2 The system performs structured processing on the environmental information acquired by the sensors. Elements such as road network information, surrounding traffic flow, and the positions and speeds of pedestrians are converted into numerical vectors. Specifically, the system acquires information such as the centerline coordinates of the road network, lane boundaries, global coordinates of surrounding obstacles, and relative speeds, and uses this information as input to a multilayer perceptron (MLP) for continuous encoding.
[0042] To accurately characterize the complex interactions between traffic participants, this embodiment constructs an interaction relationship model 220. This model introduces an attention mechanism to fuse static environmental features and dynamic traffic participant features. Interaction Features The calculation process is as follows:
[0043]
[0044] In the formula, , , These represent the query vector, key vector, and value vector, respectively, constructed from scene features and traffic participant features. The attention mechanism automatically learns the degree of attention a human driver pays to different traffic elements in the current scene by calculating the correlation between the query vector, key vector, and value vector. For example, in a scenario merging into the main lane, this mechanism assigns higher weights to vehicles approaching from behind. This attention-based interaction modeling significantly improves the system's cognitive depth of the scene and generates interactive features. It contains rich spatial structural relationships and dynamic interactive information.
[0045] Step S3: Integrate evolutionary trend features and interaction features to form a scene feature vector, which serves as a scene-driven behavioral cognitive representation.
[0046] The system will use the evolutionary trend features obtained in step S1 Interaction features obtained in step S2 The features are then fused to form a unified scene feature vector. Scene feature vector Represented as:
[0047]
[0048] in, This is the concatenation function; this vector serves as a scene-driven behavioral cognition representation, fully describing the spatial structural relationships, lane topology, and dynamic interaction states of traffic participants. Subsequently, the system trains a human-like control policy network 230 based on a deep imitation learning method.
[0049] The human-like control policy network 230 employs a fully connected neural network structure (including an input layer, at least one hidden layer, and an output layer). During training, the network's observation space includes scene feature vectors. The environmental characteristics are represented, along with the position, speed, and acceleration of the main vehicle. The motion space includes acceleration used to regulate longitudinal behavior. and steering angle used to adjust lateral behavior The system uses the actual driving behavior of a human driver as a supervisory signal, outputs predicted control actions through forward propagation, and compares the predicted control actions with the actual driving behavior of the human driver in the same scenario. Based on the batch supervised learning method, the network parameters are iteratively optimized using the gradient descent algorithm to minimize the error between the predicted values and the actual driving behavior. After sufficient training, the humanoid control policy network 230 can learn the decision-making logic of a human driver in different scenarios.
[0050] Step S4: After the network training is completed, online autonomous decision-making is performed. The current driving scene features are input into the human-like control strategy network to obtain the corresponding control strategy and execute vehicle control.
[0051] In the actual operation of autonomous vehicles, the system enters the online autonomous decision-making phase. The vehicle acquires real-time information about the current driving environment and vehicle status through sensing devices such as LiDAR and cameras. This information is then structured and encoded to form scene feature vectors. The data is then input into the pre-trained humanoid control policy network 230.
[0052] The human-like control strategy network 230 calculates and outputs the optimal vehicle control actions in real time based on the input scene characteristics. These vehicle control actions include longitudinal acceleration and steering angle. Upon receiving these commands, the underlying control module controls the vehicle's throttle opening, braking pressure, and steering angle through the actuators. Because the control commands are derived from a deep imitation of human driving patterns, the vehicle's driving performance will be smoother and more in line with the expectations of road users, thereby effectively improving the rationality and safety of driving behavior.
[0053] Example 2
[0054] This embodiment, based on Embodiment 1 above, further details the specific working mechanism of the humanoid driving behavior control system under different typical driving scenarios. Referring to claim 2, the driving scenarios include following scenarios, lane changing scenarios, overtaking scenarios, merging scenarios, and curve scenarios.
[0055] In a following vehicle scenario, the human-like driving behavior control system monitors the distance and relative speed of the target vehicle ahead in real time using sensors. At this point, the evolutionary trend features extracted by the temporal neural network model 210... This is mainly reflected in the prediction of changes in the acceleration of the vehicle in front. Scene feature vector Interactive features in The system then focuses its attention on the brake light status of the vehicle in front and the distance between the two vehicles. The humanoid control strategy network 230 outputs vehicle control actions with longitudinal acceleration. Primarily, through fine-tuning acceleration The size of the vehicle allows autonomous vehicles to maintain a following distance similar to that of human drivers, avoiding frequent sudden acceleration and deceleration, thereby improving passenger comfort.
[0056] In lane-changing scenarios, the system not only focuses on the driving environment of its own lane but also models vehicles approaching from behind in the target lane using an attention mechanism. The structured encoding process encodes the distance and speed of vehicles behind in the target lane, as well as the relative position of the vehicle to the lane lines, into vectors. After determining that lane-changing conditions are met, the human-like control strategy network 230 outputs a smoothly evolving steering angle. Because the temporal neural network model 210 has learned the evolutionary trends of humans in the initial stage of lane changing—micro-steering wheel movements, constant speed rotation in the middle stage, and rapid return to center in the later stage. Therefore, the output steering angle The commands can make the vehicle trajectory appear natural and anthropomorphic. Type curve.
[0057] In merging scenarios, such as merging from a ramp into a main road, the scene feature vector... It can characterize complex traffic flow topology. Interactive features. The system will focus on the yielding intentions of vehicles on the main road. If a vehicle behind slows down on the main road, the attention mechanism will capture this dynamic interaction, and the human-like control policy network 230 will output an appropriate acceleration. The system guides vehicles to merge decisively. If vehicles on the main road show no signs of slowing down, the system will adjust its acceleration accordingly. Slowing down and waiting. This scenario-driven behavioral cognition enables autonomous vehicles to understand the game-theoretic relationships between traffic participants, just like human drivers.
[0058] In curve scenarios, structured coding extracts the lane topology of the road network, especially curvature information. The system obtains the coordinates of path points within a 50m range ahead of the vehicle and uses them as input to the MLP. The temporal neural network model 210 then uses historical driving behavior sequences... Extract the speed switching patterns during curve entry and exit. The humanoid control strategy network 230 outputs longitudinal acceleration in a coordinated manner. and lateral steering angle It achieves human-like control logic of decelerating when entering a curve, maintaining a constant speed in the curve, and accelerating when exiting a curve, ensuring the vehicle's driving stability in curves.
[0059] Example 3
[0060] This embodiment mainly describes the training details of the human-like control policy network 230 and the construction of the system observation space. Referring to claims 7 and 8, the human-like control policy network 230 adopts a deep imitation learning framework during the training phase.
[0061] The observation space required for the training process consists of highly integrated features. Among these, the environmental features are the scene feature vectors generated in step S3 above. This vector, with its fixed dimension, provides a unified representation of different scenarios. Furthermore, the observation space includes the real-time operating status of the main vehicle, specifically its position in the global coordinate system, current speed, and lateral and longitudinal acceleration. All values are represented using Arabic numerals; for example, speed is expressed in m / s and acceleration in m / s².
[0062] The training process is controlled by the processor. First, the system initializes the weight parameters of the human-like control strategy neural network. In each training iteration, the system randomly selects a batch of sample data from the training sample set in memory. Each sample contains a scene feature vector at a specific time step. And the corresponding true values of the human driver's control actions. The humanoid control policy network 230 calculates the predicted acceleration through forward propagation. and predicted steering angle .
[0063] Subsequently, the processor calculates the supervision error between the predicted and true values. This embodiment uses the mean squared error loss function for measurement. Using gradient descent, the system updates the weights and bias parameters of each layer of the network along the inverse direction of the loss function gradient. To ensure training stability, the learning rate is set to 0.001 and adjusted according to the error reduction during training based on a preset learning rate scheduling strategy. Through tens of thousands of batch supervised learning iterations, the human-like control policy network 230 converges to a stable state, making the output predicted control actions highly consistent with the actual driving behavior of a human driver in the same scenario.
[0064] Referring to claim 9, after the network training is completed, the humanoid driving behavior control system operates at a fixed control cycle, for example, performing a decision calculation every 20ms. Within each control cycle, the processor acquires raw data collected by sensors through an interface, and obtains the current scene feature vector after structured encoding and feature fusion. The vector is immediately fed into the deployed humanoid control policy network 230, which outputs longitudinal acceleration in real time. and steering angle This high-frequency, real-time decision-making output mechanism ensures that autonomous vehicles can quickly respond to dynamically changing environments and achieve smooth, human-like autonomous driving.
[0065] This embodiment also provides a humanoid driving behavior control system based on scene-driven and imitation learning. Referring to claim 10, the system includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the method steps described in the above embodiments. In a specific hardware implementation, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), or an application-specific integrated circuit (ASIC). The memory may include random access memory (RAM) and read-only memory (ROM). The system can be deployed on the onboard computing platform of an autonomous vehicle and connected to the vehicle's radar sensors, camera equipment, and underlying actuators via a bus.
[0066] Furthermore, this invention also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the aforementioned human-like driving behavior control method based on scene-driven and imitation learning. This storage medium includes, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical discs.
[0067] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the various embodiments of the present invention. The present invention is not limited to specific implementation methods, and the scope of protection of the present invention should be determined by the scope of the claims.
[0068] In summary, this invention provides a human-like driving behavior control method based on scene-driven and imitation learning, which classifies typical driving scenarios in a driving dataset and extracts evolutionary trend features. Interaction features obtained by combining attention mechanism modeling This forms a scene-driven behavioral cognitive representation. The human-like control policy network 230, trained through deep imitation learning, can generate scene feature vectors based on real-time perception. Output longitudinal acceleration and steering angle This invention effectively enhances the autonomous decision-making capabilities of unmanned vehicles in typical driving scenarios such as following other vehicles, changing lanes, overtaking, merging, and cornering, making vehicle control smoother and more in line with human driving habits, and enhancing the rationality of driving behavior.
Claims
1. A human-like driving behavior control method based on scene-driven and imitation learning, characterized in that, Includes the following steps: Step S1: Classify typical driving scenarios in the driving dataset, model historical driving behavior sequences using a temporal neural network, and extract the evolution trend features of driving behavior as the scenario changes. Step S2: Structure the human-vehicle-environment elements in the driving scene, and perform interactive modeling of scene features and traffic participant features based on the attention mechanism to obtain interactive features; Step S3: Integrate the evolutionary trend features and the interaction features to form a scene feature vector, which serves as a scene-driven behavioral cognitive representation; Step S4: Input the real-time perceived scene feature vector into the humanoid control policy network that has been pre-trained through deep imitation learning, and output the vehicle control actions by the humanoid control policy network, which include longitudinal acceleration and steering angle.
2. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 1, characterized in that, The driving scenarios include following other vehicles, changing lanes, overtaking, merging, and cornering.
3. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 1, characterized in that, In step S1, the temporal neural network employs a long short-term memory network with a length of [missing information]. Within the time window, the sequence of historical driving behaviors is represented as follows: ； in, Indicates time The vehicle state / driving behavior vector, which is composed of vehicle speed and steering angle constitute.
4. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 3, characterized in that, The long short-term memory network uses the historical driving behavior sequence As input, the output is a prediction of future driving behavior. By minimizing the prediction result Compared to real driving behavior loss function between Training is performed, and the loss function is... Defined as: 。 5. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 1, characterized in that, In step S2, the structured coding includes: acquiring the topological / geometric information of the road network, as well as the position and speed information of vehicle flow and pedestrians, and continuously encoding them as input to the multilayer perceptron.
6. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 1, characterized in that, In step S3, the scene feature vector Represented as: In the formula, For the interaction features, The evolutionary trend characteristics; The scene feature vector It is used to characterize the spatial structural relationships, lane topology, and dynamic interaction states of traffic participants.
7. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 1, characterized in that, The observation space of the humanoid control strategy network during training includes environmental features represented by the scene feature vector, as well as the vehicle's position, speed, and acceleration; its action space includes acceleration for adjusting longitudinal behavior and steering angle for adjusting lateral behavior.
8. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 1, characterized in that, The training process of the human-like control strategy network includes: taking driving scene features as input, outputting predicted control actions through forward propagation, comparing the predicted control actions with the actual operation behavior of human drivers in the same scene, minimizing the error between the two based on batch supervised learning method, and iteratively optimizing network parameters using gradient descent method.
9. The human-like driving behavior control method based on scene-driven and imitation learning according to claim 1, characterized in that, In step S4, the vehicle perceives the current driving scene status information in real time during each control cycle. The status information includes scene structure features and vehicle operating status. The humanoid control strategy network outputs the vehicle control actions in real time based on the status information, which are then executed by the underlying control module.
10. A human-like driving behavior control system based on scene-driven and imitation learning, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in any one of claims 1 to 9.