Vehicle human-machine collaborative driving method, device, equipment and storage medium

Abstract

The present invention discloses a vehicle human-machine cooperative driving method, device, equipment and storage medium, which relates to the field of intelligent driving technology, and includes the following steps: obtaining a driving system output angle based on error information; simulating the driver's steering to obtain the driver's steering wheel angle; inputting the driving system output angle, the driver's steering wheel angle, the vehicle state information and the driver's state information into a reinforcement learning algorithm to obtain the driving weight of the driving system; linearly adding the driving weight of the driving system to the driver's steering wheel angle and the driving system output angle to obtain the final vehicle control angle, and controlling the vehicle through the final vehicle control angle. The present invention comprehensively considers the driver's state and the driving system steering information, respects the driver's wishes, and reduces the conflict between the driver's wishes and the final steering of the vehicle.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a vehicle human-machine collaborative driving method, device, equipment and storage medium. Background Art

[0002] The "Driving Automation Classification" standard proposed by the International Society of Automotive Engineers defines six driving automation levels from L0 to L5, namely no driving automation, driver assistance, partial driving automation, conditional driving automation, high driving automation, and full driving automation. With the rise of a new round of scientific and technological revolution, artificial intelligence has entered the era of big data, big computing, and big model algorithms. The deep integration of automobiles with artificial intelligence, information and communication and other fields has led to the continuous improvement of automobile automation and the development towards full driving automation. Although self-driving cars are gradually being implemented and have achieved some results, there are still some restrictions such as scenarios, road sections, and operating time. Technical difficulties such as perception and decision-making, as well as functional and network security issues need to be solved urgently. Therefore, it is difficult to achieve full driving automation in the short and medium term, and human-machine collaborative driving is still the current research focus.

[0003] Human-machine collaborative driving refers to the process in which the driver and the autonomous driving system jointly intervene in the driving process from the perception layer, decision layer, and control layer through master-slave control, switching control, shared control, etc., so as to collaboratively complete the driving task. In the driving task, the driver is good at reasoning and decision-making in complex environments, but his operating behavior is easily affected by factors such as the environment and state, showing randomness and instability. The driving system is good at refined perception and precise control, but its decision-making intelligence, learning and adaptability, and understanding of complex environments are weaker than humans. Human-machine collaborative driving enables the driver and the driving system to cooperate with each other, give full play to their respective advantages, and make up for their respective shortcomings, thereby improving overall safety and decision-making capabilities.

[0004] Human-machine collaborative driving can be divided into three categories according to the ownership of driving rights: master-slave control, switching control, and shared control. The driving rights of master-slave control belong to the driver. The driving system obtains traffic information that the driver cannot understand or does not fully understand through sensors, and analyzes and gives the driver multi-faceted warnings through the driving system, thereby guiding the driver to control. The driving rights of switching control are switched between the driver and the driving system in a time-sharing manner. In some scenarios, when the vehicle control exceeds the driver's ability, the driving system obtains control rights; on the contrary, when the vehicle control exceeds the driving system's ability, the system needs to wake up the driver and transfer the driving rights, such as automatic emergency braking system, adaptive cruise system and automatic parking system. However, in the above two control methods, the instability of driver control and the uncertainty of takeover can easily lead to accidents. Therefore, the relationship between the driver and the driving system cannot be limited to reminders, warnings or switching between humans and machines, but needs to form a complex dynamic interactive relationship of parallel control between humans and machines. Shared control means that both the driver and the driving system have driving rights at the same time. Through the control method of man-machine simultaneous in-loop, the driving rights can be flexibly transferred between the driver and the driving system, so as to achieve a common driving task and ensure driving safety and stability. Shared control, a special control method of man-machine simultaneous in-loop, faces the following difficulties:

[0005] (1) On what basis are driving rights allocated?

[0006] (2) What methods are used to achieve the allocation of driving rights;

[0007] (3) How to alleviate human-machine conflicts while improving driving comfort and tracking performance.

[0008] The paper "A Steering Decision-making Method for Human-Machine Cooperative Driving of Intelligent Vehicles Based on Reinforcement Learning" uses the driver model and the autonomous steering model of the intelligent vehicle as the interactive environment, and uses the Deep Deterministic Policy Gradient (DDPG) algorithm in reinforcement learning to continuously and dynamically allocate driving rights in human-machine cooperative driving. The state space of this model includes the control input of the driver and the driving system, as well as the vehicle and environment states. The action space is the driving weight μ of the driving system. The overall architecture is as follows: Figure 1 As shown. The driving right allocation strategy makes the optimal driving right allocation based on the real-time feedback of the driver's steering angle δ1, the vehicle's autonomous steering angle δ2 and the vehicle-road information, where the vehicle-road information includes the vehicle's tracking errors e1 and e2, the center of mass sideslip angle β, the lateral acceleration a and the expected path curvature c. The mathematical representation of the driving right allocation strategy is as follows:

[0009]

[0010] Where f(·) is the driving right allocation strategy, μ is the driving weight of the driving system, 1-μ is the driving weight of the driver, and δ3 is the comprehensive steering angle output by the driver and the driving system.

[0011] The model takes into account driving comfort, tracking accuracy, and road curvature, and can effectively improve tracking performance and comfort performance under different road curvatures. However, the diversity of driver states and the uncertainty of driving intentions in human-machine interaction can easily cause conflicts between the driver and the driving system, but the model does not consider the driver's real-time state and intentions in the loop, as well as the human-machine conflicts that may arise, which will lead to a reduction in driving performance and driving experience. Summary of the invention

[0012] The present invention provides a vehicle human-machine collaborative driving method, device, equipment and storage medium, which solves the problem that the existing model does not take into account the real-time status and intention of the driver in the loop and the possible human-machine conflict, resulting in reduced driving performance and driving experience.

[0013] In a first aspect, the present invention provides a vehicle human-machine collaborative driving method, comprising the following steps:

[0014] Collect error information when the driving system tracks the planned path, and obtain the output turning angle of the driving system based on the error information;

[0015] Simulating the driver's steering to obtain a simulated steering wheel angle, and combining the simulated steering wheel angle with the reaction lag time under different driver states to obtain the driver's steering wheel angle;

[0016] Collect vehicle status information and driver status information, input the driving system output angle, the driver's steering wheel angle, the vehicle status information and the driver status information into the reinforcement learning algorithm to obtain the driving weight of the driving system;

[0017] The driving weight of the driving system is linearly added to the driver's steering wheel angle and the driving system output angle to obtain the final vehicle control angle, and the vehicle is controlled by the final vehicle control angle.

[0018] Preferably, the collecting of error information when the driving system tracks the planned path comprises the following steps:

[0019] Predict the position and heading angle of the next control cycle based on the current speed and heading angle of the vehicle;

[0020] Find the matching point on the planned path based on the position and heading angle of the next control cycle;

[0021] Obtaining error information between the vehicle and the matching point in the Frenet coordinate system, wherein the error information includes a lateral error and a longitudinal error;

[0022] Obtaining the output steering angle of the driving system based on the error information includes the following steps:

[0023] The lateral error and longitudinal error are modeled by the vehicle dynamics model to obtain the continuous state space expression of the lateral error and longitudinal error;

[0024] Discretize the continuous state space expression to obtain a discrete state space;

[0025] Set the prediction time domain and control time domain, and predict the tracking error in the time domain based on the discrete state space from the current moment;

[0026] With the goal of minimizing the tracking error and its rate of change and minimizing the control increment, the objective function of the quadratic programming problem in the prediction time domain is established to solve the output steering system steering angle.

[0027] Preferably, the driver state includes a highly focused state, a moderately focused state, and an inattentive state, with attention scores decreasing in sequence;

[0028] Simulating the driver's steering to obtain a simulated steering wheel angle includes the following steps:

[0029] A two-point preview mechanism is used on the driver's expected trajectory to obtain a far preview point and a near preview point, and the lateral distance error between the vehicle and the far preview point and the near preview point is obtained;

[0030] PID control is used to output the steering wheel angles required for the near preview point and the far preview point respectively, and the simulated steering wheel angle is obtained;

[0031] The simulated steering wheel angle is combined with the driver's reaction lag time in different states by the following formula:

[0032]

[0033] In the formula, δ h is the driver's steering wheel angle, PID is the PID controller, e far and e near are the lateral distance errors between the vehicle and the far and near preview points, respectively; t is the reaction lag time under different driving conditions; e -ts is the transfer function of the delay link.

[0034] Preferably, the driving system output angle, the driver's steering wheel angle, the vehicle state information and the driver state are input into a reinforcement learning algorithm to obtain the driving weight of the driving system, comprising the following steps:

[0035] The driving system output angle, the driver's steering wheel angle, the vehicle state information and the driver state are used as the system state; the vehicle state information includes the lateral acceleration of the vehicle, the lateral distance error between the vehicle and the track, and the heading angle error;

[0036] For a time step t, get the current system state s t ;

[0037] Input the current system state into the new Actor network to generate the corresponding action probability distribution π θ (a t |s t ), sample the action probability distribution and get the current action a t ;

[0038] Get the current action a t The reward value r t , get the next system state s t+1 ,Will t ,a t ,r t ,s t+1 >Store to experience pool;

[0039] Input the current system state and the next system state into the Critic network and calculate the corresponding value;

[0040] Repeat the above iterations in multiple time steps until the data in the experience pool is greater than the set value; calculate the objective function of the new Actor network and the loss function of the Critic network, and then update the parameters of the new Actor network and the Critic network through back propagation;

[0041] After a certain time step, the parameters of the new Actor network are passed to the old Actor network to update the network;

[0042] Outputs the driving weights of the driving system.

[0043] Preferably, the objective function of the new Actor network is as follows:

[0044]

[0045] Where J(θ) is the objective function of the new Actor network, θ is the new parameter of the policy network, and θ′ is the old parameter of the policy network. is the expectation, ρ t (θ) is the ratio of the new and old strategies, is the estimate of the advantage function, clip is the cutoff function, and ε is the hyperparameter of the cutoff range;

[0046] ​The loss function of the Critic network is as follows:

[0047] Loss(θ)=[V(s t )-V target ] 2

[0048] Where Loss(θ) is the loss function of the Critic network, V(s t ) is the state value estimated by the Critic network, V target is the target value.

[0049] Preferably, the current action a is obtained by the following formula: t The reward value r t :

[0050] r=w1r tracking +w2r comfort +w3r safe +w4r conflict

[0051] in,

[0052] r tracking =-(|e lat |+|e yaw |)

[0053]

[0054] r safe =-200if collision

[0055] r conflict =-(|δ-δ h |+|s h -μ|)

[0056] In the formula, r is the reward function, r tracking To track the performance reward function, r comfort is the comfort performance reward function, r safe is the safety performance reward function, r conflict is the human-machine conflict reward function, w1, w2, w3 and w4 are the corresponding weights, e lat is the vehicle lateral distance error, e yaw is the heading angle error, a lat is the lateral acceleration, is the lateral acceleration, is the driving system weight change rate, δ is the final vehicle control angle, δ h is the driver's steering wheel angle, s h is the driver state, and μ is the driving weight of the driving system.

[0057] Preferably, the final vehicle control angle is as follows:

[0058] δ=μδ v +(1-μ)δ h

[0059] In the formula, δ is the final vehicle control angle, μ is the driving weight of the driving system, and δ v Output steering angle for the driving system, δ h is the driver's steering wheel angle.

[0060] In a second aspect, the present invention further provides a vehicle human-machine cooperative driving device, comprising:

[0061] A collection module, used to collect error information when the driving system tracks the planned path, and obtain the output angle of the driving system based on the error information;

[0062] A simulation module, used to simulate the driver's steering to obtain a simulated steering wheel angle, and combine the simulated steering wheel angle with the reaction lag time under different driver states to obtain the driver's steering wheel angle;

[0063] An input module is used to collect vehicle status information and driver status information, and input the driving system output angle, the driver's steering wheel angle, the vehicle status information and the driver status information into the reinforcement learning algorithm to obtain the driving weight of the driving system;

[0064] The control module is used to linearly add the driving weight of the driving system to the driver's steering wheel angle and the driving system output angle to obtain a final vehicle control angle, and control the vehicle through the final vehicle control angle.

[0065] In a second aspect, the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned vehicle human-machine collaborative driving method when executing the program.

[0066] In a second aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned vehicle human-machine collaborative driving method is implemented.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] The present invention first obtains the driving system output angle and the driver's steering wheel angle, and collects vehicle state information and driver state information. The driving system output angle, the driver's steering wheel angle, the vehicle state information and the driver's state information are input into the reinforcement learning algorithm to obtain the driving weight of the driving system. The present invention comprehensively considers the driver's state and the steering information of the driving system, respects the driver's wishes, and reduces the conflict between the driver's wishes and the final steering of the vehicle. At the same time, when obtaining the driver's steering wheel angle, the reaction lag time of the driver in different states is combined with the simulated steering wheel angle, so that the driving right distribution is more in line with the driver's manipulation preferences in different states, reducing the conflict between the driver's state and the weight distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1 To provide an architecture diagram for the existing human-machine integration transition;

[0071] Figure 2 A flowchart of a vehicle human-machine collaborative driving method of the present invention;

[0072] Figure 3 It is the block diagram of the PPO algorithm of the present invention;

[0073] Figure 4 A roadmap for an embodiment of the present invention;

[0074] Figure 5 A schematic diagram of weight distribution and turning angles for drivers in different states by a driving system according to an embodiment of the present invention;

[0075] in, Figure 5 (a): Schematic diagram of the turning angle when the driver is highly focused. Figure 5 (b): Schematic diagram of weight distribution when the driver is highly focused. Figure 5 (c): Schematic diagram of the turning angle when the driver's attention is moderately focused. Figure 5 (d) Schematic diagram of weight distribution when the driver's attention is moderately concentrated. Figure 5 (e): Schematic diagram of turning angle when the driver is not paying attention. Figure 5 (f): Schematic diagram of weight distribution when the driver is not paying attention;

[0076] Figure 6It is a schematic diagram of the assisting effect of the driving system of an embodiment of the present invention on drivers in different states;

[0077] in, Figure 6 (a): Comparison of tracking error between driver-assisted driving and driving system-assisted driving. Figure 6 (b): Comparison of lateral acceleration between driver alone driving and driving with the assistance of the driving system. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] Reference Figure 2 The present invention provides a vehicle human-machine collaborative driving method, comprising the following steps:

[0080] Step 1: The driving system tracks the planned path and uses MPC control to output the steering angle δ of the driving system v .

[0081] Model Predictive Control (MPC) is used as the control algorithm of the driving system. It specifically includes the following steps:

[0082] S1: Predict the position and heading angle of the next control cycle based on the current speed and heading angle of the vehicle.

[0083] S2: Find the matching point on the planned path based on the position and heading angle of the next control cycle.

[0084] S3: Calculate the lateral error and longitudinal error between the vehicle and the matching point in the Frenet coordinate system.

[0085] S4: Use the vehicle dynamics model to model the trajectory tracking error and obtain its continuous state space expression. Then use the midpoint Euler method and the forward Euler method to discretize it and obtain the state space of the discrete system. Set the prediction time domain and the control time domain, and predict the tracking error in the future prediction time domain from the current moment. Then transform it into a quadratic programming problem, with the goal of minimizing the tracking error and its rate of change and minimizing the control increment, establish the objective function of the quadratic programming problem in the prediction time domain, and solve the output driving system steering angle δ v .

[0086] Step 2: Simulate the driver's steering to obtain a simulated steering wheel angle, and combine the simulated steering wheel angle with the reaction lag time under different driver states to obtain the driver's steering wheel angle.

[0087] A PID controller based on two-point preview is used to simulate the driver's steering, and the driver's reaction lag time under different states is taken into account. A two-point preview mechanism is used on the driver's expected trajectory to obtain the far and near preview points, and the lateral distance error between the vehicle and the far and near preview points is calculated. In order to be closer to the error compensation behavior of actual human drivers, PID control is used to output the steering wheel angles required for the near and far preview points respectively. The steering wheel angles required for the far and near preview points and the reaction lag time are comprehensively considered to output the final driver's steering wheel angle δ h :

[0088]

[0089] Among them, PID is the PID controller, e far and e near are the lateral distance errors between the vehicle and the far and near preview points, t is the reaction lag time, e -ts is the transfer function of the delay link.

[0090] Step 3: Collect vehicle status information and driver status information.

[0091] Vehicle status information is collected through the Carla autonomous driving simulation platform.

[0092] The driver's EEG signal is collected by a head-mounted EEG acquisition device, and the collected raw EEG data is preprocessed and denoised. Then, the EEG signal features related to attention are extracted from the raw EEG signals through feature extraction, and the features are input into a neural network model for EEG signal classification to calculate the attention score and perform classification. The classification threshold includes a first threshold and a second threshold, the first threshold is less than the second threshold, and the categories are set to a highly concentrated state, a moderately concentrated state, and an inattentive state, wherein an attention state with an attention score less than the first threshold is an inattentive state, an attention score greater than the first threshold and less than the second threshold is a moderately concentrated state, and an attention state with an attention score greater than the second threshold is a highly concentrated state.

[0093] Step 4: Reference Figure 3 The driving system output angle, the driver's steering wheel angle, the vehicle state information and the driver state information are input into the PPO algorithm in reinforcement learning. The output action is the driving weight of the driving system, and the strategy in reinforcement learning is updated by maximizing the reward function.

[0094] The Proximal Policy Optimization (PPO) algorithm in reinforcement learning is used to implement the driving rights allocation strategy in human-machine collaborative driving.

[0095] The Actor network is a policy network, whose input is the current state s t , the output is the probability distribution of continuous actions π θ (a t |s t ), obtain the current action a by sampling t , its objective function is:

[0096]

[0097] in, is the ratio of the new and old strategies, For expectations, is the estimate of the advantage function, clip is the cutoff function, and ε is the hyperparameter of the cutoff range.

[0098] The Critic network is a value network, and its input is the current state s t , the output is the value of the state. The role of the Critic network is to calculate the current state s t and the selected action a t Advantages A t On the other hand, the parameters of the Critic network are updated by minimizing the loss function to more accurately estimate the value of the state. The loss function is:

[0099] Loss(θ)=[V(s t )-V target ] 2 (2)

[0100] Where V(s t ) represents the state value estimated by the Critic network, V target is the target value.

[0101] The training process is that the Actor network is in state s at time step t t Generate the corresponding action probability distribution π θ (a t |s t ), and then sample according to the probability distribution to get the current action a t , and obtain the reward value r t Then get the next state s t+1 For the collected data t ,a t ,r t ,s t+1 ​>, store it in the experience pool, stop collecting when enough data is collected in the experience pool, then calculate the objective function of the Actor network and the loss function of the Critic network, and then update the parameters of the new Actor network and the Critic network through back propagation. After a certain time step, the parameters of the new Actor network are passed to the old Actor network, so that the new and old Actor networks collect data alternately and continuously update the network, thereby improving the strategic performance of the high-intelligence agent.

[0102] The state space S = [s h ,δ h ,δ v ,s v ], by the driver state s h 、Driver steering wheel angle δ h 、 Driving system angle δ v , vehicle status v The vehicle state includes the lateral acceleration a of the vehicle. lat , the lateral distance error between the vehicle and the track e lat , heading angle error e yaw .

[0103] Action space A = [μ], where μ is the driving weight of the driving system, ranging from [0,1].

[0104] In order to ensure tracking performance, comfort performance, safety performance and reduce human-machine conflict, the reward function is set as:

[0105] r=w1r tracking +w2r comfort +w3r safe +w4r conflict (3)

[0106] where r tracking To track the performance reward function, r comfort is the comfort performance reward function, r safe is the safety performance reward function, r conflict For the human-machine conflict reward function, the weights of each item are w1=2, w2=w3=w4=1 respectively.

[0107] The reward for tracking performance is determined by the vehicle lateral distance error e lat and heading angle error e yaw The smaller the absolute values ​​of the lateral distance error and the heading angle error are, the higher the tracking accuracy during driving.

[0108] r tracking =-(|e lat |+|e yaw |) (4)

[0109] The comfort performance is rewarded by the lateral acceleration a lat , lateral acceleration Driving system weight change rate Composition, lateral acceleration a lat , lateral acceleration Driving system weight change rate The smaller the absolute value, the better the comfort performance.

[0110]

[0111] The safety performance is the detection of a collision, and if a collision occurs, the reward for the safety performance is reduced.

[0112] r safe =-200if collision (6)

[0113] The reward for human-machine conflict is composed of the difference between the vehicle's final control angle and the driver's angle, and the difference between the driver's state and the driving weight of the driving system. The smaller the absolute value, the smaller the human-machine conflict.

[0114] r conflict =-(|δ-δ h |+|s h -μ|) (7)

[0115] Step 5: Use the driving weight of the driving system output in step 4 to linearly add the driver's steering wheel angle and the driving system output angle to obtain the final vehicle control angle.

[0116] The output driving weight μ of the driving system is used to adjust the driver's steering wheel angle δ h 、Drive system output angle δ v Perform linear addition to obtain the final vehicle control angle, and the calculation formula is:

[0117] δ=μδ v +(1-μ)δ h (8)

[0118] Where 1-μ is the driver's driving weight.

[0119] Step 6: Apply the final vehicle control angle and update the environment state.

[0120] Based on the same concept, the present invention also provides a vehicle human-machine collaborative driving device, including an acquisition module, a simulation module, an input module and a control module.

[0121] The acquisition module is used to collect the current speed and heading angle of the vehicle when the driving system is tracking the planned path, and obtain the output turning angle of the driving system based on the current speed and heading angle of the vehicle.

[0122] The simulation module is used to simulate the driver's steering to obtain a simulated steering wheel angle, and combine the simulated steering wheel angle with the reaction lag time under different driver states to obtain the driver's steering wheel angle.

[0123] The input module is used to collect vehicle status information and driver status information, and input the driving system output angle, the driver's steering wheel angle, the vehicle status information and the driver status information into the reinforcement learning algorithm to obtain the driving weight of the driving system.

[0124] The control module is used to linearly add the driving weight of the driving system to the driver's steering wheel angle and the driving system output angle to obtain the final vehicle control angle, and control the vehicle through the final vehicle control angle.

[0125] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above-mentioned vehicle human-machine collaborative driving method is implemented when the processor executes the program.

[0126] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned vehicle human-machine collaborative driving method is implemented.

[0127] Example

[0128] In order to verify the effectiveness of this method, this embodiment uses the Carla autonomous driving simulation platform for verification. Figure 4 As shown, the selected route length is about 600m, and the starting point is selected in the straight part of the path, which contains six curves with different curvatures.

[0129] First, drivers in different states (high concentration, medium concentration, and lack of concentration) drive along the above trajectory alone. Then, drivers in different states drive along the above trajectory with the assistance of the driving system. The vehicle steering angle, driving weight of the driving system, comfort performance, and tracking performance are compared.

[0130] Depend on Figure 5It can be seen that, first of all, the driving weights assigned to drivers in different states of driving assistance by the driving system are different. Drivers with higher concentration levels have higher initiative and higher safety. The driving system does not need to intervene too much, but only needs to assist in adjusting the tracking and comfort of driving. Therefore, the driving system's driving authority is relatively low overall, mainly concentrated around 0.2. Drivers with medium concentration levels have medium initiative and safety. The driving system's driving authority and the driver's weight each account for half, mainly concentrated around 0.5. Drivers with lower concentration levels have lower initiative and safety, and require excessive assistance and intervention from the driving system to prevent danger. Therefore, the driving system's driving authority is relatively high, mainly concentrated around 0.8, thereby ensuring the driver's safety.

[0131] Secondly, at every bend, the driving weight of the driving system first decreases and then increases. The decrease indicates that the driver has the intention to turn, and the driver's intention needs to be respected to reduce human-machine conflict, so the driving weight of the driving system needs to be reduced. The subsequent increase indicates that the driving system begins to play an auxiliary role to help the driver turn better.

[0132] Depend on Figure 6 It can be seen that no matter what state the driver is in, with the assistance of the driving system, the vehicle's tracking error and jerk are reduced, indicating that the driving system can effectively improve the overall tracking performance and comfort performance.

[0133] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0134] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A vehicle human-machine collaborative driving method, characterized in that: The following steps are involved: Collect error information when the driving system tracks the planned path, and obtain the output turning angle of the driving system based on the error information; Simulating the driver's steering to obtain a simulated steering wheel angle, and combining the simulated steering wheel angle with the reaction lag time under different driver states to obtain the driver's steering wheel angle; Collect vehicle status information and driver status information, input the driving system output angle, the driver's steering wheel angle, the vehicle status information and the driver status information into the reinforcement learning algorithm to obtain the driving weight of the driving system; The driving weight of the driving system is linearly added to the driver's steering wheel angle and the output angle of the driving system to obtain a final vehicle control angle, and the vehicle is controlled by the final vehicle control angle; The driver status includes a highly focused state, a moderately focused state, and an inattentive state, with attention scores decreasing in sequence; Simulating the driver's steering to obtain a simulated steering wheel angle includes the following steps: A two-point preview mechanism is used on the driver's expected trajectory to obtain a far preview point and a near preview point, and the lateral distance error between the vehicle and the far preview point and the near preview point is obtained; PID control is used to output the steering wheel angles required for the near preview point and the far preview point respectively, and the simulated steering wheel angle is obtained; The simulated steering wheel angle is combined with the driver's reaction lag time in different states by the following formula: In the formula, δ h is the driver's steering wheel angle, PID is the PID controller, e far and e near are the lateral distance errors between the vehicle and the far and near preview points, respectively; t is the reaction lag time under different driving conditions; e -ts is the transfer function of the delay link, and s is a complex variable.

2. A vehicle human-machine collaborative driving method as claimed in claim 1, characterized in that: The step of collecting error information when the driving system tracks the planned path includes the following steps: Predict the position and heading angle of the next control cycle based on the current speed and heading angle of the vehicle; Find the matching point on the planned path based on the position and heading angle of the next control cycle; Obtaining error information between the vehicle and the matching point in the Frenet coordinate system, wherein the error information includes a lateral error and a longitudinal error; Obtaining the output steering angle of the driving system based on the error information includes the following steps: The lateral error and longitudinal error are modeled by the vehicle dynamics model to obtain the continuous state space expression of the lateral error and longitudinal error; Discretize the continuous state space expression to obtain a discrete state space; Set the prediction time domain and control time domain, and predict the tracking error in the time domain based on the discrete state space from the current moment; With the goal of minimizing the tracking error and its rate of change and minimizing the control increment, the objective function of the quadratic programming problem in the prediction time domain is established to solve the output steering system steering angle.

3. The vehicle human-machine collaborative driving method according to claim 1, characterized in that: The output angle of the driving system, the steering wheel angle of the driver, the vehicle state information and the driver state are input into the reinforcement learning algorithm to obtain the driving weight of the driving system, including the following steps: The driving system output angle, the driver's steering wheel angle, the vehicle state information and the driver state are used as the system state; the vehicle state information includes the lateral acceleration of the vehicle, the lateral distance error between the vehicle and the track, and the heading angle error; For a time step t, get the current system state s t ; Input the current system state into the new Actor network to generate the corresponding action probability distribution π θ (a t |s t ), sample the action probability distribution and get the current action a t ; Get the current action a t The reward value r t , get the next system state s t+1 ,Will t ,a t ,r t ,s t+1 >Store to experience pool;​ Input the current system state and the next system state into the Critic network and calculate the corresponding value; Repeat the process of obtaining the current system state and the next system state and the calculation process of the corresponding value in multiple time steps until the data in the experience pool is greater than the set value; calculate the objective function of the new Actor network and the loss function of the Critic network, and then update the parameters of the new Actor network and the Critic network through back propagation; After a certain time step, the parameters of the new Actor network are passed to the old Actor network to update the network; Outputs the driving weights of the driving system.

4. A vehicle human-machine collaborative driving method as claimed in claim 3, characterized in that: The objective function of the new Actor network is as follows: Where J(θ) is the objective function of the new Actor network, θ is the new parameter of the policy network, and θ′ is the old parameter of the policy network. is the expectation, ρ t (θ) is the ratio of the new and old strategies, is the estimate of the advantage function, clip is the cutoff function, and ε is the hyperparameter of the cutoff range; The loss function of the Critic network is as follows: Loss(θ)=[V(s t )-V target ] 2 Where Loss(θ) is the loss function of the Critic network, V(s t ) is the state value estimated by the Critic network, V target is the target value.

5. The vehicle human-machine collaborative driving method according to claim 3, characterized in that: Get the current action a by the following formula t The reward value r t : r=w1r tracking +w2r comfort +w3r safe +w4r conflict in, r tracking =-(|e lat |+|and yaw |) r safe =-200if collision r conflict =-(|δ-δ h |+|s h -μ|) In the formula, r is the reward function, r tracking To track the performance reward function, r comfort is the comfort performance reward function, r safe is the safety performance reward function, r conflict is the human-machine conflict reward function, w1, w2, w3 and w4 are the corresponding weights, e lat is the vehicle lateral distance error, e yaw is the heading angle error, a lat is the lateral acceleration, is the lateral acceleration, is the driving system weight change rate, δ is the final vehicle control angle, δ h is the driver's steering wheel angle, s h is the driver state, and μ is the driving weight of the driving system.

6. The vehicle human-machine cooperative driving method according to claim 1, characterized in that: The final vehicle control angle is as follows: d=μδ v +(1-μ)d h In the formula, δ is the final vehicle control angle, μ is the driving weight of the driving system, and δ v Output steering angle for the driving system, δ h is the driver's steering wheel angle.

7. A vehicle human-machine cooperative driving device, characterized in that: include: A collection module, used to collect error information when the driving system tracks the planned path, and obtain the output angle of the driving system based on the error information; A simulation module, used to simulate the driver's steering to obtain a simulated steering wheel angle, and combine the simulated steering wheel angle with the reaction lag time under different driver states to obtain the driver's steering wheel angle; An input module is used to collect vehicle status information and driver status information, and input the driving system output angle, the driver's steering wheel angle, the vehicle status information and the driver status information into the reinforcement learning algorithm to obtain the driving weight of the driving system; A control module, used for linearly adding the driving weight of the driving system to the driver's steering wheel angle and the driving system output angle to obtain a final vehicle control angle, and controlling the vehicle through the final vehicle control angle; The driver status includes a highly focused state, a moderately focused state, and an inattentive state, with attention scores decreasing in sequence; Simulating the driver's steering to obtain a simulated steering wheel angle includes the following steps: A two-point preview mechanism is used on the driver's expected trajectory to obtain a far preview point and a near preview point, and the lateral distance error between the vehicle and the far preview point and the near preview point is obtained; PID control is used to output the steering wheel angles required for the near preview point and the far preview point respectively, and the simulated steering wheel angle is obtained; The simulated steering wheel angle is combined with the driver's reaction lag time in different states by the following formula: In the formula, δ h is the driver's steering wheel angle, PID is the PID controller, e far and e near are the lateral distance errors between the vehicle and the far and near preview points, respectively; t is the reaction lag time under different driving conditions; e -ts is the transfer function of the delay link, and s is a complex variable.

8. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle human-machine collaborative driving method described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the vehicle human-machine collaborative driving method described in any one of claims 1 to 6 is implemented.

Images