A method for generating autonomous driving test scenarios based on personalized driver model

Abstract

The present invention relates to a method for generating an automatic driving test scenario based on a personalized driver model, including: based on genetic and evolutionary ideas, combined with reinforcement learning, simulating various stages of human drivers' driving experience, training step by step to obtain driver models at different growth stages, including a novice driver model, an experienced driver model, and a skilled driver model; based on driver models at different growth stages, according to different personalized driving characteristics of human drivers, further training to obtain personalized driver models, including an aggressive driver model, a conservative driver model, a provocative driver model, and a cooperative driver model; according to test requirements, combining driver models at different growth stages and different personalized driver models according to corresponding demand ratios to generate corresponding target test scenarios. Compared with the prior art, the present invention can effectively improve the authenticity and complexity of generated test scenarios, and improve test effects and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method for generating an autonomous driving test scenario based on a personalized driver model. Background Art

[0002] With the improvement of the level of autonomous driving, the test tools and methods for traditional cars can no longer meet the needs of autonomous driving car testing. The scenario-based virtual testing method has huge technical advantages in terms of test efficiency and test cost. It is an important means of testing and verifying autonomous driving cars in the future and has become a current research hotspot.

[0003] Currently, there are two types of autonomous driving vehicle testing: real vehicle testing and virtual simulation testing. Real vehicle testing requires a lot of manpower and time. With the improvement of driving automation level, the limitations of real vehicle testing are becoming more and more obvious. People are beginning to pay more attention to virtual simulation testing. Virtual simulation testing can efficiently test the safety performance of autonomous driving systems, but the test scenarios generated by existing virtual simulation technology are often fixed, and the behaviors of other vehicles in the scenarios are also fixed. There is a lack of interactivity between vehicles, which means that the autonomous driving system that passes the test may only respond correctly to specific test behaviors, resulting in low credibility of the test results and low test accuracy. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an automatic driving test scenario generation method based on a personalized driver model. The driver models at different growth stages and different personalized driver models are combined as needed to generate a target test scenario, thereby ensuring the accuracy of subsequent tests and improving test results.

[0005] The objective of the present invention can be achieved by the following technical solution: A method for generating an autonomous driving test scenario based on a personalized driver model, comprising the following steps:

[0006] S1. Based on genetic and evolutionary ideas and combined with reinforcement learning, the various stages of human drivers' driving experience are simulated, and driver models of different growth stages are obtained through step-by-step training. The driver models of different growth stages include novice driver models, experienced driver models and skilled driver models.

[0007] S2. Based on driver models at different growth stages, and according to different personalized driving characteristics of human drivers, further training to obtain personalized driver models, wherein the personalized driver models include an aggressive driver model, a conservative driver model, a provocative driver model, and a cooperative driver model;

[0008] S3. According to the test requirements, driver models at different growth stages and different personalized driver models are combined in accordance with corresponding demand ratios to generate corresponding target test scenarios.

[0009] Furthermore, the step S1 specifically includes the following steps:

[0010] S11. Define and build a safe driver model;

[0011] S12. According to the safe driver model, based on genetic and evolutionary ideas and combined with reinforcement learning, a novice driver model, an experienced driver model and a skilled driver model are obtained through step-by-step training.

[0012] Furthermore, the driving strategy of the safety driver model is specifically as follows:

[0013] when When the vehicle takes the action of "sudden deceleration",

[0014] when When , the vehicle takes the action of "deceleration";

[0015] when When , the vehicle takes the action of "deceleration";

[0016] Among them, d rel_front is the relative distance between the vehicle and the preceding vehicle in the same lane, v rel_front is the relative speed between the vehicle and the preceding vehicle in the same lane, v ego is the vehicle speed.

[0017] Furthermore, the action space of the novice driver model only has longitudinal actions, including five actions: longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration and longitudinal rapid deceleration;

[0018] The novice driver model is based on the longitudinal relative distance d between the vehicle and the other vehicle. rel , longitudinal relative velocity v rel , vehicle speed v ego and lane information to define the state set. The other vehicle includes the vehicle in a different lane from the vehicle. When the other vehicle is in front of the vehicle, the longitudinal speed of the other vehicle is recorded as v f , when the other car is behind your car, the longitudinal speed of the other car is v r , then the longitudinal relative velocity is defined as follows:

[0019]

[0020] Among them, d rel Included rel_front d rel_crear d rel_left_front d rel_left_reard rel_rigth_front and d rel_right_rear , which are the relative distance between the vehicle and the front vehicle in the same lane, the relative distance between the vehicle and the rear vehicle in the same lane, the relative distance between the vehicle and the front vehicle in the left lane, the relative distance between the vehicle and the rear vehicle in the left lane, the relative distance between the vehicle and the front vehicle in the right lane, and the relative distance between the vehicle and the rear vehicle in the right lane; v rel Include v rel_front 、v rel_crear 、v rel_left_front 、v rel_left_rear 、v rel_rigth_front and v rel_right_rear , which are the relative speed between the vehicle and the vehicle in front of it in the same lane, the relative speed between the vehicle and the vehicle behind it in the same lane, the relative speed between the vehicle and the vehicle in front of it in the left lane, the relative speed between the vehicle and the vehicle behind it in the left lane, the relative speed between the vehicle and the vehicle in front of it in the right lane, and the relative speed between the vehicle and the vehicle behind it in the right lane;

[0021] The novice driver model defines the lane number line_id according to the lane in which the vehicle is located, and calculates the lane number line_id according to the speed of the vehicle and the relative distance between the vehicle in front. It is divided into three states, including:

[0022]

[0023] Calculated based on the vehicle speed and relative speed It is divided into three states, including:

[0024]

[0025] In the novice driver model, the offset of the vehicle in the lane relative to the lane centerline is quantified as:

[0026] "Left deviation" means that the vehicle is on the left side of the lane centerline;

[0027] “Centered” means the vehicle is located on the center line of the lane;

[0028] "Deviated to the right" means that the vehicle is on the right side of the center line of the lane.

[0029] Furthermore, the specific process of training the novice driver model in step S12 is as follows:

[0030] First, establish a vehicle motion model over time, select an appropriate time step, initialize parameters, and generate a simulation environment. The initialization parameters include the total number of simulated vehicles n c , Maximum initialization distance allowed And the simulation time t f ;

[0031] Then, the safe driver model is used as the background car for training to obtain the prototype of the novice driver model;

[0032] Then, the prototype of the novice driver model is used to replace the safe driver model as the background vehicle for training, thereby obtaining a novice driver model;

[0033] The reward function in the novice driver model training process is specifically:

[0034] R 1 =ω s R safety +ω e R efficiency

[0035] Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor;

[0036] R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance. The specific responsibility judgment mechanism is: when a vehicle collision occurs, if both vehicles do not change lanes, the front vehicle has no traffic accident responsibility, and the rear vehicle bears all traffic accident responsibility; if one vehicle changes lanes, the vehicle with lane change behavior bears full responsibility; if both vehicles change lanes, the two vehicles share the responsibility;

[0037] R efficiency is the reward function that takes into account the traffic efficiency;

[0038] During the training process, the novice driver model obtains environmental information by interacting with the environment. The novice driver model obtains the state quantity s at the current time t. t After that, select a state variable s t The most valuable action During specific training, the novice driver model is in s t Under the state, according to Select the action to be performed t Afterwards, you get a reward r t+1 , and enter the next state s t+1 , according to the corresponding and r t+1 To update Q(s t ,a t ), through continuous iterative training, the novice driver model can eventually make the optimal action corresponding to the current environmental state.

[0039] Furthermore, the action space of the experienced driver model inherits five actions of the novice driver model, and combines the lateral uniform speed with the five longitudinal actions to evolve into five new lane changing actions, including longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration, longitudinal rapid deceleration, longitudinal uniform speed and lateral uniform speed, longitudinal acceleration and lateral uniform speed, longitudinal rapid acceleration and lateral uniform speed, longitudinal deceleration and lateral uniform speed, and longitudinal rapid deceleration and lateral uniform speed, a total of ten actions;

[0040] The state set of the experienced driver model is consistent with the state set of the novice driver model;

[0041] The specific process of training and verifying the driver model in step S12 is as follows:

[0042] Firstly, the safe driver model is used as the background vehicle for training to obtain the prototype of the experienced driver model;

[0043] Then, the trained novice driver model and the prototype of the experienced driver model are used in equal proportion to replace the safe driver model as the background vehicle for training, thereby obtaining the experienced driver model;

[0044] The reward function in the training process of the empirical driver model is specifically:

[0045] R 2 =ω s R safety +ω e R efficiency +ω r R riding_comfort

[0046] Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor, ω r is the weight of the riding comfort factor;

[0047] R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism, following distance and driving experience;

[0048] R efficiency is the reward function that takes into account the traffic efficiency;

[0049] R riding_comfort is the reward function that takes riding comfort into consideration.

[0050] Further, the action space of the skilled driver model inherits ten actions of the experienced driver model, and combines lateral acceleration and lateral deceleration with five longitudinal actions to evolve ten new lane changing actions, including: longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration, longitudinal rapid deceleration, longitudinal uniform speed and lateral uniform speed, longitudinal acceleration and lateral uniform speed, longitudinal rapid acceleration and lateral uniform speed, longitudinal deceleration and lateral uniform speed, longitudinal rapid deceleration and lateral uniform speed, longitudinal uniform speed and lateral deceleration, longitudinal acceleration and lateral deceleration, longitudinal rapid acceleration and lateral deceleration, longitudinal deceleration and lateral deceleration, longitudinal rapid deceleration and lateral deceleration, longitudinal uniform speed and lateral acceleration, longitudinal acceleration and lateral acceleration, longitudinal rapid acceleration and lateral acceleration, longitudinal deceleration and lateral acceleration, and longitudinal rapid deceleration and lateral acceleration, a total of twenty actions;

[0051] The state set of the skilled driver model is consistent with the state set of the novice driver model;

[0052] The specific process of training the skilled driver model in step S12 is as follows:

[0053] First, use the safe driver model as the background car for training to obtain the prototype of the skilled driver model;

[0054] Then, the trained novice driver model, the experienced driver model and the prototype of the skilled driver model are used in equal proportions to replace the safe driver model as background vehicles for training to generate a skilled driver model;

[0055] The reward function in the skilled driver model training process is specifically:

[0056] R 3 =ω s R safety +ω e R efficiency +ω r R riding_comfort

[0057] Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor, ω r is the weight of the riding comfort factor;

[0058] R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance;

[0059] R efficiency is the reward function that takes into account the traffic efficiency;

[0060] R riding_comfort is the reward function that takes riding comfort into consideration.

[0061] Furthermore, the step S2 specifically includes the following steps:

[0062] S21, determining the category of the personalized driver model to be trained, so that the personalized driver model inherits the action space of the skilled driver model, and the state set of the personalized driver type is consistent with the state set of the novice driver model, wherein the category of the personalized driver model includes an aggressive driver model, a conservative driver model, a provocative driver model, and a cooperative driver model;

[0063] S22, using the safe driver model as a background vehicle for training to obtain a prototype of a personalized driver model, and then replacing the safe driver model with a novice driver model, an experienced driver model, and a skilled driver model in equal proportions as the background vehicle for training to generate a corresponding personalized driver model.

[0064] Furthermore, during the training process of the personalized driver model, the reward function of the aggressive driver model is specifically:

[0065] R 4 =ω s R safety +ω d R driving_feeling +ω e R efficiency

[0066] Among them, ω s is the weight of the safety factor, ω d is the weight of driving experience, ω e is the weight of traffic efficiency;

[0067] R safety In order to consider the safety of the reward function, the design of this part of the reward function needs to consider the responsibility judgment mechanism;

[0068] R driving_feeling In order to consider the reward function of driving experience, the design of this part of the reward function needs to consider the driving experience related to traffic flow density and vehicle speed within a certain range of the vehicle;

[0069] R efficiency is the reward function that takes into account the traffic efficiency;

[0070] The reward function of the conservative driver model is as follows:

[0071] R 5 =ω s R safety +ω r R riding_comfort +ω e R efficiency

[0072] Among them, ω s is the weight of the safety factor, ω r is the weight of the riding comfort factor, ω e is the weight of traffic efficiency;

[0073] R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance;

[0074] R riding_comfort is the reward function that takes riding comfort into consideration;

[0075] R efficiency is the reward function that takes into account the traffic efficiency;

[0076] The reward function of the aggressive driver model is as follows:

[0077] R 6 =ω s R safety +ω d R driving_feeling +ω e R efficiency

[0078] Among them, ω s is the safety weight, ω d is the weight of driving experience, ω e is the weight of traffic efficiency;

[0079] R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance;

[0080] R driving_feeling is the reward function that takes driving experience into account;

[0081] R efficiency is the reward function that takes into account the traffic efficiency;

[0082] The reward function of the cooperative driver model is as follows:

[0083] R 7 =ω s E safety +ω d E driving_feeling +ω e E efficiency

[0084] Among them, ω s is the safety weight, ω d is the weight of driving experience, ω e is the weight of traffic efficiency;

[0085] R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance;

[0086] R driving_feeling is the reward function that takes driving experience into account;

[0087] R efficiency is the reward function that takes into account the traffic efficiency.

[0088] Compared with the prior art, the present invention proposes a scheme based on genetic and evolutionary ideas combined with reinforcement learning to train a driver model. By defining a safe driver model, a novice driver model is first generated, then an experienced driver model with certain experience is generated step by step, and finally a skilled driver model is generated. Then, based on the skilled driver model, human personalized driving characteristics are given to train a personalized driver model. Finally, the generated driver models are combined in a certain proportion according to the test requirements to generate corresponding test scenarios.

[0089] The present invention fully considers the different driving proficiency levels of different drivers in reality and the different driving styles among skilled drivers, and trains a variety of driver models, so that the generated test scenarios have strong authenticity.

[0090] In the present invention, since the driving styles and driving proficiency of the background vehicles in the generated test scene are different, and since the driver models in the background vehicles are different when training each driver model, the test scene finally generated has high complexity.

[0091] The present invention utilizes a reinforcement learning method to train a driver model, so that the trained driver model has high autonomy, interactivity, and evolvability, thereby making the behavior of background vehicles in the generated test scene have high uncertainty, which can better reflect the performance of the tested autonomous driving system and improve the test effect.

[0092] When training each target driver model, the present invention first uses the safe driver model as the background car to train a prototype of the target driver model. The prototype of the target driver model means that the currently obtained driver model preliminarily has the behavioral characteristics of the target driver, but the control of the behavior is not accurate enough. Then, the trained driver model and the trained target driver model prototype are used to replace the safe driver model to be trained as the background car, so that the target driver model can be trained faster and more accurately.

[0093] The driver model used for the background vehicle in the test scene generated by the present invention can be selected from the trained personalized driver models as needed, so that the test scene has strong generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0095] Figure 2 It is a schematic diagram of the application process of the embodiment;

[0096] Figure 3 Schematic diagram of the driver model training process at different growth stages in the embodiment;

[0097] Figure 4 Schematic diagram of the personalized driver model training process in the embodiment;

[0098] Figure 5 Generate a schematic diagram for a challenging test scenario in an embodiment;

[0099] Figure 6 The figure is a flow chart of applying the method of the present invention to test the automatic driving system in an embodiment. DETAILED DESCRIPTION

[0100] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0101] Example

[0102] like Figure 1 As shown, a method for generating an autonomous driving test scenario based on a personalized driver model comprises the following steps:

[0103] S1. Based on genetic and evolutionary ideas and combined with reinforcement learning, the various stages of human drivers' driving experience are simulated, and driver models of different growth stages are obtained through step-by-step training. The driver models of different growth stages include novice driver models, experienced driver models and skilled driver models.

[0104] S2. Based on driver models at different growth stages, and according to different personalized driving characteristics of human drivers, further training to obtain personalized driver models, wherein the personalized driver models include an aggressive driver model, a conservative driver model, a provocative driver model, and a cooperative driver model;

[0105] S3. According to the test requirements, driver models at different growth stages and different personalized driver models are combined in accordance with corresponding demand ratios to generate corresponding target test scenarios.

[0106] In the early stage of driver model training, a safe driver model needs to be designed as a background vehicle for training. The strategy of the safe driver model proposed in the present invention is:

[0107] when When the vehicle takes the action of "sudden deceleration",

[0108] when When , the vehicle takes the action of "deceleration";

[0109] when When , the vehicle takes the action of "deceleration";

[0110] Among them, d rel_front represents the relative distance between the vehicle and the preceding vehicle in the same lane, v rel_front represents the relative speed between the vehicle and the preceding vehicle in the same lane, v ego Indicates the vehicle speed.

[0111] In order to effectively improve the efficiency and accuracy of model training, the present invention first uses the safe driver model as the background car to train a prototype of the target driver model when training each target driver model (the target driver model prototype here refers to the driver model currently obtained that preliminarily has the behavioral characteristics of the target driver, but the control of the behavior is not accurate enough), and then uses the trained driver model (there is no trained driver model when training a novice driver model) and the trained target driver model prototype to replace the safe driver model as the background car for training. Compared with always using the safe driver model as the background car for training or always using the trained driver model as the background car (except when training a novice driver model), this method can obtain the target driver model faster.

[0112] In addition, the present invention also proposes a responsibility determination mechanism when constructing a reinforcement learning reward function for driver model training: when a vehicle collision occurs, if neither vehicle changes lanes, the front vehicle has no traffic accident responsibility, and the rear vehicle bears all traffic accident responsibility; if one vehicle changes lanes, the vehicle that changes lanes bears full responsibility; if both vehicles change lanes, the two vehicles share the responsibility equally.

[0113] The present invention designs driver models at different growth stages into novice drivers, experienced drivers with certain experience, and skilled drivers based on different considerations of driving experience. Among them, novice drivers are insufficient in driving experience, do not change lanes subjectively, pay attention to driving safety, and moderately pursue traffic efficiency; experienced drivers with certain experience pay attention to safety, change lanes only when necessary, and moderately pursue ride comfort; skilled drivers pay attention to driving safety, ride comfort and traffic efficiency, and can skillfully perform various actions such as lane changes.

[0114] Specifically, the driver model interacts with traffic status information by selecting discrete actions to train a decision network that meets the requirements. The action space of the vehicle is defined according to the lateral and longitudinal movements of the vehicle. The action space of the novice driver model (only longitudinal actions) includes: "longitudinal uniform speed", "longitudinal acceleration", "longitudinal rapid acceleration", "longitudinal deceleration", and "longitudinal rapid deceleration";

[0115] The driver model with certain experience, in addition to inheriting the above longitudinal actions, combines "lateral uniform speed" with the above longitudinal actions to evolve new lane-changing actions, including "longitudinal uniform speed and lateral uniform speed", "longitudinal acceleration and lateral uniform speed", "longitudinal rapid acceleration and lateral uniform speed", "longitudinal deceleration and lateral uniform speed" and "longitudinal rapid deceleration and lateral uniform speed".

[0116] In addition to inheriting the above-mentioned longitudinal movements and lane-changing movements, the skilled driver model also combines "lateral acceleration" and "lateral deceleration" with the above-mentioned longitudinal movements to evolve new lane-changing movements, specifically including "longitudinal uniform lateral deceleration", "longitudinal acceleration lateral deceleration", "longitudinal rapid acceleration lateral deceleration", "longitudinal deceleration lateral deceleration", "longitudinal rapid deceleration lateral deceleration", "longitudinal uniform lateral acceleration", "longitudinal acceleration lateral acceleration", "longitudinal rapid acceleration lateral acceleration", "longitudinal deceleration lateral acceleration" and "longitudinal rapid deceleration lateral acceleration".

[0117] In addition to the above action set, the novice driver model, the experienced driver and the skilled driver also have a consistent state set, which is specifically based on the longitudinal relative distance d between the vehicle and the other vehicle. rel (d rel Included rel_front d rel_crear d rel_left_front d rel_left_rear d rel_rigth_front and d rel_right_rear , which respectively represent the relative distance between the vehicle and the vehicle in front of it in the same lane, the relative distance between the vehicle and the vehicle behind it in the same lane, the relative distance between the vehicle and the vehicle in front of it in the left lane, the relative distance between the vehicle and the vehicle behind it in the left lane, the relative distance between the vehicle and the vehicle in front of it in the right lane, and the relative distance between the vehicle and the vehicle behind it in the right lane), longitudinal relative speed v rel (v rel Include v rel_front 、v rel_crear 、v rel_left_front 、v rel_left_rear 、v rel_rigth_front and v rel_right_rear , sign rules and d rel Partially similar), vehicle speed v egoand lane information, where when another vehicle (including a vehicle in a different lane from the vehicle) is in front of the vehicle, the longitudinal speed of the other vehicle is recorded as v f , when the other car is behind your car, the longitudinal speed of the other car is v r , the longitudinal relative velocity is defined as follows:

[0118]

[0119] Define the lane number line_id according to the lane you are in;

[0120] Calculated based on the speed of the vehicle and the relative distance to the vehicle ahead And divide it into three states:

[0121]

[0122] Calculated based on the vehicle speed and relative speed And divide it into three states:

[0123]

[0124] The vehicle's deviation from the lane centerline is quantified as "left" (the vehicle is on the left side of the lane centerline), "center" (the vehicle is on the lane centerline), and "right" (the vehicle is on the right side of the lane centerline).

[0125] The present invention proposes a personalized driver model based on consideration of different driving styles. Specifically, according to whether the driver of the own vehicle considers the impact of the behavior of the own vehicle on surrounding vehicles, the driving style is divided into a unimodal driving style and a multimodal driving style. The so-called unimodal driving style means that the driver of the own vehicle only considers the impact of the behavior of the own vehicle on himself, and does not consider the impact of the behavior of the own vehicle on other vehicles, including an aggressive type and a conservative type; the so-called multimodal driving style means that the driver of the own vehicle will consider the impact of the behavior of the own vehicle on other vehicles during driving, including a cooperative type and a provocative type. The cooperative type means that the driver of the own vehicle will cooperate with the behavior of other vehicles during driving, and try to create a good driving environment for other vehicles to maximize the benefits of other vehicles; the provocative type means that the driver of the own vehicle likes to hinder surrounding vehicles during driving, so that other vehicles fall into a bad driving environment and prevent the benefits of other vehicles from being maximized.

[0126] The present invention designs a reward function according to the driving motivation and driving experience corresponding to the target driver type, wherein the driving motivation reflects the goal pursued by the driver in driving, and the specific goals include safety, traffic efficiency, ride comfort and driving experience. Ride comfort refers to keeping the vibration and impact environment generated by the car during driving within a certain limit on the comfort of the occupants, which is mainly evaluated by the subjective comfort feeling of the driver and passengers. The driving experience focuses on the subjective feeling of the driver. The driving experience pursued by drivers of different styles is quite different. The reward functions corresponding to different driver models are as follows:

[0127] Reward function for training novice drivers: Novice drivers have insufficient driving experience, do not change lanes subjectively, pay great attention to driving safety, and moderately pursue traffic efficiency. The emphasis on safety is reflected in avoiding responsible accidents and keeping a long following distance. Based on this, the reward function for training novice drivers is designed as follows:

[0128] R 1 =ω s R safety +ω e R efficiency

[0129] Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor, R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance, R efficiency is the reward function that takes into account the traffic efficiency;

[0130] Training an experienced driver model: Experienced drivers still focus on safety, change lanes only when necessary, and moderately pursue comfort. In order to improve traffic efficiency, they also pay attention to increasing the speed. Based on this, the reward function is designed as follows:

[0131] R 2 =ω s R safety +ω e R efficiency +ω r R riding_comfort

[0132] Among them, ω s is the weight of the safety factor, R e is the weight of the traffic efficiency factor, ω r is the weight of the riding comfort factor, R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism, following distance and driving experience, R efficiencyis the reward function considering traffic efficiency, R riding_comfort is the reward function that takes riding comfort into consideration;

[0133] Training an experienced driver model: An experienced driver focuses on driving safety, comfort, and traffic efficiency, and is proficient in performing various actions such as lane changes. Based on this, the reward function is designed as follows:

[0134] R 3 =ω s R safety +ω e R efficiency +ω r R riding_comfort

[0135] Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor, ω r is the weight of the riding comfort factor, R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism, the following distance, R efficiency is the reward function considering traffic efficiency, R riding_comfort is the reward function that takes riding comfort into consideration;

[0136] Training an aggressive driver model: In addition to having skilled driving skills, aggressive drivers like the driving experience brought by high speeds and speed changes, and do not want to stay too long in congested sections. Based on this, the reward function is designed as follows:

[0137] R 4 =ω s R safety +ω d R driving_feeling +ω e R efficiency

[0138] Among them, ω s is the weight of the safety factor, ω d is the weight of driving experience, ω e is the weight of traffic efficiency, R safety In order to consider the safety of the reward function, the design of this part of the reward function needs to consider the responsibility judgment mechanism, R driving_feeling In order to consider the reward function of driving experience, the design of this part of the reward function needs to consider the driving experience related to traffic flow density and vehicle speed within a certain range of the vehicle, R efficiency is the reward function that takes into account the traffic efficiency;

[0139] Training a conservative driver model: Conservative drivers focus on safety and comfort during driving, avoid frequent speed and lane changes, and keep a long following distance. The reward function is set as follows:

[0140] R 5 =ω s R safety +ω r R riding_comfort +ω e R efficiency

[0141] Among them, ω s is the weight of the safety factor, ω r is the weight of the riding comfort factor, ω e is the weight of traffic efficiency, R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance, R riding_comfort is the reward function considering riding comfort, R efficiency is the reward function that takes into account the traffic efficiency;

[0142] Training the aggressive driver model: In addition to some characteristics of skilled drivers, the most obvious characteristic of aggressive drivers is that they like to hinder the driving of surrounding vehicles, especially those behind them, and gain driving experience from them. Based on this, the reward function is designed as follows:

[0143] R 6 =ω s R safety +ω d R driving_feeling +ω e R efficiency

[0144] Among them, ω s is the safety weight, ω d is the weight of driving experience, ω e is the weight of traffic efficiency, R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance, R driving_feeling is the reward function considering the driving experience, R efficiency is the reward function that takes into account the traffic efficiency;

[0145] Training cooperative driver model: In addition to some characteristics of skilled drivers, the most obvious characteristic of cooperative drivers is that they will try their best to create a good driving environment for their car and gain driving experience from it. Based on this, the reward function is designed as follows:

[0146] R 7 =ω sR safety +ω d R driving_feeling +ω e R efficiency

[0147] Among them, ω s is the safety weight, ω d is the weight of driving experience, ω e is the weight of traffic efficiency, R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance, R driving_feeling is the reward function considering the driving experience, R efficiency is the reward function that takes into account the traffic efficiency.

[0148] This embodiment applies the above technical solution, and its specific application process is as follows Figure 2 As shown: Reinforcement learning is combined with genetic and evolutionary ideas to first generate a novice driver model, then gradually generate a driver model with certain experience and a skilled driver model, and then give humans personalized driving characteristics based on the skilled driver model to make it a personalized driver model. Finally, the generated driver models are combined in a certain proportion according to demand to generate corresponding test scenarios.

[0149] Figure 3 The figure is a schematic diagram of the process of training and generating driver models at different growth stages in the embodiment, including:

[0150] (1) Training a novice driver model (this driver model can only perform longitudinal control): Before training a novice driver, it is necessary to define a safe driver model, and then define the action space and state set of the novice driver model. The specific action space includes "longitudinal uniform speed", "longitudinal acceleration", "longitudinal rapid acceleration", "longitudinal deceleration", and "longitudinal rapid deceleration";

[0151] The specific state set depends on the longitudinal relative distance between the vehicle and other vehicles (d rel , d rel Can be replaced by d rel_front d rel_rear d rel_left_front d rel_left_rear d rel_rigth_front and d rel_right_rear , which respectively represent the relative distance between the vehicle and the vehicle in front of it in the same lane, the relative distance between the vehicle and the vehicle behind it in the same lane, the relative distance between the vehicle and the vehicle in front of it in the left lane, the relative distance between the vehicle and the vehicle behind it in the left lane, the relative distance between the vehicle and the vehicle in front of it in the right lane, and the relative distance between the vehicle and the vehicle behind it in the right lane), longitudinal relative speed (v rel ,v relThe replacement rule is the same as d rel Same), vehicle speed (v ego ) and lane information to define the state set, where when the other vehicle (including the vehicle in a different lane from the vehicle) is in front of the vehicle, the longitudinal speed of the other vehicle is recorded as v f ; When the other car is behind your car, the longitudinal speed of the other car is v r ; The longitudinal relative velocity is defined as follows:

[0152]

[0153] Define the lane number line_id according to the lane where the vehicle is located, and calculate it according to the speed of the vehicle and the relative distance to the vehicle in front. It is divided into three states: and Calculated based on the vehicle speed and relative speed It is divided into three states, including and

[0154] The vehicle's deviation from the lane centerline is quantified as "left" (the vehicle is on the left side of the lane centerline), "centered" (the vehicle is on the lane centerline), and "right" (the vehicle is on the right side of the lane centerline).

[0155] When training a novice driver model, we must first establish a vehicle motion model over time, select an appropriate time step, and initialize the parameters (the total number of simulated vehicles n c ; Maximum initialization distance allowed Simulation duration t f ), and then generate a simulation environment; then use the safe driver as the background car for training to obtain a prototype of a novice driver model with insufficient control, and then use the prototype of the novice driver model to replace the safe driver model as the background car for training, and combine the reward function corresponding to the novice driver model to obtain the novice driver model. The novice driver model obtains environmental information by interacting with the environment. The novice driver model obtains the state quantity s at the current time t t After that, select a state variable s t The most valuable action During specific training, the novice driver model is in s t Under the state, according to Select the action to be performed t Afterwards, you get a reward r t+1 , and enter the next state s t+1 , according to the corresponding and rt+1 To update Q(s t ,a t ), through continuous iteration, the novice driver model can eventually make the optimal action corresponding to the current environmental state.

[0156] (2) Training an experienced driver model with a certain amount of experience (this driver model can perform longitudinal control skillfully but will change lanes cautiously): Before training the experienced driver model, let this driver model inherit the action space of the novice driver model and expand it, including "longitudinal uniform speed", "longitudinal acceleration", "longitudinal rapid acceleration", "longitudinal deceleration", "longitudinal rapid deceleration", "longitudinal uniform speed and lateral uniform speed", "longitudinal acceleration and lateral uniform speed", "longitudinal rapid acceleration and lateral uniform speed", "longitudinal deceleration and lateral uniform speed" and "longitudinal rapid deceleration and lateral uniform speed"; the state set of this model is the same as that of the novice driver model.

[0157] When training a driver model with certain experience, first use the safe driver model as the background car for training to obtain a prototype of a driver model with certain experience whose control is not fine enough. Then, use the trained novice driver model and the prototype of the driver model with certain experience to replace the safe driver model as the background car for training in equal proportions. Combined with the reward function corresponding to the experienced driver model, a driver model with certain experience is obtained.

[0158] (3) Training an experienced driver model (the driver model can perform longitudinal control and lane change proficiently): Before training the experienced driver model, let the driver model inherit the action space of the experienced driver model and expand it, including "longitudinal uniform speed", "longitudinal acceleration", "longitudinal rapid acceleration", "longitudinal deceleration", "longitudinal rapid deceleration", "longitudinal uniform speed and lateral uniform speed", "longitudinal acceleration and lateral uniform speed", "longitudinal rapid acceleration and lateral uniform speed", "longitudinal deceleration and lateral uniform speed", "longitudinal rapid deceleration and lateral uniform speed", "longitudinal uniform speed and lateral deceleration", "longitudinal acceleration and lateral deceleration", "longitudinal rapid acceleration and lateral deceleration", "longitudinal deceleration and lateral deceleration", "longitudinal rapid deceleration and lateral deceleration", "longitudinal uniform speed and lateral acceleration", "longitudinal acceleration and lateral acceleration", "longitudinal rapid acceleration and lateral acceleration", "longitudinal deceleration and lateral acceleration" and "longitudinal rapid deceleration and lateral acceleration"; the state set of this model is the same as that of the novice driver model.

[0159] When training a skilled driver model, first use the safe driver model as the background car for training to obtain a prototype of a skilled driver model with less precise control. Then, use the trained novice driver model, the driver model with certain experience and the prototype of the skilled driver model in equal proportion to replace the safe driver model as the background car for training. Combined with the reward function corresponding to the skilled driver model, a skilled driver model is generated.

[0160] Figure 4 The figure shows a schematic diagram of the training process of a personalized driver model. This embodiment trains two personalized driver models, namely, radical and provocative, based on genetic and evolutionary ideas:

[0161] (1) Training the aggressive driver model: Before training the aggressive driver model, let the driver model inherit the action space of the skilled driver model, and the state set of the model is the same as that of the novice driver model. When training the aggressive driver model, first use the safe driver model as the background car for training to obtain a prototype of the aggressive driver model with less precise control. Then, use the trained novice driver model, the driver model with certain experience, and the skilled driver model in equal proportion to replace the safe driver model as the background car for training. Combined with the reward function corresponding to the aggressive driver model, the aggressive driver model is generated.

[0162] (2) Training the provocative driver model: In addition to some characteristics of skilled drivers, the most obvious characteristic of provocative drivers is that they like to hinder the movement of surrounding vehicles, especially those behind them, and gain driving experience from them. Before training the provocative driver model, let the driver model inherit the action space of the skilled driver model, and the state set of the model is the same as that of the novice driver model. When training the provocative driver model, first use the safe driver model as the background car for training to obtain a prototype of the provocative driver model with less precise control. Then, use the trained novice driver model, the driver model with certain experience, and the skilled driver model in equal proportion to replace the safe driver model as the background car for training. Combined with the reward function corresponding to the provocative driver model, the provocative driver model is generated.

[0163] Finally, the trained driver models are used to generate corresponding test scenarios by combining different driver models in proportion according to the test requirements. This embodiment uses the trained driver models to generate a challenging test scenario to test the safety, comfort and traffic efficiency of the tested autonomous driving system. Figure 5 As shown in the figure, the test scenario is generated with the aggressive driver model and the provocative driver model in the background car accounting for 50% each. In actual applications, if you want to generate a complex test scenario, you can set the novice driver model, the experienced driver model, the skilled driver model, the cooperative driver model, the provocative driver model, and the conservative driver model in the background car to account for 14% each, and the aggressive driver model to account for 16%.

[0164] This embodiment also tests the autonomous driving system based on the generated challenging test scenarios, such as Figure 6 As shown:

[0165] Step 1: Initialize the road information in the simulation environment, including the number of lanes n line = 3, lane width d = 3.5m and lane length l = 400m;

[0166] Step 2: Determine the number of background vehicles n in the environment car =16, determine the initial speed range of the background vehicles to be [70,120] km / h, assign the driving strategy of the provocative driver model to 50% of the background vehicles, and assign the driving strategy of the aggressive driver model to the other 50% of the background vehicles;

[0167] Step 3: Determine the initial velocity v of the target vehicle 0 and the initial lateral position x 0 , initialize the number of loops n loop = 1. The target vehicle is equipped with the tested autonomous driving system. The target vehicle is randomly assigned to the center of a lane according to a random number, and the initial longitudinal speed of the target is randomly given according to a uniform distribution in the range of [70,120];

[0168] Step 4: Place the target vehicle. According to the initial velocity v of the target vehicle 0 and initial position (x 0 , 0) place the target vehicle in the test scene, where 0 means the initial longitudinal position of the target vehicle is at 0m;

[0169] Step 5: Determine the initial velocity and initial position of the background vehicle. The background vehicle is randomly assigned to a lane according to a random number. The longitudinal position of the vehicle in the designated lane is randomly determined according to a uniform distribution in [0,400]. The lateral position of the vehicle is at the center line of the lane. The initial longitudinal velocity of the car is randomly given according to a uniform distribution in the range of [70,120] to form a test scenario.

[0170] Step 6: Test and record relevant parameters and collision times of the target vehicle during driving;

[0171] Step 7: When the target vehicle reaches 400m or collides, save the lateral position x of the target vehicle 1 and speed v 1 ;

[0172] Step 8: Determine n loop >100, if not, let x 0 =x 1 , v 0 =v 1 , then go to step 3; if it is greater than 100, the test ends and evaluation is carried out.

[0173] Step 9: Evaluate the safety of the autonomous driving system under test based on the number of collisions of the target vehicle during the test, evaluate the traffic efficiency of the autonomous driving system under test based on the average speed of the target vehicle during the test, and evaluate the comfort of the autonomous driving system under test based on the average absolute value of the acceleration of the target vehicle during the test.

[0174] In summary, this embodiment first generates a novice driver model, then generates a driver model with certain experience and a skilled driver model step by step, and then gives human personalized driving characteristics on the basis of the skilled driver model to make it a personalized driver model, and finally combines the generated driver models in a certain proportion according to demand to generate corresponding test scenarios. Taking the generation of a challenging test scenario as an example, the generated aggressive driver model and provocative driver model are combined in equal proportions as background vehicles to simulate a challenging traffic flow, and the autonomous driving system to be tested is placed in the traffic flow. The operating efficiency (travel time, average speed), comfort (average of the absolute value of acceleration) and safety (number of collisions) of the system in the scenario are tested to evaluate the autonomous driving system to be tested.

[0175] Compared with other autonomous driving test scene generation methods, the driver model trained by this technical solution has intelligence and autonomous interaction capabilities. The personalized driver model significantly improves the test effect of the scene. At the same time, due to its unique training process, the final generated test scene has good authenticity, complexity and robustness, and is highly scalable and generalizable.

Claims

1. A method for generating an autonomous driving test scenario based on a personalized driver model, characterized in that: The following steps are involved: S1. Based on genetic and evolutionary ideas and combined with reinforcement learning, the various stages of human drivers' driving experience are simulated, and driver models of different growth stages are obtained through step-by-step training. The driver models of different growth stages include novice driver models, experienced driver models and skilled driver models. S2. Based on driver models at different growth stages, and according to different personalized driving characteristics of human drivers, further training to obtain personalized driver models, wherein the personalized driver models include an aggressive driver model, a conservative driver model, a provocative driver model, and a cooperative driver model; S3. According to the test requirements, driver models at different growth stages and different personalized driver models are combined according to the corresponding demand ratios to generate corresponding target test scenarios; The step S1 specifically includes the following steps: S11. Define and build a safe driver model; S12. Based on the safe driver model, based on genetic and evolutionary ideas and combined with reinforcement learning, a novice driver model, an experienced driver model and a skilled driver model are obtained through step-by-step training; The driving strategy of the safety driver model is specifically as follows: when When the vehicle takes the action of "sudden deceleration"; when When , the vehicle takes the action of "deceleration"; when When , the vehicle takes the action of "deceleration"; Among them, d rel_front is the relative distance between the vehicle and the preceding vehicle in the same lane, v rel_front is the relative speed between the vehicle and the preceding vehicle in the same lane, v ego is the vehicle speed; The step S2 specifically includes the following steps: S21. Determine the category of the personalized driver model to be trained, so that the personalized driver model inherits the action space of the skilled driver model, and the state set of the personalized driver type is consistent with the state set of the novice driver model, wherein the personalized driver model specifically divides the driving style into a unimodal driving style and a multimodal driving style according to whether the driver of the vehicle considers the impact of the vehicle's behavior on surrounding vehicles. The unimodal driving style means that the driver of the vehicle only considers the impact of the vehicle's behavior on himself, and does not consider the impact of the vehicle's behavior on other vehicles, including an aggressive type and a conservative type; the multimodal driving style means that the driver of the vehicle considers the impact of the vehicle's behavior on other vehicles during driving, including a cooperative type and a provocative type. The cooperative driver model means that the vehicle will cooperate with other vehicles during driving, and try to create a good driving environment for other vehicles to maximize the benefits of the vehicle; the provocative driver model means that the vehicle likes to hinder surrounding vehicles during driving, so that the vehicle falls into a bad driving environment and prevents the vehicle from maximizing its benefits; S22, using the safe driver model as a background vehicle for training to obtain a prototype of a personalized driver model, and then replacing the safe driver model with a novice driver model, an experienced driver model, and a skilled driver model in equal proportions as the background vehicle for training to generate a corresponding personalized driver model.

2. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 1, characterized in that: The action space of the novice driver model only has longitudinal actions, including five actions: longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration and longitudinal rapid deceleration; The novice driver model is based on the longitudinal relative distance d between the vehicle and the other vehicle. rel , longitudinal relative velocity v rel , vehicle speed v ego and lane information to define the state set. The other vehicle includes the vehicle in a different lane from the vehicle. When the other vehicle is in front of the vehicle, the longitudinal speed of the other vehicle is recorded as v f , when the other car is behind your car, the longitudinal speed of the other car is v r , then the longitudinal relative velocity is defined as follows: Among them, d rel Included rel_front ,d rel_crear ,d rel_left_front ,d rel_left_rear ,d rel_rigth_front and d rel_right_rear , which are the relative distance between the vehicle and the vehicle in front of it in the same lane, the relative distance between the vehicle and the vehicle behind it in the same lane, the relative distance between the vehicle and the vehicle in front of it in the left lane, the relative distance between the vehicle and the vehicle behind it in the left lane, the relative distance between the vehicle and the vehicle in front of it in the right lane, and the relative distance between the vehicle and the vehicle behind it in the right lane; v rel Include v rel_front 、v rel_crear 、v rel_left_front 、v rel_left_rear 、v rel_rigth_front and v rel_right_rear , which are the relative speed between the vehicle and the vehicle in front of it in the same lane, the relative speed between the vehicle and the vehicle behind it in the same lane, the relative speed between the vehicle and the vehicle in front of it in the left lane, the relative speed between the vehicle and the vehicle behind it in the left lane, the relative speed between the vehicle and the vehicle in front of it in the right lane, and the relative speed between the vehicle and the vehicle behind it in the right lane; The novice driver model defines the lane number line_id according to the lane in which the vehicle is located, and calculates the lane number line_id according to the speed of the vehicle and the relative distance between the vehicle in front. It is divided into three states, including: Calculated based on the vehicle speed and relative speed It is divided into three states, including: In the novice driver model, the offset of the vehicle in the lane relative to the lane centerline is quantified as: "Left bias" means the vehicle is on the left side of the lane centerline; "Centered" means the vehicle is located on the center line of the lane; "Off to the right" means that the vehicle is on the right side of the center line of the lane.

3. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 2, characterized in that: The specific process of training the novice driver model in step S12 is: First, establish a vehicle motion model over time, select a suitable time step, initialize parameters, and generate a simulation environment. The initialization parameters include the total number of simulated vehicles n c , Maximum initialization distance allowed And the simulation time t f ; Then, the safe driver model is used as the background car for training to obtain the prototype of the novice driver model; Then, the prototype of the novice driver model is used to replace the safe driver model as the background vehicle for training, thereby obtaining a novice driver model; The reward function in the novice driver model training process is specifically: R1=ω s R safety +ω e R efficiency Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor; R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance. The specific responsibility judgment mechanism is: when a vehicle collision occurs, if both vehicles do not change lanes, the front vehicle has no traffic accident responsibility, and the rear vehicle bears all traffic accident responsibility; if one vehicle changes lanes, the vehicle with lane change behavior bears full responsibility; if both vehicles change lanes, the two vehicles share the responsibility; R efficiency is the reward function that takes into account the traffic efficiency; During the training process, the novice driver model obtains environmental information by interacting with the environment. The novice driver model obtains the state quantity s at the current time t. t After that, select a state variable s t The most valuable action During specific training, the novice driver model is in s t Under the state, according to Select the action to be performed t Afterwards, you get a reward r t+1 , and enter the next state s t+1 , according to the corresponding and r t+1 To update Q(s t , a t ), through continuous iterative training, the novice driver model can eventually make the optimal action corresponding to the current environmental state.

4. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 2, characterized in that: The action space of the experienced driver model inherits the five actions of the novice driver model, and combines the lateral uniform speed with the five longitudinal actions to evolve five new lane change actions, including longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration, longitudinal rapid deceleration, longitudinal uniform speed and lateral uniform speed, longitudinal acceleration and lateral uniform speed, longitudinal rapid acceleration and lateral uniform speed, longitudinal deceleration and lateral uniform speed, and longitudinal rapid deceleration and lateral uniform speed, a total of ten actions; The state set of the experienced driver model is consistent with the state set of the novice driver model; The specific process of training and verifying the driver model in step S12 is as follows: Firstly, the safe driver model is used as the background vehicle for training to obtain the prototype of the experienced driver model; Then, the trained novice driver model and the prototype of the experienced driver model are used in equal proportion to replace the safe driver model as the background vehicle for training, thereby obtaining the experienced driver model; The reward function in the training process of the empirical driver model is specifically: R2=ω s R safety +oh e R efficiency +oh r R riding_comfort Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor, ω r is the weight of the riding comfort factor; R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism, following distance and driving experience; R efficiency is the reward function that takes into account the traffic efficiency; R riding_comfort is the reward function that takes riding comfort into consideration.

5. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 2, characterized in that: The action space of the skilled driver model inherits ten actions of the experienced driver model, and combines lateral acceleration and lateral deceleration with five longitudinal actions to evolve ten new lane changing actions, including: longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration, longitudinal rapid deceleration, longitudinal uniform speed and lateral uniform speed, longitudinal acceleration and lateral uniform speed, longitudinal rapid acceleration and lateral uniform speed, longitudinal deceleration and lateral uniform speed, longitudinal rapid deceleration and lateral uniform speed, longitudinal uniform speed and lateral deceleration, longitudinal acceleration and lateral deceleration, longitudinal rapid acceleration and lateral deceleration, longitudinal deceleration and lateral deceleration, longitudinal rapid deceleration and lateral deceleration, longitudinal uniform speed and lateral acceleration, longitudinal acceleration and lateral acceleration, longitudinal rapid acceleration and lateral acceleration, longitudinal deceleration and lateral acceleration, and longitudinal rapid deceleration and lateral acceleration, a total of twenty actions; The state set of the skilled driver model is consistent with the state set of the novice driver model; The specific process of training the skilled driver model in step S12 is as follows: First, use the safe driver model as the background car for training to obtain the prototype of the skilled driver model; Then, the trained novice driver model, the experienced driver model and the prototype of the skilled driver model are used in equal proportions to replace the safe driver model as the background vehicle for training to generate the skilled driver model; The reward function in the skilled driver model training process is specifically: R3=ω s R safety +oh e R efficiency +oh r R riding_comfort Among them, ω s is the weight of the safety factor, ω e is the weight of the traffic efficiency factor, ω r is the weight of the riding comfort factor; R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance; R efficiency is the reward function that takes into account the traffic efficiency; R riding_comfort is the reward function that takes riding comfort into consideration.

6. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 1, characterized in that: During the training process of the personalized driver model, the reward function of the aggressive driver model is specifically: R4=ω s R safety +oh d R driving_feeling +oh e R efficiency Among them, ω s is the weight of the safety factor, ω d is the weight of driving experience, ω e is the weight of traffic efficiency; R safety In order to consider the safety of the reward function, the design of this part of the reward function needs to consider the responsibility judgment mechanism; R driving_feeling In order to consider the reward function of driving experience, the design of this part of the reward function needs to consider the driving experience related to traffic flow density and vehicle speed within a certain range of the vehicle; R efficiency is the reward function that takes into account the traffic efficiency; The reward function of the conservative driver model is as follows: R5=ω s R safety +oh r R riding_comfort +oh e R efficiency Among them, ω s is the weight of the safety factor, ω r is the weight of the riding comfort factor, ω e is the weight of traffic efficiency; R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance; R riding_comfort is the reward function that takes riding comfort into consideration; R efficiency is the reward function that takes into account the traffic efficiency.

7. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 1, characterized in that: During the training process of the personalized driver model, the reward function of the provocative driver model is specifically: R6=ω s R safety +oh d R driving_feeling +oh e R efficiency Among them, ω s is the safety weight, ω d is the weight of driving experience, ω e is the weight of traffic efficiency; R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance; R driving_feeling is the reward function that takes driving experience into account; R efficiency is the reward function that takes into account the traffic efficiency; The reward function of the cooperative driver model is as follows: R7=ω s R safety +oh d R driving_feeling +oh e R efficiency Among them, ω s is the safety weight, ω d is the weight of driving experience, ω e is the weight of traffic efficiency; R safety In order to consider the reward function of safety, the design of this part of the reward function needs to consider the responsibility judgment mechanism and the following distance; R driving_feeling is the reward function that takes driving experience into account; R efficiency is the reward function that takes into account the traffic efficiency.

Images