Autonomous driving test scenario generation method based on personalized driver model

Abstract

The present invention relates to a method for generating autonomous driving test scenarios based on a personalized driver model, comprising: if the interactive object being tested is an autonomous driving system, obtaining natural driving data on Chinese highways and their ramps based on an existing natural driving dataset; if the interactive object being tested is known to be an autonomous driving system, collecting driving data on the driver's interaction with the autonomous driving system by combining a questionnaire with a driving simulator; training driver agent models of different styles based on the acquired driving data using a maximum entropy inverse reinforcement learning method and a deep reinforcement learning method; and combining driver models of different styles according to corresponding demand ratios to generate corresponding target test scenarios according to test requirements. Compared to the prior art, the present invention uses human driving data as the data source and can effectively improve the authenticity and complexity of test scenarios by constructing personalized driver models with different styles.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method for generating autonomous driving test scenarios based on a personalized driver model. Background Art

[0002] With the rise of deep learning and computer vision technologies, autonomous driving offers new solutions for improving traffic safety and efficiency. Future autonomous vehicles will possess comprehensive driving perception systems capable of identifying external objects and people, including traffic signs, cars, pedestrians, and road conditions. Intelligent decision-making systems rapidly process and analyze this perceived information, while execution systems precisely execute commands such as acceleration, braking, parking, and steering.

[0003] Autonomous driving vehicle testing is an important part of autonomous driving research and development, and an important support for the development of autonomous driving technology. Scenarios are a very important part of the autonomous driving test system. The diversity, coverage, and typicality of test scenarios can affect the accuracy of test results, thereby ensuring the safety and quality of autonomous driving.

[0004] Scenario-based simulation testing offers significant technical advantages in terms of efficiency, cost, and safety. It is a crucial method for testing and verifying autonomous vehicles and has become a current research hotspot. However, existing simulation testing methods still have some flaws. For example, the behavior of the surrounding vehicles in the scenario is often defined by humans based on experience and cannot accurately simulate the decision-making style and level of actual human drivers. Furthermore, test scenarios generated based on natural driving data cannot capture the situations in which human drivers, curious about and competitive with autonomous vehicles, navigate the roads. This results in low reliability and accuracy in test results. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an autonomous driving test scenario generation method based on a personalized driver model, which can improve the authenticity and complexity of the test scenario and thus achieve better test results.

[0006] The purpose of the present invention can be achieved by the following technical solution: A method for generating an autonomous driving test scenario based on a personalized driver model, comprising the following steps:

[0007] S1. Obtain corresponding driving data based on whether the tested interactive object is known to be an autonomous driving system. If the tested interactive object is unknown to be an autonomous driving system, obtain naturalistic driving data on Chinese highways and their ramps based on existing naturalistic driving datasets.

[0008] If the tested interaction object is known to be an autonomous driving system, a questionnaire and a driving simulator are combined to collect driving data of the driver's interaction with the autonomous driving system.

[0009] S2. Based on the acquired driving data, use the maximum entropy inverse reinforcement learning method and deep reinforcement learning method to train driver agent models with different styles;

[0010] S3. Based on the test requirements, driver models of different styles are combined according to the corresponding demand ratio to generate corresponding target test scenarios.

[0011] Furthermore, in step S1, if it is unknown that the tested interaction object is an autonomous driving system, the driver's style is classified into three styles: conservative, normal, and aggressive after obtaining natural driving data.

[0012] Furthermore, in step S1, if it is known that the tested interactive object is an autonomous driving system, the specific process of obtaining the corresponding driving data is as follows:

[0013] Construct road scenarios for driving simulators based on the road structure and environment of natural driving datasets;

[0014] Through questionnaire screening, the human driver styles interacting with autonomous vehicles were classified into four styles: close competitors, close curious, long-distance curious, and non-curious. Competition refers to the behavior of each participant to maximize personal interests at the expense of the interests of others. During driving, close competitive drivers will actively compete with autonomous vehicles (AVs) for drivable space in the same time and space domain, maximizing their own interests while ensuring safety. For example, when a close competitive driver encounters an AV while driving, in order to maximize their own drivable space, including improving safety, speed, driving efficiency and other interests, they will actively take actions that sacrifice the interests of the AV, such as compressing the AV's driving space, forcing it to slow down, avoid it, and overtake it;

[0015] Curiosity means finding novelty and interest in things that one does not understand. During driving, curious drivers find the surrounding AVs novel because they are unfamiliar with them, and are curious about their driving behaviors, so they follow and observe them, or actively interact with them. For example, when an AV interacts with other vehicles (excluding the current curious driver), the curious driver will stay behind or beside the AV to observe; when the AV does not interact with other vehicles, the curious driver will actively merge with it and try to overtake it. The difference between close-range curious people and long-range curious people is reflected in the distance from the AV car. Compared with close-range curious people, long-range curious people are more worried about the reliability and safety of AV cars, so they always keep a safe distance from AV cars, while close-range curious people are willing to shorten the distance from AV cars to better observe their interactive behaviors.

[0016] Let three types of drivers (close competitors, close curious drivers, and long-distance curious drivers) drive on the generated road scenarios using a driving simulator and collect corresponding driving trajectory data;

[0017] The driving data corresponding to non-curious style drivers are directly obtained from the natural driving dataset.

[0018] Furthermore, the specific process of constructing the road scene of the driving simulator is as follows:

[0019] 1) Place the main vehicle, i.e., the virtual car driven by the human driver on the driving simulator, at a fixed distance directly behind and to the side of the autonomous vehicle, and randomly place some follower model cars around it;

[0020] 2) Generate the environment settings of the road scene based on the environment settings of the natural driving dataset, including but not limited to the vehicle density, vehicle distribution, and vehicle speed of the following model.

[0021] Furthermore, the step S2 specifically includes the following steps:

[0022] S21. Setting the eigenvalue function of the reward function according to the environment model and functional requirements;

[0023] S22. Calculate the characteristic expectations of the human driver in the driving scenario;

[0024] S23. Obtaining optimal reward functions of different styles through deep inverse reinforcement learning;

[0025] S24. Based on the optimal reward function, a stylized driver agent model is trained using deep reinforcement learning.

[0026] Furthermore, the characteristic value function in step S21 is specifically:

[0027] R=θ s f s +θ e f e +θ c f c +θ i f i +θ u_d f unique_d +θ u_v f unique_v

[0028] Among them, f s is the safety characteristic value, which is equal to the ttc (time to collision) from the main vehicle to the front vehicle;

[0029] f e is the efficiency characteristic value, which is equal to the main vehicle speed;

[0030] f c is the collision characteristic value, which is equal to a fixed value when a collision occurs and equal to 0 when no collision occurs;

[0031] f i is the interaction eigenvalue, which is equal to the absolute value of the deceleration when the following vehicle is decelerated;

[0032] f unique_d Equal to the distance between the host vehicle and the autonomous vehicle;

[0033] f unique_v Equal to the speed difference between the host vehicle and the autonomous vehicle;

[0034] θ s ,θ e ,θ c ,θ i ,θ u_d ,θ u_v f s 、f e 、f c 、f i 、f unique_d 、f unique_v The corresponding weight.

[0035] Furthermore, the step S23 specifically includes the following steps:

[0036] S231, randomly generate the weight θ of the reward function;

[0037] S232. Train an agent using deep reinforcement learning, run it in a corresponding driving scenario, and store n trajectories generated during training and after training into a trajectory library;

[0038] S233. Select the first m groups of trajectories with the largest reward function in the trajectory library, apply the maximum entropy inverse reinforcement algorithm to obtain the probability of each trajectory in the m groups of trajectories, and perform weighted average calculation on the obtained probability and the characteristic value of each trajectory to obtain the expected characteristic statistics of the generated trajectory.

[0039] S234, using the maximum likelihood function method to find the gradient of θ, and using the gradient ascent method to update θ;

[0040] S235. Repeat steps S232 to S234 until θ converges to the optimum, and obtain the optimal reward function based on the optimal weight θ.

[0041] Furthermore, the specific process of deep reinforcement learning in step S24 is:

[0042] S241, initialize the network parameters of the two agents, namely Main Net and Target Net;

[0043] S242, initializing the simulation environment and the state of the environment vehicle;

[0044] S243, Main Net interacts with the simulation environment and stores the information of each step in the experience area, where the information of each step includes reward, state space, and action information;

[0045] S244. When the experience area is full, a certain number of records are randomly taken out and a gradient descent algorithm is performed on the loss function with respect to the grid parameters of the MainNet.

[0046] S245, every N times of storage, copy the network parameters of the Main Net to the Target Net to update the TargetNet;

[0047] S246. Repeat steps S243 to S245 until the Target Net grid parameters converge.

[0048] Furthermore, in step S24, the environment vehicle is first initialized using IDM (intelligent driver model), and primary intelligent agents of various styles are trained and obtained; then, the primary intelligent agents trained in various styles are used as background vehicles to train and obtain advanced intelligent agents.

[0049] Furthermore, the set of action information when the Main Net interacts with the simulation environment is discrete, including the following actions: longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration, longitudinal rapid deceleration, lateral uniform speed, lateral deceleration, and lateral acceleration. Each action performed by the vehicle is composed of a combination of actions in two directions.

[0050] Compared to existing technologies, this paper proposes a solution based on maximum entropy inverse reinforcement learning (IRL) combined with deep reinforcement learning (DRL) to train driver models with different styles. Using natural driving trajectory data collected from questionnaires and driving simulators, and employing IRRL and DRL algorithms, we train driving agent models with different styles, generating corresponding test scenarios. The behavior of the surrounding vehicles in these test scenarios accurately simulates the decision-making style and performance of actual human drivers.

[0051] The present invention fully considers the fact that different drivers have different driving styles in reality, and trains intelligent agent models with multiple styles, so that the generated test scenarios have strong authenticity and complexity. When faced with whether the interactive object is an autonomous vehicle, human drivers will have different interactive reactions. In order to show this difference, the present invention designs two virtual and real data collection methods. When the unknown interactive object is an autonomous driving system, natural driving data of three interactive styles (conservative, normal, and aggressive) are collected; when the known interactive system is an autonomous driving system, human drivers with different characteristics (close competition, close curiosity, long-distance curiosity, and long-distance non-curious) are first summarized and classified through questionnaires, and then real human interaction data is collected from the simulation simulator. The above two methods are complementary, reflecting the completeness and sufficiency of the data, which enables the generated test scenarios to cover the situation where human drivers are curious and competitive about autonomous vehicles while driving on the road, thereby improving the credibility of the test results and the accuracy of the test.

[0052] In the test scenario generated by the present invention, the driving style of the background vehicle is different from its reaction to the autonomous vehicle, which makes the final generated test scenario highly complex. In addition, the driver model used for the background vehicle in the test scenario generated by the present invention can be selected from the trained personalized driver models as needed, making the test scenario highly generalizable.

[0053] The present invention not only uses the maximum entropy inverse reinforcement learning method to learn the expert driving trajectory, making the trained reward function more interpretable and solving the problem of quantitatively setting the stylized reward function, but also uses the deep reinforcement learning method to train the driver model, so that the trained driver model has higher autonomy, interactivity and evolution, thereby making the behavior of the background vehicle in the generated test scene have higher uncertainty, which can better reflect the performance of the autonomous driving system under test and improve the test effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Schematic diagram of the method flow of the present invention;

[0055] Figure 2 A driver style classification diagram during application of the embodiment;

[0056] Figure 3 Schematic diagram of the deep inverse reinforcement learning process in the embodiment;

[0057] Figure 4 This is a schematic diagram of the proportion of stylized intelligent agents when generating a test scene in the embodiment;

[0058] Figure 5 Generate a schematic diagram for a challenging test scenario in an embodiment. DETAILED DESCRIPTION

[0059] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Example

[0061] like Figure 1 As shown, a method for generating an autonomous driving test scenario based on a personalized driver model includes the following steps:

[0062] S1. From the perspective of whether the interactive object being tested is an autonomous driving system, driving data is obtained in two ways (e.g. Figure 2 shown), respectively:

[0063] S11. The unknown interaction object being tested is an autonomous driving system: natural driving data of Chinese highways and their ramps are obtained based on the existing natural driving dataset;

[0064] The system categorizes driver styles into conservative, average, and aggressive based on natural driving data. It also uses data such as average speed, number of lane changes, and the minimum time-to-travel distance to the vehicle ahead as the basis for categorizing a driver's style.

[0065] S12. It is known that the interactive object being tested is an autonomous driving system: Based on the questionnaire, the test subjects are classified into four types of drivers: close competitor, close curious, long-distance curious, and non-curious. Among them, the driving data of the first three types when facing the autonomous driving car is obtained in a simulated environment based on the driving simulator. The data of the non-curious is similar to the natural driving data, so there is no need to collect this type of data on the driving simulator.

[0066] In this embodiment, the process of executing step S12 includes:

[0067] 1) Generate road scenes for the driving simulator based on the road structure and environment in the natural driving dataset obtained by S11.

[0068] Specifically, the simulator acquisition hardware device uses the Logitech G29 steering wheel series, and the software is based on the simulation software VTD; the road structure in the natural driving dataset specifically refers to the road width, number of lanes, and the average speed and speed variance of the cars in the dataset.

[0069] Requirements for constructing driving scenarios: Place the main vehicle (a virtual car driven by a human driver on a simulator) 250 meters behind the autonomous vehicle (directly behind and to the side), and randomly place 0 to 3 following vehicle models around it; the environment settings for generating the scenario are based on the environment settings of the natural driving dataset, specifically including the vehicle density, vehicle distribution, and vehicle speed of the following vehicle models.

[0070] 2) The styles of human drivers interacting with autonomous vehicles are divided into close competitors, close curious, long-distance curious, and non-curious. These four different styles of drivers are screened through questionnaires; among them, competition refers to the behavior of each participant to maximize personal interests at the expense of the interests of others. During the driving process, close-competitive drivers will actively compete with autonomous vehicles (AVs) for drivable space in the same time and space domain, and maximize their own interests while ensuring safety. For example, when a close-competitive driver encounters an AV while driving, in order to maximize the drivable space for himself, including improving safety, speed, driving efficiency and other interests, he will take the initiative to compress the AV's driving space, force it to slow down, avoid it, and overtake it, and other actions that sacrifice the AV's interests;

[0071] Curiosity means finding novelty and interest in things that one does not understand. During driving, curious drivers find the surrounding AVs novel because they are unfamiliar with them, and are curious about their driving behaviors, so they follow and observe them, or actively interact with them. For example, when an AV interacts with other vehicles (excluding the current curious driver), the curious driver will stay behind or beside the AV to observe; when the AV does not interact with other vehicles, the curious driver will actively merge with it and try to overtake it. The difference between close-range curious people and long-range curious people is reflected in the distance from the AV car. Compared with close-range curious people, long-range curious people are more worried about the reliability and safety of AV cars, so they always keep a safe distance from AV cars, while close-range curious people are willing to shorten the distance from AV cars to better observe their interactive behaviors.

[0072] 3) Let drivers of different styles use the driving simulator to drive in the scene generated in step 1) and collect their driving trajectory data.

[0073] S2. Using maximum entropy inverse reinforcement learning and deep reinforcement learning methods, we train driver agent models of different styles, including a close competitor model, a close curious agent model, and a long-distance curious agent model based on driving data collected from a simulator, as well as a conservative driver model, a normal driver model, and an aggressive driver model based on natural driving data.

[0074] like Figure 3 As shown, the specific process includes the following:

[0075] S21. Set the eigenvalue function of the reward function according to the environment model and functional requirements. The eigenvalue function is set to:

[0076] R=θ s f s +θ e f e +θ c f c +θ i f i +θ u_d f unique_d +θ u_v f unique_v

[0077] Among them, θ x , x=s,e,c,i,u_d,u_v, represents the weights corresponding to different eigenvalues;

[0078] f s is the safety characteristic value, which is equal to the ttc of the distance between the main vehicle and the preceding vehicle;

[0079] f e is the efficiency characteristic value, which is equal to the host vehicle speed;

[0080] f c is the collision characteristic value, which is equal to a fixed value when a collision occurs and equal to 0 when no collision occurs;

[0081] f i is the interaction eigenvalue, which is equal to the absolute value of the deceleration when the main vehicle slows down the following vehicle;

[0082] f unique_d Equal to the distance between the host vehicle and the autonomous vehicle;

[0083] f unique_v Equal to the speed difference between the host vehicle and the autonomous vehicle.

[0084] S22. Calculate the characteristic expectations of the human driver in the driving scenario;

[0085] In this embodiment, the method for calculating the feature expectation is: first, based on the driving trajectory data, the values ​​of the various feature functions of the vehicle at every 0.02 seconds until the end of the trajectory, and then averaging the accumulated results.

[0086] S23. Obtaining optimal reward functions of different styles through deep inverse reinforcement learning;

[0087] The steps of deep inverse reinforcement learning are as follows:

[0088] 1) Randomly generate the weight θ of the reward function;

[0089] 2) Use deep reinforcement learning to train an intelligent agent, run it in the corresponding driving scenario, and store the n trajectories generated during training and after training into the trajectory library;

[0090] 3) Select the top m groups of trajectories with the largest reward function in the trajectory library, and then use the principle of the maximum entropy inverse reinforcement algorithm to obtain the probability of each trajectory. The obtained probability and the characteristic value of each trajectory are weighted averaged to obtain the expected characteristic statistics of the generated trajectory.

[0091] 4) Use the maximum likelihood function method to find the gradient of θ and use the gradient ascent method to update θ;

[0092] 5) Repeat steps 2) to 4) until θ converges to the optimal

[0093] S24. Use the reward function obtained by inverse reinforcement learning to obtain the intelligent agent using the deep reinforcement algorithm;

[0094] The steps of the deep reinforcement learning method are as follows:

[0095] 1) Obtain a reward function based on the given weights;

[0096] 2) Initialize the parameters of two agent networks, namely Main Net and Target Net;

[0097] 3) Initialize the simulation environment and the vehicle status;

[0098] 4) Main Net interacts with the simulation environment and records and stores the information of each step (reward, state space, and action information) in the experience area;

[0099] 5) When the experience area is full, randomly select a certain number of records and perform a gradient descent algorithm on the loss function with respect to the grid parameters of the Main Net;

[0100] 6) Every N times of storage, copy the network parameters of Main Net to Target Net and update Target Net;

[0101] 7) Repeat steps 4) to 6) until the Target Net mesh parameters converge.

[0102] In this embodiment, the simulation environment in step 3) is generated based on a VTD (Virtual Test Drive) environment. In the deep reinforcement learning process, the environment car first uses the IDM model to train and obtain primary intelligent agents of various styles. Then, the primary intelligent agents trained in various styles are used as background vehicles to train and obtain advanced intelligent agents.

[0103] Among them, in order to adapt to the DQN algorithm, the action space used in the deep reinforcement learning environment model is discrete (that is, the set of action information when MainNet interacts with the simulation environment), which includes the following actions: longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration, longitudinal rapid deceleration; lateral uniform speed, lateral deceleration, lateral acceleration; each action performed by a car is composed of a combination of actions in two directions.

[0104] S3. Based on the test requirements, driver models of different styles are combined according to the corresponding demand ratio to generate corresponding target test scenarios.

[0105] In this embodiment, according to Figure 4 The ratios shown assign different styles of agents to the background environment car; it should be noted that in order to increase the challenge of the test scene, the proportion of close-range curious agents and close-range competitors is larger than that of other styles of agents.

[0106] In addition, the placement of agents of different styles is not completely random when generating scenes, e.g. Figure 5 As shown, agents with the "close distance" label have specific location requirements when they are generated: they must be generated within 100 meters before and after the main vehicle (the tested autonomous vehicle); agents with the "long distance" label or conservative, normal, and aggressive types have specific location requirements when they are generated: they must be generated within 250 meters before and after the main vehicle (the tested autonomous vehicle); and the probability of generating a background environment vehicle with a non-"close distance" style within 100 meters before and after the main vehicle will be lower, and the probability of generating a background environment vehicle with a non-"close distance" style within 250 meters outside 100 meters before and after the main vehicle will be higher.

[0107] In summary, this technical solution uses an agent for testing derived from human driving data. Its decision-making behavior is more consistent with that of human drivers and is stylized, effectively enhancing the realism and complexity of autonomous driving test scenarios. This solution first learns a reward function from expert trajectories using an inverse reinforcement learning algorithm. It then trains stylized agents through deep reinforcement learning, and finally combines these stylized agents to generate test scenarios. This method achieves excellent test results while ensuring test continuity, authenticity, and accuracy.

Claims

1. A method for generating autonomous driving test scenarios based on a personalized driver model, characterized in that: The following steps are involved: S1. Obtain corresponding driving data based on whether the tested interactive object is known to be an autonomous driving system. If the tested interactive object is unknown to be an autonomous driving system, obtain naturalistic driving data on Chinese highways and their ramps based on existing naturalistic driving datasets. If the tested interaction object is known to be an autonomous driving system, a questionnaire and a driving simulator are combined to collect driving data of the driver's interaction with the autonomous driving system. S2. Based on the acquired driving data, use the maximum entropy inverse reinforcement learning method and deep reinforcement learning method to train driver agent models with different styles; S3. Based on the test requirements, driver models of different styles are combined according to the corresponding demand ratio to generate corresponding target test scenarios.

2. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 1, characterized in that: In step S1, if it is unknown that the tested interaction object is an autonomous driving system, the driver's style is classified into three styles: conservative, normal, and aggressive after obtaining natural driving data.

3. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 1, wherein: In step S1, if it is known that the tested interactive object is an autonomous driving system, the specific process of obtaining the corresponding driving data is as follows: Construct road scenarios for driving simulators based on the road structure and environment of natural driving datasets; Through questionnaire screening, the human driver interaction styles with autonomous vehicles were categorized into four types: close competitor, close curious, long-distance curious, and non-curious. During driving, close competitor drivers actively compete with autonomous vehicles for drivable space in the same space-time domain, maximizing their own interests while ensuring safety. Curious drivers are unfamiliar with the AVs around them and find them novel and curious about their driving behavior. This leads them to follow and observe the AVs, or actively interact with them. The difference between close-range curious and distant-range curious drivers lies in the distance they drive from the AVs. Distant-range curious drivers are more concerned about the reliability and safety of AVs than close-range curious drivers, so they always maintain a safe distance from them. Close-range curious drivers, on the other hand, prefer to move closer to AVs to better observe their interactions. Let three types of drivers (close competitors, close curious people, and long-distance curious people) drive on the generated road scenarios using a driving simulator and collect corresponding driving trajectory data; The driving data corresponding to non-curious style drivers are directly obtained from the natural driving dataset.

4. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 3, wherein: The specific process of constructing the road scene of the driving simulator is as follows: 1) Place the main vehicle, i.e., the virtual car driven by the human driver on the driving simulator, at a fixed distance directly behind and to the side of the autonomous vehicle, and randomly place some follower model cars around it; 2) Generate the environment settings of the road scene based on the environment settings of the natural driving dataset, including but not limited to the vehicle density, vehicle distribution, and vehicle speed of the following model.

5. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 4, characterized in that: The step S2 specifically includes the following steps: S21. Setting the eigenvalue function of the reward function according to the environment model and functional requirements; S22. Calculate the characteristic expectations of human drivers in driving scenarios; S23. Obtaining optimal reward functions of different styles through deep inverse reinforcement learning; S24. Based on the optimal reward function, a stylized driver agent model is trained using deep reinforcement learning.

6. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 5, characterized in that: The characteristic value function in step S21 is specifically: R=θ s f s +θ e f e +θ c f c +θ i f i +θ u_d f unique_d +θ u_v f unique_v Among them, f s is the safety characteristic value, which is equal to the ttc from the main vehicle to the preceding vehicle; f e is the efficiency characteristic value, which is equal to the main vehicle speed; f c is the collision characteristic value, which is equal to a fixed value when a collision occurs and equal to 0 when no collision occurs; f i is the interaction eigenvalue, which is equal to the absolute value of the deceleration when the following vehicle is decelerated; f unique_d Equal to the distance between the host vehicle and the autonomous vehicle; f unique_v Equal to the speed difference between the host vehicle and the autonomous vehicle; θ s ,θ e ,θ c ,θ i ,θ u_d ,θ u_v f s 、f e 、f c 、f i 、f unique_d 、f unique_v The corresponding weight.

7. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 6, characterized in that: The step S23 specifically includes the following steps: S231, randomly generate the weight θ of the reward function; S232. Train an agent using deep reinforcement learning, run it in a corresponding driving scenario, and store n trajectories generated during training and after training into a trajectory library; S233. Select the first m groups of trajectories with the largest reward function in the trajectory library, apply the maximum entropy inverse reinforcement algorithm to obtain the probability of each trajectory in the m groups of trajectories, and perform weighted average calculation on the obtained probability and the characteristic value of each trajectory to obtain the expected characteristic statistics of the generated trajectory. S234, using the maximum likelihood function method to find the gradient of θ, and using the gradient ascent method to update θ; S235. Repeat steps S232 to S234 until θ converges to the optimum, and obtain the optimal reward function based on the optimal weight θ.

8. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 7, characterized in that: The specific process of deep reinforcement learning in step S24 is: S241, initialize the network parameters of the two agents, namely Main Net and Target Net; S242, initializing the simulation environment and the state of the environment vehicle; S243, Main Net interacts with the simulation environment and stores the information of each step in the experience area, where the information of each step includes reward, state space, and action information; S244. When the experience area is full, a certain number of records are randomly taken out and a gradient descent algorithm is performed on the loss function with respect to the grid parameters of the Main Net. S245, every N times of storage, copy the network parameters of the Main Net to the Target Net to update the Target Net; S246. Repeat steps S243 to S245 until the Target Net grid parameters converge.

9. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 8, characterized in that: In step S24, the environment vehicles are first initialized using IDM, and primary intelligent agents of various styles are trained to obtain them. Then, the primary intelligent agents trained in various styles are used as background vehicles to train and obtain advanced intelligent agents.

10. The method for generating an autonomous driving test scenario based on a personalized driver model according to claim 8, characterized in that: The set of action information when the Main Net interacts with the simulation environment is discrete and includes the following actions: longitudinal uniform speed, longitudinal acceleration, longitudinal rapid acceleration, longitudinal deceleration, longitudinal rapid deceleration, lateral uniform speed, lateral deceleration, and lateral acceleration. Each action performed by the vehicle is a combination of actions in two directions.

Images