Simulation background vehicle behavior control method, device, equipment and computer program product

Abstract

The present application relates to a method, device, equipment and computer program product for controlling the behavior of simulated background vehicles, including: obtaining the status data of background vehicles and main vehicles in the simulated traffic flow during the intelligent driving algorithm simulation test; inputting the status data of the background vehicles and main vehicles into a pre-trained driving behavior model for processing to obtain the behavior decision of the background vehicles; controlling the driving of the background vehicles in the simulated traffic flow according to the behavior decision; during the model training process, using an interactive reward function to give interactive rewards to the model's behavior decisions to guide the model's behavior decisions to increase the interaction with the main vehicle as much as possible, and using a authenticity reward function to give authenticity rewards to the model's behavior decisions to guide the model's behavior decisions to be as consistent as possible with natural driving data, so that the model's behavior decisions can maximize the sum of interactive rewards and authenticity rewards. Through this application, the interactivity between the main vehicle and the background vehicle in the simulated traffic flow can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent driving algorithm simulation testing, and specifically to a method, device, electronic equipment and computer program product for controlling simulated background vehicle behavior. Background Art

[0002] In the process of developing and testing intelligent driving algorithms, the testing of intelligent driving algorithms is an indispensable link, because intelligent driving algorithms need to be tested in various traffic environments to ensure that they can operate safely and effectively. However, actual road testing may be limited by many factors such as time, cost, and safety. Therefore, simulation testing provides a virtual environment that can simulate various complex traffic scenarios, allowing intelligent driving algorithms to be tested without being restricted by actual road conditions.

[0003] Simulated traffic flow refers to the virtual traffic flow used in the intelligent driving algorithm simulation test. It is generated by a computer model and simulates the dynamic behavior of vehicles, pedestrians and other traffic participants in the real traffic environment. The main function of simulated traffic flow is to provide a real traffic environment for the intelligent driving algorithm to evaluate its performance under different traffic conditions.

[0004] There are many ways to model simulated traffic flow, such as heuristic-based rule-driven models, learning-based deep learning models, etc. The design focus of the rule-driven model is generally based on achieving a certain behavior pattern, and the design focus of the deep learning model is generally based on matching the trajectory of the simulated traffic flow with the trajectory of natural driving data to achieve a certain degree of authenticity. Both of them make it difficult to consider the interactivity between the main vehicle (the tested vehicle using the intelligent driving algorithm) and the background vehicle (other vehicles in the simulation environment except the main vehicle) during the modeling and design stage. In fact, the interactivity between the main vehicle and the background vehicle is the most concerned aspect of intelligent driving algorithm testing. Summary of the invention

[0005] The purpose of the present application is to provide a method, device, electronic device and computer program product for controlling the behavior of simulated background vehicles, so as to improve the interactivity between the main vehicle and the background vehicles in the simulated traffic flow.

[0006] To achieve the above-mentioned purpose, according to a first aspect of the present application, a method for controlling a simulated background vehicle behavior is provided, the method comprising:

[0007] During the intelligent driving algorithm simulation test, the status data of the background vehicles and the main vehicle in the simulated traffic flow are obtained;

[0008] Inputting the status data of the background vehicle and the host vehicle into a pre-trained driving behavior model for processing to obtain a behavior decision of the background vehicle;

[0009] Controlling the movement of background vehicles in the simulated traffic flow according to the behavior decision;

[0010] The driving behavior model is trained based on natural driving data. During the training process, an interactive reward function is used to give interactive rewards to the behavioral decisions of the driving behavior model to guide the behavioral decisions of the driving behavior model to increase the interaction with the host vehicle as much as possible, and an authenticity reward function is used to give authenticity rewards to the behavioral decisions of the driving behavior model to guide the behavioral decisions of the driving behavior model to be as consistent as possible with the natural driving data, so that the behavioral decisions of the driving behavior model after training can maximize the sum of the interactive rewards and the authenticity rewards.

[0011] In the above method, the driving behavior model uses an interactive reward function to guide the behavioral decisions of the driving behavior model to increase the interaction with the main vehicle as much as possible during the training process, and uses a authenticity reward function to guide the behavioral decisions of the driving behavior model to be as consistent as possible with the natural driving data. Therefore, the behavioral decisions of the driving behavior model after training can meet the interactivity requirements and authenticity requirements. The driving behavior model is used to simulate the driving behavior decisions of background vehicles in traffic flow. On the premise of satisfying authenticity, the interactivity between the main vehicle and the background vehicles in the simulated traffic flow can be improved as much as possible, thereby providing a test environment for the intelligent driving algorithm that is both real and interactive, which is conducive to comprehensive testing and verification of the intelligent driving algorithm.

[0012] According to a second aspect of the present application, a simulated background vehicle behavior control device is provided, comprising a module for executing the method according to the first aspect.

[0013] According to a third aspect of the present application, an electronic device is provided, including:

[0014] A communication interface, used to communicate with other electronic devices;

[0015] a memory for storing computer program instructions;

[0016] A processor is used to execute the computer program instructions to support the electronic device to implement the method according to the first aspect.

[0017] According to a fourth aspect of the present application, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions instruct a computer device to perform operations corresponding to the method according to the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a flow chart of a method for controlling simulated background vehicle behavior in one embodiment of the present application.

[0020] Figure 2 This is a schematic diagram of a simulated background vehicle behavior control device in one embodiment of the present application. DETAILED DESCRIPTION

[0021] The detailed description of the drawings is intended as an illustration of the current embodiment of the present application, and is not intended to represent the only form in which the present application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included in the spirit and scope of the present application.

[0022] An embodiment of the present application provides a method for controlling the behavior of a simulated background vehicle. Figure 1 , the method of this embodiment includes the following steps:

[0023] Step S10, during the intelligent driving algorithm simulation test, obtain the status data of the background vehicles and the main vehicle in the simulated traffic flow.

[0024] Specifically, the main vehicle refers to a virtual vehicle that uses the intelligent driving algorithm under test for automatic driving in the simulated traffic flow, and the background vehicle refers to other vehicles in the simulated traffic flow except the main vehicle. When controlling the driving behavior of the background vehicle, it is necessary to obtain the status data of the background vehicle and the main vehicle as a basis for decision-making, such as the speed, acceleration, position and other information of the background vehicle and the main vehicle.

[0025] Step S20, inputting the status data of the background vehicle and the host vehicle into a pre-trained driving behavior model for processing to obtain a behavior decision of the background vehicle.

[0026] Specifically, inputting the status data of the background vehicle and the host vehicle into the pre-trained driving behavior model for processing means using a model that has been trained with a large amount of data to analyze the status data and generate appropriate behavior decisions for the background vehicle based on the data. A simulated traffic flow usually includes multiple background vehicles, and these background vehicles can all use the pre-trained driving behavior model to make behavior decisions.

[0027] The driving behavior model is trained based on natural driving data. During the training process, an interactive reward function is used to give interactive rewards to the behavioral decisions of the driving behavior model to guide the behavioral decisions of the driving behavior model to increase the interaction with the host vehicle as much as possible, and an authenticity reward function is used to give authenticity rewards to the behavioral decisions of the driving behavior model to guide the behavioral decisions of the driving behavior model to be as consistent as possible with the natural driving data, so that the behavioral decisions of the driving behavior model after training can maximize the sum of the interactive rewards and the authenticity rewards.

[0028] Specifically, this embodiment uses a reinforcement learning method to train the driving behavior model. In reinforcement learning, the driving behavior model learns how to make optimal decisions by interacting with the environment (simulated traffic flow). This learning process involves reward functions, which are the key factors guiding the learning of the driving behavior model.

[0029] The natural driving data refers to the behavior data of drivers in the real world. The driving behavior model is trained by analyzing the behavior data of drivers in the real world so that the model can learn and simulate the decision-making process of human drivers.

[0030] The interactive reward function is used to give interactive rewards to the behavioral decisions of the driving behavior model. This means that when the driving behavior model makes behavioral decisions, it will be rewarded according to the degree to which these decisions interact with the main vehicle. The interactive reward function is designed to encourage the driving behavior model (background vehicle) to make decisions that can generate more interactions with the main vehicle, such as changing lanes, overtaking, etc., so that the simulation environment can better test the performance of the intelligent driving algorithm in complex interactive situations.

[0031] The use of a realistic reward function to give realistic rewards to the behavioral decisions of the driving behavior model means that when the driving behavior model makes behavioral decisions, it will be rewarded based on the consistency of these decisions with the real driving data. The design of the realistic reward function is to encourage the model to make decisions that are consistent with the real driver behavior to ensure the authenticity of the simulation environment.

[0032] The goal of the entire training process is to ensure that the behavioral decisions of the driving behavior model after training can maximize the sum of the interactive reward and the authenticity reward. That is, the driving behavior model needs to find a balance between interactivity and authenticity so that its behavioral decisions can generate sufficient interaction with the host vehicle while maintaining consistency with the real driving data. By optimizing the sum of these two reward functions, the driving behavior model can learn the most appropriate behavioral strategy in the simulation environment.

[0033] Step S30, controlling the movement of background vehicles in the simulated traffic flow according to the behavior decision.

[0034] Specifically, the background vehicle in the simulation environment operates according to the driving strategy determined by the driving behavior model, such as accelerating, decelerating, keeping lane or changing lanes.

[0035] In the simulation test, the main vehicle (i.e., the vehicle to which the intelligent driving algorithm is applied) will respond to the situation in the simulation environment based on its own perception, decision-making, and control logic. The intelligent driving algorithm of the main vehicle will process information from the surrounding environment (including background vehicles) and make decisions based on this information to control the driving behavior of the main vehicle. Based on the driving behavior of the main vehicle, the intelligent driving algorithm of the main vehicle can be verified, that is, by observing and analyzing the performance of the main vehicle in the simulation environment, the performance and effectiveness of the intelligent driving algorithm can be evaluated, thereby achieving comprehensive testing and verification of the intelligent driving algorithm without affecting the safety of the real world. Through such testing, R&D personnel can discover and fix potential problems in the algorithm, improve the reliability and safety of the algorithm, and ultimately prepare for real vehicle road testing and commercial deployment.

[0036] In summary, in the method of this embodiment, the driving behavior model uses an interactive reward function during training to guide the behavioral decisions of the driving behavior model to increase the interaction with the main vehicle as much as possible, and uses a authenticity reward function to guide the behavioral decisions of the driving behavior model to be as consistent as possible with the natural driving data. Therefore, the behavioral decisions of the driving behavior model after training can meet the interactivity requirements and authenticity requirements. The driving behavior model is used to simulate the driving behavior decisions of background vehicles in traffic flow. On the premise of satisfying authenticity, the interactivity between the main vehicle and the background vehicles in the simulated traffic flow can be improved as much as possible, thereby providing a test environment for the intelligent driving algorithm that is both real and interactive, which is conducive to comprehensive testing and verification of the intelligent driving algorithm.

[0037] In some embodiments, the status data of the background vehicle refers to the status information of other vehicles in the simulation environment except the main vehicle, including but not limited to:

[0038] Position refers to the specific coordinates of the background car in the simulation environment;

[0039] The lane is the lane where the background vehicle is currently traveling;

[0040] The heading angle refers to the direction of the background vehicle, usually expressed as an angle;

[0041] Speed ​​refers to the instantaneous speed of the background vehicle at a certain moment;

[0042] Acceleration refers to the acceleration of the background car and reflects how fast the speed changes.

[0043] The status data of the host vehicle refers to the status information of the vehicle to which the intelligent driving algorithm is applied, including but not limited to:

[0044] Position refers to the specific coordinates of the host vehicle in the simulation environment;

[0045] The lane in which the vehicle is currently traveling is the lane in which the vehicle is currently traveling.

[0046] The heading angle refers to the direction of the main vehicle's forward movement;

[0047] Speed ​​refers to the instantaneous speed of the host vehicle at a certain moment;

[0048] Acceleration refers to the acceleration of the host vehicle.

[0049] It should be noted that the status data of the background vehicles and the main vehicle are dynamically updated to reflect the movement and changes of the vehicles in the simulation environment. In simulation testing, collecting and analyzing this data is crucial to evaluating and improving the intelligent driving algorithm because they provide the environmental context required for the algorithm to make decisions. For example, the intelligent driving algorithm needs to decide when to accelerate, decelerate, or change lanes based on the speed and position of the surrounding background vehicles. By simulating these dynamic states, R&D personnel can test the performance of the algorithm in different traffic scenarios and ensure that it can make safe driving decisions in various situations.

[0050] In some embodiments, the using the interactive reward function to provide interactive rewards to the behavior decisions of the driving behavior model further includes:

[0051] The interactive reward is the sum of the first interactive reward and the second interactive reward;

[0052] The interactive reward function gives a first interactive reward according to the interaction frequency between the background vehicle and the main vehicle. The higher the interaction frequency, the greater the corresponding first interactive reward.

[0053] The interactivity reward function gives a second interactivity reward according to the similarity between the duration of the background vehicle entering the interactive state and the natural driving data, and the higher the similarity, the greater the corresponding second interactivity reward;

[0054] The interactive state is that the driving behaviors of the background vehicle and the main vehicle influence each other.

[0055] Specifically, in this embodiment, the interactivity reward is decomposed into two parts: a first interactivity reward and a second interactivity reward. These two parts of rewards jointly guide the model to generate background vehicle behaviors with stronger interactivity.

[0056] The first interaction reward is based on the frequency of interaction between the background car and the main car. If the background car frequently interacts with the main car, such as when changing lanes, overtaking or following the main car, it will receive a higher first interaction reward. This design encourages the model to generate more behaviors that interact with the main car, making the simulation environment more complex and realistic.

[0057] The second interactivity reward is based on the similarity between the duration of the background car entering the interactive state and the natural driving data. The interactive state refers to the situation where the behavior of the background car and the host car affects each other. If the behavior of the background car in the interactive state is similar to the behavior pattern in the natural driving data, it will receive a higher second interactivity reward. This design ensures that the interactive behavior of the background car is consistent with the driving behavior in the real world.

[0058] In some embodiments, during the driving behavior model training process, if the distance between the background vehicle and the main vehicle is less than or equal to a preset distance threshold, and the background vehicle trajectory will intersect with the main vehicle trajectory without changing the current operating state, it is determined that the background vehicle enters an interactive state.

[0059] Specifically, if the following conditions are met, the background vehicle is considered to have entered the interactive state:

[0060] Condition 1: The distance between the background vehicle and the main vehicle is less than or equal to a preset distance threshold; that is, the distance between the background vehicle and the main vehicle is close enough to reach a level that is considered "dangerous" or "requires special attention". This preset distance threshold is set based on actual traffic rules and safety considerations. For example, it may be set to a safe following distance between two vehicles.

[0061] Condition 2: Without changing the current running state, the background vehicle trajectory will intersect with the main vehicle trajectory; that is, if both the background vehicle and the main vehicle maintain their current driving state (including speed and direction), their driving paths will intersect or overlap, resulting in potential collision or conflict.

[0062] When the above two conditions are met at the same time, the driving behavior model will determine that the background vehicle has entered an interactive state. In this case, the driving behavior model needs to generate appropriate behavior decisions to avoid potential collisions and ensure smooth traffic, which may include slowing down, changing lanes, or other measures to avoid conflicts.

[0063] During the training process, the driving behavior model learns how to make the best decisions when entering an interactive state and reinforces these decisions through an interactive reward function. Such training helps the model better understand how to interact with the host vehicle in complex traffic conditions and provides a more challenging and realistic testing environment for intelligent driving algorithms.

[0064] In some embodiments, the interaction frequency between the background vehicle and the main vehicle is determined according to the time between two consecutive times when the background vehicle enters the interaction state. The smaller the time, the higher the interaction frequency.

[0065] Specifically, in this embodiment, the interaction frequency is determined based on the time interval between two consecutive background vehicles entering the interaction state, which is an indicator to measure the frequency of interaction between the background vehicle and the main vehicle. In intelligent driving simulation, a high interaction frequency means that the background vehicle often interacts with the main vehicle, which may include overtaking, changing lanes, following vehicles, etc.

[0066] The time between two consecutive background vehicle entering the interaction state refers to the time interval between the current time when the background vehicle enters the interaction state and the last time it entered the interaction state. If this time interval is very short, it means that the interaction between the background vehicle and the main vehicle occurs frequently. If the interaction between the background vehicle and the main vehicle occurs more frequently, the time interval between them will be shorter and the interaction frequency will be higher.

[0067] When training the driving behavior model, by increasing the reward for the frequency of interaction between the background vehicle and the main vehicle, the model can be encouraged to generate more behaviors that interact with the main vehicle. In this way, the background vehicle behavior in the simulation environment will be more diverse and realistic, providing a more comprehensive test scenario for the main vehicle's intelligent driving algorithm.

[0068] In some embodiments, the using of the authenticity reward function to give authenticity reward to the behavior decision of the driving behavior model further comprises:

[0069] The authenticity reward function gives authenticity rewards based on the similarity between the speed distribution, acceleration distribution, lane distribution and / or distance distribution of the background vehicle and the natural driving data. The higher the similarity, the greater the corresponding authenticity reward.

[0070] Specifically, the authenticity reward function is designed to ensure that the background vehicle behavior generated by the model is as close as possible to the driving behavior in the real world. Since simulated traffic flow needs to be heterogeneous and random, the speed distribution, acceleration distribution, lane distribution, and distance distribution of the background vehicle to the front and rear vehicles are key factors in evaluating the authenticity of the background vehicle behavior. Specifically, speed distribution, acceleration distribution, lane distribution, and vehicle distance distribution refer to the statistical characteristics of the background vehicle in these aspects at a specific time or scene.

[0071] The similarity with natural driving data refers to the degree of match between the background vehicle behavior generated by the driving behavior model and the behavior pattern in the actual driving data. The higher the similarity, the closer the behavior generated by the model is to the real driving behavior, and the greater the authenticity reward obtained. Such a design encourages the driving behavior model to generate behavioral decisions that are consistent with real driving data.

[0072] In this way, the driving behavior model is motivated to generate behaviors that conform to real driving habits during the training process, making the background vehicle behavior in the simulation environment more realistic and credible, which is crucial for testing and evaluating the performance of intelligent driving algorithms under real-world conditions.

[0073] In some embodiments, the speed distribution is the mean and standard deviation of the background vehicle speeds;

[0074] The acceleration distribution is the mean and standard deviation of the background vehicle acceleration;

[0075] The lane distribution is the mean and standard deviation of the number of times or time that the background vehicle travels in each lane;

[0076] The distance distribution from the front and rear vehicles is the mean and standard deviation of the distances between the background vehicle and the front and rear vehicles.

[0077] Specifically, the speed distribution refers to the statistical characteristics of the speed of the background vehicles, including the mean and standard deviation of the speed. The mean of the speed reflects the average speed level of the background vehicles, while the standard deviation of the speed reflects the speed fluctuation of the background vehicles.

[0078] The acceleration distribution refers to the statistical characteristics of the acceleration of the background vehicle, and also includes the mean and standard deviation of the acceleration. The mean of the acceleration reflects the average acceleration level of the background vehicle, and the standard deviation of the acceleration reflects the acceleration variation range or fluctuation degree of the background vehicle.

[0079] Lane distribution refers to the statistical characteristics of the number of times or time distribution of background vehicles traveling in each lane, including the mean and standard deviation. The mean of the lane distribution may reflect the preference of the background vehicle to travel in a specific lane, while the standard deviation of the lane distribution reflects the changes or fluctuations in the lane usage of the background vehicle.

[0080] The distance distribution to the front and rear vehicles refers to the statistical characteristics of the distance between the background vehicle and the front and rear vehicles, including the mean and standard deviation of the distance to the front vehicle, as well as the mean and standard deviation of the distance to the rear vehicle. The mean of the distance to the front and rear vehicles reflects the average distance between the background vehicle and the front and rear vehicles, and the standard deviation of the distance to the front and rear vehicles reflects the range of variation or degree of fluctuation of the distance between the front and rear vehicles.

[0081] When training the driving behavior model, the authenticity of the behavior generated by the driving behavior model can be evaluated by comparing the similarity between the background vehicle behavior generated by the driving behavior model and the natural driving data in these distributions; if the behavior generated by the model is highly similar to the natural driving data in these distributions, the driving behavior model will receive a higher authenticity reward, which helps the model learn to generate more realistic and natural background vehicle behavior.

[0082] In some embodiments, when training the driving behavior model, a variety of different map scenarios can be used for training. These scenarios simulate various roads and traffic environments in the real world to ensure that the driving behavior model can learn and optimize its behavior decisions under diverse conditions.

[0083] Specifically, these scenarios include but are not limited to the following:

[0084] One-way straight road scenario, with the number of lanes ; This scenario simulates a straight road section with one direction of travel, and the number of lanes can range from 1 to 4;

[0085] Two-way straight road scenario, and the number of one-way lanes ; This scenario simulates a straight road section with two-way traffic, and the number of lanes in each direction can range from 1 to 4;

[0086] One-way curve scene, and the number of lanes ; This scenario simulates a curved road section with one-way driving, and the number of lanes can range from 1 to 4;

[0087] Two-way curve scene, and the number of one-way lanes ; This scenario simulates a two-way curved road section, and the number of lanes in each direction can range from 1 to 4;

[0088] An intersection scene with traffic lights and the minimum angle of intersection of roads ; This scenario simulates an intersection controlled by traffic lights, and the minimum angle of road intersection can be 30°, 45°, 60° or 90°;

[0089] Intersection scene without traffic lights and minimum intersection angle ; This scenario simulates an intersection without traffic lights, and the minimum angle of road intersection can also be 30°, 45°, 60° or 90°;

[0090] Roundabout scene, and the number of lanes This scenario simulates a roundabout traffic environment, where the number of lanes around the roundabout can range from 3 to 6;

[0091] The confluence and exit scenarios of expressways and auxiliary roads; this scenario simulates the connection points between expressways or expressways and auxiliary roads, including the confluence and exits of vehicles entering and exiting the expressway.

[0092] It should be noted that this embodiment selects a series of representative map scenes as templates. These scenes can cover the main types of traffic environments in the real world. Training in these template scenes can enable the model to learn common driving rules and strategies, so that it can perform well in different but related actual scenes. It can solve the problem of strong dependence on map scenes and meet the generalization requirements of driving behavior models.

[0093] Another embodiment of the present application also provides a simulated background vehicle behavior control device, including a module for executing the simulated background vehicle behavior control method described in the above embodiment. This device can be a hardware device, a software system, or a combination of hardware and software.

[0094] For example, see Figure 2 , the device of this embodiment may include the following modules:

[0095] Data acquisition module 1, used to obtain status data of background vehicles and main vehicles in simulated traffic flow during intelligent driving algorithm simulation test;

[0096] Behavior decision module 2, used for inputting the status data of the background vehicle and the host vehicle into a pre-trained driving behavior model for processing to obtain a behavior decision of the background vehicle;

[0097] A driving control module 3, used for controlling the driving of background vehicles in the simulated traffic flow according to the behavior decision;

[0098] The driving behavior model is trained based on natural driving data. During the training process, an interactive reward function is used to give interactive rewards to the behavioral decisions of the driving behavior model to guide the behavioral decisions of the driving behavior model to increase the interaction with the host vehicle as much as possible, and an authenticity reward function is used to give authenticity rewards to the behavioral decisions of the driving behavior model to guide the behavioral decisions of the driving behavior model to be as consistent as possible with the natural driving data, so that the behavioral decisions of the driving behavior model after training can maximize the sum of the interactive rewards and the authenticity rewards.

[0099] It should be noted that the various modules of the device of this embodiment correspond one-to-one to the various steps of the method of the above embodiment. Therefore, the specific principle content of the modules of this embodiment can be obtained by referring to the content of the method of the above embodiment, so it will not be repeated here.

[0100] Another embodiment of the present application further provides an electronic device, including:

[0101] A communication interface, used to communicate with other electronic devices;

[0102] a memory for storing computer program instructions;

[0103] The processor is used to execute the computer program instructions to support the electronic device to implement the simulated background vehicle behavior control method described in the above embodiment.

[0104] In this embodiment, the memory mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, and a flash card (FlashCard), etc., or the memory can also be other volatile solid-state storage devices.

[0105] The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor. The processor is the control center of the electronic device, and various parts of the electronic device are connected using various interfaces and lines.

[0106] An embodiment of the present application also provides a computer program product, including computer program instructions, wherein the computer program instructions instruct a computer device to perform operations corresponding to the method described in the above embodiment.

[0107] Specifically, the computer program product includes a series of computer program instructions, which can instruct a computer device to execute the simulated background vehicle behavior control method described in this application. These instructions are codes written in a computer program, which define how to perform specific operations. In this embodiment, these instructions are used to execute the simulated background vehicle behavior control method.

[0108] These program instructions are designed to be loaded onto a computer device (such as an on-board computing platform, etc.) and instruct the device to perform specific operations, which are the various steps in the simulated background vehicle behavior control method described above. In this way, the computer program product provides a complete software solution that can be run on various computer devices to implement the simulated background vehicle behavior control method described in the above embodiment.

[0109] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for controlling the behavior of a simulated background vehicle, characterized in that: The method comprises: During the intelligent driving algorithm simulation test, the status data of the background vehicles and the main vehicle in the simulated traffic flow are obtained; Inputting the status data of the background vehicle and the host vehicle into a pre-trained driving behavior model for processing to obtain a behavior decision of the background vehicle; Controlling the movement of background vehicles in the simulated traffic flow according to the behavioral decision; The driving behavior model is trained based on natural driving data. During the training process, an interactive reward function is used to give interactive rewards to the behavior decisions of the driving behavior model, so as to guide the behavior decisions of the driving behavior model to increase the interaction with the host vehicle as much as possible. The authenticity reward function is used to give authenticity rewards to the behavior decisions of the driving behavior model, so as to guide the behavior decisions of the driving behavior model to be consistent with the natural driving data as much as possible, so that the behavior decisions of the driving behavior model after the training are completed can maximize the sum of the interactive rewards and the authenticity rewards. The authenticity reward function gives an authenticity reward according to the similarity between the speed distribution, acceleration distribution, lane distribution and / or distance distribution from the front and rear vehicles of the background vehicle and the natural driving data. The higher the similarity, the greater the corresponding authenticity reward. The interactive reward is the sum of the first interactive reward and the second interactive reward; the interactive reward function gives the first interactive reward according to the interaction frequency between the background car and the main car, and the higher the interaction frequency, the larger the corresponding first interactive reward; the interactive reward function gives the second interactive reward according to the similarity between the duration of the background car entering the interactive state and the natural driving data, and the higher the similarity, the larger the corresponding second interactive reward; wherein, the interactive state is the mutual influence between the driving behaviors of the background car and the main car; During the training of the driving behavior model, if the distance between the background vehicle and the main vehicle is less than or equal to a preset distance threshold, and the trajectory of the background vehicle will intersect with the trajectory of the main vehicle without changing the current running state, it is determined that the background vehicle enters the interactive state; The interaction frequency between the background vehicle and the main vehicle is determined according to the time between two consecutive times when the background vehicle enters the interaction state. The smaller the time, the higher the interaction frequency.

2. The method according to claim 1, characterized in that: The speed distribution is the mean and standard deviation of the background vehicle speed; The acceleration distribution is the mean and standard deviation of the background vehicle acceleration; The lane distribution is the mean and standard deviation of the number of times or time that the background vehicle travels in each lane; The distance distribution from the front and rear vehicles is the mean and standard deviation of the distances between the background vehicle and the front and rear vehicles.

3. The method according to claim 1, characterized in that The driving behavior model is trained using a variety of different map scenarios, including: One-way straight road scenario, with the number of lanes ; Two-way straight road scenario, and the number of one-way lanes ; One-way curve scene, and the number of lanes ; Two-way curve scene, and the number of one-way lanes ; An intersection scene with traffic lights and the minimum angle of intersection of roads ; Intersection scene without traffic lights and minimum intersection angle ; Roundabout scene, and the number of lanes and / or The entrance and exit scenarios of expressways and auxiliary roads.

4. A simulated background vehicle behavior control device, characterized in that: Comprising modules for executing the method according to any one of claims 1 to 3.

5. An electronic device, characterized in that: include: A communication interface, used to communicate with other electronic devices; a memory for storing computer program instructions; A processor, configured to execute the computer program instructions to support the electronic device to implement the method as claimed in any one of claims 1 to 3.

6. A computer program product, characterized in that The method comprises computer program instructions, wherein the computer program instructions instruct a computer device to perform operations corresponding to the method according to any one of claims 1 to 3.

Images