Traffic flow-based simulation test method, device and equipment

Abstract

The application discloses a simulation test method and device based on traffic flow, and relates to the technical field of simulation test. The simulation test method based on traffic flow comprises the following steps: acquiring quantity proportion information of intelligent agents of different driving style types; determining a disappearing distance threshold and an appearing distance threshold of the intelligent agents and a measured vehicle according to a sensing range of the measured vehicle; determining an initial vehicle speed of the intelligent agents according to a current speed and a relative position type of the measured vehicle; generating a random traffic flow based on the quantity proportion information, the disappearing distance threshold, the appearing distance threshold and the initial vehicle speed; and performing simulation test on the measured vehicle based on the random traffic flow. The scheme can improve the test capability of the automatic driving function in a complex scene.

Description

Technical Field

[0001] This application relates to the field of simulation testing technology, and in particular to simulation testing methods, apparatus and equipment based on traffic flow. Background Technology

[0002] Currently, traffic flow design in autonomous driving simulation testing mainly employs two approaches. One approach involves test engineers manually designing the specific movement logic for each traffic participant based on the autonomous driving function under test, such as the vehicle in front moving slowly, cutting in from the left, and the vehicle behind overtaking. The other approach generates a large number of randomly moving traffic participants by triggering random actions.

[0003] However, while manually designed solutions are suitable for testing low-level driver assistance systems, they are limited in the number of traffic participants that can be simulated when facing high-level autonomous driving systems, and the scenario complexity is far lower than actual needs. For solutions based on random actions, the movement patterns of the generated traffic participants are difficult to control precisely, and their trajectories and driving intentions differ significantly from real traffic flow conditions, resulting in poor test results. Therefore, improving the testing capabilities of autonomous driving functions in complex scenarios has become a technical problem that needs to be solved.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a traffic flow-based simulation testing method, apparatus, and equipment, aiming to solve the technical problem of how to improve the testing capability of autonomous driving functions in complex scenarios.

[0006] To achieve the above objectives, this application proposes a traffic flow-based simulation testing method, which includes: Obtain information on the proportion of trained agents with different driving styles; The disappearance distance threshold and appearance distance threshold between the trained agent and the tested vehicle are determined based on the perception range of the tested vehicle. The initial speed of the trained agent is determined based on the current speed and relative position type of the tested vehicle. Random traffic flow is generated based on the quantity ratio information, the disappearance distance threshold, the appearance distance threshold, and the initial vehicle speed; The test vehicle is simulated based on the random traffic flow.

[0007] In one embodiment, a speed increment is determined based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the maximum allowable speed threshold, and the simulated chasing speed of the trained agent is determined based on the current speed of the tested vehicle and the speed increment. The speed reduction is determined based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the current speed of the tested vehicle. The simulated speed at which the trained agent is being chased is determined based on the current speed of the tested vehicle and the speed reduction. When the relative position type between the trained agent and the tested vehicle is rear, the initial speed of the trained agent is determined based on the simulated chasing speed and the maximum permissible speed threshold. When the relative position type between the trained agent and the tested vehicle is "ahead", the initial speed of the trained agent is determined based on the simulated speed being chased and the minimum allowable speed threshold.

[0008] In one embodiment, before the step of obtaining the percentage information of the number of trained agents of different driving style types, the method further includes: The training environment for the trained intelligent agent is constructed based on traffic-participating vehicles and the road environment; In the training environment, the weights of the preset expectation function network are updated using gradient descent to obtain the target training model; Based on the target training model, the trained agent is iteratively trained to obtain a traffic flow motion model; Based on the traffic flow motion model, the number and proportion of trained agents of different driving styles are determined.

[0009] In one embodiment, the step of constructing the training environment for the trainable agent based on the traffic-participating vehicle and the road environment includes: Determine acceleration and yaw rate as the set of actions; Define the vehicle's longitudinal velocity, lateral velocity, collision time with the lateral target, and collision time with the longitudinal target as a set of states; The trained agent is constructed based on the set of actions and the set of states; Define one-way multi-lane roads as the road environment; The training environment for the trained agent is constructed based on the traffic-participating vehicle and the road environment.

[0010] In one embodiment, the step of updating the weights of a preset expectation function network using gradient descent in the training environment to obtain the target training model includes: Initialize the historical experience data pool, the expected value function network, and the target value function network; Obtain the expected target reward value output by the expected value function network under the current state; The action or random action corresponding to the target reward expectation value is determined as the current action based on a preset allocation probability. Perform the current action in the training environment to obtain an immediate reward and the next state; The historical experience data pool is updated based on the current state, the current action, the instant reward, and the next state. Training samples are drawn from the updated historical experience data pool; The cumulative discount reward of the target value function network is determined based on the training samples and the expected value function network. The weights of the expected value function network are updated using gradient descent based on the cumulative discount reward to obtain the target training model.

[0011] In one embodiment, the step of iteratively training the trained agent based on the target training model to obtain a traffic flow motion model includes: Obtain the driving style type of the trained intelligent agent; The safety reward is determined based on whether the trained agent experiences a collision event. The longitudinal motion reward is determined based on the driving style type, the longitudinal speed of the trained agent, and a preset speed statistic. The yaw motion reward is determined based on the number of lane changes made by the trained agent. The target reward function of the trained agent is determined based on the security reward, the longitudinal motion reward, and the lateral motion reward. The target reward function is substituted into the target training model, and the trained agent is iteratively trained to obtain the traffic flow motion model.

[0012] In one embodiment, the step of simulating the test vehicle based on the random traffic flow includes: Responding to the control commands of the module under test, it generates motion state information of the vehicle under test; The traffic participant status information of the tested vehicle is determined based on the motion state of the tested vehicle and the random traffic flow. A dynamic scene is generated based on the motion state information of the tested vehicle and the state information of the traffic participants. The dynamic scene is processed to generate simulated sensor data, which is then sent to the module under test (DUT) so that the DUT can update the control commands of the vehicle under test based on the simulated sensor data, thereby performing simulation testing.

[0013] In one embodiment, the method further includes: Obtain scene parameter information, traffic flow configuration information, algorithm version information, and number of repeated executions to obtain test task information; The test task information is split into multiple simulation subtasks, and the multiple simulation subtasks are inserted into a message queue; The number of worker node instances is determined based on the number of pending simulation subtasks in the message queue, and a dynamic worker node cluster is created. When the worker node instance starts, a simulation subtask is retrieved from the message queue; Based on the task parameter information of the simulation sub-task, the traffic flow motion model is loaded from the model repository; A random traffic flow is initialized based on the traffic flow motion model, and the vehicle under test is simulated based on the random traffic flow.

[0014] Furthermore, to achieve the above objectives, this application also proposes a traffic flow-based simulation testing device, which includes: The data acquisition module is used to acquire information on the proportion of trained agents with different driving styles. The data processing module is used to determine the disappearance distance threshold and the appearance distance threshold between the trained intelligent agent and the tested vehicle based on the perception range of the tested vehicle. The data processing module is also used to determine the initial vehicle speed of the trained intelligent agent based on the current speed and relative position type of the tested vehicle; The data generation module is used to generate random traffic flow based on the quantity ratio information, the disappearance distance threshold, the appearance distance threshold, and the initial vehicle speed; The simulation test module is used to perform simulation tests on the vehicle under test based on the random traffic flow.

[0015] Furthermore, to achieve the above objectives, this application also proposes a traffic flow-based simulation testing device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the traffic flow-based simulation testing method described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the traffic flow-based simulation test method described above.

[0017] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the traffic flow-based simulation testing method described above.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: By acquiring information on the proportion of trained agents with different driving styles, the behavioral characteristics of traffic flow can be configured, enabling simulated traffic flow to accurately reflect the differences in driving behavior on real roads. Determining the vanishing and reappearing distance thresholds based on the perception range of the tested vehicle effectively reduces computational resource consumption while ensuring the realism of the simulation scenario. The initial speed of the trained agent is determined based on the current speed and relative position type of the tested vehicle, ensuring real-time matching of traffic flow speed with the tested vehicle and improving the rationality of the simulation scenario. Generating random traffic flow by combining various core parameters and traffic flow motion models allows for the rapid construction of complex and realistic test scenarios. Simulation testing based on random traffic flow comprehensively verifies the autonomous driving function of the tested vehicle in complex traffic environments, while simultaneously improving the realism, execution efficiency, and reproducibility of the test results. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an embodiment of the traffic flow-based simulation testing method of this application. Figure 2 This is a schematic diagram of a two-degree-of-freedom vehicle motion model provided in Embodiment 1 of the traffic flow-based simulation testing method of this application; Figure 3 This is a schematic diagram of a random traffic flow scenario provided in Embodiment 1 of the traffic flow-based simulation testing method of this application; Figure 4 This is a flowchart illustrating Embodiment 2 of the traffic flow-based simulation testing method of this application. Figure 5 This is a schematic diagram of module interaction provided in Embodiment 2 of the traffic flow-based simulation testing method of this application; Figure 6This is a schematic diagram of the simulation test architecture provided in Embodiment 2 of the traffic flow-based simulation test method of this application; Figure 7 A simplified flowchart illustrating the traffic flow-based simulation testing method provided in Embodiment 2 of this application; Figure 8 This is a schematic diagram of the module structure of a traffic flow-based simulation testing device according to an embodiment of this application; Figure 9 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the traffic flow-based simulation testing method in this application embodiment.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The main solution of this application embodiment is as follows: obtaining the proportion information of trained intelligent agents of different driving style types; determining the disappearance distance threshold and appearance distance threshold between the trained intelligent agent and the test vehicle according to the perception range of the test vehicle; determining the initial speed of the trained intelligent agent according to the current speed and relative position type of the test vehicle; generating random traffic flow based on the proportion information, the disappearance distance threshold, the appearance distance threshold and the initial speed; and performing simulation testing on the test vehicle based on the random traffic flow.

[0026] Manually designed solutions are suitable for testing low-level driver assistance systems, but when facing high-level autonomous driving systems, the number of traffic participants that can be simulated is limited, and the scenario complexity is far lower than actual needs. For solutions based on random actions, the movement patterns of the generated traffic participants are difficult to control precisely, and their trajectories and driving intentions differ significantly from real traffic flow conditions, resulting in poor test results. Therefore, improving the testing capabilities of autonomous driving functions in complex scenarios has become an unresolved technical problem.

[0027] This application provides a solution that, by acquiring information on the proportion of trained agents with different driving styles, allows for the configuration of traffic flow behavior characteristics, enabling simulated traffic flow to accurately reflect the differences in driving behavior on real roads. Determining disappearance and reappearance distance thresholds based on the perception range of the tested vehicle effectively reduces computational resource consumption while ensuring the realism of the simulation scenario. Determining the initial speed of the trained agent based on the current speed and relative position type of the tested vehicle ensures real-time matching of traffic flow speed with the tested vehicle, enhancing the rationality of the simulation scenario. Generating random traffic flow by combining various core parameters and traffic flow motion models allows for the rapid construction of complex and realistic test scenarios. Simulation testing based on random traffic flow comprehensively verifies the autonomous driving function of the tested vehicle in complex traffic environments, while simultaneously improving the realism, execution efficiency, and reproducibility of the test results.

[0028] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a traffic flow-based simulation testing device. The following description uses a traffic flow-based simulation testing device as an example to illustrate this embodiment and the subsequent embodiments.

[0029] Based on this, embodiments of this application provide a traffic flow-based simulation testing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the traffic flow-based simulation testing method of this application.

[0030] In this embodiment, the traffic flow-based simulation testing method includes steps S10 to S50: Step S10: Obtain the percentage information of the number of trained agents of different driving styles; It should be noted that driving style types are different categories categorized based on vehicle driving behavior and decision-making characteristics. Different driving style types correspond to different speed control and lane-changing behavior rules, and can at least include cautious driving style, balanced driving style, and aggressive driving style. Specifically, the cautious driving style is characterized by lower speeds and very few lane-changing operations, the balanced driving style is characterized by moderate speeds and normal lane-changing operations, and the aggressive driving style is characterized by higher speeds and frequent lane-changing operations.

[0031] In addition, the trained agent is a traffic participant model built on a two-degree-of-freedom vehicle kinematics model and iteratively trained using a deep Q-network (DQN) reinforcement learning algorithm. The trained agent has a fixed driving style type and can execute vehicle driving actions according to the behavioral logic of the corresponding driving style type.

[0032] Additionally, the quantity percentage information refers to the proportion of trained agents of different driving styles among all simulated traffic participants. This value is used to determine the quantity distribution ratio of trained agents of various driving styles in the simulation test.

[0033] It should be understood that, based on the specific objectives and scenario requirements of autonomous driving simulation testing, traffic participants of different driving styles can be set, such as the proportion and number of vehicles present, thereby obtaining information on the proportion of trained agents of different driving styles. For example, the proportion of trained agents with a cautious driving style is 20%, the proportion of trained agents with a balanced driving style is 50%, and the proportion of trained agents with an aggressive driving style is 30%. The total number of trained agents, i.e., the total number of traffic vehicles around the tested vehicle, is 80-100.

[0034] In one feasible implementation, steps A10 to A40 may be included before step S10: Step A10: Construct the training environment for the trainee agent based on the traffic-participating vehicle and the road environment; It should be noted that traffic participants are other vehicle participants in the simulation scenario besides the trained agent. They can adopt a simple action-triggered motion model. The motion state of the traffic participants comes from the statistical results of data collected from actual roads. In the training environment, the traffic participants act as objects that interact with the trained agent. To simplify the computation, traffic participants can be simplified to the motion of rigid bodies.

[0035] Furthermore, the road environment refers to the road space scenario set in the simulation training. A one-way three-lane road can be selected as the road environment, which can simplify the computational complexity of the training model and ensure that the training effect of the trained agent covers common road scenarios. The training environment is a comprehensive scenario carrier used for reinforcement learning training of the trained agent. The training environment integrates traffic participation vehicle motion rules, road environment parameters, and agent interaction logic.

[0036] It should be understood that a one-way three-lane road environment can be selected and the road parameters can be set to consider various situations such as vehicles in the agent's own lane, vehicles on the left, vehicles on the right, the left side of the road, and the right side of the road. Then, traffic-participating vehicles using a simple action-triggered motion model are deployed into the road environment. A single trainee agent and multiple traffic-participating vehicles are placed in the set road environment at the same time. By integrating the motion rules of the traffic-participating vehicles with the spatial parameters of the road environment, the training environment of the trainee agent is constructed to ensure that the training environment can meet the scene interaction requirements of reinforcement learning training.

[0037] In one feasible implementation, step A10 may include: Determine acceleration and yaw rate as the set of actions; Define the vehicle's longitudinal velocity, lateral velocity, collision time with the lateral target, and collision time with the longitudinal target as a set of states; The trained agent is constructed based on the set of actions and the set of states; Define one-way multi-lane roads as the road environment; The training environment for the trained agent is constructed based on the traffic-participating vehicle and the road environment.

[0038] It should be noted that the action set is the sum of all driving actions that the trained agent can autonomously choose to execute at each decision node. It serves as the basis for the trained agent's behavioral decisions, and the setting of the action set limits the types of driving operations that the trained agent can perform. The state set is the sum of all environmental states that the trained agent can perceive in the traffic flow environment. It serves as the basis for the trained agent to perceive the external environment and make corresponding decisions. The setting of the state set determines the dimension and accuracy of the trained agent's perception of the environment.

[0039] Additionally, acceleration is a physical quantity that describes how quickly a vehicle's speed changes, including longitudinal acceleration. Lateral acceleration of the vehicle Longitudinal acceleration is used to control vehicle acceleration and braking, while lateral acceleration is used to assist in controlling the vehicle's lateral motion. Yaw rate is a physical quantity that describes how quickly a vehicle rotates about an axis perpendicular to the ground. The yaw rate is used to control the steering wheel operation of the vehicle, reflecting the rate of change of the vehicle's lateral attitude.

[0040] Additionally, the longitudinal velocity u of this vehicle is the longitudinal component of the trainable agent's own speed along the road; the lateral velocity of this vehicle... This is the lateral component of the trainee agent's own speed along the road. The lateral object collision time, also known as the lateral nearest object collision time (TTC), is the remaining time before the trainee agent collides with the nearest lateral traffic participant or road boundary; the longitudinal object collision time, also known as the longitudinal nearest object collision time, is the remaining time before the trainee agent collides with the nearest longitudinal traffic participant.

[0041] In addition, a one-way multi-lane road is a road scenario that only allows vehicles to travel in one direction and includes multiple lanes. One-way multi-lane roads can cover a variety of common road interaction situations, such as vehicles in this lane, vehicles on the left, vehicles on the right, vehicles on the left side of the road, vehicles on the right side of the road, and vehicles on the left and right sides of the road.

[0042] It should be understood that, by combining the motion characteristics of the vehicle's two-degree-of-freedom kinematic model, acceleration and yaw rate can be integrated to determine the set of actions of the trained agent. At this time, the trained agent can choose acceleration and yaw rate as the lateral and longitudinal actions respectively at each decision node. At the same time, the vehicle's longitudinal velocity, lateral velocity, lateral object collision time, and longitudinal object collision time are determined as the set of states of the trained agent.

[0043] For example, the formula for defining the action set A is as follows:

[0044] In the formula, acceleration 'a' is a physical quantity describing how quickly a vehicle's speed changes, including longitudinal acceleration. Lateral acceleration of the vehicle ; This indicates the vehicle's yaw rate.

[0045] The definition formula for the state set S is as follows:

[0046] In the formula, u represents the longitudinal component of the vehicle speed, i.e., the longitudinal speed of the vehicle itself. This represents the lateral component of the vehicle's velocity, i.e., the vehicle's lateral velocity. Indicates the collision time of the longitudinal target object; This indicates the collision time with a lateral target.

[0047] Further, please refer to Figure 2 , Figure 2 This is a schematic diagram of a two-degree-of-freedom vehicle motion model provided in Embodiment 1 of the traffic flow-based simulation testing method of this application. Figure 2 As shown, at time t, the velocity vector of the vehicle's center of mass is decomposed into the longitudinal velocity u along the x-axis and the lateral velocity v along the y-axis in the vehicle coordinate system. After a time interval Δt, at time t+Δt, the vehicle moves along the path, and the vehicle coordinate system rotates by an angle Δθ around the vertical axis. At this time, the vehicle's velocity vector is updated to v1+Δv1.

[0048] The formula for the two-degree-of-freedom kinematic model of a vehicle is as follows:

[0049]

[0050] In the formula, Indicates the longitudinal acceleration of the vehicle; u represents the vehicle's lateral acceleration; u represents the vehicle's longitudinal velocity. Indicates the lateral speed of this vehicle; Indicates the vehicle's yaw rate; This represents the first derivative of u with respect to time t; This represents the first derivative of v with respect to time t.

[0051] Furthermore, the determined set of actions and states are then used as core construction parameters, and a trainable agent with environmental perception and autonomous decision-making capabilities is built by combining a two-degree-of-freedom vehicle kinematic model. Considering the practical needs of reinforcement learning training and the computational efficiency requirements of the model, a one-way multi-lane road is chosen as the road environment for training the trainable agent. Traffic-participating vehicles using a simple action-triggered motion model are deployed in this one-way multi-lane road environment. The trainable agent and all traffic-participating vehicles are placed together within the road environment, integrating road driving rules and vehicle interaction rules to complete the construction of the training environment for the trainable agent.

[0052] Step A20: In the training environment, the weights of the preset expectation function network are updated by gradient descent to obtain the target training model; It should be noted that the expectation value function network is a convolutional neural network in a deep Q-network used to estimate the value of a state and action. It is a computational unit in reinforcement learning training. By adjusting the weight parameters, it outputs the corresponding value of the state and action. The weight parameters of the expectation value function network determine the accuracy of the value estimation.

[0053] Additionally, gradient descent update refers to an optimization method that adjusts network weight parameters by calculating the gradient of the loss function, aiming to minimize the error between the predicted and target values. This allows the expectation function network to gradually approach the optimal value estimation effect. The target training model refers to the expectation function network model after weight parameter optimization, which possesses a stable state-action value estimation capability and can guide the trained agent to output driving actions that meet expectations.

[0054] It should be understood that, firstly, the initial weight parameters of the expectation function network and the weight parameters of the target value function network are initialized in the completed training environment. Then, the actions of the trained agent are executed in the training environment and data samples such as state rewards are collected. After storing the data samples in the historical experience data pool, batch data is extracted to calculate the target value function. The weight parameters of the expectation function network are continuously adjusted through the gradient descent update method to continuously reduce the error between the network prediction value and the target value. When the weight parameters are adjusted to meet the training convergence condition, the optimized target training model is obtained.

[0055] In one possible implementation, step A20 may include: Initialize the historical experience data pool, the expected value function network, and the target value function network; Obtain the expected target reward value output by the expected value function network under the current state; The action or random action corresponding to the target reward expectation value is determined as the current action based on a preset allocation probability. Perform the current action in the training environment to obtain an immediate reward and the next state; The historical experience data pool is updated based on the current state, the current action, the instant reward, and the next state. Training samples are drawn from the updated historical experience data pool; The cumulative discount reward of the target value function network is determined based on the training samples and the expected value function network. The weights of the expected value function network are updated using gradient descent based on the cumulative discount reward to obtain the target training model.

[0056] It should be noted that the historical experience data pool is a container for storing the interaction experience of intelligent agents, used to achieve data reuse and break the temporal correlation of data; the expected value function network is the main network, responsible for estimating the expected value of the reward corresponding to the state-action; the target value function network is an auxiliary network with the same structure as the expected value function network, used to calculate stable training target values. These two types of networks are the core computational units of reinforcement learning.

[0057] Furthermore, the current state represents the environmental information perceived in real time by the trained agent, and the expected target reward is the estimated action gain output by the network. Preset allocation probabilities are used to balance the agent's optimal action utilization with random action exploration, preventing training from getting stuck in local optima. The current action is the final driving operation performed by the agent. After executing the action, the agent receives an immediate reward (the real-time gain of the action) and the next state (the new state after the environment update). The immediate reward and the next state constitute a complete interactive experience, used to update the historical experience data pool. Training samples are small batches of experience data randomly drawn from the data pool, and the cumulative discounted reward is a target reference value combining immediate and long-term gains.

[0058] It should be understood that the initialization of the historical experience data pool, the expected value function network, and the target value function network is completed first to establish the training foundation. Then, the current state of the agent is input into the expected value function network to obtain the expected value of the target reward corresponding to each action. Combined with the preset allocation probability, the current action is selected from the optimal action and random actions. The current action is executed in the training environment to obtain the corresponding immediate reward and the next state. The current state, current action, immediate reward, and next state are integrated into experience data and updated to the historical experience data pool. After that, a small batch of training samples is randomly selected from the updated data pool, and the cumulative discount reward is calculated using the sample data and the target value function network. Using the cumulative discount reward as a reference standard, the weights of the expected value function network are continuously optimized and adjusted through the gradient descent algorithm until the network converges, resulting in the optimized target training model.

[0059] For example, the movement of an agent in traffic flow can be viewed as a sequence of states consisting of a series of decision nodes, i.e., a typical Markov decision process. The formula for expressing a Markov decision process is as follows:

[0060] In the formula, T represents a Markov decision process used to model sequential decision problems, where the outcome is partly random and partly under the control of the decision-maker.<S, A, P, R, γ> A consists of five elements that constitute a complete Markov decision process; S represents the set of states, which includes all discrete or continuous states that the agent may encounter in the traffic flow environment; A represents the set of actions, which includes all possible actions that the agent can take in any given state; P represents the state transition probability matrix, which defines the probability that the agent will transition to the next state s' after taking an action a in a certain state s, i.e., P(s' | s, a) reflects the randomness of the environment; R represents the reward function, which defines the immediate reward R(s, a, s') obtained by the agent after taking action a in state s and transitioning to state s'; γ represents the discount factor, which is a value between 0 and 1 used to measure the importance of future rewards relative to current rewards. A higher γ value indicates that the agent values ​​the long-term total reward more, while a lower γ value indicates that the agent prefers the immediate reward.

[0061] In the specific implementation, the historical experience data pool D is initialized to store data samples generated during training; a convolutional neural network is used as an approximator of the expected value function Q, and the weight parameters θ of the neural network are initialized; the target value function Q' and its weight parameters θ' are initialized, and the termination condition (such as reaching the maximum number of training steps) can be set consistent with Q, and iteration begins; the action corresponding to the maximum Q value output by the current expected value function is executed with probability i, and random actions are executed with probability 1-i. The introduction of random action execution ensures that the agent has a certain degree of exploratory nature during the training process; the selected action is executed in the constructed training environment, and the state s' of the next time step and the immediate reward r(s) after the execution of this action are calculated; the obtained data is then processed.<s,a,r,s'> Add the data to the historical experience data pool D, where s represents the current state, a represents the current action, r represents the immediate reward, and s' represents the next state; randomly sample a batch of data d from the historical experience data pool D; calculate the target Q-value based on the objective function, and update the Q-network using gradient descent to minimize the loss function. ,in, This represents the Q-value predicted by the main Q-network for the current state s and action a; This represents the target Q value calculated using the target value function; The difference between the Q-value predicted by the main Q-network and the target Q-value is measured. This loss is minimized by gradient descent, and the Q-network gradually learns to make its predicted Q-value closer to the target Q-value. When the end-of-training condition is met, the iteration stops, the Q-network model is returned, and the target trained model is obtained.

[0062] For example, the objective function can be expressed as follows:

[0063] In the formula, The target value function is used to calculate the output of the target network for the desired Q value, and is the target value that the target value function approximates. For target value information, it represents the state s i Next, execute action a i Afterwards, the expected cumulative discount rewards; This represents the current state of the i-th sample; This represents the action performed in the i-th sample; Indicates that in state s i Perform action a i The instant reward obtained afterward; This represents the discount factor. It is a value between 0 and 1. This represents the state of the i-th sample at the next time step, i.e., in state s. i Perform action a i Afterwards, the environment transitioned to a new state; The set of all possible actions, representing the next state. The following are all the actions that an intelligent agent can perform; Indicates the next state In this context, the trained agent selects the optimal action from all available actions using the expected value function, which yields the maximum state-action value, and uses that value as the optimal future reward. Q represents the Q-function, which refers to the Q-value estimated by the expected value function.

[0064] Step A30: Iteratively train the trained agent based on the target training model to obtain a traffic flow motion model; It should be noted that iterative training refers to the process of repeatedly executing action collection data to update parameters based on the target training model until the model converges and the behavior of the trained agent conforms to the expected action characteristics. For example, at least one thousand training iterations can be set.

[0065] Furthermore, the traffic flow motion model is a comprehensive control model that integrates the target training model with behavioral rules for agents with different driving styles. It can control trained agents to execute driving actions according to preset styles such as cautious, balanced, and aggressive. The traffic flow motion model is an execution model that generates random traffic flow and can be used to initialize traffic flow density and obtain the proportion of various agents. Expected action features refer to the pre-defined driving behavior characteristics of trained agents, including three types: cautious, balanced, and aggressive. Different types correspond to different vehicle speed and lane-changing behavior rules.

[0066] It should be understood that, firstly, the expected action characteristics of the trained agent are set according to the testing requirements, and a corresponding reward function is designed. The reward function is then combined with the target training model for iterative training. During the iterative training process, the convergence status of the training target is continuously monitored. If convergence is not achieved, the reward function and training parameters are adjusted and the training is retrained. If convergence is achieved and the behavior of the trained agent conforms to the expected action characteristics, the trained agent model is saved. After integrating the trained agent models of various driving styles, a traffic flow motion model that can be used to control traffic participants is obtained.

[0067] In one possible implementation, step A30 may include: Obtain the driving style type of the trained intelligent agent; The safety reward is determined based on whether the trained agent experiences a collision event. The longitudinal motion reward is determined based on the driving style type, the longitudinal speed of the trained agent, and a preset speed statistic. The yaw motion reward is determined based on the number of lane changes made by the trained agent. The target reward function of the trained agent is determined based on the security reward, the longitudinal motion reward, and the lateral motion reward. The target reward function is substituted into the target training model, and the trained agent is iteratively trained to obtain the traffic flow motion model.

[0068] It should be noted that a collision event is an abnormal driving event in which the trained agent comes into contact with other traffic participants or road boundaries in the training environment. Collision events are a criterion for measuring the driving safety of the trained agent; their occurrence indicates a safety defect in the agent's driving behavior. The safety reward is a numerical value used to evaluate the driving safety of the trained agent and is a component of the objective reward function. The value of the safety reward is determined based on whether a collision event occurs, in order to ensure vehicle driving safety and avoid collisions.

[0069] Additionally, longitudinal speed is the longitudinal component of the trained agent's driving speed along the road, representing the speed of the agent's longitudinal movement. The magnitude of the longitudinal speed needs to be consistent with the requirements of the driving style type. Preset speed statistics are normal distribution parameters of vehicle speed obtained by fitting data from actual road conditions, including speed mean and standard deviation information, used to define reasonable intervals for longitudinal speed. Longitudinal motion reward is a reward value that evaluates the reasonableness of the trained agent's longitudinal driving behavior and is an important component of the target reward function. The value of the longitudinal motion reward is determined based on the matching relationship between the driving style type, longitudinal speed, and preset speed statistics, ensuring consistency between the agent's longitudinal speed and acceleration and the sampling interval of the actual statistical results, according to the requirements of the action characteristics.

[0070] Additionally, the number of lane changes is the total number of times the trained agent changes lanes during driving in the training environment. It is a parameter characterizing the frequency of the trained agent's lateral movement, and the number of lane changes needs to be consistent with the yaw motion characteristics of the driving style type. The yaw motion reward is a reward value used to evaluate the rationality of the trained agent's lateral driving behavior and is an important component of the objective reward function. The value of the yaw motion reward is determined based on the matching relationship between the number of lane changes and the driving style type to ensure that the agent's yaw motion change characteristics are consistent with the defined expected action characteristic type.

[0071] In addition, the target reward function is a function that guides the direction of reinforcement learning of the trained agent. The target reward function integrates three evaluation dimensions: safety, longitudinal movement rationality, and lateral movement rationality. The target reward function combines the three reward values ​​into a unified reward feedback through weighted calculation.

[0072] It should be understood that, firstly, the driving style type corresponding to the trained agent is obtained according to the simulation test requirements. Then, the driving status of the trained agent is monitored in real time during the training process. The corresponding safety reward is determined based on whether a collision event occurs. Then, the longitudinal motion reward is determined by combining the driving style type, the longitudinal speed of the trained agent and the preset speed statistics. At the same time, the number of lane changes of the trained agent is counted and the yaw motion reward is determined. Subsequently, the safety reward, longitudinal motion reward and yaw motion reward are weighted and integrated to determine the target reward function of the trained agent. Finally, the target reward function is substituted into the target training model, and the trained agent is iteratively trained in a loop. The training parameters are continuously adjusted until the model converges. After integrating the trained agent models that meet the driving style requirements, the traffic flow motion model is obtained.

[0073] For example, the formula for the target reward function is as follows:

[0074] In the formula, r sIndicates a security reward, r v Represents longitudinal motion reward, r y w1 represents the weight of the lateral motion reward; w2 represents the weight of the longitudinal motion reward; and w3 represents the weight of the lateral motion reward.

[0075] In practical implementation, it is considered that cautious vehicles have lower speeds and generally do not change lanes, balanced vehicles have moderate speeds and generally change lanes normally, and aggressive vehicles have higher speeds and frequently change lanes. Regarding the safety bonus, when no collision occurs, the safety bonus value can be set to a preset default value, such as 0; when a collision occurs, the safety bonus value can be set to a preset negative value, such as -1, and the preset negative value must be less than the preset default value.

[0076] For longitudinal motion rewards, the preset statistical interval is (μ-σ, μ+σ), where μ is the mean of the normal distribution fitted to the actual vehicle speed, and σ is the standard deviation of the normal distribution fitted to the actual vehicle speed. When the speed v is less than the lower limit of the preset statistical interval, the longitudinal motion reward value for a cautious driving style is determined to be 1, and the longitudinal motion reward value for a balanced or aggressive driving style is determined to be -|v-μ|. When the speed v is within the preset statistical interval, the longitudinal motion reward value for a balanced driving style is determined to be 1, and the longitudinal motion reward value for a cautious or aggressive driving style is determined to be -|v-μ|. When the speed v is greater than the upper limit of the preset statistical interval, the longitudinal motion reward value for an aggressive driving style is determined to be 1, and the longitudinal motion reward value for a cautious or balanced driving style is determined to be -|v-μ|.

[0077] For yaw motion rewards, when the trained agent does not change lanes within a preset time period, the yaw motion reward value is determined to be 1 for the cautious driving style, -2 for the balanced driving style, and -3 for the aggressive driving style. When the trained agent changes lanes within the preset time period and the number of lane changes n is less than or equal to 2, the yaw motion reward value is determined to be -1000 for the cautious driving style, n for the balanced driving style, and -3 for the aggressive driving style. When the trained agent changes lanes within the preset time period and the number of lane changes n is greater than 2, the yaw motion reward value is determined to be -1000 for the cautious driving style, -n for the balanced driving style, and 1 for the aggressive driving style.

[0078] Step A40: Determine the proportion of trained agents of different driving styles based on the traffic flow motion model.

[0079] It should be understood that, based on the specific objectives and scenario requirements of autonomous driving simulation testing, and combined with the proportional values ​​corresponding to the driving style types currently configured in the traffic flow motion model, the proportion of trained intelligent agents of different driving style types in the overall simulated traffic participants is determined.

[0080] Step S20: Determine the disappearance distance threshold and appearance distance threshold between the trained agent and the tested vehicle based on the perception range of the tested vehicle. It should be noted that the vehicle under test is the main vehicle used to verify the performance of autonomous driving functions in the autonomous driving simulation test. The operating state of the vehicle under test is controlled by the algorithm under test, and driving interaction and function verification are completed in random traffic flow.

[0081] In addition, the sensing range is the spatial distance range within which the various sensors on the vehicle under test can effectively detect and identify surrounding traffic participants. This range is determined by the hardware performance parameters of the sensors and the scenario setting parameters of the simulation test.

[0082] Additionally, the vanishing distance threshold is the maximum effective interaction distance between the trained agent and the vehicle under test. This threshold is used to remove trained agents that are more than the distance to the vehicle under test during simulation testing. When the actual distance between the trained agent and the vehicle under test exceeds this value, the trained agent will be removed to reduce the resource consumption of the simulation computation.

[0083] Additionally, the occurrence distance threshold is the minimum safe generation distance between the trained agent and the vehicle under test, while the disappearance distance threshold is used to prevent the addition of new trained agents within areas where the distance to the vehicle under test is greater than the occurrence distance threshold during simulation testing. This avoids the sudden appearance of trained agents near the vehicle under test, which could affect the realism of the simulation scenario. New trained agents can only be generated within areas where the distance to the vehicle under test is greater than or equal to the occurrence distance threshold and less than or equal to the disappearance distance threshold.

[0084] Furthermore, the vanishing distance threshold can be determined by combining the computing power of the simulation equipment and the sensing range of the vehicle under test. The vanishing distance threshold should not exceed the maximum distance that the sensors on the vehicle under test can detect, and the occurrence distance threshold should be less than the vanishing distance threshold. For example, the vanishing distance threshold could be 500 meters, and the occurrence distance threshold could be 50 meters.

[0085] It should be understood that, firstly, the specific parameters of the perception range of the vehicle under test in the simulation scenario are collected and determined. Then, combined with the computing power of the simulation computing equipment, the corresponding distance values ​​are set based on the perception range parameters, and the disappearance distance threshold and appearance distance threshold between the trained agent and the vehicle under test are determined.

[0086] Step S30: Determine the initial vehicle speed of the trained agent based on the current speed and relative position type of the tested vehicle; It should be noted that the current speed of the vehicle under test is the real-time driving speed of the vehicle under test at the current moment during the simulation test. This value is calculated and output in real time by the dynamics simulation module in the simulation test.

[0087] Additionally, the relative position type refers to the relative orientation category between the trained agent and the tested vehicle on the simulated road. The relative position type can include at least the front and rear of the tested vehicle, and can also include the lane adjacent to the left and the lane adjacent to the right of the tested vehicle.

[0088] In addition, the initial vehicle speed is the starting speed value of the trained agent when it is generated in the simulation scenario. The initial vehicle speed needs to conform to the statistical distribution of vehicle speed on real roads, and at the same time, it needs to be adapted to the real-time driving status of the tested vehicle.

[0089] It should be understood that the real-time driving speed of the vehicle under test is first obtained from the dynamics simulation module to obtain the current speed of the vehicle under test. Then, the specific orientation type of each trainable agent relative to the vehicle under test is determined. The relative position type and the current speed of the vehicle under test are used as the basis for calculation to determine the initial speed of the corresponding trainable agent.

[0090] In one feasible implementation, step S30 may include steps S31 to S34: Step S31: Determine the speed increment based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the maximum allowable speed threshold; and determine the pursuit simulation speed of the trained agent based on the current speed of the tested vehicle and the speed increment. It should be noted that the speed increment is an adjustment value used to increase the speed of the trained agent, simulating the chasing state of the trained agent approaching the test vehicle. The specific magnitude of the speed increment is calculated jointly by the distance between the test vehicle and the trained agent, the disappearance distance threshold, and the maximum allowable speed threshold. The chasing simulation speed is the speed value of the traffic vehicle behind the test vehicle in a chasing state, which is formed by adding the speed increment to the current speed of the test vehicle, and is used to reflect the real driving characteristics of the following vehicle chasing the test vehicle in front.

[0091] In addition, the maximum permissible speed threshold is the maximum speed limit set for the trained intelligent agent in the simulated road environment. This value is determined with reference to the speed limit standards of real roads and the requirements of the simulation test scenario. The speed of the trained intelligent agent must not exceed this limit.

[0092] It should be understood that when randomly generating the trainee agent behind the test vehicle, the distance between the test vehicle and the trainee agent is obtained. Then, the distance between the test vehicle and the trainee agent, the disappearance distance threshold, and the maximum allowable speed threshold are substituted into a preset traffic vehicle speed initialization model to calculate the speed increment. Subsequently, the current speed of the test vehicle is obtained, and the current speed of the test vehicle is added to the speed increment to obtain the simulated chasing speed of the trainee agent.

[0093] Step S32: Determine the speed reduction based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the current speed of the tested vehicle; and determine the simulated speed at which the trained agent is being chased based on the current speed of the tested vehicle and the speed reduction. It should be noted that the speed reduction is an adjustment value used to decrease the speed of the trained agent, simulating the driving state of the trained agent being chased by the test vehicle. The specific magnitude of the speed reduction is calculated jointly by the distance between the test vehicle and the trained agent, the disappearance distance threshold, and the current speed of the test vehicle. The simulated speed of being chased is the speed value of the traffic vehicle in front of the test vehicle that the trained agent is chasing. This value is formed by subtracting the speed reduction from the current speed of the test vehicle, and is used to reflect the realistic driving characteristics of the vehicle in front being chased by the test vehicle behind.

[0094] It should be understood that when randomly generating the trainee agent in front of the test vehicle, the distance between the test vehicle and the trainee agent is obtained. Then, the distance between the test vehicle and the trainee agent, the disappearance distance threshold, and the current speed of the test vehicle are substituted into a preset traffic vehicle speed initialization model to calculate the speed reduction. Subsequently, the current speed value of the test vehicle is obtained, and the speed reduction is subtracted from the current speed of the test vehicle to obtain the simulated speed of the trainee agent being chased.

[0095] Step S33: When the relative position type between the trained agent and the tested vehicle is rear, determine the initial vehicle speed of the trained agent based on the chase simulation speed and the maximum allowable speed threshold. It should be noted that the relative position type is "behind," meaning that the trainee agent is positioned behind the test vehicle in the simulated road space. In this position, the core driving behavior of the trainee agent is to catch up with the test vehicle in front.

[0096] It should be understood that when the relative position type between the randomly generated training agent and the tested vehicle is determined to be "behind", the simulated speed value and the preset maximum permissible speed threshold value are first retrieved. The simulated speed and the maximum permissible speed threshold are compared and constrained, and the smaller of the simulated speed and the maximum permissible speed threshold is determined as the initial speed of the training agent.

[0097] Step S34: When the relative position type between the trained agent and the tested vehicle is forward, determine the initial vehicle speed of the trained agent based on the simulated speed being chased and the minimum allowable speed threshold.

[0098] It should be noted that the relative position type is "ahead", meaning that the trainee agent is positioned in front of the test vehicle in the simulated road space. Under this position, the core driving behavior of the trainee agent is to be overtaken by the test vehicle behind.

[0099] In addition, the minimum permissible speed threshold is the minimum driving speed limit set for the trained agent in the simulated road environment. This value is determined with reference to the driving regulations of real roads and the requirements of the simulation test scenario. The driving speed of the trained agent must not be lower than this limit.

[0100] It should be understood that when the relative position type between the randomly generated training agent and the tested vehicle is determined to be "ahead," the simulated speed value of the being chased is first retrieved and compared with the preset minimum allowable speed threshold value. The simulated speed of the being chased is compared with the minimum allowable speed threshold, and the smaller of the two values ​​is determined as the initial speed of the training agent.

[0101] For example, the formula for the vehicle speed initialization model is as follows:

[0102] In the formula, the vehicle is the object simulated by the trained intelligent agent, and v represents the initial speed of the vehicle; d represents the current speed of the vehicle being tested; d represents the distance between the trained agent and the vehicle being tested. This indicates the disappearance distance threshold, such as 500 (unit: m). This indicates the maximum speed threshold allowed for the initial speed of the vehicle, i.e., the maximum permissible speed threshold, which can be set to 130 (unit: km / h). This represents the minimum speed threshold allowed for the initial speed of the vehicle, i.e., the minimum permissible speed threshold, which can be set to 20 (unit: km / h).

[0103] In this implementation, by distinguishing between chasing and being chased states and calculating speed increments and decrements separately, the speed adjustment logic of the trained agent can be made to closely resemble the interaction behavior of vehicles on real roads, improving the rationality of speed settings. Constraining the initial vehicle speed by combining the maximum and minimum permissible speed thresholds can prevent the trained agent from exhibiting abnormal speeds, ensuring that the simulated traffic flow conforms to real-world regulations. Determining the initial vehicle speed based on differences in relative position types allows the trained agent to form a natural driving interaction with the tested vehicles, further enhancing the realism of random traffic flow and providing a more realistic scenario basis for subsequent simulation tests.

[0104] Step S40: Generate random traffic flow based on the quantity ratio information, the disappearance distance threshold, the appearance distance threshold, and the initial vehicle speed; It should be noted that the traffic flow motion model is a comprehensive behavior control model that integrates the motion rules of the trained agent, the interaction rules of the road environment, and the results of reinforcement learning. The traffic flow motion model can accurately control the trained agent to perform driving actions according to the preset driving style.

[0105] In addition, random traffic flow is a dynamic set of traffic participants composed of trained agents with different driving styles. It can be generated by training reinforcement learning algorithms such as deep Q-networks. The behavioral characteristics, density distribution and speed status of random traffic flow closely resemble the traffic operation status of real roads. Random traffic flow is the interactive object for simulation testing of the vehicle under test.

[0106] It should be understood that the quantity ratio information, disappearance distance threshold, appearance distance threshold, and initial vehicle speed are uniformly imported into the traffic flow motion model as configuration parameters. The traffic flow motion model completes the type allocation, quantity generation, location deployment, and speed setting of the trained intelligent agents according to the configuration parameters, and generates random traffic flow that meets the simulation test requirements through dynamic behavior control.

[0107] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a random traffic flow scenario provided in Embodiment 1 of the traffic flow-based simulation testing method of this application. Figure 3 As shown, SV represents the vehicle under test, which is the main vehicle used to test and verify the autonomous driving system. In the simulation test, all traffic flows and scenarios are built around the vehicle under test; EP represents other traffic participants, i.e., the trained intelligent agents.

[0108] Step S50: Perform simulation testing on the vehicle under test based on the random traffic flow.

[0109] It should be noted that simulation testing is a complete testing process for verifying the autonomous driving function of the vehicle under test in a closed-loop simulation environment. Simulation testing simulates real-world driving interaction scenarios, comprehensively testing the vehicle's driving safety, decision-making rationality, and functional robustness in random traffic flow. The closed-loop simulation environment integrates scenario simulation modules, sensor simulation modules, dynamics simulation modules, random traffic flow generation modules, and traffic flow recording modules, enabling real-time interaction of test data and complete reproduction of test scenarios.

[0110] It should be understood that the generated random traffic flow is completely imported into the closed-loop simulation environment, enabling the vehicle under test to drive autonomously in the dynamic simulation scenario composed of random traffic flow. Real-time data on the driving status of the vehicle under test, its relative position data with the trained intelligent agent, and collision risk data are collected. Based on the collected data, the simulation test of the autonomous driving function of the vehicle under test is completed.

[0111] This embodiment provides a traffic flow-based simulation testing method. By acquiring the proportion of trained agents with different driving styles, the behavioral characteristics of traffic flow can be configured, ensuring that the simulated traffic flow accurately reflects the differences in driving behavior on real roads. Determining the disappearance and reappearance distance thresholds based on the perception range of the tested vehicle effectively reduces computational resource consumption while ensuring the realism of the simulation scenario. The initial speed of the trained agent is determined based on the current speed and relative position type of the tested vehicle, ensuring real-time matching of the traffic flow speed state with the tested vehicle, thus improving the rationality of the simulation scenario. Generating random traffic flow by combining various core parameters and traffic flow motion models allows for the rapid construction of complex and realistic test scenarios. Simulation testing based on random traffic flow comprehensively verifies the autonomous driving function of the tested vehicle in complex traffic environments, while improving the realism, execution efficiency, and reproducibility of the results.

[0112] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 Step S50 may include steps S51 to S54: Step S51: Respond to the control command of the module under test and generate motion state information of the vehicle under test; It should be noted that the module under test (DUT) is the real control unit or virtual algorithm model whose functional performance needs to be verified in autonomous driving simulation testing. It is the object of the simulation test, collecting various environmental perception information, completing decision calculations, and then outputting corresponding vehicle control signals. Control commands are vehicle driving control signals output by the DUT after information processing and decision calculations, including acceleration control signals, braking control signals, steering control signals, etc. These control commands determine the driving behavior and state change trends of the DUT. The motion state information of the DUT is a set of various parameters characterizing the real-time driving state of the DUT, including its position, attitude, speed, and acceleration information, reflecting the driving situation of the DUT.

[0113] It should be understood that the processing unit used to simulate vehicle dynamics response receives control commands output from the module under test and parses and executes these commands according to the dynamics response rules of a real vehicle. This processing unit simulates the actual driving response process of the vehicle under test after receiving the control commands, and through numerical calculations and state deduction, generates motion state information of the vehicle under test, including position, attitude, velocity, and acceleration, thus completing the generation operation of the vehicle under test's motion state information.

[0114] Step S52: Determine the traffic participant status information of the tested vehicle based on the motion state of the tested vehicle and the random traffic flow; It should be noted that traffic participant state information is a set of parameters that characterize the real-time driving state of each trained agent in a random traffic flow. It includes the position information, attitude information, vehicle speed information, driving intention information, etc. of the trained agent, and is used to determine the real-time driving situation of all traffic participants around the tested vehicle in order to construct a dynamic interactive scenario.

[0115] It should be understood that the process begins by retrieving the already generated motion state information of the tested vehicle, while simultaneously acquiring the overall operational data of the constructed random traffic flow. Using the motion state of the tested vehicle as the benchmark for traffic interaction, and combining it with the preset driving rules and relative positional relationships of each trained agent in the random traffic flow, the real-time position, attitude, speed, and driving intention of each traffic participant are calculated and determined one by one. Finally, the state data of all traffic participants are integrated to obtain complete traffic participant state information for the tested vehicle.

[0116] Step S53: Generate a dynamic scene based on the motion state information of the tested vehicle and the state information of the traffic participants; It should be noted that dynamic scenes are simulated scene images that include static road elements and dynamic driving elements, presented by the rendering engine. They integrate static scene information such as roads, lanes, ground markings, road facilities, trees and buildings, as well as dynamic driving information such as the real-time movement status of the tested vehicle and traffic participants. They can completely restore the driving interaction environment of real roads and provide a visual scene foundation for sensor simulation.

[0117] It should be understood that the motion state information of the vehicle under test and the state information of traffic participants are synchronously transmitted to the processing unit responsible for scene rendering. First, the pre-drawn static road scene elements are loaded, and then the real-time motion state data of the vehicle under test and traffic participants are accurately superimposed onto the static scene. The rendering engine integrates and renders the static scene elements and dynamic motion elements to generate a dynamic scene that can reflect the vehicle's driving interaction in real time.

[0118] Step S54: Process the dynamic scene to generate simulation sensor data, and send the simulation sensor data to the module under test so that the module under test can update the control command of the vehicle under test based on the simulation sensor data and perform simulation testing.

[0119] It should be noted that simulated sensor data refers to environmental data collected by perception devices onboard an autonomous vehicle, including visual sensors, millimeter-wave radar sensors, and lidar sensors. The format of simulated sensor data is consistent with that of real sensors. Simulated sensor data contains target location information, distance information, image information, point cloud information, etc., and serves as input data for the module under test (DUT) to perform environmental perception and driving decisions, thus influencing the control command output of the DUT.

[0120] It should be understood that the processing unit responsible for sensor simulation receives the generated dynamic scene and extracts, processes, and repackages the image information, point cloud information, distance information, and other content in the dynamic scene. This processing unit generates corresponding simulated sensor data according to the data format specifications of the real sensor, and then sends the simulated sensor data to the module under test (DUT) via a specified transmission protocol. After receiving the simulated sensor data, the DUT performs environmental perception and decision calculations, and updates and outputs new control commands based on the perception results, thus forming a closed-loop operation process to continuously complete the simulation test of the DUT.

[0121] Please refer to Figure 5 , Figure 5 This is a schematic diagram of module interaction provided in Embodiment 2 of the traffic flow-based simulation testing method of this application. Figure 5 As shown, the random traffic flow generation module generates random traffic flow data which is input into the scenario simulation module. The traffic flow recording module can provide traffic flow playback data to the scenario simulation module. The dynamic scenario generated by the scenario simulation module is input into the sensor simulation module. The simulation sensor data generated by the sensor simulation module is input into the module under test. The module under test and the dynamics simulation module interact bidirectionally with each other to exchange control commands and vehicle state data. The motion state information of the vehicle under test output by the dynamics simulation module is input into the scenario simulation module. At the same time, the dynamics simulation module feeds back the vehicle state data to the random traffic flow generation module, thus forming a complete closed-loop simulation test process to realize the continuous verification of autonomous driving functions based on random traffic flow.

[0122] In one feasible implementation, the method may further include steps B10 to B60: Step B10: Obtain scene parameter information, traffic flow configuration information, algorithm version information, and number of repeated executions to obtain test task information; It should be noted that the scene parameter information is a complete set of parameters that define the autonomous driving simulation test scene, including map information, weather information, lighting information, etc. It determines the static environment and environmental state of the simulation test and is the basic data for constructing the simulation test scene.

[0123] In addition, traffic flow configuration information is a set of parameters that set the composition and operation status of random traffic flow. It includes the proportion of trained agents with different driving styles, the initial density of traffic flow, the disappearance distance threshold, the appearance distance threshold, etc., which determine the behavioral characteristics and distribution status of random traffic flow.

[0124] In addition, the version information of the algorithm under test is the information that identifies the iteration version of the autonomous driving control algorithm in the module under test, distinguishes the algorithm program at different development stages and optimization levels. Different versions of the algorithm have different decision logic and control effects, and serve as the object identifier for simulation testing.

[0125] In addition, the number of repetitions is a value that sets the number of times the simulation test task is run repeatedly. It is set according to the robustness verification requirements of autonomous driving functions. The more times it is repeated, the more comprehensively the stability of the algorithm can be verified in random traffic flow.

[0126] In addition, test task information is a complete set of test data that integrates scenario parameter information, traffic flow configuration information, the version information of the algorithm under test, and the number of repeated executions. It is the core task basis for cloud-based distributed testing and is used to determine the execution conditions and requirements of simulation testing.

[0127] It should be understood that the test management platform receives user-inputted scenario parameter information, traffic flow configuration information, tested algorithm version information, and repeated execution count information through its web interface. The platform then standardizes and integrates these four types of information to form test task information containing all test execution elements, thus completing the acquisition and generation of test task information.

[0128] Step B20: Divide the test task information into multiple simulation subtasks and insert the multiple simulation subtasks into the message queue; It should be noted that a simulation subtask is an independent simulation test unit formed by breaking down the overall test task information. Each simulation subtask contains complete test execution parameters and is assigned a unique random seed. Simulation subtasks can run independently and complete the corresponding test work. Executing multiple simulation subtasks in parallel can greatly improve the overall test efficiency.

[0129] Additionally, the message queue is a message transmission component for temporarily storing and scheduling simulation subtasks. It manages simulation subtasks using a first-in, first-out (FIFO) scheduling rule, enabling asynchronous scheduling and buffering of simulation subtasks. The message queue ensures the stability and orderliness of task distribution in distributed testing. The splitting operation is the process of dividing the overall test task into a corresponding number of independent simulation subtasks based on the number of repetitions and parallel testing requirements. Each split simulation subtask has independent execution parameters. The insertion operation is the process of adding multiple generated simulation subtasks sequentially to the message queue. The insertion operation follows the message queue's scheduling rules, ensuring that simulation subtasks wait for scheduling and execution in an orderly manner.

[0130] It should be understood that the task scheduler reads the generated test task information, and according to the repetition count information and parallel testing requirements in the test task information, it breaks down the overall test task into a corresponding number of independent simulation subtasks, assigns a unique random seed information to each simulation subtask, and then inserts all simulation subtasks into the message queue in a preset order to complete the splitting and enqueueing operation of simulation subtasks.

[0131] Step B30: Determine the number of worker node instances based on the number of unprocessed simulation subtasks in the message queue, and create a dynamic worker node cluster; It should be noted that the number of simulation subtasks to be processed is the total number of simulation subtasks waiting to be scheduled for execution in the message queue. This reflects the task load of distributed testing; the more simulation subtasks to be processed, the more computing resources are required. A worker node instance is an independent computing unit that executes simulation subtasks. It runs in a containerized manner, hosting the simulation closed-loop environment and test programs. The worker node instance is the core execution unit of distributed testing.

[0132] Additionally, a dynamic worker node cluster is a computing cluster composed of multiple elastically scalable worker node instances. Instance creation and destruction are implemented based on a container orchestration management platform. The number of instances can be automatically adjusted according to task load, balancing computing resources and testing efficiency. The creation operation is the process of generating a corresponding number of worker node instances from cloud computing resources based on a determined number of worker node instances. This creation operation is completed using the container orchestration management platform, ensuring standardized deployment of worker node instances.

[0133] It should be understood that the container orchestration management platform monitors the number of pending simulation subtasks in the message queue in real time. Based on the number of pending simulation subtasks and the preset single instance processing capacity, it calculates the required number of worker node instances, creates the corresponding worker node instances in the cloud according to the calculated number, and integrates all worker node instances to form a dynamic worker node cluster, thus completing the creation of the dynamic worker node cluster.

[0134] Step B40: When the worker node instance starts, retrieve the simulation subtask from the message queue; It should be noted that worker node instance startup is the process from the completion of worker node instance creation to loading the runtime environment and having the ability to execute tasks. The worker node instance startup process will complete operations such as loading container images and configuring the runtime environment. After the worker node instance is started, it can actively apply for simulation subtasks.

[0135] Additionally, retrieving simulation subtasks involves worker node instances retrieving unprocessed simulation subtasks from the message queue. This retrieval operation follows the message queue's scheduling rules, with each worker node instance retrieving one simulation subtask at a time. This retrieval operation ensures accurate matching between tasks and execution units. A simulation subtask is an independent simulation test unit containing all test parameters. After retrieving a simulation subtask, a worker node instance can execute the corresponding test process.

[0136] It should be understood that when a worker node instance in a dynamic worker node cluster completes startup and enters the ready state, the worker node instance actively sends a task retrieval request to the message queue. After receiving the request, the message queue distributes the simulation subtasks to be processed to the corresponding worker node instances according to the first-in-first-out rule. The worker node instances then retrieve the simulation subtasks and prepare to execute the test.

[0137] Step B50: Load the traffic flow motion model from the model repository based on the task parameter information of the simulation subtask; It should be noted that the task parameter information includes all test execution parameters contained in the simulation subtask, such as scene parameter information, traffic flow configuration information, and random seed information. This information is the core basis for loading the corresponding model and executing the test. The model repository is a storage component that stores and manages trained traffic flow motion models. The model repository categorizes and stores traffic flow motion models with different driving styles, such as cautious, balanced, and aggressive. It provides fast model retrieval and loading services, ensuring efficient model access.

[0138] In addition, the loading operation is the process of retrieving and calling the corresponding traffic flow motion model from the model repository based on the task parameter information. The loading operation imports the model into the runtime environment of the working node instance, enabling the model to have the conditions for real-time operation.

[0139] It should be understood that the working node instance parses the simulation subtask obtained, extracts the task parameter information, determines the required traffic flow motion model type based on the traffic flow configuration information in the task parameter information, sends a model loading request to the model repository, the model repository retrieves the corresponding model according to the request and transmits it to the working node instance, and the working node instance loads the traffic flow motion model into the local runtime environment.

[0140] Step B60: Initialize random traffic flow based on the traffic flow motion model, and conduct simulation tests on the vehicle under test based on the random traffic flow.

[0141] It should be noted that initializing random traffic flow is the process of generating initial state random traffic flow based on traffic flow configuration information and traffic flow motion model. It includes operations such as type allocation, quantity setting, location deployment, and initial vehicle speed setting of trained intelligent agents, providing a standard initial interaction scenario for simulation testing.

[0142] It should be understood that the working node instance calls the loaded traffic flow motion model, and based on the traffic flow configuration information in the simulation subtask, completes the allocation of the number of trained intelligent agents, the type ratio, the location deployment and the initial vehicle speed setting, realizes the initialization of random traffic flow, and imports the initialized random traffic flow into the closed-loop simulation environment, so that the test vehicle can drive autonomously in the random traffic flow and conduct simulation tests of the autonomous driving function.

[0143] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the simulation test architecture provided in Embodiment 2 of the traffic flow-based simulation test method of this application. This distributed autonomous driving simulation test architecture consists of four layers: a test task definition layer, a distributed scheduling and core service layer, a dynamic simulation execution cluster, and a data storage layer. In the test task definition layer, users configure test parameters and generate test tasks through the web user interface (Web UI) of the test management platform. The tasks are then submitted to the task scheduler of the distributed scheduling and core service layer through the application programming interface gateway (API Gateway). The task scheduler orchestrates tasks and distributes simulation tasks through the web user interface of the test management platform. Simultaneously, it calls the scene management service to obtain static scenes from the scene library and calls the traffic flow generation service to read trained intelligent models from the model repository. The distributed simulation tasks enter the container cluster (C_Cluster) of the dynamic simulation execution cluster. This cluster contains multiple simulation worker containers (WorkerPods, where a Pod is the smallest deployable computing unit in Kubernetes). Each simulation work container executes a single Worker task process, sequentially pulling task parameters, loading the simulation closed-loop system, integrating dynamic traffic flow, performing simulation tests, and collecting results and logs. The dynamic simulation execution cluster uploads test results to object storage to store results and logs, and updates the status to a relational database to store task status and metadata. The modules at all levels work together to realize the complete execution process of distributed autonomous driving simulation testing.

[0144] This embodiment provides a traffic flow-based simulation testing method. By generating motion state information of the vehicle under test in response to control commands from the module under test, it can accurately simulate the real dynamic response of the vehicle under test, ensuring the authenticity and rationality of the vehicle state in the simulation test. Based on the motion state of the vehicle under test and random traffic flow, the method determines the state information of traffic participants, which can determine the driving state of surrounding traffic participants in real time and construct a natural and realistic vehicle interaction relationship. Combining the two types of state information to generate dynamic scenes can completely restore the real road driving environment, improving the realism and credibility of the simulation scene. The dynamic scene is processed to generate simulation sensor data and sent back to the module under test, forming a complete closed-loop simulation testing process. This allows the module under test to complete decision-making and control under near-real perception data, effectively improving the accuracy of autonomous driving simulation testing.

[0145] For example, to help understand the implementation process of the traffic flow-based simulation testing method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 7 , Figure 7 A simplified flowchart of a traffic flow-based simulation testing method is provided, specifically: Design the motion rules and traffic environment of the trained agent and determine the state set and action set. Construct a reinforcement learning training model method, design the expected result and corresponding reward function, and perform iterative training. Then, determine whether the model has converged. If it has not converged, return to the steps of constructing the reinforcement learning training model and retrain. If it has converged, combine the training results into a trained traffic flow model according to the testing purpose. Then, integrate the traffic flow model into the cloud testing system and execute simulation testing. Determine whether the simulation test passes. If it fails and the reason is that the traffic flow settings are unreasonable, return to the steps of constructing a trained traffic flow model according to the testing purpose to adjust the model. If it fails and the reason is that the tested object has missing functions, return to the steps of pre-testing to check the functional problems of the tested object. If the test passes, the process ends.

[0146] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the traffic flow-based simulation test method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0147] This application also provides a traffic flow-based simulation testing device, please refer to... Figure 8 The traffic flow-based simulation testing device includes: Data acquisition module 10 is used to acquire information on the proportion of trained intelligent agents of different driving styles; Data processing module 20 is used to determine the disappearance distance threshold and appearance distance threshold between the trained intelligent agent and the tested vehicle based on the perception range of the tested vehicle. The data processing module 20 is also used to determine the initial vehicle speed of the trained intelligent agent based on the current speed and relative position type of the tested vehicle; Data generation module 30 is used to generate random traffic flow based on the quantity ratio information, the disappearance distance threshold, the appearance distance threshold and the initial vehicle speed; The simulation test module 40 is used to perform simulation tests on the vehicle under test based on the random traffic flow.

[0148] In one embodiment, the data processing module 20 is further configured to determine a speed increment based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the maximum allowable speed threshold, and to determine the simulated chasing speed of the trained agent based on the current speed of the tested vehicle and the speed increment. The speed reduction is determined based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the current speed of the tested vehicle. The simulated speed at which the trained agent is being chased is determined based on the current speed of the tested vehicle and the speed reduction. When the relative position type between the trained agent and the tested vehicle is rear, the initial speed of the trained agent is determined based on the simulated chasing speed and the maximum permissible speed threshold. When the relative position type between the trained agent and the tested vehicle is "ahead", the initial speed of the trained agent is determined based on the simulated speed being chased and the minimum allowable speed threshold.

[0149] In one embodiment, the data processing module 20 is further configured to construct a training environment for the trained intelligent agent based on the traffic-participating vehicle and the road environment; In the training environment, the weights of the preset expectation function network are updated using gradient descent to obtain the target training model; Based on the target training model, the trained agent is iteratively trained to obtain a traffic flow motion model; Based on the traffic flow motion model, the number and proportion of trained agents of different driving styles are determined.

[0150] In one embodiment, the data processing module 20 is further configured to determine the acceleration and yaw rate as a set of actions; Define the vehicle's longitudinal velocity, lateral velocity, collision time with the lateral target, and collision time with the longitudinal target as a set of states; The trained agent is constructed based on the set of actions and the set of states; Define one-way multi-lane roads as the road environment; The training environment for the trained agent is constructed based on the traffic-participating vehicle and the road environment.

[0151] In one embodiment, the data processing module 20 is further configured to initialize a historical experience data pool, an expected value function network, and a target value function network; Obtain the expected target reward value output by the expected value function network under the current state; The action or random action corresponding to the target reward expectation value is determined as the current action based on a preset allocation probability. Perform the current action in the training environment to obtain an immediate reward and the next state; The historical experience data pool is updated based on the current state, the current action, the instant reward, and the next state. Training samples are drawn from the updated historical experience data pool; The cumulative discount reward of the target value function network is determined based on the training samples and the expected value function network. The weights of the expected value function network are updated using gradient descent based on the cumulative discount reward to obtain the target training model.

[0152] In one embodiment, the data processing module 20 is further configured to obtain the driving style type of the trained intelligent agent; The safety reward is determined based on whether the trained agent experiences a collision event. The longitudinal motion reward is determined based on the driving style type, the longitudinal speed of the trained agent, and a preset speed statistic. The yaw motion reward is determined based on the number of lane changes made by the trained agent. The target reward function of the trained agent is determined based on the security reward, the longitudinal motion reward, and the lateral motion reward. The target reward function is substituted into the target training model, and the trained agent is iteratively trained to obtain the traffic flow motion model.

[0153] In one embodiment, the simulation test module 40 is further configured to respond to the control commands of the module under test and generate motion state information of the vehicle under test; The traffic participant status information of the tested vehicle is determined based on the motion state of the tested vehicle and the random traffic flow. A dynamic scene is generated based on the motion state information of the tested vehicle and the state information of the traffic participants. The dynamic scene is processed to generate simulated sensor data, which is then sent to the module under test (DUT) so that the DUT can update the control commands of the vehicle under test based on the simulated sensor data, thereby performing simulation testing.

[0154] In one embodiment, the simulation test module 40 is further used to acquire scene parameter information, traffic flow configuration information, the version information of the algorithm under test, and the number of repeated executions to obtain test task information; The test task information is split into multiple simulation subtasks, and the multiple simulation subtasks are inserted into a message queue; The number of worker node instances is determined based on the number of pending simulation subtasks in the message queue, and a dynamic worker node cluster is created. When the worker node instance starts, a simulation subtask is retrieved from the message queue; Based on the task parameter information of the simulation sub-task, the traffic flow motion model is loaded from the model repository; A random traffic flow is initialized based on the traffic flow motion model, and the vehicle under test is simulated based on the random traffic flow.

[0155] The traffic flow-based simulation testing device provided in this application, employing the traffic flow-based simulation testing method described in the above embodiments, can solve the technical problem of how to improve the testing capability of autonomous driving functions in complex scenarios. Compared with the prior art, the beneficial effects of the traffic flow-based simulation testing device provided in this application are the same as those of the traffic flow-based simulation testing method provided in the above embodiments, and other technical features in the traffic flow-based simulation testing device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0156] This application provides a traffic flow-based simulation test device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the traffic flow-based simulation test method in Embodiment 1 above.

[0157] The following is for reference. Figure 9 The diagram illustrates a structural schematic of a traffic flow-based simulation test device suitable for implementing embodiments of this application. The traffic flow-based simulation test device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9The traffic flow-based simulation test equipment shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0158] like Figure 9 As shown, the traffic flow-based simulation test device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the traffic flow-based simulation test device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the traffic flow-based simulation test equipment to communicate wirelessly or wiredly with other devices to exchange data. Although a traffic flow-based simulation test equipment with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0159] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0160] The traffic flow-based simulation testing equipment provided in this application, employing the traffic flow-based simulation testing method described in the above embodiments, can solve the technical problem of how to improve the testing capability of autonomous driving functions in complex scenarios. Compared with the prior art, the beneficial effects of the traffic flow-based simulation testing equipment provided in this application are the same as those of the traffic flow-based simulation testing method provided in the above embodiments, and other technical features of this traffic flow-based simulation testing equipment are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0161] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0163] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the traffic flow-based simulation test method in the above embodiments.

[0164] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), or flash memory, optical fiber, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0165] The aforementioned computer-readable storage medium may be included in a traffic flow-based simulation test device; or it may exist independently and not assembled into a traffic flow-based simulation test device.

[0166] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a traffic flow-based simulation testing device, cause the traffic flow-based simulation testing device to: acquire information on the proportion of trained agents of different driving styles; determine the disappearance distance threshold and appearance distance threshold between the trained agent and the tested vehicle based on the perception range of the tested vehicle; determine the initial speed of the trained agent based on the current speed and relative position type of the tested vehicle; generate a random traffic flow based on the proportion information, the disappearance distance threshold, the appearance distance threshold, and the initial speed; and perform simulation testing on the tested vehicle based on the random traffic flow.

[0167] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0169] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0170] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described traffic flow-based simulation testing method, thereby solving the technical problem of how to improve the testing capability of autonomous driving functions in complex scenarios. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the traffic flow-based simulation testing method provided in the above embodiments, and will not be repeated here.

[0171] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the traffic flow-based simulation testing method described above.

[0172] The computer program product provided in this application can solve the technical problem of how to improve the testing capability of autonomous driving functions in complex scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the traffic flow-based simulation testing method provided in the above embodiments, and will not be repeated here.

[0173] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or / indirect applications in other related technical fields are included in the patent protection scope of this application.

Claims

1. A simulation testing method based on traffic flow, characterized in that, The traffic flow-based simulation testing method includes: Obtain information on the proportion of trained agents with different driving styles; The disappearance distance threshold and appearance distance threshold between the trained agent and the tested vehicle are determined based on the perception range of the tested vehicle. The initial speed of the trained agent is determined based on the current speed and relative position type of the tested vehicle. Random traffic flow is generated based on the quantity ratio information, the disappearance distance threshold, the appearance distance threshold, and the initial vehicle speed; The test vehicle is simulated based on the random traffic flow.

2. The method as described in claim 1, characterized in that, The step of determining the initial vehicle speed of the trained agent based on the current speed and relative position type of the tested vehicle includes: The speed increment is determined based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the maximum allowable speed threshold. The pursuit simulation speed of the trained agent is determined based on the current speed of the tested vehicle and the speed increment. The speed reduction is determined based on the distance between the tested vehicle and the trained agent, the disappearance distance threshold, and the current speed of the tested vehicle. The simulated speed at which the trained agent is being chased is determined based on the current speed of the tested vehicle and the speed reduction. When the relative position type between the trained agent and the tested vehicle is rear, the initial speed of the trained agent is determined based on the simulated chasing speed and the maximum permissible speed threshold. When the relative position type between the trained agent and the tested vehicle is "ahead", the initial speed of the trained agent is determined based on the simulated speed being chased and the minimum allowable speed threshold.

3. The method as described in claim 1, characterized in that, Before the step of obtaining the percentage information of trained agents of different driving styles, the method further includes: The training environment for the trained intelligent agent is constructed based on traffic-participating vehicles and the road environment; In the training environment, the weights of the preset expectation function network are updated using gradient descent to obtain the target training model; Based on the target training model, the trained agent is iteratively trained to obtain a traffic flow motion model; Based on the traffic flow motion model, the number and proportion of trained agents of different driving styles are determined.

4. The method as described in claim 3, characterized in that, The steps of constructing the training environment for the trainable intelligent agent based on traffic-participating vehicles and the road environment include: Determine acceleration and yaw rate as the set of actions; Define the vehicle's longitudinal velocity, lateral velocity, collision time with the lateral target, and collision time with the longitudinal target as a set of states; The trained agent is constructed based on the set of actions and the set of states; Define one-way multi-lane roads as the road environment; The training environment for the trained agent is constructed based on the traffic-participating vehicle and the road environment.

5. The method as described in claim 3, characterized in that, The step of updating the weights of the preset expectation function network using gradient descent in the training environment to obtain the target training model includes: Initialize the historical experience data pool, the expected value function network, and the target value function network; Obtain the expected target reward value output by the expected value function network under the current state; The action or random action corresponding to the target reward expectation value is determined as the current action based on a preset allocation probability. Perform the current action in the training environment to obtain an immediate reward and the next state; The historical experience data pool is updated based on the current state, the current action, the instant reward, and the next state. Training samples are drawn from the updated historical experience data pool; The cumulative discount reward of the target value function network is determined based on the training samples and the expected value function network. The weights of the expected value function network are updated using gradient descent based on the cumulative discount reward to obtain the target training model.

6. The method as described in claim 3, characterized in that, The step of iteratively training the trained agent based on the target training model to obtain the traffic flow motion model includes: Obtain the driving style type of the trained intelligent agent; The safety reward is determined based on whether the trained agent experiences a collision event. The longitudinal motion reward is determined based on the driving style type, the longitudinal speed of the trained agent, and a preset speed statistic. The lateral movement reward is determined based on the number of lane changes made by the trained agent. The target reward function of the trained agent is determined based on the security reward, the longitudinal motion reward, and the lateral motion reward. The target reward function is substituted into the target training model, and the trained agent is iteratively trained to obtain the traffic flow motion model.

7. The method as described in claim 1, characterized in that, The steps for simulating the test vehicle based on the random traffic flow include: Responding to the control commands of the module under test, it generates motion state information of the vehicle under test; The traffic participant status information of the tested vehicle is determined based on the motion state of the tested vehicle and the random traffic flow. A dynamic scene is generated based on the motion state information of the tested vehicle and the state information of the traffic participants. The dynamic scene is processed to generate simulated sensor data, which is then sent to the module under test (DUT) so that the DUT can update the control commands of the vehicle under test based on the simulated sensor data, thereby performing simulation testing.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Obtain scene parameter information, traffic flow configuration information, algorithm version information, and number of repeated executions to obtain test task information; The test task information is split into multiple simulation subtasks, and the multiple simulation subtasks are inserted into a message queue; The number of worker node instances is determined based on the number of pending simulation subtasks in the message queue, and a dynamic worker node cluster is created. When the worker node instance starts, a simulation subtask is retrieved from the message queue; Based on the task parameter information of the simulation sub-task, the traffic flow motion model is loaded from the model repository; A random traffic flow is initialized based on the traffic flow motion model, and the vehicle under test is simulated based on the random traffic flow.

9. A traffic flow-based simulation testing device, characterized in that, The device includes: The data acquisition module is used to acquire information on the proportion of trained agents with different driving styles. The data processing module is used to determine the disappearance distance threshold and the appearance distance threshold between the trained intelligent agent and the tested vehicle based on the perception range of the tested vehicle. The data processing module is also used to determine the initial vehicle speed of the trained intelligent agent based on the current speed and relative position type of the tested vehicle; The data generation module is used to generate random traffic flow based on the quantity ratio information, the disappearance distance threshold, the appearance distance threshold, and the initial vehicle speed; The simulation test module is used to perform simulation tests on the vehicle under test based on the random traffic flow.

10. A traffic flow-based simulation testing device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the traffic flow-based simulation testing method as described in any one of claims 1 to 8.

Images