Automatic driving lane changing decision-making method fusing safety integrity framework and reinforcement learning

Abstract

The invention discloses an automatic driving lane changing decision-making method fusing a safety integrity framework and reinforcement learning, and belongs to the technical field of automatic driving control. The objective of the invention is to solve the problem of collaborative optimization of safety and driving efficiency. The method comprises the following steps: constructing quantitative indexes of a security integrity framework, wherein the quantitative indexes comprise exposure degree, severity and controllability; designing a total reward function for reinforcement learning by combining and considering the environmental basic influence; the total reward function is fused into a target function of the PPO algorithm, and an automatic driving lane changing model based on the PPO algorithm is obtained after training; and a multi-dimensional evaluation index system is constructed, and evaluation of the automatic driving lane changing model based on the PPO algorithm is completed. According to the invention, collaborative optimization of safety and driving efficiency is finally realized.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving control technology, specifically involving an autonomous driving lane-changing decision-making method that integrates a safety integrity framework and reinforcement learning. Background Technology

[0002] With the development of autonomous driving technology, the automatic lane-changing decision-making capability of autonomous vehicles has become one of the important indicators for measuring their intelligence level. However, the safety of lane-changing decisions in real-world driving environments still faces many challenges, especially the non-stationarity and uncertainty factors in real-world scenarios with continuously changing real-time conditions. These factors may lead to dangerous driving behaviors by autonomous vehicles, or even fatal accidents. Existing safe lane-changing models suffer from fragmented methodologies and do not strictly adhere to functional safety standards in their safety constraint design.

[0003] Reinforcement learning, through agent-environment interaction and maximizing cumulative rewards to learn behavioral decision-making strategies, has achieved significant results in gaming and robotics. Therefore, reinforcement learning-based autonomous driving behavior decision-making algorithms have gained widespread recognition and application in the field of autonomous driving, providing efficient and intelligent behavioral decision-making solutions for dealing with complex and dynamic traffic environments. However, the safety constraint design in existing safe lane-changing models suffers from fragmented methodologies and a lack of strict adherence to functional safety standards. Specifically, this manifests as inconsistent design paradigms and a failure to translate integrity level requirements from safety standards into concrete design constraints. Summary of the Invention

[0004] The problem this invention aims to solve is to achieve synergistic optimization of safety and driving efficiency, and proposes an autonomous driving lane-changing decision-making method that integrates a safety integrity framework and reinforcement learning.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] An autonomous driving lane-changing decision-making method integrating a safety integrity framework and reinforcement learning includes the following steps:

[0007] S1. Construct a quantitative indicator framework for security integrity, including exposure, severity, and controllability;

[0008] S2. Based on the security integrity framework quantification index obtained in step S1, and taking into account the environmental fundamental impact, design the total reward function for reinforcement learning.

[0009] S3. Integrate the total reward function obtained in step S2 into the objective function of the PPO algorithm, and after training, obtain an autonomous driving lane-changing model based on the PPO algorithm;

[0010] S4. Construct a multi-dimensional evaluation index system to evaluate the autonomous driving lane-changing model based on the PPO algorithm obtained in step S3.

[0011] Furthermore, the specific implementation method of step S1 includes the following steps:

[0012] S1.1. Based on the Responsibility Sensitive Safety Model (RSS), the minimum safe distance is introduced to calculate exposure. Minimum safe distance... Defined as:

[0013]

[0014] in, They represent the vehicle itself and the vehicle in front of it respectively. The speed of time Indicates the reaction time of the driver or autonomous driving system. These represent the maximum deceleration of the vehicle and the vehicle in front, respectively.

[0015] Then the exposure level is obtained. The calculation formula is:

[0016]

[0017] in, Indicates the longitudinal distance between the vehicle and the vehicle in front. The minimum safe distance calculated for RSS;

[0018] S1.2. Using collision kinetic energy loss as a severity index, the total amount of mechanical energy converted into structural deformation and heat during the collision process is characterized, and the results at time points are calculated for vehicles 1 and 2. Predicted total kinetic energy loss for:

[0019]

[0020]

[0021] in, For vehicle 1 at time The velocity component along the expected collision direction, For the mass of vehicle 1, For vehicle 2 at time The velocity component along the expected collision direction, For the mass of vehicle 2, To reduce quality;

[0022] Then the severity is obtained. The calculation formula is:

[0023] ;

[0024] S1.3. The Remaining Collision Distance (SRCD) is introduced as a controllability index, measuring the remaining operable distance to the potential collision point while the vehicle maintains its current motion. Under the extreme braking assumption, Remaining collision distance at time Represented as:

[0025]

[0026] in, This indicates the longitudinal relative distance between the vehicle and the vehicle in front. Indicates the current speed of the vehicle. This represents the maximum achievable deceleration of the vehicle.

[0027] Then the controllability index is obtained. Defined as:

[0028] .

[0029] Furthermore, the specific implementation method of step S2 includes the following steps:

[0030] S2.1. Design an exposure reward function based on exposure. The expression is:

[0031]

[0032] in, , The intensity coefficient for constraining exposure. For exposure indicators;

[0033] S2.2. Design a severity reward function based on severity The expression is:

[0034]

[0035] in, , The strength coefficient is the constraint severity. , To control the transition of the function from 0 to a threshold near the threshold Smoothness; As a severity indicator, This is the severity threshold;

[0036] S2.3. Design a controllable reward function based on controllability The expression is:

[0037]

[0038] in, , The controllability adjustment coefficient. As a controllability indicator;

[0039] S2.4. Considering the fundamental environmental impacts, including velocity constraints, time constraints, and collision constraints, the comprehensive environmental reward function is obtained. The expression is:

[0040]

[0041] in, For speed constraints, Due to time constraints, For collision constraints, The strengthening coefficient for velocity constraints, The time constraint reinforcement coefficient;

[0042]

[0043] in, For the vehicle's speed, Speed ​​limits on roads;

[0044]

[0045] in, This refers to the vehicle's travel time.

[0046] ;

[0047] S2.5. Design the total reward function for reinforcement learning The expression is;

[0048] .

[0049] Furthermore, the specific implementation method of step S3 includes the following steps:

[0050] S3.1. Defining Strategies in Reinforcement Learning The core objective, or objective function, is to maximize the expected cumulative reward, which is expressed as:

[0051]

[0052] in, , This refers to the reward function, i.e., the defined total reward function; As expected, The performance objective function is... The optimal policy parameters are given, and T is the maximum time step. To accumulate rewards, For a moment The environmental conditions, For a moment The action chosen by the agent;

[0053] S3.2. The PPO algorithm obtains the objective function by cutting the objective function in step S3.1. for:

[0054]

[0055] in, , For strategy ratio, To reward the advantage function, The shear coefficient is... For the expectation of the empirical sample, It is the shearing function;

[0056] S3.3. The total reward function obtained in step S2 is designed and integrated into the objective function of the PPO algorithm. In the simulation environment, the PPO algorithm interacts with the environment through state perception, action execution, and receives the immediate reward calculated by the total reward function. Subsequently, using this interaction data, the PPO algorithm optimizes the parameters of the policy network and the value network to maximize the long-term cumulative reward and finally obtain the autonomous driving lane-changing model.

[0057] Furthermore, the specific implementation method of step S4 includes the following steps:

[0058] S4.1. Construct safety metrics including collision rate and collision time. ;

[0059]

[0060] ;

[0061] S4.2. After calculating the safety indicators in step S4.1, and assuming that safety is guaranteed, calculate the driving score and task completion efficiency;

[0062] Driving points A comprehensive reward function defined by the environment. The weighted fusion of forward speed, arrival time, and collision penalty factors is used as the overall performance evaluation standard, expressed as:

[0063]

[0064] Task completion efficiency Defined as the average time required for an agent to complete one round or reach the target state, expressed as:

[0065]

[0066] in, This indicates the total number of test rounds. Indicates the index of the test round. Indicates the first The total number of time steps or actual time taken by the agent from the start to the successful completion of the task in each test round:

[0067] The driving score uses an environment-defined comprehensive reward function, which weights and integrates forward speed, arrival time, and collision penalty factors as the evaluation standard for overall performance.

[0068] Task completion efficiency is defined as the average time required for an agent to complete one round or reach the target state.

[0069] The beneficial effects of this invention are:

[0070] This invention presents an autonomous driving lane-changing decision-making method that integrates a safety integrity framework and reinforcement learning. It transforms the ISO 26262 safety integrity framework into explicit and quantifiable safety constraints. These safety constraints are deeply embedded into the underlying logic of the reinforcement learning lane-changing decision-making model, ensuring that the lane-changing behavior of autonomous vehicles in highly complex dynamic traffic environments is always confined within the absolute safety boundaries defined by functional safety, ultimately achieving synergistic optimization of safety and driving efficiency.

[0071] This invention presents an autonomous driving lane-changing decision-making method that integrates a safety integrity framework and reinforcement learning, addressing the shortcomings of existing autonomous driving lane-changing decision-making methods that lack a unified paradigm and standard for safety constraint design. The core of this method lies in providing a lane-changing decision-making model based on a functional safety integrity framework. First, based on the functional safety (ISO 26262, Road vehicles — Functional safety) standard, a model capable of dynamically quantifying and evaluating safety integrity indicators is constructed. Then, this dynamic indicator is used as a key parameter and embedded into the reward function of the reinforcement learning agent, forming adaptive safety constraints. Ultimately, this invention ensures the inherent safety of lane-changing behavior in complex and ever-changing traffic environments, while achieving an optimal balance between safety and driving efficiency.

[0072] This invention presents an autonomous driving lane-changing decision-making method that integrates a safety integrity framework and reinforcement learning. It constructs a unified safety integrity framework, systematically transforming the functional safety requirements of ISO 26262 into a series of explicit, quantifiable, and verifiable safety constraints, and deeply integrating these constraints into the reinforcement learning-based lane-changing decision-making method for autonomous vehicles. This method provides stringent theoretical safety constraints for the lane-changing behavior of autonomous vehicles in complex dynamic traffic scenarios, thereby significantly improving overall traffic efficiency while ensuring absolute priority for driving safety. Attached Figure Description

[0073] Figure 1 This is a flowchart of an autonomous driving lane-changing decision-making method that integrates a safety integrity framework and reinforcement learning, as described in this invention. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0075] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0076] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 Detailed explanation is as follows:

[0077] Example 1:

[0078] An autonomous driving lane-changing decision-making method integrating a safety integrity framework and reinforcement learning includes the following steps:

[0079] S1. Construct a quantitative indicator framework for security integrity, including exposure, severity, and controllability;

[0080] Furthermore, the specific implementation method of step S1 includes the following steps:

[0081] S1.1. Based on the Responsibility Sensitive Safety Model (RSS), the minimum safe distance is introduced to calculate exposure. Minimum safe distance... Defined as:

[0082]

[0083] in, They represent the vehicle itself and the vehicle in front of it respectively. The speed of time Indicates the reaction time of the driver or autonomous driving system. These represent the maximum deceleration of the vehicle and the vehicle in front, respectively.

[0084] Then the exposure level is obtained. The calculation formula is:

[0085]

[0086] in, Indicates the longitudinal distance between the vehicle and the vehicle in front. The minimum safe distance calculated for RSS;

[0087] Furthermore, exposures are used to measure the likelihood of a potential hazardous event occurring. A minimum safe distance is introduced based on the Responsibility-Sensitive Safety (RSS) model to achieve a real-time exposure metric. At that time, exposure This indicates that the vehicle is in a safe condition; when At that time, exposure The higher the value, the closer the vehicle is to danger and the higher the degree of exposure.

[0088] S1.2. Using collision kinetic energy loss as a severity index, the total amount of mechanical energy converted into structural deformation and heat during the collision process is characterized, and the results at time points are calculated for vehicles 1 and 2. Predicted total kinetic energy loss for:

[0089]

[0090]

[0091] in, For vehicle 1 at time The velocity component along the expected collision direction, For the mass of vehicle 1, For vehicle 2 at time The velocity component along the expected collision direction, For the mass of vehicle 2, To reduce quality;

[0092] Then the severity is obtained. The calculation formula is:

[0093] ;

[0094] Furthermore, severity measures the degree of personal injury by the level of energy loss once a potentially hazardous event occurs. Using collision kinetic energy loss as a severity indicator approximates the collision as a perfectly inelastic process, where kinetic energy loss reaches its maximum. The higher the value, the more energy the vehicle needs to dissipate, and the stronger the impact on potential occupants, resulting in a higher degree of severity.

[0095] S1.3. The Remaining Collision Distance (SRCD) is introduced as a controllability index, measuring the remaining operable distance to the potential collision point while the vehicle maintains its current motion. Under the extreme braking assumption, Remaining collision distance at time Represented as:

[0096]

[0097] in, This indicates the longitudinal relative distance between the vehicle and the vehicle in front. Indicates the current speed of the vehicle. This represents the maximum achievable deceleration of the vehicle.

[0098] Then the controllability index is obtained. Defined as:

[0099] .

[0100] Furthermore, controllability is used to assess the ability of a driver or autonomous driving system to effectively intervene in hazardous scenarios to avoid harm. When When the vehicle has sufficient space to avoid a collision even under maximum braking, it has a certain degree of controllability; while when When the maximum braking force is applied, it indicates that a collision cannot be avoided and the system is deemed uncontrollable.

[0101] S2. Based on the security integrity framework quantification index obtained in step S1, and taking into account the environmental fundamental impact, design the total reward function for reinforcement learning.

[0102] Furthermore, the specific implementation method of step S2 includes the following steps:

[0103] S2.1. Design an exposure reward function based on exposure. The expression is:

[0104]

[0105] in, , The intensity coefficient for constraining exposure. For exposure metrics; S2.2. Design a severity reward function based on severity. The expression is:

[0106]

[0107] in, , The strength coefficient is the constraint severity. , To control the transition of the function from 0 to a threshold near the threshold Smoothness; As a severity indicator, This is the severity threshold; The smaller the value, the steeper the transition, approximating a step function.

[0108] S2.3. Design a controllable reward function based on controllability The expression is:

[0109]

[0110] in, , This is the controllability adjustment coefficient, which controls the sensitivity of the constraint to changes in controllability. As a controllability indicator;

[0111] S2.4. Considering the fundamental environmental impacts, including velocity constraints, time constraints, and collision constraints, the comprehensive environmental reward function is obtained. The expression is:

[0112]

[0113] in, For speed constraints, Due to time constraints, For collision constraints, The strengthening coefficient for velocity constraints, The time constraint reinforcement coefficient;

[0114]

[0115] in, For the vehicle's speed, Speed ​​limits on roads;

[0116]

[0117] in, This refers to the vehicle's travel time.

[0118] ;

[0119] S2.5. Design the total reward function for reinforcement learning The expression is;

[0120] .

[0121] S3. Integrate the total reward function obtained in step S2 into the objective function of the PPO algorithm, and after training, obtain an autonomous driving lane-changing model based on the PPO algorithm;

[0122] Furthermore, the specific implementation method of step S3 includes the following steps:

[0123] S3.1. Defining Strategies in Reinforcement Learning The core objective, or objective function, is to maximize the expected cumulative reward, which is expressed as:

[0124]

[0125] in, , This refers to the reward function, i.e., the defined total reward function; As expected, The performance objective function is... The optimal policy parameters are given, and T is the maximum time step. To accumulate rewards, For a moment The environmental conditions, For a moment The action chosen by the agent;

[0126] S3.2. The PPO algorithm obtains the objective function by cutting the objective function in step S3.1. for:

[0127]

[0128] in, , For strategy ratio, To reward the advantage function, The shear coefficient is... For the expectation of the empirical sample, It is the shearing function;

[0129] Furthermore, the Proximal Policy Optimization (PPO) algorithm can effectively suppress excessive policy updates by pruning the objective function, thus ensuring training stability.

[0130] S3.3. The total reward function obtained in step S2 is designed and integrated into the objective function of the PPO algorithm. In the simulation environment, the PPO algorithm interacts with the environment through state perception, action execution, and receives the immediate reward calculated by the total reward function. Subsequently, using this interaction data, the PPO algorithm optimizes the parameters of the policy network and the value network to maximize the long-term cumulative reward and finally obtain the autonomous driving lane-changing model.

[0131] S4. Construct a multi-dimensional evaluation index system to evaluate the autonomous driving lane-changing model based on the PPO algorithm obtained in step S3.

[0132] Furthermore, the specific implementation method of step S4 includes the following steps:

[0133] S4.1. Construct safety metrics including collision rate and collision time. ;

[0134]

[0135] ;

[0136] Furthermore, the Collision Rate (CR) is defined as the proportion of rounds in which at least one collision occurs across all test rounds. A lower CR value indicates a higher level of safety for the algorithm's strategy. Time to Collision (TCT) represents the time required for the vehicle to collide with an obstacle ahead, assuming the vehicle maintains its current speed and direction. This indicates that the vehicle does not have sufficient safety margin. This indicates that the vehicle has a more ample safety margin.

[0137] S4.2. After calculating the safety indicators in step S4.1, and assuming that safety is guaranteed, calculate the driving score and task completion efficiency;

[0138] Driving points A comprehensive reward function defined by the environment. The weighted fusion of forward speed, arrival time, and collision penalty factors is used as the overall performance evaluation standard, expressed as:

[0139]

[0140] Task completion efficiency Defined as the average time required for an agent to complete one round or reach the target state, expressed as:

[0141]

[0142] in, This indicates the total number of test rounds. Indicates the index of the test round. Indicates the first In each test round, the total number of time steps or actual time taken by the agent from the start to successfully completing the task. The lower the value, the faster the task is completed and the higher the efficiency.

[0143] The driving score uses an environment-defined comprehensive reward function, which weights and integrates forward speed, arrival time, and collision penalty factors as the evaluation standard for overall performance.

[0144] Task completion efficiency is defined as the average time required for an agent to complete one round or reach the target state.

[0145] Furthermore, the method in this embodiment conforms to the safety constraint design method of the ISO 26262 standard. The safety integrity indicators—exposure, severity, and controllability—from the standard are formally constructed into a set of explicit and quantifiable safety constraints, which are then integrated into the reinforcement learning lane-changing decision model of the autonomous vehicle. By enforcing these constraints in complex traffic environments, this invention can systematically improve the driving efficiency of autonomous vehicles while ensuring functional safety.

[0146] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0147] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A lane-changing decision-making method for autonomous driving that integrates a safety integrity framework and reinforcement learning, characterized in that, Includes the following steps: S1. Construct a quantitative indicator framework for security integrity, including exposure, severity, and controllability; S2. Based on the security integrity framework quantification index obtained in step S1, and taking into account the environmental fundamental impact, design the total reward function for reinforcement learning. S3. Integrate the total reward function obtained in step S2 into the objective function of the PPO algorithm, and after training, obtain the autonomous driving lane-changing model based on the PPO algorithm; S4. Construct a multi-dimensional evaluation index system to evaluate the autonomous driving lane-changing model based on the PPO algorithm obtained in step S3.

2. The autonomous driving lane-changing decision-making method integrating a safety integrity framework and reinforcement learning as described in claim 1, characterized in that, The specific implementation method of step S1 includes the following steps: S1.

1. Based on the Responsibility Sensitive Safety Model (RSS), the minimum safe distance is introduced to calculate exposure. Minimum safe distance... Defined as: in, They represent the vehicle itself and the vehicle in front of it respectively. The speed of time, Indicates the reaction time of the driver or autonomous driving system. These represent the maximum deceleration of the vehicle and the vehicle in front, respectively. Then the exposure level is obtained. The calculation formula is: in, Indicates the longitudinal distance between the vehicle and the vehicle in front. The minimum safe distance calculated for RSS; S1.

2. Using collision kinetic energy loss as a severity index, the total amount of mechanical energy converted into structural deformation and heat during the collision process is characterized, and the results at time points are calculated for vehicles 1 and 2. Predicted total kinetic energy loss for: in, For vehicle 1 at time The velocity component along the expected collision direction, For the mass of vehicle 1, For vehicle 2 at time The velocity component along the expected collision direction, For the mass of vehicle 2, To reduce quality; Then the severity is obtained. The calculation formula is: ； S1.

3. The Remaining Collision Distance (SRCD) is introduced as a controllability index, measuring the remaining operable distance to the potential collision point while the vehicle maintains its current motion. Under the extreme braking assumption, Remaining collision distance at time Represented as: in, This indicates the longitudinal relative distance between the vehicle and the vehicle in front. Indicates the current speed of the vehicle. This represents the maximum achievable deceleration of the vehicle. Then the controllability index is obtained. Defined as: 。 3. The autonomous driving lane-changing decision-making method integrating a safety integrity framework and reinforcement learning as described in claim 2, characterized in that, The specific implementation method of step S2 includes the following steps: S2.

1. Design an exposure reward function based on exposure. The expression is: in, , The intensity coefficient for constrained exposure. For exposure indicators; S2.

2. Design a severity reward function based on severity The expression is: in, , The strength coefficient is the constraint severity. , To control the transition of the function from 0 to a threshold near the threshold Smoothness; As a severity indicator, This is the severity threshold; S2.

3. Design a controllable reward function based on controllability The expression is: in, , The controllability adjustment coefficient. As a controllability indicator; S2.

4. Considering the fundamental environmental impacts, including velocity constraints, time constraints, and collision constraints, the comprehensive environmental reward function is obtained. The expression is: in, For speed constraints, Due to time constraints, For collision constraints, The strengthening coefficient for velocity constraints, The time constraint reinforcement coefficient; in, For the vehicle's speed, Speed ​​limits on roads; in, This refers to the vehicle's travel time. ； S2.

5. Design the total reward function for reinforcement learning The expression is; 。 4. The autonomous driving lane-changing decision-making method integrating a safety integrity framework and reinforcement learning as described in claim 3, characterized in that, The specific implementation method of step S3 includes the following steps: S3.

1. Defining Strategies in Reinforcement Learning The core objective, or objective function, is to maximize the expected cumulative reward, which is expressed as: in, , This refers to the reward function, i.e., the defined total reward function; For the expectation, The performance objective function is... The optimal policy parameters are given, and T is the maximum time step. To accumulate rewards, For a moment The environmental conditions, For a moment The action chosen by the agent; S3.

2. The PPO algorithm obtains the objective function by cutting the objective function in step S3.

1. for: in, , For strategy ratio, To reward the advantage function, The shear coefficient is... For the expectation of the empirical sample, It is the shearing function; S3.

3. The total reward function obtained in step S2 is designed and integrated into the objective function of the PPO algorithm. In the simulation environment, the PPO algorithm interacts with the environment through state perception, action execution, and receives the immediate reward calculated by the total reward function. Subsequently, using this interaction data, the PPO algorithm optimizes the parameters of the policy network and the value network to maximize the long-term cumulative reward and finally obtain the autonomous driving lane-changing model.

5. The autonomous driving lane-changing decision-making method integrating a safety integrity framework and reinforcement learning as described in claim 4, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. Construct safety metrics including collision rate and collision time. ; ； S4.

2. After calculating the safety indicators in step S4.1, and assuming that safety is guaranteed, calculate the driving score and task completion efficiency; Driving points A comprehensive reward function defined by the environment. The weighted fusion of forward speed, arrival time, and collision penalty factors is used as the overall performance evaluation standard, expressed as: Task completion efficiency Defined as the average time required for an agent to complete one round or reach the target state, expressed as: in, This indicates the total number of test rounds. Indicates the index of the test round. Indicates the first In each test round, the total number of time steps or actual time taken by the agent from the start to the successful completion of the task.

Images