Intelligent learning evaluation method and system for driving strategy of self-driving automobile

Abstract

The invention provides an intelligent learning evaluation method and system for a driving strategy of an automatic driving automobile. Screening safety-critical scene data in the automatic driving big data, constructing a safety-critical scene set, and obtaining a safety-critical scene set explicit function in combination with explicit function operation; carrying out automatic differential calculation on a basic mean square error loss function, and obtaining a safety-critical scene loss function in combination with a safety-critical scene set representation function; according to the safety-critical scene loss function, through driving strategy probability distribution training, forming a driving strategy probability distribution intelligent learning model; verifying the effectiveness and accuracy of the driving strategy probability distribution intelligent learning model, and evaluating the safety precision of the driving strategy probability distribution intelligent learning model; and the efficiency and the accuracy of intelligent learning evaluation of the driving strategy of the autonomous vehicle are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent learning testing and evaluation technology for autonomous driving safety scenarios, and more specifically, to an intelligent learning and evaluation method and system for autonomous vehicle driving strategies. Background Technology

[0002] Currently, autonomous vehicles face serious safety issues. With the large-scale popularization of autonomous vehicles, serious traffic accidents are increasing. These safety issues fundamentally hinder the technological development and large-scale application of autonomous vehicles. Therefore, it is urgent to conduct safety testing and evaluation of autonomous vehicles. The basic process of autonomous vehicle testing and evaluation is to generate a series of test scenarios to test autonomous vehicles, then collect the test results and evaluate the safety of autonomous vehicles, which can usually yield estimated values ​​of test indicators such as accident rate. However, the real-road test evaluation method based on Monte Carlo sampling theory is very inefficient because safety-critical scenarios (such as collision accident scenarios) in natural driving environments are very rare, which means that obtaining an estimate of the accident rate requires a large amount of test mileage. It is estimated that an autonomous vehicle needs to accumulate more than 10 billion kilometers of testing in natural driving environments to obtain a high-confidence safety performance test result. To address the inefficiency of real-world road testing, intelligent equivalent accelerated test scenario generation methods improve the efficiency of autonomous vehicle testing and evaluation by increasing the sampling probability of safety-critical scenarios. A key step in this process is identifying safety-critical scenarios for autonomous vehicles. This step requires estimating the autonomous vehicle's driving strategy, but how to learn the autonomous vehicle's driving strategy in safety-critical scenarios during testing remains to be solved. Therefore, it is necessary to propose an intelligent learning and evaluation method and system for autonomous vehicle driving strategies to at least partially address the problems existing in the current technology. Summary of the Invention

[0003] The summary of this invention introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. The summary of this invention does not mean that it attempts to limit the key features and essential technical features of the claimed technical solution, nor does it mean that it attempts to determine the scope of protection of the claimed technical solution.

[0004] To at least partially solve the above problems, the present invention provides an intelligent learning and evaluation method for driving strategies of autonomous vehicles, comprising:

[0005] S100: Filter safety-critical scenario data from autonomous driving big data, construct a set of safety-critical scenarios, and obtain the indicator function of the set of safety-critical scenarios by combining indicator function operation;

[0006] S200 performs automatic differential calculation of the basic mean square error loss function, and combines it with the characteristic function of the safety-critical scenario set to obtain the safety-critical scenario loss function;

[0007] The S300, based on the loss function for safety-critical scenarios, trains a driving strategy probability distribution intelligent learning model through driving strategy probability distribution training.

[0008] S400 verifies the effectiveness and accuracy of the intelligent learning model of driving strategy probability distribution, evaluates the safety accuracy of the driving strategy of the intelligent learning model of driving strategy probability distribution, and improves the efficiency and accuracy of intelligent learning evaluation of driving strategy for autonomous vehicles.

[0009] Preferably, S100 includes:

[0010] S101 uses autonomous driving big data capture and processing to filter safety-critical scenario data from autonomous driving big data.

[0011] S102, Construct a set of security-critical scenarios based on security-critical scenario data;

[0012] S103, Based on the set of safety-critical scenarios, and combined with the characteristic function operation, obtain the characteristic function of the set of safety-critical scenarios.

[0013] Preferably, S200 includes:

[0014] S201, the basic mean square error loss function is automatically calculated by differentiation using a neural network training tool; the neural network for the autonomous vehicle driving strategy is set to a multi-layer perceptron architecture.

[0015] S202, Calculate the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy;

[0016] S203, based on the product of the characteristic function of the set of safety-critical scenarios and the basic mean square error loss function, the safety-critical scenario loss function.

[0017] Preferably, S300 includes:

[0018] S301, Based on the safety-critical scenario loss function, train the probability distribution of driving strategy in safety-critical scenarios where driving behavior data is scarce;

[0019] S302 is trained using the probability distribution of driving strategies, and after training, an intelligent learning model of the probability distribution of driving strategies is formed.

[0020] Preferably, S400 includes:

[0021] S401, Extract safety-critical driving data of autonomous vehicles in safety-critical scenarios, input the safety-critical driving data of autonomous vehicles into the intelligent driving following model, and form a safety-critical intelligent driving following verification model.

[0022] S402, based on the safety-critical intelligent driving following verification model, verify the effectiveness and accuracy of the intelligent learning model of the driving strategy probability distribution of driving data critical to the safety of autonomous vehicles; evaluate the safety accuracy of the driving strategy of the intelligent learning model of the driving strategy probability distribution; and improve the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

[0023] This invention provides an intelligent learning and evaluation system for driving strategies of autonomous vehicles, comprising:

[0024] The safety-critical scenario set module filters safety-critical scenario data from autonomous driving big data, constructs a safety-critical scenario set, and obtains the indicator function of the safety-critical scenario set by combining indicator function operations.

[0025] The safety-critical scenario loss function module automatically differentiates and calculates the basic mean square error loss function, and combines it with the characteristic function of the safety-critical scenario set to obtain the safety-critical scenario loss function.

[0026] The driving strategy probability distribution learning module trains a driving strategy probability distribution intelligent learning model based on the loss function of safety-critical scenarios and the driving strategy probability distribution.

[0027] The driving strategy probability model evaluation module verifies the effectiveness and accuracy of the intelligent learning model of driving strategy probability distribution, evaluates the safety accuracy of the driving strategy of the intelligent learning model of driving strategy probability distribution, and improves the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

[0028] Preferred, the safety-critical scenario set module includes:

[0029] The safety-critical scenario data filtering unit filters safety-critical scenario data from autonomous driving big data through autonomous driving big data capture and processing.

[0030] The security-critical scenario collection architecture unit constructs a collection of security-critical scenarios based on security-critical scenario data.

[0031] The scenario set indicator function unit obtains the indicator function of the safety-critical scenario set by combining indicator function operations based on the safety-critical scenario set.

[0032] Preferably, the loss function module for safety-critical scenarios includes:

[0033] The basic error automatic differentiation unit automatically differentiates and calculates the basic mean square error loss function using a neural network training tool; the neural network for the autonomous vehicle driving strategy is set as a multi-layer perceptron architecture.

[0034] The parameter gradient computation unit calculates the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy;

[0035] The safety-critical loss function calculation unit calculates the safety-critical scenario loss function by multiplying the characteristic function of the safety-critical scenario set with the basic mean square error loss function.

[0036] Preferably, the driving strategy probability distribution learning module includes:

[0037] The driving strategy probability training unit trains the driving strategy probability distribution in safety-critical scenarios based on the loss function of the safety-critical scenario, taking into account the scarcity of driving behavior data.

[0038] The strategy distribution learning intelligent model unit is trained by the probability distribution of driving strategies, and after training, it forms an intelligent learning model of the probability distribution of driving strategies.

[0039] Preferably, the driving strategy probability model evaluation module includes:

[0040] The intelligent driving following verification unit extracts safety-critical driving data of autonomous vehicles in safety-critical scenarios, inputs the safety-critical driving data of autonomous vehicles into the intelligent driving following model, and forms a safety-critical intelligent driving following verification model.

[0041] The distributed learning intelligent model evaluation unit verifies the effectiveness and accuracy of the driving strategy probability distribution intelligent learning model for autonomous vehicles based on the safety-critical intelligent driving follow-up verification model; evaluates the driving strategy safety accuracy of the driving strategy probability distribution intelligent learning model; and improves the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

[0042] Compared with the prior art, the present invention has at least the following beneficial effects:

[0043] This invention discloses an intelligent learning and evaluation method and system for autonomous vehicle driving strategies. It constructs a set of safety-critical scenarios by filtering safety-critical scenario data from autonomous driving big data, and obtains an indicator function for the set of safety-critical scenarios through indicator function calculation. It then performs automatic differential calculation of the basic mean square error loss function, and obtains a safety scenario loss function by combining the indicator function of the safety scenario set. Based on the safety scenario loss function, it trains a driving strategy probability distribution intelligent learning model using driving strategy probability distribution training. The effectiveness and accuracy of the intelligent learning model are verified, and the safety accuracy of the driving strategy is evaluated. This improves the efficiency and accuracy of intelligent learning and evaluation of autonomous vehicle driving strategies; significantly improves the learning efficiency of autonomous vehicle driving strategy probability distribution in safety-critical scenarios; and significantly reduces the learning mean square error of the autonomous vehicle driving strategy probability distribution. This invention improves the accuracy of simulated road test evaluations in key safety scenarios for autonomous driving intelligent learning; it proposes an online learning method for autonomous vehicle driving strategies; during safety testing of autonomous vehicles, deep learning technology is used to learn the driving strategies of autonomous vehicles using only driving behavior data from safety-critical scenarios, significantly reducing the variance of the loss function relative to the gradient of neural network parameters during training, thus effectively achieving online learning of autonomous vehicle driving strategies; in the process of safety testing of autonomous vehicles, this patent uses deep learning technology to learn the driving strategies of autonomous vehicles using only driving behavior data from safety-critical scenarios; effectively achieving online learning of autonomous vehicle driving strategies, especially for driving strategies in safety-critical scenarios.

[0044] The present invention provides an intelligent learning and evaluation method and system for driving strategies of autonomous vehicles. Other advantages, objectives and features of the present invention will be partly apparent from the following description, and partly understood by those skilled in the art through research and practice of the present invention. Attached Figure Description

[0045] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0046] Figure 1 This is a diagram of an embodiment of the intelligent learning and evaluation system for driving strategies of autonomous vehicles according to the present invention.

[0047] Figure 2 This is a diagram of another embodiment of the intelligent learning and evaluation system for driving strategies of autonomous vehicles described in this invention.

[0048] Figure 3 This is a curve illustration of an intelligent learning and evaluation method and system for driving strategies of autonomous vehicles according to the present invention. Detailed Implementation

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, so that those skilled in the art can implement it based on the specification; as shown in the figures, the present invention provides an intelligent learning and evaluation method for driving strategies of autonomous vehicles, including:

[0050] S100: Filter safety-critical scenario data from autonomous driving big data, construct a set of safety-critical scenarios, and obtain the indicator function of the set of safety-critical scenarios by combining indicator function operation;

[0051] S200 performs automatic differential calculation of the basic mean square error loss function, and combines it with the characteristic function of the safety-critical scenario set to obtain the safety-critical scenario loss function;

[0052] The S300, based on the loss function for safety-critical scenarios, trains a driving strategy probability distribution intelligent learning model through driving strategy probability distribution training.

[0053] S400 verifies the effectiveness and accuracy of the intelligent learning model of driving strategy probability distribution, evaluates the safety accuracy of the driving strategy of the intelligent learning model of driving strategy probability distribution, and improves the efficiency and accuracy of intelligent learning evaluation of driving strategy for autonomous vehicles.

[0054] The principle and effect of the above technical solution are as follows: This invention provides an intelligent learning and evaluation method for driving strategies of autonomous vehicles, including: screening safety-critical scenario data from autonomous driving big data, constructing a set of safety-critical scenarios, and obtaining an indicator function for the set of safety-critical scenarios by combining indicator function calculations; performing automatic differential calculation of the basic mean square error loss function, and obtaining a safety scenario loss function by combining the indicator function for the set of safety-critical scenarios; training a driving strategy probability distribution intelligent learning model based on the safety scenario loss function through driving strategy probability distribution training; verifying the effectiveness and accuracy of the driving strategy probability distribution intelligent learning model, and evaluating the driving strategy safety accuracy of the driving strategy probability distribution intelligent learning model; improving the efficiency and accuracy of intelligent learning and evaluation of driving strategies of autonomous vehicles; significantly improving the learning efficiency of the driving strategy probability distribution of autonomous vehicles in safety-critical scenarios; and reducing the learning mean square error of the driving strategy probability distribution of autonomous vehicles. The difference is significantly reduced, improving the accuracy of simulated road test evaluation in key safety scenarios for autonomous driving intelligent learning; an online learning method for autonomous vehicle driving strategies is proposed; during the safety testing of autonomous vehicles, deep learning technology is used to learn the driving strategy of autonomous vehicles using only driving behavior data in safety-critical scenarios, significantly reducing the variance of the loss function relative to the gradient of the neural network parameters during training, thus effectively realizing online learning of autonomous vehicle driving strategies; during the safety testing of autonomous vehicles, this patent uses deep learning technology to learn the driving strategy of autonomous vehicles using only driving behavior data in safety-critical scenarios; effectively realizing online learning of autonomous vehicle driving strategies, especially for driving strategies in safety-critical scenarios.

[0055] Constructing an autonomous vehicle overtaking scenario 1, such as Figure 1 As shown, the tested autonomous vehicle is denoted as AV1, the background vehicle as BV1, and the vehicle in front of BV1 as LV1. During the process of AV1 overtaking BV1 and LV1, if BV1 also changes lanes to overtake LV1, BV1 may collide with AV1; the state of the overtaking scenario is defined as follows: Where v BV R1 represents the velocity of BV1, and R1 represents the relative distance between LV1 and BV1. R1 represents the relative velocity between LV1 and BV1, and R2 represents the relative distance between BV1 and AV1. This represents the relative speed between BV1 and AV1; the action in the overtaking scenario is defined as a = [a...]. LV ,a BV ], where a LV a represents the acceleration at LV1. BVThis represents the acceleration of BV1;

[0056] Constructing an autonomous vehicle overtaking scenario 2, such as Figure 2 As shown, the tested autonomous vehicle is denoted as AV2, the background vehicle as BV2, and the vehicle in front of BV2 as LV2. During the process of AV2 overtaking BV2 and approaching LV2, if BV2 also changes lanes to follow LV2, a collision may occur between BV2 and AV2. The state of the overtaking scenario is defined as follows: Where v BV2 R represents the velocity of BV2. 12 This represents the relative distance between LV2 and BV2. R represents the relative velocity between LV1 and BV1. 22 This represents the relative distance between BV2 and AV2. The relative speeds of BV2 and AV2 are represented; the action in the overtaking scenario is defined as a. 12 =[a LV2 ,a BV2 ], where a LV2 a represents the acceleration at LV2. BV2 This represents the acceleration of BV2;

[0057] The probability distribution of the driving strategy of an autonomous vehicle is π(a|s), where a is the action variable, including the lateral and longitudinal accelerations of the autonomous vehicle, and s is the state variable, including the positions and velocities of all vehicles in the driving environment. During the testing of autonomous vehicles, machine learning methods are used to utilize the driving behavior data of the autonomous vehicles. The driving strategy is learned; where the positive integer N is the number of driving behavior data; the rarity of safety-critical scenarios in the test environment results in very scarce driving behavior data for autonomous vehicles in safety-critical scenarios; utilizing driving behavior data collected online. This patent proposes an online learning method for autonomous vehicle driving strategies, denoted as π, which uses a deep neural network to learn the probability distribution of the autonomous vehicle's driving strategy. θ Where θ represents the parameters of the neural network. Addressing the scarcity of driving behavior data in safety-critical scenarios during driving strategy training, this patent proposes training the driving strategy π using only driving data from autonomous vehicles in safety-critical scenarios. θ Specifically, a batch of training data The loss function is:

[0058]

[0059] Where B is the positive integer, representing the batch size of the training data, and L is the loss function, which can be the mean squared error function. The basic training method applies the loss function L to the driving strategy π.θ The gradient of the parameter θ is calculated as follows:

[0060]

[0061] Due to the rarity of safety-critical scenarios, the variance of the gradient g is very large and needs to be further reduced; the loss function L with respect to the driving policy π θ The gradient of parameter θ is calculated using the following formula:

[0062]

[0063] in, S is calculated using the automatic differentiation function provided by the neural network training tool. c This represents a set of security-critical scenarios. For S c Indicator functions:

[0064]

[0065] Use TTC to determine the set S of security-critical scenarios. c TTC is the time required for an autonomous vehicle to collide with another vehicle while maintaining its current speed and direction. The formula for TTC is:

[0066]

[0067] Where s represents the current state, Δx represents the relative distance between the autonomous vehicle and other vehicles, and Δv represents the relative speed between the autonomous vehicle and other vehicles; when TTC is below a certain threshold, it indicates that the distance between the autonomous vehicle and other vehicles is too close, posing a risk of collision; let τ be the threshold for TTC; then the set S of safety-critical scenarios... c for:

[0068] S c ={s∈S:TTC(s)<τ}; where S represents the set of all possible states; by training using only driving data of autonomous vehicles in safety-critical scenarios, the variance of the loss function gradient G can be significantly reduced, i.e., Var(G)≤Var(g), enabling the proposed method to effectively learn the driving strategy of autonomous vehicles.

[0069] In one embodiment, S100 includes:

[0070] S101 uses autonomous driving big data capture and processing to filter safety-critical scenario data from autonomous driving big data.

[0071] S102, Construct a set of security-critical scenarios based on security-critical scenario data;

[0072] S103, Based on the set of safety-critical scenarios, and combined with the characteristic function operation, obtain the characteristic function of the set of safety-critical scenarios.

[0073] The principle and effect of the above technical solution are as follows: by capturing and processing big data of autonomous driving, safety-critical scenario data in autonomous driving big data is filtered; based on the safety-critical scenario data, a set of safety-critical scenarios is constructed; based on the set of safety-critical scenarios, combined with the operation of indicator functions, the indicator function of the set of safety-critical scenarios is obtained.

[0074] In one embodiment, S200 includes:

[0075] S201, the basic mean square error loss function is automatically calculated by differentiation using a neural network training tool; the neural network for the autonomous vehicle driving strategy is set to a multi-layer perceptron architecture.

[0076] S202, Calculate the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy;

[0077] S203, based on the product of the characteristic function of the set of safety-critical scenarios and the basic mean square error loss function, the safety-critical scenario loss function.

[0078] The principle and effect of the above technical solution are as follows: the basic mean square error loss function is automatically calculated by using a neural network training tool; the neural network of the autonomous vehicle driving strategy is set as a multilayer perceptron architecture; the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy is calculated; and the safety-critical scenario loss function is calculated by multiplying the characteristic function of the safety-critical scenario set with the basic mean square error loss function.

[0079] In one embodiment, S300 includes:

[0080] S301, Based on the safety-critical scenario loss function, train the probability distribution of driving strategy in safety-critical scenarios where driving behavior data is scarce;

[0081] S302 is trained using the probability distribution of driving strategies, and after training, an intelligent learning model of the probability distribution of driving strategies is formed.

[0082] The principle and effect of the above technical solution are as follows: based on the loss function of safety-critical scenarios, the probability distribution of driving strategies is trained to account for the scarcity of driving behavior data in safety-critical scenarios; through the training of the probability distribution of driving strategies, an intelligent learning model of the probability distribution of driving strategies is formed after training.

[0083] In one embodiment, S400 includes:

[0084] S401, Extract safety-critical driving data of autonomous vehicles in safety-critical scenarios, input the safety-critical driving data of autonomous vehicles into the intelligent driving following model, and form a safety-critical intelligent driving following verification model.

[0085] S402, based on the safety-critical intelligent driving following verification model, verify the effectiveness and accuracy of the intelligent learning model of the driving strategy probability distribution of driving data critical to the safety of autonomous vehicles; evaluate the safety accuracy of the driving strategy of the intelligent learning model of the driving strategy probability distribution; and improve the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

[0086] The principle and effects of the above technical solution are as follows: Extract safety-critical driving data of autonomous vehicles in safety-critical scenarios; input this data into an intelligent driving follow-up model to form a safety-critical intelligent driving follow-up verification model; verify the effectiveness and accuracy of the intelligent learning model for the probability distribution of driving strategies based on the safety-critical driving data of autonomous vehicles; evaluate the safety accuracy of the driving strategy based on the intelligent learning model for the probability distribution of driving strategies; improve the efficiency and accuracy of intelligent learning and evaluation of driving strategies of autonomous vehicles; the learning results of the intelligent learning model for the probability distribution of driving strategies of autonomous vehicles are as follows: Figure 3 As shown; Meansquared error represents the mean squared error; Ordinary method represents the mean squared error curve of the prior art; Our method represents the mean squared error curve of this patent; Safety-critical states represent safety-critical scenarios; Figure 3 The curves represent the mean square error curves in safety-critical scenarios, where curve 1 represents the mean square error curve of the prior art in safety-critical scenarios, and curve 2 represents the mean square error curve of the present invention in safety-critical scenarios; by Figure 3 The mean square error of the curve in this patent is significantly reduced; the method proposed in this patent can significantly and effectively learn the driving strategy of autonomous vehicles in safety-critical scenarios.

[0087] This invention provides an intelligent learning and evaluation system for driving strategies of autonomous vehicles, comprising:

[0088] The safety-critical scenario set module filters safety-critical scenario data from autonomous driving big data, constructs a safety-critical scenario set, and obtains the indicator function of the safety-critical scenario set by combining indicator function operations.

[0089] The safety-critical scenario loss function module automatically differentiates and calculates the basic mean square error loss function, and combines it with the characteristic function of the safety-critical scenario set to obtain the safety-critical scenario loss function.

[0090] The driving strategy probability distribution learning module trains a driving strategy probability distribution intelligent learning model based on the loss function of safety-critical scenarios and the driving strategy probability distribution.

[0091] The driving strategy probability model evaluation module verifies the effectiveness and accuracy of the intelligent learning model of driving strategy probability distribution, evaluates the safety accuracy of the driving strategy of the intelligent learning model of driving strategy probability distribution, and improves the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

[0092] The principle and effect of the above technical solution are as follows: This invention provides an intelligent learning and evaluation system for autonomous vehicle driving strategies, comprising: a safety-critical scenario set module, which filters safety-critical scenario data from autonomous driving big data, constructs a safety-critical scenario set, and obtains a safety-critical scenario set indicator function by combining indicator function calculation; a safety-critical scenario loss function module, which performs automatic differential calculation of the basic mean square error loss function and obtains a safety-critical scenario loss function by combining the safety-critical scenario set indicator function; a driving strategy probability distribution learning module, which, based on the safety-critical scenario loss function, trains the driving strategy probability distribution to form an intelligent learning model of the driving strategy probability distribution; and a driving strategy probability model evaluation module, which verifies the effectiveness and accuracy of the intelligent learning model of the driving strategy probability distribution and evaluates the driving strategy safety accuracy of the intelligent learning model of the driving strategy probability distribution; thereby improving the efficiency and accuracy of intelligent learning and evaluation of autonomous vehicle driving strategies; and increasing the learning efficiency of autonomous vehicle driving strategy probability distribution in safety-critical scenarios. Significant improvements are achieved; the learning mean square error of the probability distribution of autonomous vehicle driving strategies is significantly reduced, improving the accuracy of simulated road test evaluation in key safety scenarios for autonomous driving intelligent learning; an online learning method for autonomous vehicle driving strategies is proposed; during the safety testing of autonomous vehicles, deep learning technology is used to learn the driving strategy of autonomous vehicles using only driving behavior data of autonomous vehicles in safety-critical scenarios, which significantly reduces the variance of the gradient of the loss function relative to the neural network parameters during training, thereby effectively realizing the online learning of autonomous vehicle driving strategies; during the safety testing of autonomous vehicles, this patent uses deep learning technology to learn the driving strategy of autonomous vehicles using only driving behavior data of autonomous vehicles in safety-critical scenarios; effectively realizing the online learning of autonomous vehicle driving strategies, especially for driving strategies in safety-critical scenarios.

[0093] Constructing an autonomous vehicle overtaking scenario 1, such as Figure 1As shown, the tested autonomous vehicle is denoted as AV1, the background vehicle as BV1, and the vehicle in front of BV1 as LV1. During the process of AV1 overtaking BV1 and LV1, if BV1 also changes lanes to overtake LV1, BV1 may collide with AV1; the state of the overtaking scenario is defined as follows: Where v BV R1 represents the velocity of BV1, and R1 represents the relative distance between LV1 and BV1. R1 represents the relative velocity between LV1 and BV1, and R2 represents the relative distance between BV1 and AV1. This represents the relative speed between BV1 and AV1; the action in an overtaking scenario is defined as a = [a...]. LV ,a BV ], where a LV a represents the acceleration at LV1. BV This represents the acceleration of BV1;

[0094] Constructing an autonomous vehicle overtaking scenario 2, such as Figure 2 As shown, the tested autonomous vehicle is denoted as AV2, the background vehicle as BV2, and the vehicle in front of BV2 as LV2. During the process of AV2 overtaking BV2 and approaching LV2, if BV2 also changes lanes to follow LV2, a collision may occur between BV2 and AV2. The state of the overtaking scenario is defined as follows: Where v BV2 R represents the velocity of BV2. 12 This represents the relative distance between LV2 and BV2. R represents the relative velocity between LV1 and BV1. 22 This represents the relative distance between BV2 and AV2. The relative velocity between BV2 and AV2 is represented; the action in the overtaking scenario is defined as a. 12 =[a LV2 ,a BV2 ], where a LV2 a represents the acceleration at LV2. BV2 This represents the acceleration of BV2;

[0095] The probability distribution of the driving strategy of an autonomous vehicle is π(a|s), where a is the action variable, including the lateral and longitudinal accelerations of the autonomous vehicle, and s is the state variable, including the positions and velocities of all vehicles in the driving environment. During the testing of autonomous vehicles, machine learning methods are used to utilize the driving behavior data of the autonomous vehicles. The driving strategy is learned; where the positive integer N is the number of driving behavior data; the rarity of safety-critical scenarios in the test environment results in very scarce driving behavior data for autonomous vehicles in safety-critical scenarios; utilizing driving behavior data collected online. This patent proposes an online learning method for autonomous vehicle driving strategies, denoted as π, which uses a deep neural network to learn the probability distribution of the autonomous vehicle's driving strategy. θ Where θ represents the parameters of the neural network. Addressing the scarcity of driving behavior data in safety-critical scenarios during driving strategy training, this patent proposes training the driving strategy π using only driving data from autonomous vehicles in safety-critical scenarios. θ Specifically, a batch of training data The loss function is:

[0096]

[0097] Where B is the positive integer, representing the batch size of the training data, and L is the loss function, which can be the mean squared error function. The basic training method applies the loss function L to the driving strategy π. θ The gradient of the parameter θ is calculated as follows:

[0098]

[0099] Due to the rarity of safety-critical scenarios, the variance of the gradient g is very large and needs to be further reduced; the loss function L with respect to the driving policy π θ The gradient of parameter θ is calculated using the following formula:

[0100]

[0101] in, S is calculated using the automatic differentiation function provided by the neural network training tool. c This represents a set of security-critical scenarios. For S c Indicator functions:

[0102]

[0103] Use TTC to determine the set S of security-critical scenarios. c TTC is the time required for an autonomous vehicle to collide with another vehicle while maintaining its current speed and direction. The formula is as follows:

[0104]

[0105] Where s represents the current state, Δx represents the relative distance between the autonomous vehicle and other vehicles, and Δv represents the relative speed between the autonomous vehicle and other vehicles; when TTC is below a certain threshold, it indicates that the distance between the autonomous vehicle and other vehicles is too close, posing a risk of collision; let τ be the threshold for TTC; then the set S of safety-critical scenarios... c for:

[0106] S c ={s∈S:TTC(s)<τ}; where S represents the set of all possible states; by training using only driving data of autonomous vehicles in safety-critical scenarios, the variance of the loss function gradient G can be significantly reduced, i.e., Var(G)≤Var(g), enabling the proposed method to effectively learn the driving strategy of autonomous vehicles.

[0107] In one embodiment, the security-critical scenario collection module includes:

[0108] The safety-critical scenario data filtering unit filters safety-critical scenario data from autonomous driving big data through autonomous driving big data capture and processing.

[0109] The security-critical scenario collection architecture unit constructs a collection of security-critical scenarios based on security-critical scenario data.

[0110] The scenario set indicator function unit obtains the indicator function of the safety-critical scenario set by combining indicator function operations based on the safety-critical scenario set.

[0111] The principle and effect of the above technical solution are as follows: The safety-critical scenario collection module includes: a safety-critical scenario data filtering unit, which filters safety-critical scenario data in autonomous driving big data through autonomous driving big data capture and big data processing; a safety-critical scenario collection architecture unit, which constructs a safety-critical scenario collection based on the safety-critical scenario data; and a scenario collection indicator function unit, which obtains the safety-critical scenario collection indicator function based on the safety-critical scenario collection and indicator function operation.

[0112] In one embodiment, the loss function module for safety-critical scenarios includes:

[0113] The basic error automatic differentiation unit automatically differentiates and calculates the basic mean square error loss function using a neural network training tool; the neural network for the autonomous vehicle driving strategy is set as a multi-layer perceptron architecture.

[0114] The parameter gradient computation unit calculates the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy;

[0115] The safety-critical loss function calculation unit calculates the safety-critical scenario loss function by multiplying the characteristic function of the safety-critical scenario set with the basic mean square error loss function.

[0116] The principle and effect of the above technical solution are as follows: The safety-critical scenario loss function module includes: an automatic differential unit for basic error, which automatically differentiates and calculates the basic mean square error loss function using a neural network training tool; the neural network for the autonomous vehicle driving strategy is set as a multilayer perceptron architecture; a parameter gradient operation unit, which calculates the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy; and a safety-critical scenario loss function operation unit, which calculates the safety-critical scenario loss function by multiplying the characteristic function of the safety-critical scenario set with the basic mean square error loss function.

[0117] In one embodiment, the driving strategy probability distribution learning module includes:

[0118] The driving strategy probability training unit trains the driving strategy probability distribution in safety-critical scenarios based on the loss function of the safety-critical scenario, taking into account the scarcity of driving behavior data.

[0119] The strategy distribution learning intelligent model unit is trained by the probability distribution of driving strategies, and after training, it forms an intelligent learning model of the probability distribution of driving strategies.

[0120] The principle and effect of the above technical solution are as follows: The driving strategy probability distribution learning module includes: a driving strategy probability training unit, which trains the driving strategy probability distribution in safety-critical scenarios based on the loss function of the safety-critical scenario, taking into account the scarcity of driving behavior data; and a strategy distribution learning intelligent model unit, which forms a driving strategy probability distribution intelligent learning model after training through the driving strategy probability distribution.

[0121] In one embodiment, the driving strategy probability model evaluation module includes:

[0122] The intelligent driving following verification unit extracts safety-critical driving data of autonomous vehicles in safety-critical scenarios, inputs the safety-critical driving data of autonomous vehicles into the intelligent driving following model, and forms a safety-critical intelligent driving following verification model.

[0123] The distributed learning intelligent model evaluation unit verifies the effectiveness and accuracy of the driving strategy probability distribution intelligent learning model for autonomous vehicles based on the safety-critical intelligent driving follow-up verification model; evaluates the driving strategy safety accuracy of the driving strategy probability distribution intelligent learning model; and improves the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

[0124] The principle and effect of the above technical solution are as follows: The driving strategy probability model evaluation module includes: an intelligent driving following verification unit, which extracts safety-critical driving data of autonomous vehicles in safety-critical scenarios, inputs the safety-critical driving data of autonomous vehicles into the intelligent driving following model, and forms a safety-critical intelligent driving following verification model; a distributed learning intelligent model evaluation unit, which verifies the effectiveness and accuracy of the driving strategy probability distribution intelligent learning model of autonomous vehicles based on the safety-critical intelligent driving following verification model; evaluates the safety accuracy of the driving strategy of the driving strategy probability distribution intelligent learning model; and improves the efficiency and accuracy of intelligent learning evaluation of driving strategies of autonomous vehicles; the learning results of the driving strategy probability distribution intelligent learning model of autonomous vehicles are as follows: Figure 3 As shown; Meansquared error represents the mean squared error; Ordinary method represents the mean squared error curve of the prior art; Our method represents the mean squared error curve of this patent; Safety-critical states represent safety-critical scenarios; Figure 3 The curves represent the mean square error curves in safety-critical scenarios, where curve 1 represents the mean square error curve of the prior art in safety-critical scenarios, and curve 2 represents the mean square error curve of the present invention in safety-critical scenarios; by Figure 3 The mean square error of the curve in this patent is significantly reduced; the method proposed in this patent can significantly and effectively learn the driving strategy of autonomous vehicles in safety-critical scenarios.

[0125] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. Other modifications can be easily made by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A method for intelligent learning and evaluation of driving strategies for autonomous vehicles, characterized in that, include: S100: Filter safety-critical scenario data from autonomous driving big data, construct a set of safety-critical scenarios, and obtain the indicator function of the set of safety-critical scenarios by combining indicator function operation; S200 performs automatic differential calculation of the basic mean square error loss function, and combines it with the characteristic function of the set of safety-critical scenarios to obtain the safety-critical scenario loss function; The S300, based on the loss function for safety-critical scenarios, forms an intelligent learning model of driving strategy probability distribution through training on the driving strategy probability distribution. S400 verifies the effectiveness and accuracy of the intelligent learning model for the probability distribution of driving strategies, and evaluates the safety accuracy of the intelligent learning model for the probability distribution of driving strategies. Improve the efficiency and accuracy of intelligent learning and evaluation of driving strategies for autonomous vehicles.

2. The intelligent learning and evaluation method for driving strategies of autonomous vehicles according to claim 1, characterized in that, S100 includes: S101, through the capture and processing of autonomous driving big data, filters safety-critical scenario data from autonomous driving big data; S102, Construct a set of security-critical scenarios based on security-critical scenario data; S103, Based on the set of safety-critical scenarios, and combined with the characteristic function operation, obtain the characteristic function of the set of safety-critical scenarios.

3. The intelligent learning and evaluation method for driving strategies of autonomous vehicles according to claim 1, characterized in that, S200 includes: S201, the basic mean square error loss function is automatically calculated by differentiation using a neural network training tool; the neural network for the autonomous vehicle driving strategy is set to a multi-layer perceptron architecture. S202, Calculate the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy; S203, based on the product of the characteristic function of the set of safety-critical scenarios and the basic mean square error loss function, the safety-critical scenario loss function.

4. The intelligent learning and evaluation method for driving strategies of autonomous vehicles according to claim 1, characterized in that, The S300 includes: S301, Based on the safety-critical scenario loss function, train the probability distribution of driving strategy in safety-critical scenarios where driving behavior data is scarce; S302 is trained using the probability distribution of driving strategies, and after training, an intelligent learning model of the probability distribution of driving strategies is formed.

5. The intelligent learning and evaluation method for driving strategies of autonomous vehicles according to claim 1, characterized in that, The S400 includes: S401, Extract safety-critical driving data of autonomous vehicles in safety-critical scenarios, input the safety-critical driving data of autonomous vehicles into the intelligent driving following model, and form a safety-critical intelligent driving following verification model. S402, based on the safety-critical intelligent driving following verification model, verify the effectiveness and accuracy of the intelligent learning model of the driving strategy probability distribution of driving data critical to the safety of autonomous vehicles; evaluate the safety accuracy of the driving strategy of the intelligent learning model of the driving strategy probability distribution; and improve the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

6. An intelligent learning and evaluation system for driving strategies of autonomous vehicles, characterized in that, include: The safety-critical scenario set module filters safety-critical scenario data from autonomous driving big data, constructs a safety-critical scenario set, and obtains the indicator function of the safety-critical scenario set by combining indicator function operations. The safety-critical scenario loss function module automatically differentiates and calculates the basic mean square error loss function, and combines it with the characteristic function of the safety-critical scenario set to obtain the safety-critical scenario loss function. The driving strategy probability distribution learning module trains a driving strategy probability distribution intelligent learning model based on the loss function of safety-critical scenarios and the driving strategy probability distribution. The driving strategy probability model evaluation module verifies the effectiveness and accuracy of the intelligent learning model of driving strategy probability distribution and evaluates the safety accuracy of the driving strategy based on the intelligent learning model of driving strategy probability distribution. Improve the efficiency and accuracy of intelligent learning and evaluation of driving strategies for autonomous vehicles.

7. The intelligent learning and evaluation system for driving strategies of autonomous vehicles according to claim 6, characterized in that, The security-critical scenario collection module includes: The safety-critical scenario data filtering unit filters safety-critical scenario data from autonomous driving big data through autonomous driving big data capture and processing. The security-critical scenario collection architecture unit constructs a collection of security-critical scenarios based on security-critical scenario data. The scenario set indicator function unit obtains the indicator function of the safety-critical scenario set by combining indicator function operations based on the safety-critical scenario set.

8. The intelligent learning and evaluation system for driving strategies of autonomous vehicles according to claim 6, characterized in that, The loss function module for safety-critical scenarios includes: The basic error automatic differentiation unit automatically differentiates and calculates the basic mean square error loss function using a neural network training tool; the neural network for the autonomous vehicle driving strategy is set as a multi-layer perceptron architecture. The parameter gradient computation unit calculates the parameter gradient of the training data loss function with respect to the probability distribution of the driving strategy; The safety-critical loss function calculation unit calculates the safety-critical scenario loss function by multiplying the characteristic function of the safety-critical scenario set with the basic mean square error loss function.

9. The intelligent learning and evaluation system for driving strategies of autonomous vehicles according to claim 6, characterized in that, The driving strategy probability distribution learning module includes: The driving strategy probability training unit trains the driving strategy probability distribution in safety-critical scenarios based on the loss function of the safety-critical scenario, taking into account the scarcity of driving behavior data. The strategy distribution learning intelligent model unit is trained by the probability distribution of driving strategies, and after training, it forms an intelligent learning model of the probability distribution of driving strategies.

10. The intelligent learning and evaluation system for driving strategies of autonomous vehicles according to claim 6, characterized in that, The driving strategy probabilistic model evaluation module includes: The intelligent driving following verification unit extracts safety-critical driving data of autonomous vehicles in safety-critical scenarios, inputs the safety-critical driving data of autonomous vehicles into the intelligent driving following model, and forms a safety-critical intelligent driving following verification model. The distributed learning intelligent model evaluation unit verifies the effectiveness and accuracy of the driving strategy probability distribution intelligent learning model for autonomous vehicles based on the safety-critical intelligent driving follow-up verification model; evaluates the driving strategy safety accuracy of the driving strategy probability distribution intelligent learning model; and improves the efficiency and accuracy of intelligent learning evaluation of driving strategies for autonomous vehicles.

Images