Test scene generation and screening method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a test scene generation and screening method, and relates to the technical field of intelligent driving test. The method comprises the steps of generating a test scene set according to a design operation boundary of an intelligent driving algorithm and influence factors of operation of the intelligent driving algorithm; under the condition that real vehicle test conditions or virtual driving platform test conditions exist, training a logistic regression classifier by using part of the test scene sets, determining risk levels of all scenes in the test scene sets by using the logistic regression classifier, and dividing the risk levels according to risk degrees perceived by a user for the test scene sets; and under the condition that a real vehicle test condition and a virtual driving platform test condition do not exist, constructing a user risk expectation model according to the design operation boundary and the dynamic risk source in the influence factors, and determining the risk level of each scene in the test scene set by using the user risk expectation model. The risk level of the test scene is determined based on the psychological risk expectation of the user, and the test efficiency of the intelligent driving algorithm is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving test technology, and more specifically, to a test scenario generation and screening method, device, electronic device and storage medium. Background Art

[0002] In this technical field, intelligent driving algorithms or functions (for example, autonomous driving algorithms or functions, assisted driving algorithms or functions) are required to be reliable when they are merged into the main line or integrated for release. That is, it is necessary to conduct sufficient and comprehensive tests when the intelligent driving algorithms or functions are completed, and modify the problems in the intelligent driving algorithms or functions based on the test results.

[0003] In the existing methods for generating and screening intelligent driving test scenarios, test scenario generation is generally generalized without purpose, resulting in a large number of test scenarios, low test efficiency, a wide screening range of test scenarios, and failure to generate specific scenarios for different intelligent driving algorithms or functions. In addition, the existing methods only screen the test scenarios for common and dangerous scenarios based on whether the vehicle has collided. This method is relatively simple and does not consider non-collision but dangerous scenarios, nor does it consider the driving experience of the user (i.e., user) of the intelligent driving algorithm or function when using it (for example, if the intelligent driving algorithm or function does not operate correctly in the driver's expectation of risk, it will affect the driver's driving experience, and even the driver may choose to actively intervene in the steering wheel and brakes in a state of psychological tension). As a result, the screened test scenarios are not targeted, and only the result of whether there is no collision can be obtained from the test. The entire test process is inefficient and does not consider the influence of the actual user and user in the process. Summary of the invention

[0004] The embodiments of the present application propose a test scenario generation and screening method, device, electronic device and storage medium to solve the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present application provides a test scenario generation and screening method, the method comprising: generating a test scenario set based on the design operation boundaries of an intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm; in the case of having actual vehicle testing conditions or virtual driving platform testing conditions, using part of the test scenario set to train a logistic regression classifier, and using the logistic regression classifier to determine the risk level of each scenario in the test scenario set, wherein the risk level is divided according to the degree of risk perceived by the user for the test scenario set; in the case of not having actual vehicle testing conditions and virtual driving platform testing conditions, constructing a user risk expectation model based on the design operation boundaries and dynamic risk sources in the influencing factors, and using the user risk expectation model to determine the risk level of each scenario in the test scenario set.

[0006] In a second aspect, an embodiment of the present application provides a test scenario generation and screening device, which includes: a scenario generation module, which is used to generate a test scenario set based on the design operation boundaries of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm; a classification evaluation module, which is used to train a logistic regression classifier using part of the test scenario set when real vehicle test conditions or virtual driving platform test conditions are available, and use the logistic regression classifier to determine the risk level of each scenario in the test scenario set, and the risk level is divided according to the degree of risk perceived by the user for the test scenario set; a threshold evaluation module, which is used to construct a user risk expectation model based on the design operation boundaries and dynamic risk sources in the influencing factors when real vehicle test conditions and virtual driving platform test conditions are not available, and use the user risk expectation model to determine the risk level of each scenario in the test scenario set.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein an application is stored in the memory, and the application is used to enable the processor to execute the method provided by the embodiment of the present application when called by the processor.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program code stored thereon, wherein the program code is used to enable the processor to execute the method provided by the embodiment of the present application when called by the processor.

[0009] The test scenario generation and screening method provided in the embodiment of the present application has the following technical effects: it can generate a test scenario set of the intelligent driving algorithm in a targeted manner based on the influencing factors and design operation boundaries of the intelligent driving algorithm, taking into account the influence of users in the use of the algorithm or function, and determining the risk level of the test scenario set based on the user's psychological risk expectations perceived by the test scenario, thereby realizing the screening of test scenarios with different risk levels, having reliability and anthropomorphism, and taking into account the scenarios in the existing test scenario screening methods that are risky but do not have collisions or may bring psychological oppression to users, so that subsequent test results based on the test scenarios can improve the user experience of the intelligent driving algorithm and improve the testing efficiency of the intelligent driving algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0011] Figure 1A schematic diagram of a test scenario and a screening method provided in an embodiment of the present application is shown;

[0012] Figure 2 A schematic diagram of the process of step S110 provided in an embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram of the process of step S120 provided in an embodiment of the present application is shown;

[0014] Figure 4 A schematic diagram of the process of step S130 provided in an embodiment of the present application is shown;

[0015] Figure 5 A schematic diagram of the structure of a test scenario generation and screening device provided in an embodiment of the present application is shown;

[0016] Figure 6 A schematic structural diagram of an electronic device provided in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0018] The test scenario generation and screening method in the embodiment of the present application is used to generate a test scenario and perform a risk level assessment on the test scenario. The test scenario generation and screening method can be applied to a test scenario generation and screening device or an electronic device. The test scenario generation and screening device can be integrated in an electronic device. The electronic device is used to test the intelligent driving algorithm. The electronic device may include but is not limited to a desktop computer, a laptop computer, a tablet computer, etc. The electronic device can be connected to a vehicle, a virtual driving platform, and a server (such as a cloud server) to achieve data interaction between devices, so that the electronic device can obtain real vehicle test data from a vehicle or a virtual driving platform and obtain natural driving data from a server.

[0019] Next, the test scenarios and screening methods in the embodiments of this application are introduced. Figure 1 , Figure 1 FIG. 1 is a flow chart showing a test scenario and a screening method provided by an embodiment of the present application. Figure 1 As shown, the test scenario generation and screening method may include steps S110 to S130.

[0020] Step S110: Generate a test scenario set according to the design operation boundary of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm.

[0021] The intelligent driving algorithm in the embodiments of the present application refers to an algorithm or function related to intelligent driving for which there is a test requirement. The intelligent driving algorithm may include but is not limited to algorithms related to automatic driving and assisted driving, for example, a planning control algorithm for intelligent cruise control.

[0022] The design operation boundary (also called the design operation domain) of the intelligent driving algorithm refers to the parameters and their value ranges involved in running the intelligent driving algorithm. Taking the planning control algorithm as an example, the planning control algorithm may include but is not limited to the following operation boundaries: vehicle speed (15kph~130kph), illumination (30lux~150k lux), acceleration (-5m / s2~5m / s2), temperature (≥5℃), lane width (2.3m~5.5m), etc.

[0023] The influencing factors of the operation of the intelligent driving algorithm refer to the parameters and their value ranges that affect the operation of the intelligent driving algorithm. The influencing factors may affect the user's driving experience or driving safety, and the parameters in the influencing factors also have their own value ranges. Taking the planning control algorithm as an example, assuming that the planning control algorithm only tests the situation where there are vehicles in the adjacent lanes, the planning control algorithm may include but is not limited to the following influencing factors: the speed of the vehicle, the acceleration of the vehicle, the longitudinal speed of the vehicle in the adjacent lane, the lateral speed of the vehicle in the adjacent lane, the longitudinal distance to the adjacent vehicle, the lateral distance to the adjacent vehicle, light, temperature, lane width, etc.

[0024] A test scenario set refers to a collection of test scenarios. A test scenario refers to a scenario used to test an intelligent driving algorithm. The test scenario in the embodiment of the present application is generated by targeted generalization based on the design and operation boundaries of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm, so that targeted test scenarios can be generated for different algorithms, thereby improving the accuracy of subsequent test results and the testing efficiency of the algorithm.

[0025] See also Figure 2 , Figure 2 FIG. 2 shows a flow chart of step S110 provided in an embodiment of the present application. Figure 2 As shown, step S110 may include steps S111 to S114.

[0026] Step S111: extracting the design operation boundary of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm.

[0027] The design operation boundary of the intelligent driving algorithm can be extracted to generate a sample data set D of the design operation boundary. n =[D1, D2, ..., D n ], where D iRepresents the value range of each design operating boundary parameter. Outside the range, the algorithm will not be turned on and there is no need for testing. Taking the planning control algorithm as an example, the planning control algorithm may include but is not limited to the following operating boundaries: vehicle speed (15kph~130kph), light (30lux~150klux), acceleration (-5m / s2~5m / s2), temperature (≥5℃), lane width (2.3m~5.5m), that is, n=4.

[0028] The influencing factors of the intelligent driving algorithm can be extracted to generate a sample data set of influencing factors. n =[I1, I2, …, I n ], where I i Represents the value range of each influencing factor parameter. Taking the planning control algorithm as an example, assuming that the planning control algorithm only tests the situation where there are vehicles in the adjacent lanes, the planning control algorithm may include but is not limited to the following influencing factors: own vehicle speed, own vehicle acceleration, longitudinal speed of vehicles in adjacent lanes, lateral speed of vehicles in adjacent lanes, longitudinal distance to adjacent vehicles, lateral distance to adjacent vehicles, illumination, temperature, lane width, that is, I=9.

[0029] Step S112: Combine the design operation boundary and the influencing factors to obtain a test scenario key parameter set.

[0030] In some embodiments, it is possible to identify whether the design operation boundary and the influencing factors can be directly combined according to the requirements of the intelligent driving algorithm. If the design operation boundary and the influencing factors can be directly combined, the design operation boundary and the influencing factors are directly combined. If the design operation boundary and the influencing factors cannot be directly combined, at least one factor in the influencing factors that has the least impact on the implementation of the intelligent driving algorithm or a factor that cannot be tested is deleted.

[0031] Specifically, it is possible to identify whether the design operating boundary and the influencing factors can be directly combined based on the actual test requirements of the intelligent driving algorithm. For example, assuming that the speed value range in the design operating boundary is 30kph~100kph, and the speed value range in the influencing factor is 50kph~90kph, then the combination of the two can directly take the speed value range of 50kph~90kph. This is the case where the design operating boundary and the influencing factor can be directly combined. If the speed value ranges in the design operating boundary and the speed in the influencing factor are ambiguous, whether the speed in the design operating boundary and the influencing factor can be directly combined depends on the factors that need to be actually considered in the intelligent driving test scenario / environment.

[0032] In some embodiments, in the process of combining the design operating boundaries and influencing factors, when there are repeated parameters in the design operating boundaries and influencing factors, the parameter value range in the design operating boundaries is obtained as the value range of the repeated parameters, that is, if there is repetition in the design operating boundaries and influencing factors, the operating parameter boundaries of the intelligent driving algorithm or function shall prevail.

[0033] Sample data set D according to the design run boundary n =[D1, D2, ..., D n ] and sample dataset of influencing factors I n =[I1, I2, …, I n ], combine the design operation boundary and influencing factors to obtain the test scenario key parameter set K = [D1, D2, ..., D m , I1, I2, …, I m ]. Wherein, m represents the number of design operation boundary parameters and influencing factor parameters obtained after combination.

[0034] Taking the planning control algorithm as an example, assuming that the test is carried out on a simulation platform and the vehicle hardware is in an ideal condition, there is no need to consider the influence of temperature, etc. Therefore, the temperature factor is deleted, and finally the key parameter set of the test scenario of the planning control algorithm is obtained: K = [vehicle speed, vehicle acceleration, lane width, lighting, longitudinal speed of vehicles in adjacent lanes, lateral speed of vehicles in adjacent lanes, longitudinal distance to adjacent vehicles, lateral distance to adjacent vehicles].

[0035] Step S113: Determine the value range of each parameter in the test scenario key parameter set.

[0036] After obtaining the key parameter set of the test scenario K = [D1, D2, ..., D m , I1, I2, …, I m ] After that, the value range of each parameter in the key parameter set of the test scenario can be further determined in combination with relevant regulatory requirements and driving safety requirements.

[0037] Taking the planning control algorithm as an example, considering the vehicle speed limit, vehicle physical limit, functional safety limit, value significance and other requirements in the test scenario, the value range of each parameter in the test scenario key parameter set of the planning control algorithm can be further determined, for example, the longitudinal speed of vehicles in adjacent lanes (5kph~125kph), the lateral speed of vehicles in adjacent lanes (0kph~15kph), etc., which will not be elaborated one by one.

[0038] Step S114: Determine the sampling interval of each parameter in the test scenario key parameter set to obtain the test scenario set.

[0039] The sampling interval of each parameter in the key parameter set of the test scenario can be determined to obtain the test scenario set (i.e., the basic scenario set) of the intelligent driving algorithm or function. For example, the sampling interval of the vehicle speed can be set to 1 kph, or only some typical parameter values ​​can be extracted. The specific sampling interval or some typical parameter values ​​can be calibrated according to the test requirements of the intelligent driving algorithm, and this application does not make specific restrictions.

[0040] After obtaining the test scenario set, it can be determined whether the actual vehicle test conditions or the virtual driving platform test conditions are currently met. If the actual vehicle test conditions or the virtual driving platform test conditions are met, proceed to step S120, and a partial test scenario set can be used to train a logistic regression classifier based on the principle of logistic regression, and use the logistic regression classifier to determine the risk level of each scenario in the test scenario set, and the risk level is divided according to the degree of risk perceived by the user for the test scenario set. If the actual vehicle test conditions and the virtual driving platform test conditions are not met, proceed to step S130, and a user risk expectation model can be constructed based on the design operation boundary and the dynamic risk sources in the influencing factors, and the user risk expectation model can be used to determine the risk level of each scenario in the test scenario set in combination with natural driving data.

[0041] Step S120: When the conditions for real vehicle testing or virtual driving platform testing are met, a logistic regression classifier is trained using a partial test scenario set, and the logistic regression classifier is used to determine the risk level of each scenario in the test scenario set, wherein the risk level is divided according to the degree of risk perceived by the user for the test scenario set.

[0042] See also Figure 3 , Figure 3 FIG. 2 shows a flow chart of step S120 provided in an embodiment of the present application. Figure 3 As shown, step S120 may include steps S121 to S124.

[0043] Step S121: extracting some test scenes from the test scene set as test samples.

[0044] In some embodiments, some test scenarios can be randomly selected from the test scenario set of the intelligent driving algorithm as test samples. Randomly selecting samples can make the reliability and accuracy of the trained logistic regression classifier training samples higher, and can achieve the efficiency of selecting test samples and improve the overall test efficiency.

[0045] In some embodiments, some test scenarios may be extracted from the test scenario set of the intelligent driving algorithm as test samples according to certain rules, wherein the rules may be set according to actual needs. For example, a certain extraction interval may be set according to the sampling interval of each parameter in the design operating boundary and the influencing factors, and some test scenarios may be extracted from the test scenario set as test samples according to the extraction interval.

[0046] Step S122: obtaining test results of different users for the test sample based on real vehicle tests or virtual driving platforms, wherein the test results include multiple risk levels.

[0047] Different users can be selected to conduct real-car tests on each test scenario in the test sample, and obtain test results of different users based on real-car tests or virtual driving platforms for the test sample. Among them, the test results can be classified into the following multiple test results and multiple risk levels, and each test result corresponds to a risk level:

[0048] (1) The user completes the test normally and does not perceive any risk, and the risk level is no risk;

[0049] (2) The user perceives a certain risk but no risk occurs, and the risk level is low;

[0050] (3) If the user perceives the risk to be so great that he needs to take action and successfully avoids the collision after taking the action, the risk level is medium risk;

[0051] (4) If the user perceives the risk to be so great that he needs to take action and a collision still occurs after taking the action, the risk level is high.

[0052] Step S123: using the test results to train a logistic regression classifier to obtain a logistic regression classifier, wherein the logistic regression classifier is used to classify the risk level of the test scenario into one of the multiple risk levels.

[0053] Based on the principle of logistic regression, the test result obtained in step S122 can be used to train a logistic regression classifier. If the training result does not meet expectations (i.e., it does not meet the pre-set model training end condition, such as the model loss is too high), then continue to extract test samples for training according to the steps of step S121 to step S123 until the training result meets expectations and a logistic regression classifier is obtained. For example, the logistic regression classifier may include a user high risk expectation classifier, a user medium risk expectation classifier, a user low risk expectation classifier, and a user no risk expectation classifier. The user high risk expectation classifier can be used to screen test scenarios with high risk levels. The user medium risk expectation classifier can be used to screen test scenarios with medium risk levels. The user low risk expectation classifier can be used to screen test scenarios with low risk levels. The user no risk expectation classifier can be used to screen test scenarios with no risk levels. It is understandable that since there is no need to test the risk-free test scenario, it is not necessary to train the risk-free test scenario, and only the high risk expectation classifier, the user medium risk expectation classifier, and the user low risk expectation classifier can be trained.

[0054] Step S124: Use a logistic regression classifier to determine the risk level of each scenario in the test scenario set.

[0055] Each test scenario in the test scenario set is input into a trained logistic regression classifier. The logistic regression classifier can automatically classify the test scenario into one of a low-risk level test scenario, a medium-risk level test scenario, and a high-risk level test scenario, thereby realizing the screening of different risk levels of the test scenarios in the test scenario set.

[0056] Step S130: In the absence of real vehicle testing conditions and virtual driving platform testing conditions, a user risk expectation model is constructed based on the design operation boundaries and dynamic risk sources in the influencing factors, and the user risk expectation model is used to determine the risk level of each scenario in the test scenario set.

[0057] See also Figure 4 , Figure 4 FIG. 2 shows a flow chart of step S130 provided in an embodiment of the present application. Figure 4 As shown, step S130 may include steps S131 to S135.

[0058] Step S131: Extracting dynamic risk sources relative to the vehicle from the design operation boundary and influencing factors.

[0059] Dynamic risk sources can refer to parameters in the design operation boundary and influencing factors that may make users perceive risks. Dynamic risk sources relative to the vehicle in the design operation boundary and influencing factors can be extracted. Specifically, the key parameter set of the test scenario K = [D1, D2, ..., D m, I1, I2, …, I m ] to extract the dynamic risk sources relative to the vehicle. Taking the planning control algorithm as an example, the dynamic risk sources relative to the vehicle are only adjacent vehicles.

[0060] Step S132: Propose a risk quantification function based on the parameters related to the dynamic risk source, take the remaining parameters after removing the parameters related to the dynamic risk source from the design operation boundary and influencing factors as risk coefficients, and build a user risk expectation model.

[0061] A risk quantification function can be proposed based on the parameters related to the dynamic risk source, and the remaining parameters after removing the parameters related to the dynamic risk source from the design operation boundary and influencing factors are used as risk coefficients to quantify the degree of risk perceived by users and construct a user risk expectation model P:

[0062] P=ΠD u ΠI u ∑E o (D1, ..., D o , I1,…,I o )

[0063] in, and It represents the product of the risk coefficients of the remaining parameters for the user's psychology after removing the parameters related to the dynamic risk source. Each coefficient is calculated based on a certain formula as the parameters change. If the influencing factor is a fixed value, it can also be calibrated according to the risk level that the user can feel based on the factor; and represents the risk quantification function between each dynamic risk source and its associated parameters.

[0064] Since the risk intensity of dynamic risk sources for users will change with the relative speed between the dynamic risk source and the vehicle, this application introduces the Doppler effect influence coefficient ω o , used to quantify that when the dynamic risk source is close to the user, the pressure received by the user will increase exponentially, and when the dynamic risk source is far away from the user, the pressure received by the user will decrease exponentially. Based on this, in some embodiments, the Doppler effect influence coefficient can be determined according to the relative relationship between the dynamic risk source and the vehicle; according to the Doppler effect influence coefficient and the user risk expectation model, the final user risk expectation model is constructed. Among them, the relative relationship between the dynamic risk source and the vehicle may include but is not limited to the relative speed, relative distance, relative angle, etc. between the dynamic risk source and the vehicle.

[0065] For example, the Doppler effect influence coefficient ω can be determined according to the relative relationship between the dynamic risk source and the vehicle according to the following expression: o :

[0066]

[0067] Where β = v / v max , v represents the relative speed between the ego vehicle and the dynamic risk source. When the ego vehicle and the dynamic risk source are relatively close, v is positive, and when the ego vehicle and the dynamic risk source are relatively far away, v is negative. max It indicates the relatively larger speed between the vehicle and the dynamic risk source; θ indicates the angle (i.e., relative angle) between the line connecting the vehicle and the dynamic risk source and the speed direction.

[0068] For example, the final user risk expectation model P can be constructed according to the following expression based on the Doppler effect influence coefficient and the user risk expectation model:

[0069] P=ω o ΠD u ΠI u ∑E oi (D1, ..., D o , I1,…,I o )

[0070] The user risk expectation model can be used to output the risk quantification value for the test scenario. That is, by inputting the test scenario parameters into the user risk expectation model, the risk quantification value of the test scenario can be obtained. The risk quantification value is used to measure the risk level of the test scenario.

[0071] Taking the planning control algorithm as an example, the only dynamic risk source is the adjacent vehicle. According to the relevant parameters of the adjacent vehicles, the method of introducing the potential field of relative speed is used to calculate the risk quantification function of the dynamic risk source. The dynamic risk source is characterized as having a repulsive potential field for the ego vehicle. The repulsive potential field is affected by the relative distance and relative speed between the dynamic risk source and the ego vehicle, and there is a force on the ego vehicle in the direction of the negative gradient of its potential field. Therefore, the user risk expectation model P that the user can feel in the planning control algorithm is:

[0072] P=ω o D w D c E o

[0073] Among them, ω o Indicates the Doppler effect coefficient; D w The formula that represents the impact of lane width changes on the risk level that users can feel, D c is the formula for the impact of lighting changes on the risk level that users can feel, E o is a function of the risk level that the user can perceive from the neighboring vehicles.

[0074] Step S133: Calibrate the threshold of the user risk expectation model based on the natural driving data to obtain multiple thresholds, where the multiple thresholds correspond to different risk levels.

[0075] Natural driving scenarios refer to real scenarios where human drivers interact with surrounding traffic participants and traffic infrastructure while driving on a road. Natural driving data refers to data collected when a vehicle is driving in a natural driving scenario. Natural driving data may include, but is not limited to: user reaction information (e.g., heart rate acceleration, reaction, reaction results), vehicle status information (e.g., vehicle speed, acceleration, heading angle, etc.), traffic environment information (e.g., number of lanes, lane width), path selection, user driving habits, weather conditions, etc. Natural driving data comes from the real world and has high data diversity. It can provide basic data for the training and testing of intelligent driving algorithms (e.g., perception recognition algorithms and planning control algorithms, etc.), and provide logical ideas for the development of decision-making and control algorithms.

[0076] The threshold of the user risk expectation model can be calibrated using natural driving data. Specifically, the user risk expectation can be expressed as no reaction, accelerated heartbeat, reaction, reaction result and other specific actions. According to the user risk expectation performance, multiple thresholds of the user risk expectation model can be calibrated, and the multiple thresholds correspond to different risk levels. For example, the low risk threshold that the user can bear can be calibrated as P l , the medium risk threshold at which the user will take action is P m , the high risk threshold that the user cannot prevent the collision even if he takes action is P h Among them, the values ​​of high risk threshold, medium risk threshold, and low risk threshold decrease in sequence.

[0077] Step S134: input each test scenario in the test scenario set into the user risk expectation model to obtain a risk quantification value for each test scenario.

[0078] After calibrating the threshold of the user risk expectation model, each test scenario in the test scenario set can be input into the user risk expectation model to obtain a risk quantification value for each test scenario.

[0079] Step S135: Determine the risk level of each test scenario according to the multiple thresholds and the risk quantification value of each test scenario.

[0080] For each test scenario, the risk quantification value of the test scenario is compared with multiple thresholds. If the risk quantification value of the test scenario is less than the low risk threshold, the test scenario is a risk-free scenario. If the risk quantification value of the test scenario is greater than or equal to the low risk threshold and less than the medium risk threshold, the test scenario is a low risk scenario. If the risk quantification value of the test scenario is greater than or equal to the medium risk threshold and less than the high risk threshold, the test scenario is a medium risk scenario. If the risk quantification value of the test scenario is greater than or equal to the high risk threshold, the test scenario is a high risk scenario.

[0081] Steps S110 to S130 have the following technical effects: based on the influencing factors and design operation boundaries of the intelligent driving algorithm, a test scenario set of the intelligent driving algorithm can be generated in a targeted manner, the influence of the user on the use of the algorithm or function is taken into account, and the risk level of the test scenario set is determined based on the user's psychological risk expectations perceived by the test scenario, thereby realizing the screening of test scenarios with different risk levels, with reliability and anthropomorphism, and taking into account the existing test scenario screening methods. The scenarios that are risky but do not have collisions or may bring psychological oppression to users, so that subsequent test results based on the test scenarios can improve the user experience of the intelligent driving algorithm and improve the testing efficiency of the intelligent driving algorithm.

[0082] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram showing the structure of a test scenario generation and screening device provided by an embodiment of the present application. Figure 5 The test scenario generation and screening device 100 may include: a scenario generation module 110 , a classification evaluation module 120 , and a threshold evaluation module 130 .

[0083] The scenario generation module 110 is used to generate a test scenario set according to the design operation boundary of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm.

[0084] The classification and evaluation module 120 is used to: when real vehicle testing conditions or virtual driving platform testing conditions are met, use a partial test scenario set to train a logistic regression classifier, and use the logistic regression classifier to determine the risk level of each scenario in the test scenario set, and the risk level is divided according to the degree of risk perceived by the user for the test scenario set.

[0085] The threshold assessment module 130 is used to: in the absence of real vehicle testing conditions and virtual driving platform testing conditions, build a user risk expectation model based on the design operation boundaries and dynamic risk sources in the influencing factors, and use the user risk expectation model to determine the risk level of each scenario in the test scenario set.

[0086] In some embodiments, the classification and evaluation module 120 is used to: extract some test scenarios from the test scenario set as test samples; obtain test results of different users for the test samples based on real vehicle tests or virtual driving platforms, and the test results include multiple risk levels; use the test results to train a logistic regression classifier to obtain a logistic regression classifier, and the logistic regression classifier is used to divide the risk level of the test scenario into one of the multiple risk levels.

[0087] In some embodiments, the threshold assessment module 130 is used to: extract dynamic risk sources relative to the vehicle from the design operating boundaries and influencing factors; propose a risk quantification function based on the parameters related to the dynamic risk sources, and use the remaining parameters after removing the parameters related to the dynamic risk sources from the design operating boundaries and influencing factors as risk coefficients to construct a user risk expectation model.

[0088] In some embodiments, the threshold evaluation module 130 is used to: determine the Doppler effect influence coefficient according to the relative relationship between the dynamic risk source and the vehicle; and construct a final user risk expectation model according to the Doppler effect influence coefficient and the user risk expectation model.

[0089] In some embodiments, the threshold assessment module 130 is used to: calibrate the threshold of the user risk expectation model based on natural driving data to obtain multiple thresholds, and the multiple thresholds correspond to different risk levels; input each test scenario in the test scenario set into the user risk expectation model to obtain a risk quantification value for each test scenario; determine the risk level of each test scenario based on the multiple thresholds and the risk quantification value of each test scenario.

[0090] In some embodiments, the scenario generation module 110 is used to: extract the design operation boundaries of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm; combine the design operation boundaries and the influencing factors to obtain a test scenario key parameter set; determine the value range of each parameter in the test scenario key parameter set; determine the sampling interval of each parameter in the test scenario key parameter set to obtain a test scenario set.

[0091] In some embodiments, the scenario generation module 110 is used to: when the design operation boundary and the influencing factors cannot be directly combined, delete at least one factor among the influencing factors that has the least impact on the implementation of the intelligent driving algorithm or a factor that cannot be tested; or when there are repeated parameters in the design operation boundary and the influencing factors, obtain the parameter value range in the design operation boundary as the value range of the repeated parameter.

[0092] Those skilled in the art can clearly understand that the above device provided in the embodiment of the present application can implement the method provided in the embodiment of the present application. The specific working process of the above-described device and module can refer to the corresponding process of the method in the embodiment of the present application, which will not be repeated here.

[0093] In the embodiments provided in the present application, the coupling, direct coupling or communication connection between the modules shown or discussed may be indirect coupling or communication coupling through some interfaces, devices or modules, and may be electrical, mechanical or other forms, and the embodiments of the present application do not impose specific limitations on this.

[0094] In addition, each functional module in the embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0095] See also Figure 6 , Figure 6 FIG. 1 is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present application. Figure 6 As shown, the electronic device 200 may include a memory 210 and a processor 220, wherein the memory 210 stores an application program, and the application program is configured to enable the processor 220 to execute the test scenario generation and screening method provided in an embodiment of the present application when called by the processor 220.

[0096] The processor 220 may include one or more processing cores. The processor 220 uses various interfaces and lines to connect various parts of the entire electronic device 200, and is used to run or execute instructions, programs, code sets or instruction sets stored in the memory 210, and call to run or execute data stored in the memory 210, perform various functions of the electronic device 200 and process data.

[0097] The processor 220 can be implemented in at least one of the following hardware forms: digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 220 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 220, but may be implemented separately through a communication chip.

[0098] The memory 210 may include a random access memory (RAM) or a read-only memory (ROM). The memory 210 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 210 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may store data created by the electronic device 200 during use, etc.

[0099] The embodiment of the present application further provides a computer-readable storage medium having a program code stored thereon, the program code being configured to execute the method provided in the embodiment of the present application when called by a processor.

[0100] The computer-readable storage medium may be an electronic memory such as a flash memory, an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a hard disk or a ROM.

[0101] In some embodiments, the computer readable storage medium includes a non-volatile computer readable medium (Non-Transitory Computer-Readable Storage Medium, referred to as Non-TCRSM). The computer readable storage medium has a storage space for the program code that executes any method step in the above method. These program codes can be read from or written into one or more computer program products. The program code can be compressed in an appropriate form.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A test scenario generation and screening method, characterized in that: include: Generate a set of test scenarios based on the design and operation boundaries of the intelligent driving algorithm and the factors affecting the operation of the intelligent driving algorithm; In the case of real vehicle testing conditions or virtual driving platform testing conditions, a logistic regression classifier is trained using part of the test scenario set, and the risk level of each scenario in the test scenario set is determined using the logistic regression classifier, wherein the risk level is divided according to the degree of risk perceived by the user for the test scenario set; In the absence of real vehicle testing conditions and virtual driving platform testing conditions, a user risk expectation model is constructed based on the design operation boundaries and dynamic risk sources in the influencing factors, and the user risk expectation model is used to determine the risk level of each scenario in the test scenario set.

2. The method according to claim 1, characterized in that The method of using a partial test scene set to train a logistic regression classifier includes: Extract some test scenarios from the test scenario set as test samples; Obtaining test results of different users on the test sample based on real vehicle testing or a virtual driving platform, wherein the test results include multiple risk levels; The test results are used to train a logistic regression classifier to obtain a logistic regression classifier, and the logistic regression classifier is used to classify the risk level of the test scenario into one of the multiple risk levels.

3. The method according to claim 1, characterized in that The user risk expectation model is constructed according to the design operation boundary and the dynamic risk sources in the influencing factors, including: Extract dynamic risk sources relative to the vehicle from the design operation boundaries and influencing factors; A risk quantification function is proposed based on the parameters related to the dynamic risk sources. The remaining parameters after removing the parameters related to the dynamic risk sources from the design operation boundaries and influencing factors are used as risk coefficients to construct a user risk expectation model.

4. The method according to claim 3, characterized in that After proposing a risk quantification function based on the parameters related to the dynamic risk source, taking the remaining parameters after removing the parameters related to the dynamic risk source from the design operation boundary and the influencing factors as the risk coefficient, and constructing the user risk expectation model, the method further includes: Determining a Doppler effect influence coefficient according to a relative relationship between the dynamic risk source and the vehicle; According to the Doppler effect influence coefficient and the user risk expectation model, the final user risk expectation model is constructed.

5. The method according to claim 1, characterized in that The using the user risk expectation model to determine the risk level of each scenario in the test scenario set includes: The threshold of the user risk expectation model is calibrated based on natural driving data to obtain multiple thresholds, each of which corresponds to a different risk level. Input each test scenario in the test scenario set into the user risk expectation model to obtain a risk quantification value for each test scenario; The risk level of each test scenario is determined according to the multiple thresholds and the risk quantification value of each test scenario.

6. The method according to claim 1, characterized in that The generating of a test scenario set according to the design operation boundary of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm includes: Extract the design and operation boundaries of intelligent driving algorithms and the factors affecting the operation of intelligent driving algorithms; Combining the design operation boundary and the influencing factors to obtain a test scenario key parameter set; Determine the value range of each parameter in the key parameter set of the test scenario; Determine the sampling interval of each parameter in the test scenario key parameter set to obtain the test scenario set.

7. The method according to claim 6, characterized in that The combining of the design operation boundary and the influencing factors includes: In the case where the design operation boundary and the influencing factors cannot be directly combined, deleting at least one factor among the influencing factors that has the least impact on the implementation of the intelligent driving algorithm or a factor that cannot be tested; or When there are repeated parameters in the design operation boundary and the influencing factors, the parameter value range in the design operation boundary is obtained as the value range of the repeated parameter.

8. A test scenario generation and screening device, characterized in that: include: A scenario generation module is used to generate a test scenario set based on the design operation boundary of the intelligent driving algorithm and the influencing factors of the operation of the intelligent driving algorithm; A classification and assessment module, for training a logistic regression classifier using a portion of the test scenario set when real vehicle test conditions or virtual driving platform test conditions are available, and for determining the risk level of each scenario in the test scenario set using the logistic regression classifier, wherein the risk level is divided according to the degree of risk perceived by the user for the test scenario set; The threshold assessment module is used to build a user risk expectation model based on the dynamic risk sources in the design operation boundaries and influencing factors when there are no real vehicle testing conditions and virtual driving platform testing conditions, and use the user risk expectation model to determine the risk level of each scenario in the test scenario set.

9. An electronic device, characterized in that: include: A memory and a processor, wherein an application is stored in the memory, and the application is used to enable the processor to execute the method according to any one of claims 1 to 7 when called by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and when the program codes are called by a processor, the processor is used to execute the method according to any one of claims 1 to 7.

Images