Method, device and equipment for evaluating vehicle lane change and storage medium

Abstract

The present application relates to the technical field of automatic driving, and in particular to a vehicle lane changing evaluation method, device, equipment and storage medium. Simulation data of a vehicle automatic driving system in a lane changing scene is obtained; the type of the lane changing scene is determined based on the simulation data, and an index evaluation standard is determined based on the type; reference data is selected according to the index evaluation standard, and an index set is generated based on the reference data; the vehicle automatic driving system is evaluated based on the index set; thereby solving the problem that the existing technology cannot effectively evaluate the vehicle lane changing of the automatic driving system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to a vehicle lane changing evaluation method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of automatic driving technology, in the process of automatic driving and analysis feedback of automatic driving case data, how to evaluate the vehicle automatic driving system is an important problem in the technical field.

[0003] In the existing evaluation scheme, a comparative experiment is usually used to evaluate the artificially preset driving characteristics, but there is currently no corresponding automatic evaluation method for the case of vehicle lane changing due to path planning. In the case of vehicle lane changing due to path planning, the factors to be considered are complex, and how to evaluate the advantages and disadvantages of different automatic driving systems in vehicle lane changing through a technical scheme has become a problem to be solved.

[0004] Therefore, there is an urgent need to provide a scheme that can effectively evaluate the vehicle lane changing of the automatic driving system. SUMMARY

[0005] The main purpose of the present application is to solve the problem that the existing technology cannot effectively evaluate the vehicle lane changing of the automatic driving system.

[0006] The first aspect of the present application provides a vehicle lane changing evaluation method, which comprises: obtaining simulation data of a vehicle automatic driving system in a lane changing scene; determining the type of the lane changing scene based on the simulation data, and determining an index evaluation standard based on the type; selecting reference data according to the index evaluation standard, and generating an index set based on the reference data; and evaluating the vehicle automatic driving system based on the index set.

[0007] Optionally, in the first implementation manner of the first aspect of the present application, the obtaining of the simulation data of the vehicle automatic driving system in the lane changing scene comprises: reading a simulation data set of the vehicle automatic driving system; extracting a driving scene and lane information corresponding to the driving scene from the simulation data set, wherein the lane information comprises a lane identifier and a lane type; determining whether the lane identifier changes in a preset time period; if yes, determining whether the lane belongs to a merging lane according to the lane type; if not, determining that the driving scene is a lane changing scene, and obtaining simulation data corresponding to the lane changing scene.

[0008] Optionally, in a second implementation form of the first aspect of the present application, the reference data comprises human driving data and historical simulation data corresponding to a history of the vehicle automatic driving system, and the index set comprises a first index set and a second index set; the generating the index set based on the reference data comprises: extracting a corresponding reference lane-changing scene from the human driving data according to the lane-changing scene, and extracting a corresponding historical lane-changing scene from the historical simulation data according to the lane-changing scene; comparing the lane-changing scene with the reference lane-changing scene to obtain a first comparison result, and generating the first index set according to the first comparison result; comparing the lane-changing scene with the historical lane-changing scene to obtain a second comparison result, and generating the second index set according to the second comparison result.

[0009] Optionally, in a third implementation form of the first aspect of the present application, the generating the first index set according to the first comparison result comprises: determining whether a corresponding lane-changing scene is a correct lane-changing scene according to the first comparison result, and calculating a number of the correct lane-changing scenes and a total number of the lane-changing scenes corresponding to the lane-changing scene; extracting a lane-changing time difference and a lane-changing position difference between the reference lane-changing scene and the corresponding correct lane-changing scene from the first comparison result; generating a lane-changing time difference set based on the lane-changing time difference, and generating a lane-changing position difference set based on the lane-changing position difference; and generating the first index set based on the number of the correct lane-changing scenes, the total number of the lane-changing scenes, the lane-changing time difference set and the lane-changing position difference set.

[0010] Optionally, in a fourth implementation form of the first aspect of the present application, the second index set comprises a second number of correct lane-changing scenes, a second total number of lane-changing scenes, a historical lane-changing time difference set and a historical lane-changing position difference set; the evaluating the vehicle automatic driving system based on the index set comprises: calculating a time difference value according to the lane-changing time difference set and the historical lane-changing time difference set to obtain a time difference set, and calculating a position difference value according to the lane-changing position difference set and the historical lane-changing position difference set to obtain a position difference set; calculating a score and a corresponding score factor according to the number of the correct lane-changing scenes, the total number of the lane-changing scenes, the second number of correct lane-changing scenes, the second total number of lane-changing scenes, the time difference set and the position difference set; and evaluating the vehicle automatic driving system according to the score and the score corresponding score factor.

[0011] Optionally, in a fifth implementation form of the first aspect of the present application, the score comprises a first score, a second score and a third score, and the score factor comprises a first score factor, a second score factor and a third score factor; and the calculating the score and the corresponding score factor according to the number of correct lane-changing scenarios, the total number of scenarios, the number of second correct lane-changing scenarios, the total number of second scenarios, the time difference set and the position difference set comprises: calculating the first score and the first score factor according to the number of correct lane-changing scenarios, the total number of scenarios, the number of second correct lane-changing scenarios and the total number of second scenarios; calculating the second score and the second score factor according to the time difference set; and calculating the third score and the third score factor according to the position difference set.

[0012] Optionally, in a sixth implementation form of the first aspect of the present application, the evaluating the vehicle automatic driving system according to the score and the score factor corresponding to the score comprises: determining whether the first score factor is greater than a preset first score threshold; if the first score factor is greater than the first score threshold, evaluating the target automatic driving system according to the first score to obtain a first evaluation result; if the first score factor is not greater than the first score threshold, determining whether the second score factor is greater than a preset second score threshold and determining whether the third score factor is greater than a preset third score threshold; if the second score factor is greater than the second score threshold, evaluating the target automatic driving system according to the second score to obtain a second evaluation result; and if the third score factor is greater than the third score threshold, evaluating the target automatic driving system according to the third score to obtain a third evaluation result.

[0013] The second aspect of the present application provides an evaluation device for vehicle lane-changing, which comprises: an acquisition module configured to acquire simulation data of a vehicle automatic driving system in a lane-changing scenario; a determination module configured to determine a type of the lane-changing scenario based on the simulation data, and determine an index evaluation standard based on the type; a generation module configured to select reference data according to the index evaluation standard, and generate an index set based on the reference data; and an evaluation module configured to evaluate the vehicle automatic driving system based on the index set.

[0014] Optionally, in the first implementation manner of the second aspect of the present application, the obtaining module comprises: a reading unit, configured to read the simulation data set of the vehicle automatic driving system; a first extracting unit, configured to extract the driving scene and the lane information corresponding to the driving scene from the simulation data set, wherein the lane information comprises a lane identifier and a lane type; a judging unit, configured to judge whether the lane identifier changes in a preset time period; if yes, judge whether the lane belongs to a merging lane according to the lane type; if not, determine that the driving scene is a lane changing scene, and obtain the simulation data corresponding to the lane changing scene.

[0015] Optionally, in the second implementation manner of the second aspect of the present application, the generating module comprises: a second extracting unit, configured to extract a corresponding reference lane changing scene from the human driving data according to the lane changing scene, and extract a corresponding historical lane changing scene from the historical simulation data according to the lane changing scene; a first comparing unit, configured to compare the lane changing scene with the reference lane changing scene to obtain a first comparison result, and generate a first index set according to the first comparison result; a second comparing unit, configured to compare the lane changing scene with the historical lane changing scene to obtain a second comparison result, and generate a second index set according to the second comparison result.

[0016] Optionally, in the third implementation manner of the second aspect of the present application, the first comparing unit is further configured to: determine whether the corresponding lane changing scene is a correct lane changing scene according to the first comparison result, and calculate the number of correct lane changing scenes and the total number of scenes corresponding to the lane changing scene; extract the lane changing time difference and the lane changing position difference between the reference lane changing scene and the corresponding correct lane changing scene from the first comparison result; generate a lane changing time difference set based on the lane changing time difference, and generate a lane changing position difference set based on the lane changing position difference; generate the first index set based on the number of correct lane changing scenes, the total number of scenes, the lane changing time difference set and the lane changing position difference set.

[0017] Optionally, in the fourth implementation manner of the second aspect of the present application, the evaluation module comprises: a first calculating unit, configured to calculate a time difference value according to the lane changing time difference set and the historical lane changing time difference set to obtain a time difference set, and calculate a position difference value according to the lane changing position difference set and the historical lane changing position difference set to obtain a position difference set; a second calculating unit, configured to calculate a score and a corresponding score factor according to the number of correct lane changing scenes, the total number of scenes, the number of second correct lane changing scenes, a second total number of scenes, the time difference set and the position difference set; an evaluation unit, configured to evaluate the vehicle automatic driving system according to the score and the score corresponding score factor.

[0018] Optionally, in a fifth implementation form of the second aspect of the present application, the second calculating unit is further configured to: calculate the first score and the first score factor according to the number of correct lane changing scenarios, the total number of scenarios, the number of second correct lane changing scenarios and the second total number of scenarios; calculate the second score and the second score factor according to the set of time differences; and calculate the third score and the third score factor according to the set of position differences.

[0019] Optionally, in a sixth implementation form of the second aspect of the present application, the evaluating unit is further configured to: determine whether the first score factor is greater than a preset first score threshold; if the first score factor is greater than the first score threshold, evaluate the target automatic driving system according to the first score to obtain a first evaluation result; if the first score factor is not greater than the first score threshold, determine whether the second score factor is greater than a preset second score threshold and determine whether the third score factor is greater than a preset third score threshold; if the second score factor is greater than the second score threshold, evaluate the target automatic driving system according to the second score to obtain a second evaluation result; and if the third score factor is greater than the third score threshold, evaluate the target automatic driving system according to the third score to obtain a third evaluation result.

[0020] The third aspect of the present application provides a computer device, comprising: a memory and at least one processor, the memory storing instructions; the at least one processor invoking the instructions in the memory to enable the computer device to perform each step of the vehicle lane changing evaluation method described above.

[0021] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium storing instructions, when the instructions are run on a computer, enabling the computer to perform each step of the vehicle lane changing evaluation method described above.

[0022] In the technical solution of the present application, the method acquires simulation data of a vehicle automatic driving system in a lane changing scene; determines the type of the lane changing scene based on the simulation data, and determines an index evaluation standard based on the type; selects reference data according to the index evaluation standard, and generates an index set based on the reference data; and evaluates the vehicle automatic driving system based on the index set. The above provides a scoring method for simulated automatic driving vehicles in different sub-scenes, taking the lane changing time and position of a human driver in the reference data as a benchmark, provides three sub-indexes for judging the lane changing ability of automatic driving, and can evaluate the lane changing ability of the automatic driving system in terms of lane changing accuracy, time difference and position difference in the form of scores and score factors, thereby solving the problem that the lane changing ability of the automatic driving system cannot be effectively evaluated in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A first embodiment of the evaluation method for vehicle lane changing in the present application is shown in the figure;

[0024] Figure 2 A second embodiment of the evaluation method for vehicle lane changing in the present application is shown in the figure;

[0025] Figure 3 A third embodiment of the evaluation method for vehicle lane changing in the present application is shown in the figure;

[0026] Figure 4 An embodiment of the evaluation device for vehicle lane changing in the present application is shown in the figure;

[0027] Figure 5 Another embodiment of the evaluation device for vehicle lane changing in the present application is shown in the figure;

[0028] Figure 6 An embodiment of the computer device in the present application is shown in the figure. DETAILED DESCRIPTION

[0029] In order to solve the problem that the automatic driving system cannot be effectively evaluated in the vehicle lane changing in the prior art, the application provides a vehicle lane changing evaluation method, device, equipment and storage medium. The method obtains simulation data of the vehicle automatic driving system in the lane changing scene; determines the type of the lane changing scene based on the simulation data, and determines the index evaluation standard based on the type; selects reference data according to the index evaluation standard, and generates an index set based on the reference data; and evaluates the vehicle automatic driving system based on the index set. The above takes the lane changing time and position of the human driver in the reference data as the benchmark, provides a scoring method for the simulated automatic driving vehicle in different sub-scenes, provides three sub-indices to judge the lane changing ability of the automatic driving, and can evaluate the lane changing ability of the automatic driving system in the three aspects of lane changing accuracy, time difference and position difference in the form of score and score factor, thereby solving the problem that the automatic driving system cannot be effectively evaluated in the vehicle lane changing in the prior art.

[0030] The terms "first", "second", "third", "fourth" and the like in the description, claims, and drawings of the application, and those above and below (if any) are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is to be interpreted to allow that combinations of the features of the various embodiments (and of the various features described herein) can provide desirable means for carrying out the respective embodiments for which the features are described. It will be further understood that the data used in this description is interchangeable. Embodiments herein described can be carried out in other sequences than the one described herein for implementing the embodiments. Furthermore, the terms "comprising", "having", "including" and any other corresponding semantic alternatives are intended to cover the inclusion of one or more steps or units and that any process, method, system, product, or apparatus that includes a combination of steps or units listed in the specification or claims are not limited to the stated combination but can include other steps or units that are not expressly listed or inherent to these processes, methods, products, or apparatuses.

[0031] For the sake of understanding, the specific flow of the embodiments of the application is described below. Please refer to Figure 1 The first embodiment of the vehicle lane changing evaluation method in the embodiments of the application has the following implementation steps:

[0032] 101. Obtain simulation data of the vehicle automatic driving system in the lane changing scene.

[0033] In this step, the simulation data includes at least one of the driving data of the vehicle under the control of the automatic driving system, the simulation data of the vehicle under the control of the automatic driving system, and the simulation data in the virtual lane changing scene based on the automatic driving system.

[0034] 102. Determine the type of the lane changing scene based on the simulation data, and determine the index evaluation standard based on the type.

[0035] The type in this step includes a congestion lane-changing type, an empty lane-changing type, and a general lane-changing type.

[0036] The index evaluation standard in this step includes a standard corresponding to the congestion lane-changing type, a standard corresponding to the empty lane-changing type, and a standard corresponding to the general lane-changing type.

[0037] The process of determining the type of the lane-changing scene based on the simulation data in this step includes:

[0038] Obstacles are extracted from the lane-changing scene, and the number corresponding to the obstacles is calculated.

[0039] The type corresponding to the lane-changing scene is determined according to the number corresponding to the obstacles.

[0040] 103. Reference data is selected according to the index evaluation standard, and an index set is generated based on the reference data.

[0041] The reference data in this step includes human driving data and historical simulation data corresponding to a vehicle historical automatic driving system.

[0042] The human driving data in this step includes at least one of human driving data collected by an on-board sensor, human driving data collected by an off-board sensor, and human driving data based on a driving simulation software.

[0043] The historical simulation data corresponding to the vehicle historical automatic driving system in this step includes at least one of driving data of the vehicle under control of a historical version of the automatic driving system, simulation data of the vehicle under control of the historical version of the automatic driving system, and simulation data based on a virtual lane-changing scene under the historical version of the automatic driving system.

[0044] The index set in this step includes a first index set and a second index set.

[0045] The process of generating an index set based on reference data in this step includes:

[0046] A corresponding reference lane-changing scene is extracted from the human driving data according to the lane-changing scene, for example, a corresponding type and lane identification are extracted from the lane-changing scene, and a corresponding reference lane-changing scene is determined from the human driving data according to the type and lane identification.

[0047] A corresponding historical lane-changing scene is extracted from the historical simulation data according to the lane-changing scene, for example, a corresponding type and lane identification are extracted from the lane-changing scene, and a corresponding historical lane-changing scene is determined from the historical simulation data according to the type and lane identification.

[0048] The lane-changing scene is compared with the reference lane-changing scene to obtain a first comparison result, and a first index set is generated according to the first comparison result;

[0049] The lane-changing scene is compared with the historical lane-changing scene to obtain a second comparison result, and a second index set is generated according to the second comparison result.

[0050] 104. The vehicle automatic driving system is evaluated based on the index set.

[0051] In this step, the first index set includes the number of correct lane-changing scenes and the total number of scenes;

[0052] In this step, the second index set includes the number of second correct lane-changing scenes, the second total number of scenes, a historical lane-changing time difference set and a historical lane-changing position difference set;

[0053] For this step, the following methods can be used:

[0054] The number of correct lane-changing scenes is divided by the total number of scenes to obtain a first correct rate;

[0055] The number of second correct lane-changing scenes is divided by the second total number of scenes to obtain a second correct rate;

[0056] The difference between the first correct rate and the second correct rate is calculated to obtain a first score;

[0057] The first score factor is calculated according to the following formula:

[0058] Wherein, Z1 represents the first score factor, n represents the total number of scenes, p1 represents the first correct rate, and p2 represents the second correct rate;

[0059] It is judged whether the first score factor is greater than a preset first score threshold, for example, the first score threshold can be set to 1.96;

[0060] If yes, it is judged whether the first score is greater than 0;

[0061] If no, the result of evaluating the vehicle automatic driving system is that there is no significant change relative to the vehicle historical automatic driving system;

[0062] If yes, the result of evaluating the vehicle automatic driving system is that it is significantly better relative to the vehicle historical automatic driving system;

[0063] If not greater than, the result of evaluating the vehicle automatic driving system is that it is significantly worse relative to the vehicle historical automatic driving system.

[0064] By implementing the above method, simulation data of a vehicle automatic driving system in a lane changing scene is obtained; a type of the lane changing scene is determined based on the simulation data, and an index evaluation standard is determined based on the type; reference data is selected according to the index evaluation standard, and an index set is generated based on the reference data; the vehicle automatic driving system is evaluated based on the index set; and above, the first correct rate and the second correct rate in the lane changing scene are calculated by taking the lane changing scene of a human driver in the reference data and the lane changing scene of a historical automatic driving system of the vehicle as a benchmark, and the first score and the first score factor are calculated based on the first correct rate and the second correct rate, and the lane changing capability of the automatic driving system is evaluated according to the first score and the first score factor, thereby solving the problem that the automatic driving system cannot be effectively evaluated in terms of vehicle lane changing in the prior art.

[0065] Please refer to Figure 2 In the second embodiment of the evaluation method for vehicle lane changing in the embodiment of the present application, the implementation steps of the method are as follows:

[0066] 201, obtain simulation data of a vehicle automatic driving system in a lane changing scene;

[0067] For this step, the following method can be used for implementation:

[0068] read a simulation data set of the vehicle automatic driving system;

[0069] extract a driving scene and lane information corresponding to the driving scene from the simulation data set, wherein the lane information includes lane identification and lane type;

[0070] determine a corresponding judgment time period according to the lane type, for example, if the lane type is a main lane, the corresponding judgment time is set to 5 seconds;

[0071] determine whether the lane identification changes within the judgment time period;

[0072] if yes, determine whether the lane belongs to a merging lane according to the lane type;

[0073] if not, determine that the driving scene is a lane changing scene, and obtain simulation data corresponding to the lane changing scene.

[0074] 202, determine a type of the lane changing scene based on the simulation data, and determine an index evaluation standard based on the type;

[0075] In this step, the type includes a congestion lane changing type, an open lane changing type and a general lane changing type;

[0076] In the step, the index evaluation criteria include a standard corresponding to a congestion lane-changing type, a standard corresponding to an empty lane-changing type, and a standard corresponding to a general lane-changing type.

[0077] In the step, the process of determining the type of the lane-changing scene based on the simulation data includes:

[0078] Obstacles are extracted from the lane-changing scene, and a quantity corresponding to the obstacles is calculated.

[0079] The type corresponding to the lane-changing scene is determined according to the quantity corresponding to the obstacles.

[0080] Further, the process of extracting obstacles from the lane-changing scene includes:

[0081] The host vehicle position and host vehicle direction corresponding to the host vehicle and the vehicle type, vehicle position, and driving direction corresponding to the other vehicle are extracted from the lane-changing scene.

[0082] It is judged whether the vehicle type meets a preset type condition.

[0083] If yes, a longitudinal distance and a lateral distance are calculated according to the host vehicle position and the vehicle position.

[0084] It is judged whether the longitudinal distance and the lateral distance meet a preset distance condition.

[0085] A yaw angle difference is calculated according to the host vehicle direction and the driving direction, and it is judged whether the yaw angle difference meets a preset yaw angle condition.

[0086] If the distance condition is met and the yaw angle condition is met, the other vehicle is determined to be an obstacle, and obstacles are extracted from the lane-changing scene.

[0087] 203. Reference data is selected according to index evaluation criteria, and an index set is generated based on the reference data.

[0088] For this step, it can be implemented in the following way:

[0089] Reference lane-changing scenes corresponding to the human driving data are extracted according to the standards corresponding to the lane-changing types, for example, the standards corresponding to the lane-changing types include standards corresponding to a congested lane-changing type, standards corresponding to an empty lane-changing type, and standards corresponding to a general lane-changing type.

[0090] Corresponding historical lane-changing scenes are extracted from the historical simulation data according to the standards corresponding to the lane-changing types, for example, the type and lane identifier corresponding to the lane-changing scene are extracted, and the corresponding historical lane-changing scene is determined from the historical simulation data according to the type and lane identifier.

[0091] The lane change scenario is compared with the reference lane change scenario to obtain a first comparison result, and a first index set is generated based on the first comparison result;

[0092] The lane change scenario is compared with the historical lane change scenario to obtain a second comparison result, and a second indicator set is generated based on the second comparison result.

[0093] In practical applications, the standard for the congested lane change type is that the number of obstacles is not less than 12;

[0094] The standard for the general lane change type is that the number of obstacles is not less than 9 and not more than 12.

[0095] The standard for the open lane change type is that the number of obstacles is less than 9.

[0096] 204. Evaluate vehicle autonomous driving systems based on indicator sets.

[0097] In this step, the first index set includes the lane change time difference set and the lane change position difference set;

[0098] In this step, the second set of indicators includes a set of historical lane change time differences and a set of historical lane change position differences;

[0099] This step can be implemented in the following way:

[0100] The time difference value is calculated based on the lane change time difference set and the historical lane change time difference set to obtain the time difference set. For example, the time difference value is obtained by subtracting the corresponding historical lane change time difference from the lane change time difference in the historical lane change time difference set.

[0101] The position difference value is calculated based on the lane change position difference set and the historical lane change position difference set to obtain the position difference set. For example, the position difference value is obtained by subtracting the corresponding historical lane change position difference from the lane change position difference in the historical lane change position difference set.

[0102] Calculate the score and corresponding score factors based on the time difference set and the location difference set;

[0103] The vehicle's autonomous driving system is evaluated based on the score and the corresponding scoring factors.

[0104] Further, the step of calculating the score and corresponding scoring factors based on the time difference set and the location difference set includes:

[0105] Calculate the second score and the second score factor based on the time difference set;

[0106] calculating the third score and the third score factor according to the position difference set;

[0107] Specifically, the process of calculating the second score and the second score factor according to the time difference set comprises:

[0108] calculating the mean value, the standard deviation and the standard error corresponding to all the time difference values;

[0109] calculating the second score indicator according to the following formula:

[0110] wherein t represents the second score indicator, v represents the time difference value, m represents the mean value corresponding to all the time difference values, x represents a preset confidence factor, and d represents the standard deviation corresponding to all the time difference values;

[0111] calculating the mean value of the second score indicator to obtain the second score;

[0112] calculating the standard error of the second score indicator, and dividing the second score by the standard error of the second score indicator to obtain the second score factor;

[0113] In actual application, the confidence factor can be set as 1.96;

[0114] Specifically, the process of calculating the third score and the third score factor according to the position difference set comprises:

[0115] According to the calculation process corresponding to the calculation method of calculating the second score and the second score factor, the third score and the corresponding third score factor are calculated according to the lane-changing position difference set and the historical lane-changing position difference set.

[0116] Further, the process of evaluating the vehicle automatic driving system according to the score and the score factor corresponding to the score comprises:

[0117] determining whether the second score factor is greater than a preset second score threshold, for example, the second score threshold can be set as 1.64;

[0118] determining whether the third score factor is greater than a preset third score threshold;

[0119] if the second score factor is greater than the second score threshold, evaluating the target automatic driving system according to the second score to obtain a second evaluation result;

[0120] if the third score factor is greater than the third score threshold, evaluating the target automatic driving system according to the third score to obtain a third evaluation result;

[0121] Specifically, the process of evaluating the target automatic driving system according to the second score to obtain a second evaluation result comprises:

[0122] determining whether the second score is greater than 0;

[0123] if the second score is not greater than 0, the result of evaluating the vehicle automatic driving system is significantly worse than the vehicle historical automatic driving system;

[0124] if the second score is greater than 0, the result of evaluating the vehicle automatic driving system is significantly better than the vehicle historical automatic driving system;

[0125] Specifically, the process of evaluating the target automatic driving system according to the third score to obtain a third evaluation result comprises:

[0126] determining whether the third score is greater than 0;

[0127] if the third score is not greater than 0, the result of evaluating the vehicle automatic driving system is significantly worse than the vehicle historical automatic driving system;

[0128] if the third score is greater than 0, the result of evaluating the vehicle automatic driving system is significantly better than the vehicle historical automatic driving system.

[0129] By implementing the above method, simulation data of a vehicle automatic driving system in a lane changing scene is obtained; a type of the lane changing scene is determined based on the simulation data, and an index evaluation standard is determined based on the type; reference data is selected according to the index evaluation standard, and an index set is generated based on the reference data; the vehicle automatic driving system is evaluated based on the index set; in the above, the lane changing time and position of a human driver in the reference data are taken as a benchmark, a scoring method for a simulated automatic driving vehicle in different sub-scenes in the lane changing scene is provided, a second score, a second score factor, a third score, and a third score factor are calculated according to the lane changing time difference and the lane changing position difference, and the lane changing capability of the automatic driving system is evaluated according to the scores and the score factors, thereby solving the problem in the prior art that the automatic driving system cannot be effectively evaluated in terms of vehicle lane changing.

[0130] Referring to Figure 3 , a third embodiment of the evaluation method for vehicle lane changing in the embodiment of the application, the implementation steps of the method are as follows:

[0131] 301, simulation data of a vehicle automatic driving system in a lane changing scene is obtained;

[0132] This step is basically the same as step 201 in the foregoing embodiments, and thus will not be described here again.

[0133] 302, determine the type of the lane changing scene based on the simulation data, and determine the index evaluation criteria based on the type;

[0134] In this step, the type includes a congestion lane changing type, an open lane changing type, and a general lane changing type.

[0135] In this step, the index evaluation criteria include a standard corresponding to the lane changing type, a standard corresponding to the open lane changing type, and a standard corresponding to the general lane changing type.

[0136] In this step, the process of determining the type of the lane changing scene based on the simulation data includes:

[0137] extracting an obstacle from the lane changing scene and calculating a quantity corresponding to the obstacle;

[0138] determining the type corresponding to the lane changing scene according to the quantity corresponding to the obstacle.

[0139] Further, the process of extracting the obstacle from the lane changing scene includes:

[0140] extracting a host vehicle position and a host vehicle direction corresponding to the host vehicle and a vehicle type, a vehicle position, and a driving direction corresponding to the other vehicle from the lane changing scene;

[0141] determining whether the vehicle type satisfies a preset type condition;

[0142] If yes, calculating a longitudinal distance and a lateral distance according to the host vehicle position and the vehicle position;

[0143] determining whether the longitudinal distance and the lateral distance satisfy a preset distance condition;

[0144] calculating a yaw angle difference according to the host vehicle direction and the driving direction, and determining whether the yaw angle difference satisfies a preset yaw angle condition;

[0145] If the distance condition is satisfied and the yaw angle condition is satisfied, determining that the other vehicle is an obstacle, and extracting the obstacle from the lane changing scene.

[0146] In actual applications, the type condition can be set as: the vehicle type is any one of a bicycle, a car, a bus, and a truck.

[0147] In actual applications, the distance condition can be set as: the longitudinal distance is less than 50 m or the lateral distance is less than 5.5 m.

[0148] In practical applications, the yaw angle condition can be set as: the yaw angle difference is less than 2radius.

[0149] 303、select reference data according to the index evaluation standard, and generate an index set based on the reference data;

[0150] In this step, the process of generating an index set based on reference data includes:

[0151] According to the lane changing scene, the corresponding reference lane changing scene is extracted from the human driving data, and the corresponding historical lane changing scene is extracted from the historical simulation data according to the lane changing scene;

[0152] The lane changing scene is compared with the reference lane changing scene to obtain a first comparison result, and a first index set is generated according to the first comparison result;

[0153] The lane changing scene is compared with the historical lane changing scene to obtain a second comparison result, and a second index set is generated according to the second comparison result.

[0154] Further, the process of generating a first index set according to the first comparison result includes:

[0155] According to the first comparison result, it is determined whether the corresponding lane changing scene is a correct lane changing scene, and the number of correct lane changing scenes and the total number of lane changing scenes corresponding to the lane changing scene are calculated;

[0156] From the first comparison result, the lane changing time difference and the lane changing position difference between the reference lane changing scene and the corresponding correct lane changing scene are extracted;

[0157] Based on the lane changing time difference, a lane changing time difference set is generated, and based on the lane changing position difference, a lane changing position difference set is generated;

[0158] Based on the number of correct lane changing scenes, the total number of scenes, the lane changing time difference set and the lane changing position difference set, the first index set is generated.

[0159] 304, evaluate the vehicle automatic driving system based on the index set.

[0160] In this step, the second index set includes the number of second correct lane changing scenes, the second total scene quantity, the historical lane changing time difference set and the historical lane changing position difference set;

[0161] For this step, it can be implemented by the following way:

[0162] calculate a time difference value according to the lane-changing time difference set and the historical lane-changing time difference set to obtain the time difference set, and calculate a position difference value according to the lane-changing position difference set and the historical lane-changing position difference set to obtain the position difference set;

[0163] calculate a score and a corresponding score factor according to the number of correct lane-changing scenes, the total number of scenes, the number of second correct lane-changing scenes, the second total number of scenes, the time difference set and the position difference set;

[0164] evaluate the vehicle automatic driving system according to the score and the score corresponding score factor.

[0165] Further, the process of calculating a score and a corresponding score factor according to the number of correct lane-changing scenes, the total number of scenes, the number of second correct lane-changing scenes, the second total number of scenes, the time difference set and the position difference set, comprises:

[0166] calculate the first score and the first score factor according to the number of correct lane-changing scenes, the total number of scenes, the number of second correct lane-changing scenes and the second total number of scenes;

[0167] calculate the second score and the second score factor according to the time difference set;

[0168] calculate the third score and the third score factor according to the position difference set.

[0169] Further, the process of evaluating the vehicle automatic driving system according to the score and the score corresponding score factor, comprises:

[0170] determine whether the first score factor is greater than a preset first score threshold;

[0171] if the first score factor is greater than the first score threshold, determine whether the first score is greater than 0;

[0172] if the first score is greater than 0, determine that the first score result is significantly better than the vehicle historical automatic driving system;

[0173] if the first score is not greater than 0, determine that the first score result is significantly worse than the vehicle historical automatic driving system;

[0174] if the first score factor is not greater than the first score threshold, determine whether the second score factor is greater than a preset second score threshold;

[0175] if the second score factor is greater than the second score threshold, determine whether the first score is greater than 0;

[0176] if the second score is greater than 0, determining that the second score result is significantly better than the vehicle historical automatic driving system;

[0177] if the second score is not greater than 0, determining that the second score result is significantly worse than the vehicle historical automatic driving system;

[0178] determining whether the third score factor is greater than a preset third score threshold value;

[0179] if the third score factor is greater than the third score threshold value, determining whether the third score is greater than 0;

[0180] if the third score is greater than 0, determining that the third score result is significantly better than the vehicle historical automatic driving system;

[0181] if the third score is not greater than 0, determining that the third score result is significantly worse than the vehicle historical automatic driving system.

[0182] By implementing the above method, simulation data of a vehicle automatic driving system in a lane changing scene is obtained; a type of the lane changing scene is determined based on the simulation data, and an index evaluation standard is determined based on the type; reference data is selected according to the index evaluation standard, and an index set is generated based on the reference data; the vehicle automatic driving system is evaluated based on the index set; and above, the lane changing time and position of a human driver in the reference data are taken as a benchmark, a scoring method for a simulated autonomous vehicle in different sub-scenes is provided, three sub-indices are provided to judge the lane changing capability of the autonomous driving, and the lane changing capability of the autonomous driving system can be evaluated in terms of lane changing accuracy, time difference and position difference in the form of scores and score factors, thereby solving the problem in the prior art that the lane changing capability of the autonomous driving system cannot be effectively evaluated.

[0183] The evaluation method for vehicle lane changing in the embodiment of the application is described above, and the evaluation device for vehicle lane changing in the embodiment of the application is described below, please refer to Figure 4 One embodiment of the evaluation device for vehicle lane changing in the embodiment of the application, the device comprises:

[0184] The acquisition module 401 is configured to acquire simulation data of a vehicle automatic driving system in a lane changing scene;

[0185] The determination module 402 is configured to determine a type of the lane changing scene based on the simulation data, and determine an index evaluation standard based on the type;

[0186] The generation module 403 is configured to select reference data according to the index evaluation standard, and generate an index set based on the reference data;

[0187] an evaluation module 404, configured to evaluate the vehicle automatic driving system based on the index set.

[0188] By implementing the above device, by acquiring simulation data of a vehicle automatic driving system in a lane changing scene; determining a type of the lane changing scene based on the simulation data, and determining an index evaluation standard based on the type; selecting reference data according to the index evaluation standard, and generating an index set based on the reference data; evaluating the vehicle automatic driving system based on the index set; the above provides a scoring method for a simulated autonomous vehicle in different sub-scenes, provides three sub-indices to judge the lane changing ability of the autonomous driving, and can evaluate the lane changing ability of the autonomous driving system in terms of lane changing accuracy, time difference and position difference in the form of scores and score factors, thereby solving the problem that the lane changing ability of the autonomous driving system cannot be effectively evaluated in the prior art.

[0189] Referring to Figure 5 Another embodiment of the vehicle lane changing evaluation device in the embodiment of the present application includes:

[0190] an acquisition module 401, configured to acquire simulation data of a vehicle automatic driving system in a lane changing scene;

[0191] a determination module 402, configured to determine a type of the lane changing scene based on the simulation data, and determine an index evaluation standard based on the type;

[0192] a generation module 403, configured to select reference data according to the index evaluation standard, and generate an index set based on the reference data;

[0193] an evaluation module 404, configured to evaluate the vehicle automatic driving system based on the index set.

[0194] In the embodiment, the acquisition module 401 includes:

[0195] a reading unit 4011, configured to read a simulation data set of a vehicle automatic driving system;

[0196] a first extraction unit 4012, configured to extract a driving scene and lane information corresponding to the driving scene from the simulation data set, wherein the lane information includes a lane identifier and a lane type;

[0197] The judging unit 4013 is configured to judge whether the lane mark changes within a preset time period. If yes, it is judged according to the lane type whether the lane belongs to a merging lane. If no, it is determined that the driving scene is a lane-changing scene, and the simulation data corresponding to the lane-changing scene is obtained.

[0198] In the embodiment, the generating module 403 comprises:

[0199] The second extracting unit 4031 is configured to extract a reference lane-changing scene corresponding to the lane-changing scene from the human driving data according to the lane-changing scene, and extract a historical lane-changing scene corresponding to the lane-changing scene from the historical simulation data according to the lane-changing scene;

[0200] The first comparing unit 4032 is configured to compare the lane-changing scene with the reference lane-changing scene to obtain a first comparison result, and generate a first index set according to the first comparison result.

[0201] The first comparing unit 4032 is further configured to determine whether the corresponding lane-changing scene is a correct lane-changing scene according to the first comparison result, calculate the number of correct lane-changing scenes and the total number of lane-changing scenes corresponding to the lane-changing scene, extract a lane-changing time difference and a lane-changing position difference between the reference lane-changing scene and the corresponding correct lane-changing scene from the first comparison result, generate a lane-changing time difference set based on the lane-changing time difference, generate a lane-changing position difference set based on the lane-changing position difference, and generate the first index set based on the number of correct lane-changing scenes, the total number of lane-changing scenes, the lane-changing time difference set and the lane-changing position difference set.

[0202] The second comparing unit 4033 is configured to compare the lane-changing scene with the historical lane-changing scene to obtain a second comparison result, and generate a second index set according to the second comparison result.

[0203] In the embodiment, the evaluation module 404 comprises:

[0204] The first calculating unit 4041 is configured to calculate a time difference value according to the lane-changing time difference set and the historical lane-changing time difference set to obtain a time difference set, and calculate a position difference value according to the lane-changing position difference set and the historical lane-changing position difference set to obtain a position difference set.

[0205] The second calculating unit 4042 is configured to calculate a score and a corresponding score factor according to the number of correct lane-changing scenes, the total number of lane-changing scenes, the number of second correct lane-changing scenes, a second total number of lane-changing scenes, the time difference set and the position difference set.

[0206] The second computing unit 4042 is further configured to: calculate the first score and the first score factor according to the number of correct lane-changing scenes, the total number of scenes, the number of second correct lane-changing scenes and the second total number of scenes; calculate the second score and the second score factor according to the time difference set; and calculate the third score and the third score factor according to the position difference set.

[0207] The evaluation unit 4043 is configured to evaluate the vehicle automatic driving system according to the score and the score factor corresponding to the score.

[0208] The evaluation unit 4043 is further configured to: determine whether the first score factor is greater than a preset first score threshold; if the first score factor is greater than the first score threshold, evaluate the target automatic driving system according to the first score to obtain a first evaluation result; if the first score factor is not greater than the first score threshold, determine whether the second score factor is greater than a preset second score threshold and determine whether the third score factor is greater than a preset third score threshold; if the second score factor is greater than the second score threshold, evaluate the target automatic driving system according to the second score to obtain a second evaluation result; and if the third score factor is greater than the third score threshold, evaluate the target automatic driving system according to the third score to obtain a third evaluation result.

[0209] By implementing the above device, simulation data of a vehicle automatic driving system in a lane-changing scene is obtained; the type of the lane-changing scene is determined based on the simulation data, and an index evaluation standard is determined based on the type; reference data is selected according to the index evaluation standard, and an index set is generated based on the reference data; and the vehicle automatic driving system is evaluated based on the index set. In this way, the lane-changing time and position of a human driver in the reference data are used as a benchmark to provide a scoring method for a simulated automatic driving vehicle in different sub-scenes, and three sub-indices are provided to judge the lane-changing capability of the automatic driving vehicle. The lane-changing capability of the automatic driving system can be evaluated in terms of lane-changing accuracy, time difference and position difference in the form of a score and a score factor, thereby solving the problem that the lane-changing capability of the automatic driving system cannot be effectively evaluated in the prior art.

[0210] Please refer to Figure 6 , and the following describes an embodiment of a computer device in the embodiment of the application from the perspective of hardware processing.

[0211] Figure 6is a structural schematic diagram of a computer device provided by an embodiment of the present application. The computer device 600 can have great differences due to different configurations or performances, and can include one or more processors (central processing units, CPU) 610 (for example, one or more processors) and a memory 620, one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. The memory 620 and the storage media 630 can be temporary storage or persistent storage. The programs stored in the storage media 630 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations on the computer device 600. Furthermore, the processor 610 can be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the computer device 600.

[0212] The computer device 600 can also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and / or one or more operating systems 631, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that the computer device 600 can include more or fewer components than those shown in the figure, or combine some components, or arrange different components. Figure 6 The computer device structure shown does not constitute a limitation on the computer device provided by the present application, and can include more or fewer components than those shown in the figure, or combine some components, or arrange different components.

[0213] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium. The computer readable storage medium stores instructions, and when the instructions run on a computer, the computer executes the steps of the vehicle lane changing evaluation method described above.

[0214] In practical applications, the method provided above can be implemented based on an artificial intelligence technology, where the artificial intelligence (AI) is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving an environment, acquiring knowledge and using the knowledge to obtain optimal results. Specifically, the method can be executed based on a server, and the server can be a standalone server or a cloud server providing cloud services, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content delivery network (CDN), and basic cloud computing services such as a big data and artificial intelligence platform.

[0215] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the described devices and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein.

[0216] If the integrated units are implemented in the form of software function units and sold or used as independent products, the units can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0217] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features, without departing from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for evaluating vehicle lane changes, characterized in that, The evaluation method includes: Acquire simulation data of the vehicle's autonomous driving system in lane-changing scenarios; Based on the simulation data, the type of lane change scenario is determined, and based on the type, the evaluation criteria for the indicators are determined; Reference data is selected based on the aforementioned indicator evaluation criteria, and an indicator set is generated based on the aforementioned reference data; The vehicle's autonomous driving system is evaluated based on the aforementioned set of indicators; The reference data includes human driving data and historical simulation data corresponding to the vehicle's historical autonomous driving system. The indicator set includes a first indicator set and a second indicator set. Generating the indicator set based on the reference data includes: extracting a corresponding reference lane-changing scenario from the human driving data based on the lane-changing scenario, and extracting a corresponding historical lane-changing scenario from the historical simulation data based on the lane-changing scenario; comparing the lane-changing scenario with the reference lane-changing scenario to obtain a first comparison result, and generating a first indicator set based on the first comparison result; comparing the lane-changing scenario with the historical lane-changing scenario to obtain a second comparison result, and generating a second indicator set based on the second comparison result.

2. The evaluation method according to claim 1, characterized in that, The acquisition of simulation data of the vehicle's autonomous driving system in lane-changing scenarios includes: Read the simulation dataset of the vehicle's autonomous driving system; The driving scenario and the lane information corresponding to the driving scenario are extracted from the simulation dataset, wherein the lane information includes lane identification and lane type; Determine whether the lane markings have changed within a preset time period; If so, determine whether the lane is a lane merging lane based on the lane type. If it does not belong to the lane merging lane, the driving scenario is determined to be a lane changing scenario, and the simulation data corresponding to the lane changing scenario is obtained.

3. The evaluation method according to claim 2, characterized in that, The step of generating the first indicator set based on the first comparison result includes: Based on the first comparison result, determine whether the corresponding lane change scenario is a correct lane change scenario, and calculate the number of correct lane change scenarios and the total number of scenarios corresponding to the lane change scenarios; Extract the lane change time difference and lane change position difference between the reference lane change scenario and the corresponding correct lane change scenario from the first comparison result; A set of lane change time differences is generated based on the lane change time difference, and a set of lane change position differences is generated based on the lane change position difference; The first index set is generated based on the number of correct lane change scenarios, the total number of scenarios, the lane change time difference set, and the lane change position difference set.

4. The evaluation method according to claim 3, characterized in that, The second set of indicators includes the number of second correct lane change scenarios, the total number of second scenarios, the historical lane change time difference set, and the historical lane change position difference set; The evaluation of the vehicle's autonomous driving system based on the set of indicators includes: The time difference value is calculated based on the lane change time difference set and the historical lane change time difference set to obtain the time difference set; and the position difference value is calculated based on the lane change position difference set and the historical lane change position difference set to obtain the position difference set. The score and corresponding scoring factors are calculated based on the number of correct lane change scenarios, the total number of scenarios, the number of second correct lane change scenarios, the second total number of scenarios, the time difference set, and the position difference set. The vehicle's autonomous driving system is evaluated based on the score and the corresponding scoring factors.

5. The evaluation method according to claim 4, characterized in that, The rating includes a first rating, a second rating, and a third rating, and the rating factors include a first rating factor, a second rating factor, and a third rating factor; The calculation of scores and corresponding scoring factors based on the number of correct lane change scenarios, the total number of scenarios, the number of second correct lane change scenarios, the second total number of scenarios, the time difference set, and the position difference set includes: The first score and the first score factor are calculated based on the number of correct lane change scenarios, the total number of scenarios, the number of second correct lane change scenarios, and the second total number of scenarios. Calculate the second score and the second score factor based on the time difference set; The third score and the third score factor are calculated based on the location difference set.

6. The evaluation method according to claim 5, characterized in that, The evaluation of the vehicle's autonomous driving system based on the score and the corresponding scoring factor includes: Determine whether the first scoring factor is greater than a preset first scoring threshold; If the first scoring factor is greater than the first scoring threshold, the vehicle autonomous driving system is evaluated based on the first score to obtain a first evaluation result; If the first rating factor is not greater than the first rating threshold, then determine whether the second rating factor is greater than the preset second rating threshold, and determine whether the third rating factor is greater than the preset third rating threshold. If the second scoring factor is greater than the second scoring threshold, the vehicle's autonomous driving system is evaluated based on the second score to obtain a second evaluation result. If the third scoring factor is greater than the third scoring threshold, the vehicle's autonomous driving system is evaluated based on the third score to obtain a third evaluation result.

7. An evaluation device, characterized in that, The device includes: The acquisition module is used to acquire simulation data of the vehicle's autonomous driving system in lane-changing scenarios; The determination module is used to determine the type of lane change scenario based on the simulation data, and to determine the evaluation criteria based on the type; The generation module is used to select reference data according to the indicator evaluation criteria and generate an indicator set based on the reference data; An evaluation module is used to evaluate the vehicle's autonomous driving system based on the set of indicators; The reference data includes human driving data and historical simulation data corresponding to the vehicle's historical autonomous driving system. The indicator set includes a first indicator set and a second indicator set. The generation module includes: a second extraction unit, used to extract a corresponding reference lane change scenario from the human driving data based on the lane change scenario, and to extract a corresponding historical lane change scenario from the historical simulation data based on the lane change scenario; a first comparison unit, used to compare the lane change scenario with the reference lane change scenario to obtain a first comparison result, and to generate a first indicator set based on the first comparison result; and a second comparison unit, used to compare the lane change scenario with the historical lane change scenario to obtain a second comparison result, and to generate a second indicator set based on the second comparison result.

8. A computer device, characterized in that, include: A memory and at least one processor, wherein the memory stores instructions and the memory and the at least one processor are interconnected via a circuit; The at least one processor invokes the instructions in the memory to cause the computer device to perform the steps of the evaluation method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements the various steps of the evaluation method as described in any one of claims 1-6.

Images