Automatic driving system evaluation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an automatic driving system evaluation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining automatic driving information in a testing process of a target vehicle, determining a driving parameter and / or a way giving parameter of the target vehicle passing through an intersection according to the automatic driving information, and determining the drivability of the target vehicle according to the driving parameters and / or the yielding parameters. Therefore, by analyzing the automatic driving information in the vehicle testing process, the intersection passing performance and the intersection yielding performance of the automatic driving system can be determined, and then the performance of the automatic driving system passing through the intersection is evaluated.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to an automatic driving system evaluation method, device, electronic device and storage medium. Background Art

[0002] A typical scenario in road testing of autonomous driving systems is when an autonomous vehicle passes through an intersection. In the scenario where an autonomous vehicle passes through an intersection, the most common use of the autonomous vehicle is the intersection yield function, for example, when the vehicle makes an unprotected left turn, it yields to oncoming vehicles going straight, or when the vehicle passes through a zebra crossing at an intersection, it yields to pedestrians on the zebra crossing.

[0003] However, for L4 level autonomous driving systems, although there are many indicators and methods in the existing technology to evaluate the overall performance of the vehicle in road tests, these methods are difficult to reflect the performance of autonomous driving vehicles passing through intersections. Taking the most commonly used indicator MPI (travel distance per takeover) as an example, when the vehicle performs well on a straight road but performs poorly through intersections, it is necessary to take over each time it passes through an intersection to give way to other traffic participants (including vehicles, pedestrians, etc.). The indicator MPI has a strong correlation with the performance of the vehicle passing through intersections. The more intersections there are, the larger the MPI indicator value, and the worse the corresponding vehicle's performance through intersections; however, when the MPI indicator value is large, but the autonomous driving vehicle only passes through one intersection during the road test, the higher MPI value does not reflect the good performance of the autonomous driving vehicle at the intersection. Therefore, a new evaluation method is needed to evaluate the performance of the autonomous driving system through intersections. Summary of the invention

[0004] In view of this, an object of an embodiment of the present invention is to provide an autonomous driving system evaluation method, device, electronic device and storage medium to evaluate the performance of an autonomous driving system passing through an intersection.

[0005] In a first aspect, an embodiment of the present invention is intended to provide an automatic driving system evaluation method, the method comprising:

[0006] Acquiring autonomous driving information of the target vehicle during the test, wherein the autonomous driving information includes fully autonomous driving state information, takeover state information, and abnormal body perception information;

[0007] Determining driving parameters and / or yielding parameters for the target vehicle to pass through an intersection according to the autonomous driving information;

[0008] The driving performance of the target vehicle is determined according to the driving parameter and / or the yielding parameter.

[0009] Further, determining the driving performance of the target vehicle according to the yield parameter includes:

[0010] Determining a first execution rate and / or a second execution rate according to the yielding parameter, wherein the first execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior, and the second execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior and does not experience abnormal body sensation during the yielding process;

[0011] The driving performance of the target vehicle is determined according to the first execution rate and / or the second execution rate.

[0012] Furthermore, the yielding parameters include the number of times the vehicle yielded at intersections, the number of times the vehicle yielded in a fully autonomous driving state, and the number of times the vehicle yielded in a fully autonomous driving state without any abnormal sensation.

[0013] Furthermore, the giving way behavior includes a combination of one or more of the following: giving way at low right of way, giving way at zebra crossings and giving way at high right of way.

[0014] Further, determining the driving performance of the target vehicle according to the driving parameters includes:

[0015] Determine a first pass rate and / or a second pass rate according to the driving parameters, wherein the first pass rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state, and the second pass rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state without any abnormal sensation;

[0016] The driving performance of the target vehicle is determined according to the first pass rate and / or the second pass rate.

[0017] Furthermore, the driving parameters include: the number of times the vehicle passes through an intersection, the number of times the vehicle passes through an intersection in a fully automatic driving state, and the number of times the vehicle passes through an intersection in a fully automatic driving state without experiencing any abnormal sensation.

[0018] Furthermore, the abnormal body sensation includes a combination of one or more of the following: freezing, sudden braking, rapid movement and rapid turning.

[0019] Furthermore, the method further comprises:

[0020] The driving performance corresponding to different test processes is recorded to monitor changes in the driving performance.

[0021] In a second aspect, an embodiment of the present invention is intended to provide an automatic driving system evaluation device, the device comprising:

[0022] An acquisition module, used to acquire the autonomous driving information of the target vehicle during the test, wherein the autonomous driving information includes fully autonomous driving state information, takeover state information, and abnormal body sensing information;

[0023] A determination module, configured to determine driving parameters and / or yielding parameters for the target vehicle to pass through an intersection according to the automatic driving information;

[0024] An evaluation module is used to determine the driving performance of the target vehicle according to the driving parameter and / or the yielding parameter.

[0025] In a third aspect, an embodiment of the present invention aims to provide a computer program product, wherein the computer program product comprises a computer program / instructions, and when the computer program / instructions are executed by a processor, the method as described in any one of the above items is implemented.

[0026] In a fourth aspect, an embodiment of the present invention aims to provide an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in any one of the above items.

[0027] In a fifth aspect, an embodiment of the present invention aims to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps as described in any one of the above items are implemented.

[0028] The technical solution of the embodiment of the present invention obtains the automatic driving information of the target vehicle during the test, determines the driving parameters and / or the yielding parameters of the target vehicle passing through the intersection according to the automatic driving information, and determines the driving performance of the target vehicle according to the driving parameters and / or the yielding parameters. Therefore, by analyzing the automatic driving information during the vehicle test, the intersection passing performance and intersection yielding performance of the automatic driving system can be determined, thereby realizing the evaluation of the performance of the automatic driving system passing through the intersection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0030] Figure 1 is a flow chart of an automatic driving system evaluation method according to an embodiment of the present invention;

[0031] Figure 2 is a flow chart of evaluating the intersection yielding performance of an automatic driving system according to an embodiment of the present invention;

[0032] Figure 3 is a flow chart of evaluating the intersection traffic performance of an automatic driving system according to an embodiment of the present invention;

[0033] Figure 4 is a schematic diagram of recording driving performance in an embodiment of the present invention;

[0034] Figure 5 is a schematic diagram of an automatic driving system evaluation device according to an embodiment of the present invention;

[0035] Figure 6 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0037] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0038] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".

[0039] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0040] The solutions described in this specification and in the examples, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0041] The performance of an autonomous vehicle passing through an intersection includes the response function to traffic lights, the intersection passing performance of fully autonomous driving through an intersection without human control, the yielding performance when yielding at an intersection, etc. Evaluating the performance of an autonomous driving system passing through an intersection is of great significance for optimizing the performance of autonomous driving vehicles. In view of this, an embodiment of the present invention provides an autonomous driving system evaluation method to evaluate the intersection passing performance and yielding performance of an autonomous driving system in an autonomous driving vehicle.

[0042] Figure 1 FIG. 1 is a flow chart of an automatic driving system evaluation method according to an embodiment of the present invention. Figure 1As shown, the automatic driving system evaluation method in this embodiment includes the following steps.

[0043] In step S100, the automatic driving information of the target vehicle during the test is obtained, wherein the automatic driving information includes full automatic driving state information, takeover state information and abnormal body perception information.

[0044] In this embodiment, the target vehicle realizes the automatic driving of the vehicle through the automatic driving system. The automatic driving state of the vehicle includes the fully automatic driving state and the takeover state. In the automatic driving state, the vehicle can independently perceive the environment, make decisions and perform corresponding behaviors without the need for direct intervention and control of the vehicle by the human driver. In the takeover state, the driver takes over the control of the vehicle and is responsible for the safe operation of the vehicle.

[0045] Optionally, the test process in this embodiment is a performance test for the performance of the vehicle passing through an intersection. Generally, the excellent performance of a vehicle passing through an intersection is generally reflected in passing through the intersection in a fully automatic driving state without human takeover, and it is best to maintain normal physical sensation when passing through the intersection without causing abnormal physical sensation problems. Therefore, the automatic driving information obtained in this embodiment includes fully automatic driving state information, takeover state information, and abnormal physical sensation information, etc., so as to analyze the automatic driving process of the target vehicle through the above automatic driving information, and then realize the evaluation of the automatic driving system. Therefore, in this embodiment, the evaluation of the vehicle's automatic driving system can be realized without adding other components, which is conducive to reducing the evaluation cost of the automatic driving system.

[0046] Optionally, the autonomous driving state information in this embodiment includes information such as the start time, end time and occurrence location of the fully autonomous driving state. The takeover state information includes information such as the start time, end time and occurrence location of the takeover state. Abnormal somatosensory information is used to characterize relevant driving information that causes physical discomfort to passengers in the vehicle, including abnormal somatosensory shapes and information such as the occurrence time and location of each abnormal somatosensory shape. Abnormal somatosensory shapes include a combination of one or more of the following: freezing, sudden braking, sudden movement and sudden turning, but are not limited to the above situations.

[0047] Furthermore, in the present embodiment, during the target vehicle test, the target vehicle can determine whether the vehicle is in a fully automatic driving state or a takeover state through the vehicle monitoring system and the driver status monitoring system of the vehicle, and collect the corresponding fully automatic driving state information and takeover state information. Specifically, in the present embodiment, the vehicle can be determined to be in a fully automatic driving state through sensors in the vehicle monitoring system, including laser radar, camera, ultrasonic sensor, etc., and the relevant information in the automatic driving state information can be determined based on the sensor information collected by the vehicle monitoring system. In addition, in the present embodiment, the vehicle control unit or the driver status can be monitored by the driver status detection system to determine whether the vehicle is in a takeover state, such as when the output and state of the vehicle control unit are abnormal or inconsistent with expectations, or when the driver is determined to be in a driving state based on the detected driver's eye state, head posture and hand movement, the vehicle is determined to be in a takeover state, and the takeover state information is determined by collecting relevant information in the takeover state. In addition, in this embodiment, various sensors installed on the vehicle can be used to monitor whether abnormal body sensations occur during vehicle driving. For example, a throttle position sensor can be used to detect a jam, a steering wheel speed sensor can be used to detect a sharp turn, and a speed sensor or an acceleration sensor can be used to detect sudden braking and rapid driving. When abnormal body sensations are detected, abnormal body sensation patterns and abnormal body sensation information such as the occurrence time, end time, and occurrence location of each abnormal body sensation pattern are recorded according to sensor information. Thus, different states of the target vehicle and corresponding state information are determined in the above manner, making it more convenient to obtain automatic driving information during the target vehicle test.

[0048] In step S200, driving parameters and / or yielding parameters for the target vehicle to pass through the intersection are determined based on the automatic driving information.

[0049] In this embodiment, the driving parameters are used to evaluate the intersection performance of the target vehicle when passing through the intersection, and the yielding parameters are used to evaluate the yielding performance of the target vehicle when passing through the intersection. The driving parameters and yielding parameters of the target vehicle when passing through the intersection can be determined by analyzing and processing the automatic driving information.

[0050] In an optional implementation, in this embodiment, only the driving parameters of the target vehicle passing through the intersection are determined based on the autonomous driving information, so as to determine the intersection performance of the target vehicle passing through the intersection through the driving parameters, and then the performance of the vehicle passing through the intersection is evaluated through the intersection performance.

[0051] In another optional implementation, in this embodiment, only the yielding parameters of the target vehicle when passing through the intersection are determined based on the autonomous driving information, so as to determine the yielding performance of the target vehicle when passing through the intersection through the yielding parameters, and then evaluate the performance of the vehicle passing through the intersection through the yielding performance.

[0052] In another optional implementation, in this embodiment, the driving parameters and yielding parameters of the target vehicle passing through the intersection are determined based on the autonomous driving information, so as to determine the intersection traffic performance of the target vehicle through the intersection through the driving parameters and determine the yielding performance of the target vehicle through the intersection through the yielding parameters, and then evaluate the performance of the vehicle through the intersection through the intersection traffic performance and the yielding performance.

[0053] In step S300, the driving performance of the target vehicle is determined according to the driving parameter and / or the yielding parameter.

[0054] In this embodiment, the driving performance of the target vehicle includes intersection passability and / or yielding performance, the intersection passability is determined according to the driving parameters, and the yielding performance is determined according to the yielding parameters. Below, the determination methods of intersection passability and yielding performance are described respectively. It should be understood that when the driving performance to be evaluated includes both intersection passability and yielding performance, the methods in the following embodiments can be used simultaneously or separately to determine the intersection passability and yielding performance of the vehicle, and then the performance of the vehicle passing the intersection can be evaluated according to the intersection passability and yielding performance.

[0055] Figure 2 1 is a flow chart of evaluating the intersection yielding performance of an automatic driving system according to an embodiment of the present invention. Figure 2 As shown, in this embodiment, the intersection traffic performance of the automatic driving system is evaluated through the following steps.

[0056] In step S210, a first execution rate and / or a second execution rate is determined according to the yield parameter.

[0057] In this embodiment, the yielding parameter is used to characterize the parameters corresponding to the yielding operation performed by the target vehicle at the intersection. The first execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior, and the second execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior and does not experience abnormal sensations during the yielding process.

[0058] Optionally, the yielding behavior in this embodiment includes a combination of one or more of the following: yielding at low right of way, yielding at zebra crossings, and yielding at high right of way. Among them, yielding at low right of way is defined as yield, corresponding to the situation where the vehicle is a traffic participant with low right of way and gives way to traffic participants with high right of way, including the situation where the vehicle gives way to oncoming vehicles. Giving way at zebra crossings is defined as crosswork, corresponding to the situation where the vehicle gives way to pedestrians, bicycles, electric vehicles, etc. on the zebra crossing. Giving way at high right of way is defined as ACC, corresponding to the situation where the vehicle is a traffic participant with high right of way and gives way to traffic participants with low right of way, including the situation where the vehicle goes straight while other vehicles turn, the vehicle turns left while other vehicles turn right, etc.

[0059] Optionally, the abnormal physical sensation in this embodiment is used to characterize driving situations that cause physical discomfort to passengers in the vehicle, including one or more of the following combinations: jamming, sudden braking, rapid movement, and rapid turning, but not limited to the above situations. Among them, whether the abnormal physical sensation occurs during the driving of the vehicle can be detected by various sensors loaded on the vehicle, for example, jamming is detected by a throttle position sensor, rapid turning is detected by a steering wheel speed sensor, and rapid braking and rapid movement are detected by a speed sensor or an acceleration sensor.

[0060] Furthermore, the yielding parameters in this embodiment can be selected according to actual test requirements. In an optional implementation, in this embodiment, the first execution rate is determined only according to the yielding parameters, and the corresponding yielding parameters include the number of yielding times at the intersection when the target vehicle passes through the intersection and the number of yielding times in a fully automatic driving state. The first execution rate is determined by calculating the ratio of the number of yielding times at the intersection to the number of yielding times in a fully automatic driving state. Further, when the number of yielding times at the intersection is a0 and the number of yielding times in a fully automatic driving state is a1, the first execution rate is a0 / a1.

[0061] In another optional implementation, in this embodiment, the second execution rate is determined only based on the yielding parameters, and the corresponding yielding parameters include the number of times the target vehicle yields in a fully automatic driving state when passing through an intersection and the number of times the target vehicle yields in a fully automatic driving state without any abnormal sensation, and the second execution rate is determined by calculating the ratio of the number of times the vehicle yields in a fully automatic driving state to the number of times the target vehicle yields in a fully automatic driving state without any abnormal sensation. Further, when the number of times the target vehicle yields in a fully automatic driving state is a1 and the number of times the target vehicle yields in a fully automatic driving state without any abnormal sensation is a2, the second execution rate is a1 / a2.

[0062] In another optional implementation, in this embodiment, the first execution rate and the second execution rate are determined based on the yielding parameters, and the corresponding yielding parameters include the number of yielding times at intersections, the number of yielding times in a fully automatic driving state, and the number of yielding times in a fully automatic driving state without any abnormal sensation. The first execution rate is the ratio of the number of yielding times at intersections to the number of yielding times in a fully automatic driving state, and the second execution rate is the ratio of the number of yielding times in a fully automatic driving state to the number of yielding times in a fully automatic driving state without any abnormal sensation. Furthermore, when the number of yielding times at intersections is a0, the number of yielding times in a fully automatic driving state is a1, and the number of yielding times in a fully automatic driving state without any abnormal sensation is a2, the first execution rate is a0 / a1, and the second execution rate is a1 / a2.

[0063] Furthermore, in this embodiment, the method for determining the yielding parameters is explained by taking the yielding parameters including the number of times the target vehicle yields at the intersection when passing through the intersection, the number of times the vehicle yields in a fully automatic driving state, and the number of times the vehicle yields in a fully automatic driving state without any abnormal physical sensation as an example.

[0064] Optionally, the yield parameter in this embodiment is determined based on the ROS bag in the test process of the autonomous driving system. The ROS bag is a file format or tool for recording and replaying data in the Robot Operating System (ROS), which can store sensor data, status information, control commands, etc. in the ROS system, and record them in a time series. During the test process of the target vehicle, the initial value of the flag bit corresponding to each yield function is 0. After each yield function is triggered, the corresponding yield function flag bit is set to 1, corresponding to a rising edge from 0 to 1. After the yield function is released, the flag bit is set to 0, corresponding to a falling edge from 1 to 0. That is, each time a continuous rising edge and falling edge appear, a yield behavior is triggered and recorded. Therefore, in this embodiment, the number of yield behaviors is determined by obtaining the number of recorded yield behavior triggers. Furthermore, in order to improve the accuracy of the number of yielding behaviors, the triggering records of the yielding function are filtered in this embodiment. For example, the recorded yielding behaviors are screened according to the duration of the status bit, and the yielding behavior corresponding to the time when the status bit remains 1 for a period greater than a preset threshold is identified as a valid yielding behavior, and the number of triggering times of the valid yielding behavior is determined as the number of yielding behaviors.

[0065] Furthermore, when determining the number of times a vehicle gives way in a fully autonomous driving state when passing through an intersection, since the ROSbag records the start time and end time of each giving way behavior, as well as the state of the vehicle when it is manually taken over and the time when the takeover occurs. Therefore, by querying the giving way behavior in the record and whether there is a state label of manual takeover between the corresponding start time and end time of the giving way behavior, it is possible to determine whether the vehicle gives way in a fully autonomous driving state. Specifically, when there is manual takeover, the corresponding giving way behavior is determined as giving way in a non-fully autonomous driving state; when there is no manual takeover, the corresponding giving way behavior is determined as giving way in a fully autonomous driving state. Therefore, by determining whether the giving way behavior is a giving way behavior performed in a fully autonomous driving state, the number of times the vehicle gives way in a fully autonomous driving state when passing through an intersection is determined, that is, the number of times the vehicle gives way in a fully autonomous driving state.

[0066] Furthermore, when determining the number of times the vehicle gave way in a fully autonomous driving state without any abnormal sensation when passing through an intersection, since the ROS bag records the start time and end time of each giving way behavior, the state of the vehicle when it was manually taken over and the time of the takeover, as well as the abnormal sensation state and the time of occurrence of the abnormal sensation state. Therefore, by querying the yielding behavior in the record, whether there is human takeover between the corresponding start time and end time of the yielding behavior, and the state label of abnormal sensation, it is possible to determine whether the vehicle gave way in a fully autonomous driving state and no abnormal sensation occurred during the giving way process. Specifically, when there is no human takeover and no abnormal sensation occurs, the corresponding giving way behavior is determined as giving way in a fully autonomous driving state without any abnormal sensation, and then the number of giving way behaviors in which the giving way is performed in a fully autonomous driving state without any abnormal sensation is determined, that is, the number of times the giving way is performed in a fully autonomous driving state without any abnormal sensation.

[0067] It should be noted that, since there are many different situations corresponding to yielding behaviors, when the type of yielding behaviors is not the same (for example, including low-right-of-way yielding, zebra crossing yielding and high-right-of-way yielding), in this embodiment, different yielding behaviors can be recorded jointly, and each time a yielding behavior occurs, regardless of the type of yielding behavior, it is recorded once; or, different yielding behaviors can also be recorded separately, for example, different types of yielding behaviors can be distinguished by setting different identifiers, and each time a yielding behavior occurs, a yielding behavior record is added for the yielding behavior with the corresponding identifier according to the type of the yielding behavior.

[0068] Therefore, when determining the yielding parameters, taking the record of the number of yielding at the intersection as an example, if the joint recording method is adopted, the number of records of yielding at the intersection obtained after the test is completed is used as the number of yielding at the intersection during this test; if the differentiated recording method is adopted, the number of records of different types of yielding behaviors is first obtained, and then the number of records of the number of yielding behaviors of the types is summed up, and the sum result is the number of yielding at the intersection during this test. For example, when the yielding behaviors include three types of low road right yielding, zebra crossing yielding and high road right yielding, and the number of records corresponding to the three types are x, y and z respectively, the number of yielding at the intersection during the corresponding test is x+y+z. It should be understood that other yielding parameters except the number of yielding at the intersection can also be recorded in the above manner while adding label filtering, and then the corresponding yielding parameters are determined by the recording results.

[0069] Furthermore, when using a differentiated recording method to record the yield parameters corresponding to different types of yielding behaviors, a joint determination method or a differentiated determination method similar to the above-mentioned recording method may also be used to determine the first execution rate and the second execution rate in this embodiment. Specifically, taking the determination of the first execution rate as an example, assuming that during the test process of a vehicle passing through an intersection, the yielding behaviors include three types of low right of way yielding, zebra crossing yielding, and high right of way yielding. The number of intersection yields under the low right of way yielding situation is a01, and the number of low right of way yielding is performed in a fully automatic driving state is a11. The number of intersection yields corresponding to zebra crossing yielding is a02, and the number of zebra crossing yielding is performed in a fully automatic driving state is a12. The number of intersection yields under the high right of way yielding situation is a03, and the number of high right of way yielding is performed in a fully automatic driving state is a13. At this time, if the joint determination method is adopted, the total number of intersection yielding times a0=a01+a02+a03 and the total number of yielding times in a fully automatic driving state a1=a11+a12+a13 are first determined in this embodiment, and then the first execution rate is determined to be a0 / a1=(a01+a02+a03) / (a11+a12+a13). If the differentiated determination method is adopted, the first sub-execution rate corresponding to each yielding type is first determined in this embodiment, including the first sub-execution rate in the case of low road right yielding as a01 / a11, the first sub-execution rate in the case of zebra crossing yielding as a02 / a12, and the first sub-execution rate in the case of high road right yielding as a03 / a13, and then the average of the first sub-execution rates corresponding to different types is calculated as the first execution rate in the corresponding test process, that is, the first execution rate is (a01 / a11+a02 / a12+a03 / a13) / 3. Therefore, by determining the first execution rate when the target vehicle passes through the intersection in the above different ways, the method for determining the first execution rate is more flexible.

[0070] Furthermore, since the number of intersections that the vehicle passes through during the test is at least one, when there are multiple intersections, for the yielding behaviors occurring at different intersections, in this embodiment, the same type of yielding behaviors at different intersections can be cumulatively recorded based on a joint record similar to the above, or the same type of yielding behaviors at different intersections can be differentiated and recorded in a manner similar to the number of laps, and the ratio corresponding to each intersection is calculated according to the number of recorded yielding behaviors, and then the ratios of different intersections are averaged to determine the first execution rate or the second execution rate under the corresponding test scenario. Therefore, by providing different recording methods for different types of yielding behaviors at different intersections, the method for determining the first execution rate and the second execution rate corresponding to the test process is made more flexible.

[0071] In step S220 , the driving performance of the target vehicle is determined according to the first execution rate and / or the second execution rate.

[0072] In this embodiment, the driving performance is the yielding performance of the target vehicle when passing through an intersection. Since both the first execution rate and the second execution rate can reflect the yielding performance of the vehicle when passing through an intersection, when testing the yielding performance of the target vehicle when passing through an intersection, the driving performance of the target vehicle can be determined based on only the first execution rate, or based on only the second execution rate, or based on both the first execution rate and the second execution rate.

[0073] Optionally, in this embodiment, the numerical value corresponding to the first execution rate and / or the second execution rate is determined as the driving performance of the target vehicle to quantify the yielding performance of the vehicle when passing through the intersection, which is conducive to a more intuitive understanding of the driving performance of the vehicle.

[0074] Therefore, in this embodiment, by determining the first execution rate and / or the second execution rate based on the yielding parameters, and determining the driving performance of the target vehicle based on the first execution rate and / or the second execution rate, it is possible to evaluate the yielding performance of the vehicle when passing through the intersection, thereby facilitating the evaluation of the performance of the vehicle when passing through the intersection.

[0075] Figure 3 is a flow chart of evaluating the intersection performance of an automatic driving system according to an embodiment of the present invention. Figure 3 As shown, in this embodiment, the intersection yielding performance of the automatic driving system is evaluated through the following steps.

[0076] In step S310 , a first pass rate and / or a second pass rate is determined according to the driving parameters.

[0077] In this embodiment, the driving parameters are used to characterize the driving parameters of the target vehicle when passing through the intersection. The first passing rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state, and the second passing rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state without any abnormal sensation.

[0078] Optionally, the abnormal physical sensation in this embodiment is used to characterize driving situations that cause physical discomfort to passengers in the vehicle, including one or more of the following combinations: jamming, sudden braking, rapid movement, and rapid turning, but not limited to the above situations. Among them, whether the abnormal physical sensation occurs during the driving of the vehicle can be detected by various sensors loaded on the vehicle, for example, jamming is detected by a throttle position sensor, rapid turning is detected by a steering wheel speed sensor, and rapid braking and rapid movement are detected by a speed sensor or an acceleration sensor.

[0079] Furthermore, the driving parameters in this embodiment can be selected according to actual test requirements. In an optional implementation, in this embodiment, the first pass rate is determined only according to the driving parameters, and the corresponding driving parameters include the number of intersection passes and the number of intersections passed in a fully automatic driving state. The first pass rate is determined by calculating the ratio of the number of intersection passes to the number of intersections passed in a fully automatic driving state. Specifically, when the number of intersection passes is b0 and the number of intersections passed in a fully automatic driving state is b1, the first pass rate is b0 / b1.

[0080] In another optional implementation, in this embodiment, the second pass rate is determined only based on driving parameters, and the corresponding driving parameters include the number of times the vehicle passes through the intersection in a fully automatic driving state and the number of times the vehicle passes through the intersection in a fully automatic driving state without any abnormal sensation, and the second pass rate is determined by calculating the ratio of the number of times the vehicle passes through the intersection in a fully automatic driving state to the number of times the vehicle passes through the intersection in a fully automatic driving state without any abnormal sensation. Specifically, when the number of times the vehicle passes through the intersection in a fully automatic driving state is b1 and the number of times the vehicle passes through the intersection in a fully automatic driving state without any abnormal sensation is b2, the second pass rate is b1 / b2.

[0081] In another optional implementation, in this embodiment, the first pass rate and the second pass rate are determined based on driving parameters, and the corresponding driving parameters include the number of intersection passes, the number of times the intersection is passed in a fully automatic driving state, and the number of times the intersection is passed in a fully automatic driving state without abnormal physical sensation. The first pass rate is the ratio of the number of intersection passes to the number of times the intersection is passed in a fully automatic driving state, and the second pass rate is the ratio of the number of times the intersection is passed in a fully automatic driving state to the number of times the intersection is passed in a fully automatic driving state without abnormal physical sensation. Specifically, when the number of intersection passes is b0, the number of times the intersection is passed in a fully automatic driving state is b1, and the number of times the intersection is passed in a fully automatic driving state without abnormal physical sensation is b2, the first pass rate is b0 / b1, and the second pass rate is b1 / b2.

[0082] Furthermore, the number of intersection passing times, the number of intersection passing times in a fully autonomous driving state, and the number of intersection passing times in a fully autonomous driving state without any abnormal physical sensation in the driving parameters of this embodiment can be determined by querying the relevant data in the ROS bag in the autonomous driving system.

[0083] In step S320, the driving performance of the target vehicle is determined according to the first pass rate and / or the second pass rate.

[0084] In this embodiment, the driving performance is the intersection passability of the target vehicle through the intersection. Since both the first pass rate and the second pass rate can reflect the intersection passability of the vehicle through the intersection, when determining the intersection passability of the vehicle through the intersection, it can be determined only based on the first pass rate, or only based on the second pass rate, or it can be determined based on both the first pass rate and the second pass rate.

[0085] Optionally, in this embodiment, the numerical value corresponding to the first pass rate and / or the second pass rate is determined as the driving performance of the target vehicle to quantify the intersection performance of the vehicle passing through the intersection, which is conducive to a more intuitive understanding of the vehicle's driving performance.

[0086] Therefore, in this embodiment, by determining the first pass rate and / or the second pass rate based on the driving parameters, and determining the driving performance of the target vehicle based on the first pass rate and / or the second pass rate, it is possible to evaluate the intersection performance of the vehicle passing through the intersection, thereby facilitating the evaluation of the performance of the vehicle passing through the intersection.

[0087] Optionally, in this embodiment, after determining the yielding performance of the target vehicle when passing through the intersection and / or the intersection passing performance when passing through the intersection by the above method, in order to facilitate the stability analysis of the vehicle's driving performance, the automatic driving system evaluation method in this embodiment also includes the following steps.

[0088] In step S400, the driving performance corresponding to different test processes is recorded to monitor the change of the driving performance.

[0089] In this embodiment, by recording the driving performance determined in different test processes and then sorting out the recorded driving performance under different test processes, it is possible to monitor changes in driving performance in different test processes, which is beneficial for subsequent optimization of the automatic driving system according to the driving performance and changes in driving performance, thereby improving the overall performance of the vehicle's automatic driving.

[0090] Alternatively, if Figure 4 As shown, in this embodiment, the driving performance corresponding to each test process is displayed in the chronological order of the test process, and the change of the same driving performance is displayed in the form of a curve graph, and different driving performances correspond to different forms of display curves. For example, taking the driving performance of yielding as an example, the solid line and the dotted line in the figure respectively represent the yielding success rate and the yielding sensory normality rate of the vehicle. The yielding success rate corresponds to the aforementioned first execution rate, which is used to characterize the probability that the vehicle performs the yielding operation in a fully automatic driving state. The yielding sensory normality rate corresponds to the aforementioned second execution rate, which is used to characterize the probability that the vehicle performs the yielding operation in a fully automatic driving state without any abnormal sensory abnormality.

[0091] Furthermore, in the display interface diagram of driving performance in this embodiment, the driving parameters and yielding parameters involved in the test process, as well as the specific parameter polarity of the driving parameters and yielding parameters can also be displayed. For example, Figure 4 The display interface shown in the figure shows parameters related to yield (low road right of way yielding behavior), crosswalk (zebra crossing yielding behavior) and ACC) (high road right of way yielding behavior) during the test on x month x day, including the number of yielding behaviors, the number of successful yielding behaviors executed with fully automatic driving, the number of abnormal physical sensations, and the yielding success rate corresponding to the ratio of the number of yielding behaviors to the number of successful yielding behaviors executed with fully automatic driving, and the yielding physical sensation normality rate corresponding to the ratio of the number of yielding behaviors executed with fully automatic driving to the number of yielding behaviors executed with fully automatic driving without abnormal physical sensations.

[0092] It should be understood that the display content and display method in the display interface diagram of driving performance in this embodiment can be set according to the actual usage scenario. This is only an example and is not limited to this.

[0093] The technical solution of the embodiment of the present invention obtains the automatic driving information of the target vehicle during the test, determines the driving parameters and / or the yielding parameters of the target vehicle passing through the intersection according to the automatic driving information, and determines the driving performance of the target vehicle according to the driving parameters and / or the yielding parameters. Therefore, by analyzing the automatic driving information during the vehicle test, the intersection passing performance and intersection yielding performance of the automatic driving system can be determined, thereby realizing the evaluation of the performance of the automatic driving system passing through the intersection.

[0094] Figure 5 Schematic diagram of an automatic driving system evaluation device according to an embodiment of the present invention. Figure 5 As shown, the automatic driving system evaluation device in this embodiment includes an acquisition module 1, a determination module 2 and an evaluation module 3. Among them, the acquisition module 1 is used to obtain the automatic driving information of the target vehicle during the test process, and the automatic driving information includes full automatic driving state information, takeover state information and abnormal body perception information. The determination module 2 is used to determine the driving parameters and / or yielding parameters of the target vehicle passing through the intersection according to the automatic driving information. The evaluation module 3 is used to determine the driving performance of the target vehicle according to the driving parameters and / or yielding parameters.

[0095] Optionally, the automatic driving system evaluation device in this embodiment further includes a monitoring module 4. The monitoring module is used to record the driving performance corresponding to different test processes to monitor the changes in the driving performance.

[0096] Optionally, the evaluation module 3 in this embodiment is used to determine the first execution rate and / or the second execution rate according to the yielding parameters when determining the driving performance of the target vehicle according to the yielding parameters, and determine the driving performance of the target vehicle according to the first execution rate and / or the second execution rate. Among them, the first execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior, and the second execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior and no abnormal physical sensation occurs during the yielding process. Further, the yielding parameters in this embodiment include the number of yielding at intersections, the number of yielding in a fully automatic driving state, and the number of yielding in a fully automatic driving state without abnormal physical sensation. The yielding behavior includes a combination of one or more of the following: low road right yielding, zebra crossing yielding and high road right yielding. Abnormal physical sensation includes a combination of one or more of the following: jamming, sudden braking, rapid movement and sharp turns.

[0097] Optionally, when determining the driving performance of the target vehicle according to the driving parameters, the evaluation module 3 in this embodiment determines the first pass rate and / or the second pass rate according to the driving parameters, and determines the driving performance of the target vehicle according to the first pass rate and / or the second pass rate. Among them, the first pass rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state, and the second pass rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state without abnormal physical sensation. Further, the driving parameters in this embodiment include: the number of intersection passes, the number of times the intersection is passed in a fully automatic driving state, and the number of times the intersection is passed in a fully automatic driving state without abnormal physical sensation. Abnormal physical sensation includes a combination of one or more of the following: jamming, sudden braking, rapid movement, and sharp turns.

[0098] Figure 6 Schematic diagram of an electronic device according to an embodiment of the present invention. Figure 6 The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 61 and a memory 62. The processor 61 and the memory 62 are connected via a bus 63. The memory 62 is suitable for storing instructions or programs executable by the processor 61. The processor 61 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 61 executes the instructions stored in the memory 62, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 63 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 64 and the display device and the input / output (I / O) device 65. The input / output (I / O) device 65 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 65 is connected to the system via an input / output (I / O) controller 66.

[0099] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0101] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0102] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0103] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0104] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0105] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for evaluating an automatic driving system, characterized in that: The method comprises: Acquiring autonomous driving information of the target vehicle during the test, wherein the autonomous driving information includes fully autonomous driving state information, takeover state information, and abnormal body perception information; Determining driving parameters and / or yielding parameters for the target vehicle to pass through an intersection according to the autonomous driving information; The driving performance of the target vehicle is determined according to the driving parameter and / or the yielding parameter.

2. The method according to claim 1, characterized in that Determining the driving performance of the target vehicle according to the yielding parameter includes: Determining a first execution rate and / or a second execution rate according to the yielding parameter, wherein the first execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior, and the second execution rate is used to characterize the probability that the target vehicle correctly performs the yielding behavior and does not experience abnormal body sensation during the yielding process; The driving performance of the target vehicle is determined according to the first execution rate and / or the second execution rate.

3. The method according to claim 2, characterized in that The yielding parameters include the number of times the vehicle yielded at intersections, the number of times the vehicle yielded in a fully autonomous driving state, and the number of times the vehicle yielded in a fully autonomous driving state without any abnormal sensation.

4. The method according to claim 2, characterized in that: The giving way behavior includes one or more of the following combinations: giving way at low right of way, giving way at zebra crossings and giving way at high right of way.

5. The method according to claim 1, characterized in that Determining the driving performance of the target vehicle according to the driving parameters includes: Determine a first pass rate and / or a second pass rate according to the driving parameters, wherein the first pass rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state, and the second pass rate is used to characterize the probability that the target vehicle passes through the intersection in a fully automatic driving state without any abnormal sensation; The driving performance of the target vehicle is determined according to the first pass rate and / or the second pass rate.

6. The method according to claim 5, characterized in that The driving parameters include: the number of times the vehicle passes through an intersection, the number of times the vehicle passes through an intersection in a fully autonomous driving state, and the number of times the vehicle passes through an intersection in a fully autonomous driving state without experiencing any abnormal sensation.

7. The method according to claim 2 or 5, characterized in that: The abnormal body sensation includes one or more of the following combinations: freezing, sudden braking, rapid movement and sudden turning.

8. The method according to claim 1, characterized in that The method further comprises: The driving performance corresponding to different test processes is recorded to monitor changes in the driving performance.

9. An automatic driving system evaluation device, characterized in that: The device comprises: An acquisition module, used to acquire the autonomous driving information of the target vehicle during the test, wherein the autonomous driving information includes fully autonomous driving state information, takeover state information, and abnormal body sensing information; A determination module, configured to determine driving parameters and / or yielding parameters for the target vehicle to pass through an intersection according to the automatic driving information; An evaluation module is used to determine the driving performance of the target vehicle according to the driving parameter and / or the yielding parameter.

10. A computer program product, characterized in that The computer program product comprises a computer program / instructions, and when the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 8 are implemented.

Images