Test method of automatic driving system, electronic equipment and readable storage medium

Abstract

The invention discloses a test method of an automatic driving system, electronic equipment and a readable storage medium, and belongs to the technical field of automatic driving. The test method comprises the following steps: acquiring first test data and first abnormal information of the automatic driving system in a current test scene; identifying the first test data to obtain second abnormal information; generating a first analysis result of the second abnormal information; matching the first abnormal information with the second abnormal information; and if the first abnormal information and the second abnormal information are not successfully matched, generating a retest suggestion of the automatic driving system based on the second abnormal information, and prompting the retest suggestion and the first analysis result. According to the test method, the first abnormal information is verified through the second abnormal information, the advantages of manual identification and non-manual identification are combined, and the reliability of the test platform is effectively improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and particularly to a test method, an electronic device, and a readable storage medium for an autonomous driving system. Background Art

[0002] An autonomous driving system is a complex integrated system designed to enable a vehicle to automatically complete driving tasks without a driver or without continuous driver intervention, and has certain potential in improving traffic safety and efficiency, reducing operating costs, etc.

[0003] The performance, safety, and reliability of an autonomous driving system are the keys to whether the autonomous driving system can truly achieve commercial application. In related technologies, the verification of the performance, safety, and reliability of an autonomous driving system is usually dominated by humans, and testers need to analyze the test data of the autonomous driving system.

[0004] However, due to the subjectivity and incompleteness of humans, when testing an autonomous driving system through the above method, there may be problems such as omission and one-sidedness, resulting in the performance, safety, and reliability of the autonomous driving system not being accurately verified. Summary of the Invention

[0005] This application provides a test method, an electronic device, and a readable storage medium for an autonomous driving system to solve the technical problems existing in related technologies. Specifically, the following technical solutions are included.

[0006] In a first aspect, this application provides a test method for an autonomous driving system. The test method includes: obtaining first test data and first abnormal information of the autonomous driving system in a current test scenario, where the first abnormal information is used to indicate abnormal data in the first test data obtained after manual identification; identifying the first test data to obtain second abnormal information, where the second abnormal information is used to indicate abnormal data in the first test data obtained without manual identification; generating a first analysis result of the second abnormal information; matching the first abnormal information and the second abnormal information; if the first abnormal information and the second abnormal information do not match successfully, generating a retest suggestion for the autonomous driving system based on the second abnormal information and prompting the retest suggestion and the first analysis result.

[0007] In some possible embodiments, the test method further includes: obtaining a second analysis result of the tester for the first abnormal information; in the case where the first abnormal information and the second abnormal information match successfully, if the second analysis result is consistent with the first analysis result, saving and prompting the second analysis result and / or the first analysis result; if the second analysis result is inconsistent with the first analysis result, generating a retest suggestion for the autonomous driving system based on the second abnormal information and prompting the retest suggestion and the first analysis result.

[0008] In some possible embodiments, the identifying the first test data to obtain second abnormal information includes: when the first test data exceeds a threshold range, marking the abnormal data in the first test data to obtain the second abnormal information.

[0009] In some possible embodiments, the identifying the first test data to obtain second abnormal information includes: comparing the first test data with first historical test data of the autonomous driving system in the current test scenario, and obtaining the second abnormal information according to the comparison result.

[0010] In some possible embodiments, the first analysis result includes a horizontal comparison result for indicating the universality of the second abnormal information, and the generating the first analysis result of the second abnormal information includes: obtaining a plurality of third abnormal information according to second test data of a plurality of other autonomous driving systems in the current test scenario; matching the second abnormal information with the plurality of third abnormal information one by one, and generating the horizontal comparison result of the second abnormal information according to the matching result.

[0011] In some possible embodiments, the first analysis result includes a vertical comparison result for indicating the frequency of the second abnormal information, and the generating the first analysis result of the second abnormal information includes: obtaining a plurality of fourth abnormal information according to second historical test data of the autonomous driving system in other test scenarios; matching the second abnormal information with the plurality of fourth abnormal information one by one, and generating the vertical comparison result of the second abnormal information according to the matching result.

[0012] In some possible embodiments, the first analysis result includes a classification result for indicating the abnormal category of the second abnormal information, and the generating the first analysis result of the second abnormal information includes: extracting features of the first test data corresponding to the second abnormal information to obtain abnormal features of the second abnormal information; matching the features of the second abnormal information with an abnormal feature library, and obtaining the classification result of the second abnormal information according to the matching result.

[0013] In some possible embodiments, there are multiple pieces of the second abnormal information, and generating a retest suggestion for the autonomous driving system based on the second abnormal information includes: generating multiple retest suggestions for the autonomous driving system based on the multiple pieces of second abnormal information; the testing method further includes: dynamically adjusting the multiple retest suggestions according to the first analysis results respectively corresponding to the multiple pieces of second abnormal information; the dynamic adjustment includes a priority adjustment for adjusting the retest order of the multiple retest suggestions.

[0014] In a second aspect, the present application provides a testing device for an autonomous driving system. The testing device includes: an acquisition module configured to acquire first test data and first abnormal information of the autonomous driving system in a current test scenario, where the first abnormal information is used to indicate abnormal data in the first test data obtained after manual identification; an identification module configured to identify the first test data to obtain second abnormal information, where the second abnormal information is used to indicate abnormal data in the first test data obtained without manual identification; an analysis module configured to generate a first analysis result of the second abnormal information; a matching module configured to match the first abnormal information and the second abnormal information; if the first abnormal information and the second abnormal information do not match successfully, then generate a retest suggestion for the autonomous driving system based on the second abnormal information and prompt the retest suggestion and the first analysis result.

[0015] In some possible embodiments, the acquisition module is configured to acquire a second analysis result of the first abnormal information by a tester; the matching module is configured to, when the first abnormal and the second abnormal match successfully, save and prompt the second analysis result and / or the first analysis result if the second analysis result and the first analysis result are consistent; if the second analysis result and the first analysis result are inconsistent, then generate a retest suggestion for the autonomous driving system based on the second abnormal information and prompt the retest suggestion and the first analysis result.

[0016] In some possible embodiments, the identification module is configured to, when the first test data exceeds a threshold range, mark the abnormal data in the first test data to obtain the second abnormal information.

[0017] In some possible embodiments, the identification module is configured to compare the first test data with first historical test data of the autonomous driving system in the current test scenario to obtain the second abnormal information according to a comparison result.

[0018] In some possible embodiments, the first analysis result includes a horizontal comparison result for indicating the universality of the second abnormal information. The analysis module is configured to obtain a plurality of third abnormal information based on second test data of a plurality of other autonomous driving systems in the current test scenario; match the second abnormal information with the plurality of third abnormal information one by one, and generate the horizontal comparison result of the second abnormal information according to the matching result.

[0019] In some possible embodiments, the first analysis result includes a longitudinal comparison result for indicating the frequency of the second abnormal information. The analysis module is configured to obtain a plurality of fourth abnormal information based on second historical test data of the autonomous driving system in other test scenarios; match the second abnormal information with the plurality of fourth abnormal information one by one, and generate the longitudinal comparison result of the second abnormal information according to the matching result.

[0020] In some possible embodiments, the first analysis result includes a classification result for indicating the abnormal category of the second abnormal information. The analysis module is configured to extract features from the first test data corresponding to the second abnormal information to obtain abnormal features of the second abnormal information; match the features of the second abnormal information with an abnormal feature library, and obtain the classification result of the second abnormal information according to the matching result.

[0021] In some possible embodiments, there are a plurality of the second abnormal information. The matching module is configured to generate a plurality of retest suggestions for the autonomous driving system based on the plurality of second abnormal information; the test device further includes: an adjustment module, configured to dynamically adjust the plurality of retest suggestions according to the first analysis results respectively corresponding to the plurality of second abnormal information; the dynamic adjustment includes a priority adjustment for adjusting the retest order of the plurality of retest suggestions.

[0022] In a third aspect, the present application provides an electronic device for testing an autonomous driving system, including: a memory storing at least one program instruction for testing an autonomous driving system thereon; a processor, when the above program instruction is executed by the processor, enabling the electronic device to implement the method in the first aspect or any possible implementation manner of the first aspect of the present application.

[0023] In a fourth aspect, the present application provides a computer program (product), the computer program (product) includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, enabling a computer to implement the method in the first aspect or any possible implementation manner of the first aspect of the present application.

[0024] In a fifth aspect, the present application provides a computer-readable storage medium having stored thereon program instructions for testing an autonomous driving system. When the program instructions are executed by one or more processors, the method in the first aspect of the present application or any possible implementation of the first aspect is implemented.

[0025] The beneficial effects of the technical solution provided by this application include at least:

[0026] The technical solution disclosed in the present application can match the first abnormal information obtained by the tester from identifying the first test data of the autonomous driving system in the current test scenario with the second abnormal information obtained by non-manual identification, so that the first abnormal information identified by the tester can verify the second abnormal information to ensure the reliability and accuracy of the second abnormal information. If the first abnormal information and the second abnormal information are not matched successfully, it means that the abnormality is the second abnormal information autonomously identified by non-manual means. Considering the limitations of non-manual methods when facing complex data, the second abnormal information can be retested, thereby effectively combining the respective advantages of manual identification and non-manual identification, and improving the reliability of the test method. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 It is a schematic diagram of an implementation scenario provided by an embodiment of the present application;

[0029] Figure 2 is a flow chart of a testing method for an autonomous driving system provided in an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of the structure of a test device for an autonomous driving system provided in an embodiment of the present application;

[0031] Figure 4 It is a schematic diagram of the structure of an electronic device for testing an autonomous driving system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0034] Figure 1 is a schematic diagram of an implementation scenario provided by an embodiment of the present application. Refer to Figure 1 , the implementation scenario provided by the embodiment of the present application includes a vehicle 11 and a test device 12 for an autonomous driving system.

[0035] An autonomous driving system is installed in the vehicle 11. With the support of the autonomous driving system, the vehicle 11 can complete driving tasks without a driver or without continuous driver intervention. The test device 12 for the autonomous driving system can be connected to the vehicle 11 in a wired or wireless manner to process the data obtained from the vehicle 11, so that the test device 12 for the autonomous driving system can test the performance, safety, reliability, etc. of the autonomous driving system in the vehicle 11.

[0036] Optionally, the autonomous driving system installed in the vehicle 11 may include one or more of an in-vehicle controller, in-vehicle sensors, and an in-vehicle control terminal. The test device 12 for the autonomous driving system may be a server, a server cluster, a cloud server, etc., and the present application places no limitation thereon.

[0037] Those skilled in the art should understand that the above-mentioned vehicle 11 and the test device 12 for the autonomous driving system are only examples, and other existing or future vehicles and computer electronic devices that can be applied to the present application should also be included within the protection scope of the present application and are hereby incorporated herein by reference.

[0038] Figure 2 is a flowchart of a method for testing an autonomous driving system provided by an embodiment of the present application. This method may be executed, for example, by a server, a server cluster, or a cloud server communicatively connected to the autonomous driving system, and the present application places no limitation thereon. Refer to Figure 2 , the method for testing an autonomous driving system provided by the embodiment of the present application may include the following steps.

[0039] Step S210, obtain first test data and first exception information of the autonomous driving system in the current test scenario, where the first exception information may be used to indicate the abnormal data in the first test data obtained after manual identification.

[0040] Schematically, the first test data of the autonomous driving system in the current test scenario can be, for example, the environmental information, vehicle status information, etc. collected in real time through various sensors in the autonomous driving system during the current test process. The abnormal data in the first test data can be used, for example, to indicate the abnormal behavior of the autonomous driving system in the current test environment. The abnormal behavior can be, for example, any behavior that causes the autonomous driving system to fail to complete the driving task as required or expected, and the present application does not impose any restrictions on this. Among them, the current test scenario can be, for example, any scenario that can test the performance, safety, and reliability of the autonomous driving system, and the present application does not impose any restrictions on this.

[0041] In some embodiments, the method for obtaining the first abnormal information can include, for example: interacting with the tester in various or one way such as voice, text, etc., so that the tester can identify according to the prompted first test data of the autonomous driving system in the current test environment, and obtain the first abnormal information based on the abnormal data in the first test data; then obtaining the first abnormal information obtained by the tester after identification through the interactive feedback of the tester.

[0042] Step S220: Identify the first test data to obtain the second abnormal information. Among them, the second abnormal information can be used to indicate the abnormal data in the first test data obtained without manual identification.

[0043] Optionally, the second abnormal information can be, for example, obtained by any non-manual method such as a server, a server cluster, or a cloud server that can identify the first test data when identifying the first test data, and can be used to indicate the abnormal data corresponding to the abnormal behavior of the autonomous driving system in the current test environment.

[0044] Considering that when a vehicle equipped with an autonomous driving system is driving normally, various behavior parameters of the autonomous driving system (such as vehicle speed, longitudinal acceleration, lateral acceleration, etc.) usually exhibit characteristics such as stability, continuity, and regularity, and the first test data collected by the sensors of the autonomous driving system corresponding to various behavior parameters also usually fall within the expected threshold range; while when the vehicle equipped with the autonomous driving system exhibits abnormal behavior, various behavior parameters of the autonomous driving system will also show abnormalities, and abnormal data exceeding the expected threshold range will also appear in the first test data corresponding to various behavior parameters. In view of this, the first test data can be identified by determining whether the first test data includes abnormal data. In some embodiments, the first test data is identified to obtain second abnormal information. For example, it may include: when the first test data exceeds the threshold range, the abnormal data in the first test data is marked to obtain the second abnormal information. Among them, the value of the threshold range can be adjusted according to the actual application situation, and the present application does not make any restrictions here. For example, when marking the abnormal data in the first test data, the data exceeding the threshold range can be marked as abnormal data. By using the threshold range to identify the first test data in the embodiments of the present application, the abnormal data in the first test data can be obtained quickly and conveniently, effectively improving the comprehensiveness and timeliness of the identification of the first test data.

[0045] In some embodiments, the first test data of the autonomous driving system in the current test scenario may include, for example, the speed data of the autonomous driving system, the longitudinal acceleration data, the lateral acceleration data, the position information of the autonomous driving system on the road, and other data; the threshold range may include a first threshold range corresponding to the speed data, a second threshold range corresponding to the longitudinal acceleration data, a third threshold range corresponding to the lateral acceleration data, a fourth threshold range corresponding to the data such as the position information of the autonomous driving system on the road, and other multiple threshold ranges.

[0046] It is also considered that due to the uncertainty and instability of the current test scenarios of the autonomous driving system, it is vulnerable to interference from unpredictable factors such as weather and environment. In view of this, it is desired that the recognition of the first test data has a certain environmental adaptability to reduce the possibility of incorrect and abnormal recognition. In some embodiments, the first test data is recognized to obtain second abnormal information. For example, it may include comparing the first test data with the first historical test data of the autonomous driving system in the current test scenario to obtain the second abnormal information according to the comparison result. Among them, the first historical test data of the autonomous driving system in the current test scenario may be, for example, environmental information, vehicle status information, etc. collected by various sensors in the autonomous driving system during other test processes before the current test process. By comparing the first test data of the autonomous driving system in the current test scenario with the first historical data in the current test scenario in the embodiments of the present application, the first test data can be recognized according to the relative relationship between the environmental information and the behavior parameters of the autonomous driving system in the current test scenario, so that the recognition of the first test data has better adaptability and reliability.

[0047] Step S230, generate a first analysis result of the second abnormal information.

[0048] Schematically, the first analysis result of the second abnormal information may be generated, for example, by a non-artificial manner such as a server, a server cluster, or a cloud server that can analyze the second abnormal information, and can be used to interpret the second abnormal information.

[0049] In some embodiments, the first analysis result may include, for example, a horizontal comparison result for indicating the universality of the second abnormal information. The method for generating the first analysis result of the second abnormal information may include, for example: obtaining a plurality of third abnormal information according to the second test data of a plurality of other autonomous driving systems in the current test scenario; matching the second abnormal information of the autonomous driving system in the current test scenario with the plurality of third abnormal information of other autonomous driving systems in the current test scenario one by one to generate a horizontal comparison result of the second abnormal information according to the matching result. Among them, the horizontal comparison result can be used to explain whether the second abnormal information of the autonomous driving system in the current test scenario is universal. For example, when the number of times the second abnormal information of the autonomous driving system appears in a plurality of other tested autonomous driving systems reaches a certain amount, it is considered that the second abnormal information is universal.

[0050] In some embodiments, the method of generating a horizontal comparison result of the second abnormal information by matching the second abnormal information of the autonomous driving system in the current test scenario with multiple third abnormal information of other autonomous driving systems in the current test scenario one by one may include, for example: matching the second abnormal information of the autonomous driving system in the current test scenario with multiple third abnormal information of other autonomous driving systems in the current test scenario one by one, and recording the number of matches once when the match is successful; after the one-by-one matching is completed, obtaining the first cumulative number of matches; if the first cumulative number of matches reaches the first match number threshold, generating a first horizontal comparison result for indicating that the second abnormal information of the autonomous driving system in the current test scenario is universal; if the first cumulative number of matches does not reach the first match number threshold, generating a second horizontal comparison result for indicating that the second abnormal information of the autonomous driving system in the current test scenario is not universal. Among them, the determination of whether the second abnormal information matches the multiple third abnormal information may refer to the following criteria, for example: whether the first test data corresponding to the second abnormal information and the second test data corresponding to the multiple third abnormal information have the same characteristics.

[0051] Through the above method, the embodiments of the present application can further determine whether the second abnormal information of the autonomous driving system in the current test scenario is universal, so as to improve the fairness and accuracy of the autonomous driving system test.

[0052] In some embodiments, the first analysis result may include, for example, a vertical comparison result for indicating the frequency of the second abnormal information. The method of generating the first analysis result of the second abnormal information may include, for example: obtaining multiple fourth abnormal information according to the second historical test data of the autonomous driving system in other test scenarios; matching the second abnormal information with the multiple fourth abnormal information one by one to generate a vertical comparison result according to the matching result. Among them, the vertical comparison result can be used to indicate the frequency of the second abnormal information of the autonomous driving system in the current test scenario, so as to explain whether the second abnormal information of the autonomous driving system in the current test scenario is accidental.

[0053] In some embodiments, the method for generating a longitudinal comparison result of the second abnormal information by matching the second abnormal information of the autonomous driving system in the current test scenario with multiple fourth abnormal information of the autonomous driving system in other test scenarios one by one may include, for example: matching the second abnormal information of the autonomous driving system in the current test scenario with multiple fourth abnormal information of the autonomous driving system in other test scenarios one by one, and recording the number of matches once when the match is successful; after the one-by-one matching is completed, obtaining the second cumulative number of matches; if the second cumulative number of matches reaches the second match number threshold, generating a first longitudinal comparison result for explaining that the second abnormal information of the autonomous driving system in the current test scenario is not accidental; if the second cumulative number of matches does not reach the second match number threshold, generating a second longitudinal comparison result for explaining that the second abnormal information of the autonomous driving system in the current test scenario is accidental. Among them, the judgment of whether the second abnormal information matches the multiple fourth abnormal information may refer to the following criteria, for example: whether the first test data corresponding to the second abnormal information has the same characteristics as the second historical test data corresponding to the multiple fourth abnormal information.

[0054] Embodiments of the present application can determine whether the second abnormal information of the autonomous driving system in the current test scenario is caused by systematic reasons or environmental factors of the autonomous driving system according to whether the second abnormal information is accidental. If the second abnormal information of the autonomous driving system in the current test scenario is accidental, the second abnormal information is caused by environmental factors; if the second abnormal information of the autonomous driving system in the current test scenario is not accidental, the second abnormal information is caused by systematic factors. Through the above method, embodiments of the present application can further determine whether the second abnormal information of the autonomous driving system in the current test scenario is accidental, so as to improve the reliability and accuracy of the autonomous driving system test.

[0055] In some embodiments, the first analysis result may include, for example, a classification result for indicating the abnormal category of the second abnormal information. The method for generating the first analysis result of the second abnormal information may include, for example: extracting features of the first test data corresponding to the second abnormal information to obtain the abnormal features of the second abnormal information; matching the features of the second abnormal information with the abnormal feature library to obtain a classification result according to the matching result. Among them, the abnormal feature library can be used to indicate the relationship between abnormal features and abnormal categories. When the abnormal features of the second abnormal information are successfully matched with the abnormalities in the abnormal feature library, the abnormal category of the second abnormal information can be determined according to the relationship between the abnormal features and abnormal categories indicated by the abnormal feature library. Through the above method, embodiments of the present application can further determine the category of the second abnormal information of the autonomous driving system in the current test scenario, so as to facilitate subsequent processing and improve the test efficiency.

[0056] Step S240: Match the first abnormal information with the second abnormal information. If the first abnormal information and the second abnormal information do not match successfully, generate a retest suggestion for the autonomous driving system based on the second abnormal information, and prompt the retest suggestion and the first analysis result.

[0057] As described above, the identification method of the first abnormal information can be, for example, a manual method, and the identification method of the second abnormal information can be, for example, a non - manual method. Since the identifications of both the manual method and the non - manual method have limitations. For example, considering the huge amount of the first test data of the autonomous driving system in the current test scenario, the manual identification may be omitted; or, considering the complexity of the first test data of the autonomous driving system in the current test scenario, the non - manual identification may be incorrect. In view of this, the first abnormal information and the second abnormal information can be matched. If the first abnormal information and the second abnormal information match successfully, it means that both the manual method and the non - manual method have identified the abnormality, and the possibility of the non - manual method identifying errors is reduced. The first analysis result generated according to the second abnormal information can be saved and prompted. If the first abnormal information and the second abnormal information do not match successfully, it indicates that the abnormality is autonomously identified by the non - manual method. A retest suggestion for the second abnormal information can be generated and the retest suggestion for the second abnormal information and the first analysis result can be prompted for the tester's reference.

[0058] Through the above method, the embodiment of the present application enables the first abnormal information identified manually to verify the second abnormal information identified non - manually, reduces the error rate of the first abnormal information and / or the second abnormal information, and improves the reliability of the test method for the autonomous driving system.

[0059] To further improve the reliability of the test method for the autonomous driving system, after obtaining the first abnormal information identified by the tester for the first test data of the autonomous driving system in the current test scenario, the analysis result of the first abnormal information by the tester can be further obtained, and the second abnormal information identified non - manually can be verified again according to the analysis result. In some embodiments, the test method provided by the embodiment of the present application may further include: obtaining the second analysis result of the first abnormal information by the tester; in the case where the first abnormal information and the second abnormal information match successfully, if the second analysis result is consistent with the first analysis result, save and prompt the second analysis result and / or the first analysis result; if the second analysis result is inconsistent with the first analysis result, generate a retest suggestion for the autonomous driving system based on the second abnormal information and prompt the retest suggestion for the autonomous driving system and the first analysis result for the tester's reference.

[0060] Through the above method, the embodiment of the present application can match the second analysis result generated by the tester manually with the first analysis result generated in a non-manual manner, so as to verify the second abnormal information and the generated first analysis result obtained in the non-manual manner again, thereby improving the reliability of the automatic driving system test system.

[0061] Considering that in the actual test process, there may be multiple second abnormal information in the current test scenario of the automatic driving system, and there may also be multiple retest suggestions for the second abnormal information. In view of this, in order to improve the test efficiency, it is necessary to reasonably arrange the retest suggestions for multiple second abnormal information. In some embodiments, the second abnormal information obtained after the non-manual identification of the first test data may include multiple, and the retest suggestions for the automatic driving system are generated based on the second abnormal information. For example, it may include: generating retest suggestions for the automatic driving system based on multiple second abnormal information. When there are multiple retest suggestions, the test method provided by the embodiment of the present application may further include: dynamically adjusting multiple retest suggestions according to the first analysis results respectively corresponding to multiple second abnormal information; the dynamic adjustment includes priority adjustment for adjusting the retest order of multiple retest suggestions. For example, different priorities may be assigned to multiple second abnormal information according to the first analysis results respectively corresponding to multiple second abnormal information, and the retest order of multiple retest suggestions is arranged in an orderly manner according to the priorities.

[0062] The technical solution disclosed in the present application can match the first abnormal information obtained by the tester's identification of the first test data in the current test scenario of the automatic driving system with the second abnormal information obtained by non-manual identification, so that the first abnormal information identified by the tester can verify the second abnormal information, so as to ensure the reliability and accuracy of the second abnormal information. If the first abnormal information and the second abnormal information do not match successfully, it means that the abnormal information is the second abnormal information independently identified by the non-manual method. Considering the limitations of the non-manual method in the face of complex data, the second abnormal information can be retested again, thereby effectively combining the respective advantages of manual identification and non-manual identification and improving the reliability of the test method.

[0063] In another possible implementation manner, the present application further provides a test device for an automatic driving system. Figure 3 It is a schematic structural diagram of the test device for the automatic driving system provided by the embodiment of the present application. Refer to Figure 3 The test device for the automatic driving system provided by the embodiment of the present application includes:

[0064] An acquisition module 310, configured to acquire the first test data and the first abnormal information in the current test scenario of the automatic driving system, where the first abnormal information is used to indicate the abnormal data in the first test data obtained after manual identification.

[0065] An identification module 320 is configured to identify the first test data to obtain second exception information, where the second exception information is used to indicate the exception data in the first test data obtained after non-artificial identification.

[0066] An analysis module 330 is configured to generate a first analysis result of the second exception information.

[0067] A matching module 340 is configured to match the first exception information with the second exception information; if the first exception information and the second exception information do not match successfully, a retest recommendation for the autonomous driving system is generated based on the second exception information, and the retest recommendation and the first analysis result are prompted.

[0068] In some possible implementation manners, an acquisition module 310 is configured to acquire a second analysis result of the tester for the first exception information; the matching module 340 is configured to, when the first exception information and the second exception information match successfully, if the second analysis result is consistent with the first analysis result, save and prompt the second analysis result and / or the first analysis result; if the second analysis result is inconsistent with the first analysis result, a retest recommendation for the autonomous driving system is generated based on the second exception information, and the retest recommendation and the first analysis result are prompted.

[0069] In some possible implementation manners, the identification module 320 is configured to mark the exception data in the first test data to obtain second exception information when the first test data exceeds the threshold range.

[0070] In some possible implementation manners, the identification module 320 is configured to compare the first test data with first historical test data of the autonomous driving system in the current test scenario, and obtain second exception information according to the comparison result.

[0071] In some possible implementation manners, the first analysis result includes a horizontal comparison result for indicating the universality of the second exception information. The analysis module 330 is configured to obtain a plurality of third exception information according to second test data of a plurality of other autonomous driving systems in the current test scenario; match the second exception information with the plurality of third exception information one by one, and generate a horizontal comparison result of the second exception information according to the matching result.

[0072] In some possible implementation manners, the first analysis result includes a vertical comparison result for indicating the frequency of the second exception information. The analysis module 330 is configured to obtain a plurality of fourth exception information according to second historical test data of the autonomous driving system in other test scenarios; match the second exception information with the plurality of fourth exception information one by one, and generate a vertical comparison result of the second exception information according to the matching result.

[0073] In some possible implementation manners, the first analysis result includes a classification result for indicating the abnormal category of the second abnormal information. The analysis module 330 is configured to extract features from the first test data corresponding to the second abnormal information to obtain the abnormal features of the second abnormal information; match the features of the second abnormal information with the abnormal feature library, and obtain the classification result of the second abnormal information according to the matching result.

[0074] In some possible implementation manners, there are multiple pieces of second abnormal information. The matching module 340 is configured to generate multiple retest suggestions for the autonomous driving system based on the multiple pieces of second abnormal information; the test device further includes: an adjustment module, configured to dynamically adjust the multiple retest suggestions according to the first analysis results respectively corresponding to the multiple pieces of second abnormal information; the dynamic adjustment includes a priority adjustment for adjusting the retest order of the multiple retest suggestions.

[0075] The test device for the autonomous driving system provided in the above embodiment and the embodiment of the test method for the autonomous driving system belong to the same concept, and the specific implementation process is detailed in the embodiment of the test method for the autonomous driving system.

[0076] In some other possible implementation manners, the present application further provides an electronic device for testing an autonomous driving system. Figure 4 is a schematic structural diagram of the electronic device for testing the autonomous driving system provided in the embodiment of the present application. Refer to Figure 4 The electronic device for testing the autonomous driving system provided in the embodiment of the present application includes:

[0077] A memory 410, on which at least one program instruction for testing the autonomous driving system is stored.

[0078] A processor 420. When the above program instruction is executed by the processor 420, the electronic device implements the method and the steps of its multiple embodiments described above. According to different implementation manners, the processor 420 may be a CPU (central processing unit), a GPU (graphics processing unit), or one or more types of other general and / or special processors, including but not limited to a DSP (digital signal processor), an ASIC (application specific integrated circuit), an FPGA (field-programmable gate array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and the number thereof may be determined according to actual needs. Figure 2 ​

[0079] In some other possible embodiments, the present application further provides a computer program (product), the computer program (product) includes computer programs / instructions, and the computer programs / instructions are executed by a processor to enable the computer to implement the method and the steps of its multiple embodiments described above in conjunction with Figure 2 the description.

[0080] In some other possible embodiments, the present application further provides a computer-readable storage medium, on which program instructions for testing an autonomous driving system are stored. When the program instructions are executed by one or more processors, the method and the steps of its multiple embodiments described above in conjunction with Figure 2 the description are implemented. The computer-readable storage medium may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0081] It should be noted that the electronic device in the present application may also be referred to as a display device. In addition, the information, data (including but not limited to image data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the grayscale values involved in the present application are obtained under full authorization.

[0082] The term "and / or" in the embodiments of the present application is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone.

[0083] The above description is only for the convenience of those skilled in the art to understand the technical solution of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for testing an automatic driving system, characterized in that: The test method includes: Acquire first test data and first abnormal information of the autonomous driving system in a current test scenario, where the first abnormal information is used to indicate abnormal data in the first test data obtained after manual identification; Identify the first test data to obtain second abnormality information, where the second abnormality information is used to indicate abnormal data in the first test data obtained after non-manual identification; generating a first analysis result of the second abnormal information; Matching the first abnormal information with the second abnormal information; If the first abnormal information and the second abnormal information do not match successfully, a retest suggestion for the autonomous driving system is generated based on the second abnormal information, and the retest suggestion and the first analysis result are prompted.

2. The testing method according to claim 1, characterized in that: The test method also includes: Obtaining a second analysis result of the tester on the first abnormal information; In the case where the first abnormal information and the second abnormal information match successfully, if the second analysis result is consistent with the first analysis result, the second analysis result and / or the first analysis result is saved and prompted; If the second analysis result is inconsistent with the first analysis result, a retest suggestion for the autonomous driving system is generated based on the second abnormal information and the retest suggestion and the first analysis result are prompted.

3. The testing method according to claim 1, characterized in that: The step of identifying the first test data to obtain the second abnormal information includes: When the first test data exceeds a threshold range, abnormal data in the first test data is marked to obtain the second abnormal information.

4. The testing method according to claim 1, characterized in that: The step of identifying the first test data to obtain the second abnormal information includes: The first test data is compared with first historical test data of the automatic driving system in the current test scenario, and the second abnormal information is obtained according to the comparison result.

5. The testing method according to claim 1, characterized in that: The first analysis result includes a horizontal comparison result indicating the prevalence of the second abnormal information, and the first analysis result for generating the second abnormal information includes: Obtaining a plurality of third abnormal information according to the second test data of a plurality of other autonomous driving systems in the current test scenario; The second abnormal information is matched with the plurality of third abnormal information one by one, and the horizontal comparison result of the second abnormal information is generated according to the matching result.

6. The testing method according to claim 1, characterized in that: The first analysis result includes a longitudinal comparison result indicating the frequency of the second abnormal information, and the first analysis result for generating the second abnormal information includes: Obtaining a plurality of fourth abnormal information according to second historical test data of the autonomous driving system in other test scenarios; The second abnormal information is matched with the plurality of fourth abnormal information one by one, and the longitudinal comparison result of the second abnormal information is generated according to the matching result.

7. The testing method according to claim 1, characterized in that: The first analysis result includes a classification result for indicating an abnormal category of the second abnormal information, and the first analysis result for generating the second abnormal information includes: performing feature extraction on the first test data corresponding to the second abnormal information to obtain abnormal features of the second abnormal information; The feature of the second abnormal information is matched with an abnormal feature library, and the classification result of the second abnormal information is obtained according to the matching result.

8. The testing method according to claim 1, characterized in that: The second abnormal information includes a plurality of items, and the generating a retest suggestion for the automatic driving system based on the second abnormal information includes: generating a plurality of retest suggestions for the autonomous driving system based on the plurality of second abnormal information; The test method also includes: The multiple retest suggestions are dynamically adjusted according to the first analysis results respectively corresponding to the multiple second abnormal information; the dynamic adjustment includes a priority adjustment for adjusting the retest order of the multiple retest suggestions.

9. An electronic device, characterized in that: include: A memory having stored therein program instructions for testing the autonomous driving system; as well as The processor, when the program instructions are executed by the processor, enables the vehicle to implement the test method described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: Program instructions for testing the autonomous driving system are stored thereon, and when the program instructions are executed by one or more processors, the testing method described in any one of claims 1-8 is implemented.

Images