Method and device for generating an autonomous driving algorithm evaluation report

Abstract

The disclosure provides a method and device for generating an automatic driving algorithm evaluation report, and relates to the technical fields of automatic driving and artificial intelligence. The method comprises the following steps: obtaining first evaluation data output by a first automatic driving algorithm; determining a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm; inputting the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model. Thus, the target prompt word is determined based on different evaluation data and / or the type of the algorithm, and then the large language model is used to automatically generate the evaluation report based on the target prompt word, thereby reducing the difficulty and cost of generating the evaluation report and improving the generation efficiency and report quality of the evaluation report.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of automatic driving and artificial intelligence, and particularly relates to a method and device for generating an automatic driving algorithm evaluation report. BACKGROUND

[0002] There are many things that need to be tested in automatic driving software, such as tests of perception, planning, control and the like on modules, and tests of software packaging and delivery. After testing, it is often necessary to generate evaluation reports according to different evaluation indexes based on a large amount of test data, and to further statistically analyze the test data to determine the problems existing in the current automatic driving algorithm. In the current report generation method, the cognitive level of the reporter determines the upper limit of the report, and therefore a more efficient and high-quality evaluation report generation method is needed to complete the most time-consuming and laborious report writing task. SUMMARY

[0003] The present disclosure aims to at least partially solve one of the technical problems in the related art.

[0004] In an embodiment of the first aspect of the present disclosure, a method for generating an automatic driving algorithm evaluation report is provided, comprising:

[0005] obtaining first evaluation data output by a first automatic driving algorithm;

[0006] determining a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm;

[0007] inputting the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0008] In an embodiment of the second aspect of the present disclosure, a device for generating an automatic driving algorithm evaluation report is provided, comprising:

[0009] a first obtaining module configured to obtain first evaluation data output by a first automatic driving algorithm;

[0010] a determining module configured to determine a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm;

[0011] a second obtaining module configured to input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0012] In an embodiment of the third aspect of the present disclosure, an electronic device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for generating an automatic driving algorithm evaluation report according to the embodiments of the present disclosure when executing the program.

[0013] A computer-readable storage medium storing a computer program is provided in a fourth aspect of the present disclosure. The computer program, when executed by a processor, implements the method for generating an automatic driving algorithm evaluation report as provided in the embodiments of the present disclosure.

[0014] A computer program product is provided in a fifth aspect of the present disclosure. The computer program product includes a computer program that, when executed by a processor, implements the method for generating an automatic driving algorithm evaluation report as provided in the embodiments of the present disclosure.

[0015] The method and device for generating an automatic driving algorithm evaluation report provided by the present disclosure have the following beneficial effects:

[0016] In the present embodiment, first, first evaluation data output by a first automatic driving algorithm is obtained, then a target prompt word is determined according to the first evaluation data and / or the type of the first automatic driving algorithm, and then the target prompt word and the first evaluation data are input into a preset large language model to obtain a first evaluation report output by the model. Thus, first, a target prompt word is determined based on different evaluation data and / or the type of the algorithm, and then an evaluation report is automatically generated based on the target prompt word and using a large language model, which reduces the difficulty and cost of generating the evaluation report and improves the generation efficiency and report quality of the evaluation report.

[0017] Additional aspects and advantages of the present disclosure will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and / or additional aspects and advantages of the present disclosure will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a method for generating an automatic driving algorithm evaluation report according to an embodiment of the present disclosure is shown in FIG. 1;

[0020] Figure 2 A flowchart of a method for generating an automatic driving algorithm evaluation report according to another embodiment of the present disclosure is shown in FIG. 2;

[0021] Figure 3 A flowchart of a method for generating an automatic driving algorithm evaluation report according to another embodiment of the present disclosure is shown in FIG. 3;

[0022] Figure 4 A structural diagram of a device for generating an automatic driving algorithm evaluation report according to an embodiment of the present disclosure is shown in FIG. 4;

[0023] Figure 5A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure are described below in detail, examples of which are shown in the accompanying drawings, in which the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as a limitation of the present disclosure.

[0025] The method for generating an automatic driving algorithm evaluation report in the embodiments of the present disclosure can be executed by the device for generating an automatic driving algorithm evaluation report in the embodiments of the present disclosure, which can be configured into an electronic device, and the present disclosure does not limit this. The electronic device can be any device with computing capability, such as a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a wearable device, and the like, which are hardware devices with various operating systems, touch screens, and / or display screens. The method for generating an automatic driving algorithm evaluation report is described below by taking an algorithm evaluation system as an example.

[0026] A method and device for generating an automatic driving algorithm evaluation report provided by the present disclosure are described in detail below with reference to the accompanying drawings.

[0027] Figure 1 A flowchart of a method for generating an automatic driving algorithm evaluation report provided by an embodiment of the present disclosure.

[0028] As shown in Figure 1 The method for generating an automatic driving algorithm evaluation report can include the following steps.

[0029] In step 101, first evaluation data output by a first automatic driving algorithm is obtained.

[0030] The first automatic driving algorithm refers to a computer program for perceiving, analyzing the environment around a vehicle, or controlling the driving state of the vehicle, such as a path planning algorithm. The first evaluation data refers to test data obtained after testing the first automatic driving algorithm, which can be sensor data, vehicle state data, algorithm running logs, and the like, and the present disclosure does not limit this.

[0031] In the embodiments of the present disclosure, the algorithm evaluation system can test the first automatic driving algorithm by deploying it to the system of a real automatic driving vehicle, or can use a simulator to simulate an automatic driving scene for testing, and the like, and the present disclosure does not limit this. Then, the data output by the first automatic driving algorithm in the test is taken as the first evaluation data.

[0032] At step 102, the target prompt word is determined according to the first evaluation data and / or the type of the first automatic driving algorithm.

[0033] The type of the first automatic driving algorithm can be a perception algorithm, a positioning and map algorithm, a planning and decision algorithm, or a control algorithm, etc., which is not limited in the present disclosure. The target prompt word is one or more evaluation indexes for generating a target evaluation report, which can be safety, accuracy and stability, multi-scene adaptability, energy consumption and efficiency, etc., which is not limited in the present disclosure.

[0034] In the embodiments of the present disclosure, a prompt word library can be pre-set in the algorithm evaluation system, and then the algorithm evaluation system can filter one or more suitable prompt words from the prompt word library as the target prompt word according to the first evaluation data. For example, when the first evaluation data is sensor data, the target prompt word can be “distance accuracy” and “multi-scene perception adaptability”. Alternatively, the target prompt word can also be filtered according to the type of the first automatic driving algorithm, for example, when the first automatic driving algorithm is a control algorithm, the determined target prompt word can be “vehicle control accuracy” and “direction control stability”. The present disclosure is not limited thereto. Alternatively, the target prompt word can also be determined according to the type of the first evaluation data and the type of the first automatic driving algorithm.

[0035] Optionally, the algorithm evaluation system can determine the target prompt word associated with the first evaluation data according to the association between the evaluation data and the prompt word, and / or determine the target prompt word associated with the first automatic driving algorithm according to the association between the algorithm and the prompt word.

[0036] In the embodiments of the present disclosure, the algorithm evaluation system can determine whether the evaluation data is associated with each prompt word and whether the algorithm is associated with each prompt word by manual labeling or semantic analysis, etc. In some cases, the algorithm evaluation system can only obtain the prompt word associated with the first evaluation data as the target prompt word, or only obtain the prompt word associated with the first automatic driving algorithm as the target prompt word, or obtain the prompt word associated with both as the target prompt word, which is not limited in the present disclosure.

[0037] Optionally, the algorithm evaluation system can also first determine the first prompt word associated with the first evaluation data and the second prompt word associated with the first automatic driving algorithm, and then in the case that the first prompt word and the second prompt word are different, the first prompt word and the second prompt word are fused based on the confidence corresponding to the first prompt word and the second prompt word respectively to obtain the target prompt word.

[0038] The confidence of the prompt word can be determined according to the feasibility and rationality of the prompt word in an actual application scenario, or the frequency of occurrence in different scenarios or different data sets, and the like, for example, the higher the frequency, the higher the confidence corresponding to the prompt word, and the like, which is not limited in the present disclosure.

[0039] In the embodiments of the present disclosure, when the first prompt word and the second prompt word are different, the algorithm evaluation system can obtain the confidence corresponding to the first prompt word and the second prompt word from the prompt word library, and fuse the first prompt word and the second prompt word according to the size of the confidence, so that the finally determined target prompt word can be the first prompt word, or the second prompt word, or a superordinate word containing the meanings of the two prompt words, or a combination of the meanings of the two prompt words, and the like, which is not limited in the present disclosure.

[0040] In step 103, the target prompt word and the first evaluation data are input into a preset large language model to obtain a first evaluation report output by the model.

[0041] The preset large language model can be a generative model trained and generated according to a large amount of historical report data, or can be a model obtained by fine-tuning on the basis of a natural language processing tool ChatGPT (Chat Generative Pre-trained Transformer).

[0042] In the embodiments of the present disclosure, when the input data can contain data such as version, date, tester, optimization point, etc., the preset large language model can generate a report in a corresponding specific format, such as a comparison table, a column chart, a curve chart, and a summary of different versions, and then combine the first evaluation data to summarize, aggregate and layout the information to output the first evaluation report.

[0043] In the embodiments, the algorithm evaluation system first obtains the first evaluation data output by the first automatic driving algorithm, and then determines the target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm, and then inputs the target prompt word and the first evaluation data into the preset large language model to obtain the first evaluation report output by the model. Thus, the target prompt word is first determined based on different evaluation data and / or the type of the algorithm, and then the evaluation report is automatically generated based on the target prompt word and the large language model, which reduces the difficulty and cost of generating the evaluation report and improves the generation efficiency and report quality of the evaluation report.

[0044] Figure 2 A flowchart of a method for generating an automatic driving algorithm evaluation report provided by another embodiment of the present disclosure.

[0045] As Figure 2As shown, the method for generating the automatic driving algorithm evaluation report can include:

[0046] In step 201, first evaluation data output by the first automatic driving algorithm is obtained.

[0047] The specific implementation form of step 201 can refer to the above embodiments, which will not be described here.

[0048] In step 202, reference evaluation data associated with the type of the first automatic driving algorithm is determined.

[0049] The reference evaluation data refers to data indicators that should be included when evaluating different types of first automatic driving algorithms.

[0050] In the embodiments of the present disclosure, the algorithm evaluation system can determine corresponding reference evaluation data for different types of first automatic driving algorithms. For example, when the first automatic driving algorithm is a perception algorithm, the associated reference evaluation data can include data such as target detection, target tracking, and semantic segmentation. Or, when the first automatic driving algorithm is a control algorithm, the associated reference evaluation data can include data such as vehicle dynamics control, steering control, and brake control.

[0051] In step 203, the first evaluation data is verified based on the reference evaluation data.

[0052] In the embodiments of the present disclosure, the algorithm evaluation data can compare the vectors in the obtained first evaluation data with the vectors in the reference evaluation data one by one to determine whether the information contained in the first evaluation data is complete.

[0053] In step 204, in the case where the first data amount contained in the first evaluation data is less than the second data amount contained in the reference evaluation data, and / or in the case where the first data type contained in the first evaluation data is different from the second data type contained in the reference evaluation data, the first evaluation data missing warning information is output.

[0054] In the embodiments of the present disclosure, when the first evaluation data satisfies the condition that the first data amount contained is less than the second data amount contained in the reference evaluation data, and / or when the first data type contained in the first evaluation data is different from the second data type contained in the reference evaluation data, the algorithm evaluation system can determine that there is missing data in the first evaluation data, and thus sends the first evaluation data missing warning information to prompt the tester to supplement the missing part.

[0055] Alternatively, the data missing warning reminder can not be performed, and the first evaluation data with missing information can be directly input into the model to generate a report containing incomplete indicators.

[0056] In the embodiments of the present disclosure, after receiving the missing warning information output by the algorithm evaluation system, the tester may choose to supplement the corresponding missing data, so that the algorithm evaluation system can receive the first evaluation data update instruction sent by the tester and obtain new first evaluation data. Alternatively, the tester may choose to ignore the missing warning information, so that the algorithm test system can perform subsequent processing based on the first evaluation data.

[0057] Step 205, in the case of receiving the updated first evaluation data, determining the target prompt word according to the updated first evaluation data and / or the type of the first automatic driving algorithm.

[0058] Step 206, inputting the target prompt word and the updated first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0059] The specific implementation forms of steps 205 and 206 can refer to the above embodiments, which will not be described here.

[0060] In the embodiments, after the algorithm evaluation system obtains the first evaluation data output by the first automatic driving algorithm, the algorithm evaluation system checks the first evaluation data according to the reference evaluation data associated with the type of the first automatic driving algorithm. In the case that the first data amount contained in the first evaluation data is less than the second data amount contained in the reference evaluation data, and / or in the case that the first data type contained in the first evaluation data is different from the second data type contained in the reference evaluation data, the algorithm evaluation system outputs first evaluation data missing warning information to prompt the tester to supplement the missing information, thereby ensuring the integrity of the first evaluation data, providing conditions for improving the efficiency and quality of the generated evaluation report, and then obtaining the evaluation report based on the updated first evaluation data and the target prompt word through the preset large language model, thereby further improving the quality of the generated evaluation report.

[0061] Figure 3 A flowchart of an automatic driving algorithm evaluation report generation method provided by another embodiment of the present disclosure.

[0062] As shown in Figure 3 , the automatic driving algorithm evaluation report generation method can include:

[0063] Step 301, obtaining first evaluation data output by a first automatic driving algorithm.

[0064] Step 302, determining a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm.

[0065] Step 303, inputting the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0066] The specific implementation forms of steps 301-303 can refer to the above embodiments, which will not be described here again.

[0067] In step 304, a reference report associated with the target prompt word is determined.

[0068] The reference report can be a driving dynamic target report, a parking and berthing integrated report, a parking garage position detection report, and the like, which are not limited in the present disclosure.

[0069] In the embodiments of the present disclosure, a plurality of types of reference reports and the prompt words associated with each type of reference report can be pre-stored in the algorithm evaluation system, so that the algorithm evaluation system can find the associated reference report according to the target prompt word after obtaining the target prompt word.

[0070] In step 305, the first evaluation report is verified based on the reference report.

[0071] In step 306, in the case that the first evaluation report does not contain at least one item of information indicated in the reference report, the first evaluation report is updated based on at least one item of information indicated in the reference report to obtain a second evaluation report, wherein the display mode of an item of information indicated in the second evaluation report is different from the display mode of the remaining information.

[0072] In the embodiments of the present disclosure, in the case that the first evaluation report does not contain at least one item of information indicated in the reference report, the algorithm evaluation system can supplement the first evaluation report with the at least one item of information indicated in the reference report to obtain an updated second evaluation report.

[0073] It should be noted that the display mode of the supplemented content based on the reference report in the second evaluation report is different from the display mode of the other information, for example, the font of the supplemented content can be displayed in red, and the remaining information can be displayed in black, and the like, which are not limited in the present disclosure. The two can be distinguished by the different display modes, the readability of the generated evaluation report is improved, and the source of the data contained in the evaluation report can be determined by the tester.

[0074] In step 307, the second evaluation report is displayed.

[0075] In the embodiments of the present disclosure, the algorithm evaluation system can display the second evaluation report, so that the tester can intuitively confirm the problems exposed by the corresponding first automatic driving algorithm, and can also make adaptive modifications in the second evaluation report.

[0076] Optionally, the algorithm evaluation system can store the updated third evaluation report and the first evaluation data in the pre-set database in association with each other in the case that an update instruction for the second evaluation report is received.

[0077] In this embodiment of the disclosure, the tester can click on the second evaluation report on the display interface, or the update control in the display interface of the second evaluation report, to send an update instruction for the second evaluation report to the algorithm evaluation system. Then the algorithm evaluation system can obtain the information modified by the tester on the second evaluation report, generate the updated third evaluation report, and associate it with the first evaluation data and store it in a preset database.

[0078] Then, if the amount of data in the preset database exceeds the threshold, or if the preset model update cycle is reached, the large language model is updated using the data in the preset database to obtain the updated language model.

[0079] The threshold can be a fixed value or a variable value determined based on the actual situation. For example, if the requirements for the generated evaluation report are high, the preset threshold may be smaller. The update cycle may be a preset fixed time period, such as 1 day, 1 week, etc. This disclosure does not limit it, or it may be a variable value determined based on the frequency of algorithm evaluation.

[0080] In this embodiment, when the amount of data in the preset database exceeds a threshold, or when any of the preset model update cycles is reached, the accuracy and timeliness of the current large language model can be considered to have decreased. Therefore, the algorithm evaluation system can use the data in the preset database to update the large language model, improve the model's accuracy, and ensure the quality and reliability of the generated evaluation report.

[0081] In this embodiment of the disclosure, testers can also score the generated evaluation report, and then the algorithm evaluation system uses the reports with higher scores to adjust the large language model in real time.

[0082] In this embodiment, after obtaining the first evaluation report output by the model, the algorithm evaluation system verifies it based on the reference report associated with the target prompt words. Then, if the first evaluation report does not contain at least one piece of information indicated in the reference report, it updates the first evaluation report based on at least one piece of information indicated in the reference report to obtain a second evaluation report, which is then displayed. This allows for the supplementation and updating of missing information in the generated first evaluation report, further improving the completeness and quality of the evaluation report and providing conditions for updating the evaluation report generation model and ensuring the model's accuracy and reliability.

[0083] To implement the above embodiments, this disclosure also proposes an apparatus for generating autonomous driving algorithm evaluation reports.

[0084] like Figure 4 As shown, Figure 4A structural schematic diagram of an automatic driving algorithm evaluation report generation device provided by an embodiment of the present disclosure. The automatic driving algorithm evaluation report generation device 400 can include:

[0085] A first obtaining module 401 configured to obtain first evaluation data output by a first automatic driving algorithm;

[0086] A determining module 402 configured to determine a target prompt word according to the first evaluation data and / or a type of the first automatic driving algorithm;

[0087] A second obtaining module 403 configured to input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0088] In some embodiments, the determining module 402 can be further configured to:

[0089] determine the target prompt word associated with the first evaluation data according to an association relationship between the evaluation data and the prompt word; and / or

[0090] determine the target prompt word associated with the first automatic driving algorithm according to an association relationship between the algorithm and the prompt word.

[0091] In some embodiments, the determining module 402 can be further configured to:

[0092] determine a first prompt word associated with the first evaluation data and a second prompt word associated with the first automatic driving algorithm;

[0093] in a case where the first prompt word is different from the second prompt word, fuse the first prompt word and the second prompt word based on respective confidences of the first prompt word and the second prompt word to obtain the target prompt word.

[0094] In some embodiments, the first obtaining module 401 can be further configured to:

[0095] determine reference evaluation data associated with the type of the first automatic driving algorithm;

[0096] verify the first evaluation data based on the reference evaluation data;

[0097] in a case where a first data amount contained in the first evaluation data is less than a second data amount contained in the reference evaluation data, and / or in a case where a first data type contained in the first evaluation data is different from a second data type contained in the reference evaluation data, output a first evaluation data missing warning information.

[0098] In some embodiments, the second obtaining module 403 can be further configured to:

[0099] determine a reference report associated with the target prompt word;

[0100] verify the first evaluation report based on the reference report;

[0101] In a case where the evaluation report does not contain at least one piece of information indicated in the reference report, update the first evaluation report based on at least one piece of information indicated in the reference report to obtain a second evaluation report, wherein the display mode of one piece of information indicated in the second evaluation report is different from the display mode of the remaining information;

[0102] display the second evaluation report.

[0103] In some embodiments, the second obtaining module 403 described above can also be configured to:

[0104] In a case where an update instruction for the second evaluation report is received, store the updated third evaluation report and the first evaluation data in the preset database.

[0105] In some embodiments, the second obtaining module 403 described above can also be configured to:

[0106] In a case where the amount of data in the preset database is greater than a threshold value, or in a case where a preset model update period is reached, update the large language model using the data in the preset database to obtain an updated language model.

[0107] The functions and specific implementation principles of any of the modules described in the embodiments can refer to the method embodiments of the present disclosure described above, and will not be described here.

[0108] In the present embodiment, the algorithm evaluation system first obtains first evaluation data output by the first automatic driving algorithm, and then determines a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm. Then, the target prompt word and the first evaluation data are input into a preset large language model to obtain a first evaluation report output by the model. In this way, the target prompt word is first determined based on different evaluation data and / or the type of the algorithm, and then the evaluation report is automatically generated based on the target prompt word using the large language model, which reduces the difficulty and cost of generating the evaluation report and improves the generation efficiency and report quality of the evaluation report.

[0109] To implement the above-mentioned embodiments, the present disclosure further proposes an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the generation method of the automatic driving algorithm evaluation report proposed in the foregoing embodiments of the present disclosure is implemented.

[0110] To achieve the above-mentioned embodiments, the disclosure further proposes a computer readable storage medium, storing a computer program, the computer program being executed by a processor to implement the automatic driving algorithm evaluation report generation method proposed in the foregoing embodiments of the disclosure.

[0111] To achieve the above-mentioned embodiments, the disclosure further proposes a computer program product, comprising a computer program, the computer program being executed by a processor to implement the automatic driving algorithm evaluation report generation method proposed in the foregoing embodiments of the disclosure.

[0112] Figure 5 A block diagram of an exemplary electronic device suitable for implementing an embodiment of the disclosure is shown. Figure 5 The electronic device 500 shown is merely one example and should not be taken as limiting the scope of the functionality or use of embodiments of the disclosure.

[0113] As shown in Figure 5 The electronic device 500 is presented in the form of a general-purpose computing device. The components of electronic device 500 can include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including system memory 28 and processing unit 16.

[0114] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures including Industry Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0115] Electronic device 500 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by electronic device 500 and includes both volatile and non-volatile media, removable and non-removable media.

[0116] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Electronic device 500 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact optical disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0117] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0118] The electronic device 500 can also communicate with one or more external devices 14 such as a keyboard or pointing device, a display 24, etc.; other devices such as a storage device or a printer; and / or one or more devices that enable a user to interact with the electronic device 500; and / or one or more devices that enable the electronic device 500 to communicate with one or more other computing devices. Such communication can be via an input / output (I / O) interface 22. Still yet, the electronic device 500 can communicate with one or more networks such as a local area network (LAN), a wide area network (WAN), and / or the Internet through a network adapter 20. As an example, the network adapter 20 can include a modem, a network card (wireless or wired), or other well-known interface devices. The electronic device 500 can also contain one or more input / output (I / O) devices such as a keyboard, mouse, and / or the like. The electronic device 500 can also include one or more storage devices such as a disk drive, optical storage device, and / or the like. Such computer program products can also include any tangible computer program storage media with a computer program stored therein.

[0119] The processing unit 16 executes the various functions of the applications and data processing by running programs stored in the system memory 28, such as implementing the methods described in the foregoing embodiments.

[0120] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the description of the specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0121] In addition, the terms "first", "second", etc. are used only for the purpose of description and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly specified.

[0122] Any processes or methods described in the flowcharts or otherwise described herein can be understood as representing a module, segment, or portion of code that includes one or more executable instructions for implementing the specified logical function(s) or process(es). The scope of a preferred embodiment of this disclosure includes alternatives that can not be explicitly described or shown in the figures and that can be understood by those skilled in the art in light of the disclosure. The description of a process or method is not limited to an explicit order of steps unless explicitly stated in the claims. The steps of a process or method can be performed in an order different than the order shown or discussed, including substantially simultaneously or in reverse order, unless explicitly stated in the claims.

[0123] Logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be embodied in computer-readable instructions, segments, or a portion of a program, which includes one or more executable instructions for implementing the logic functions or processes described in the flowcharts or otherwise described herein. The scope of a preferred embodiment of this disclosure includes alternatives that can not be explicitly described or shown in the figures and that can be understood by those skilled in the art in light of the disclosure. The description of a process or method is not limited to an explicit order of steps unless explicitly stated in the claims. The steps of a process or method can be performed in an order different than the order shown or discussed, including substantially simultaneously or in reverse order, unless explicitly stated in the claims. For purposes of this description, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a computer- readable storage medium or a computer-readable communication medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (a magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (an optical device), and a portable compact disc read-only memory (CDROM). Note that the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. In this description and the following claims, a "computer-readable medium" can be one or more of the same type or types of media described above, depending upon the particular usage of or upon the particular modification of a representation of a program or portion of a program desired to be implemented.

[0124] It is to be understood that the various parts of the disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or a combination thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0125] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0126] In addition, each functional unit in each embodiment of the present disclosure can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. The integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0127] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present disclosure.

Claims

1. A method for generating an automatic driving algorithm evaluation report, the method comprising: receiving a plurality of evaluation results of an automatic driving algorithm; and generating the automatic driving algorithm evaluation report based on the plurality of evaluation results. The method comprises: acquiring first evaluation data output by a first automatic driving algorithm; determining a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm; inputting the target prompt word and the first evaluation data into a preset large language model to acquire a first evaluation report output by the model; determining a first prompt word associated with the first evaluation data and a second prompt word associated with the first automatic driving algorithm; in a case where the first prompt word is different from the second prompt word, fusing the first prompt word and the second prompt word according to confidence degrees corresponding to the first prompt word and the second prompt word respectively to acquire the target prompt word. determining a target prompt word associated with the first evaluation data according to an association relationship between evaluation data and prompt words; and / or 2. The method of claim 1, wherein, determining a target prompt word associated with the first automatic driving algorithm according to an association relationship between algorithms and prompt words. after acquiring the first evaluation data output by the first automatic driving algorithm, further comprising: determining reference evaluation data associated with the type of the first automatic driving algorithm; 3. The method of claim 1, wherein, verifying the first evaluation data based on the reference evaluation data; in a case where a first data amount contained in the first evaluation data is smaller than a second data amount contained in the reference evaluation data, and / or in a case where a first data type contained in the first evaluation data is different from a second data type contained in the reference evaluation data, outputting first evaluation data missing warning information. after acquiring the first evaluation report output by the model, further comprising: determining a reference report associated with the target prompt word; 4. The method of claim 1, wherein, verifying the first evaluation report based on the reference report; in a case where at least one item of information indicated in the reference report is not contained in the evaluation report, updating the first evaluation report based on the at least one item of information indicated in the reference report to acquire a second evaluation report, wherein a display mode of the indicated item of information in the second evaluation report is different from display modes of remaining information; displaying the second evaluation report. after displaying the second evaluation report, further comprising: in a case where an update instruction for the second evaluation report is received, storing an updated third evaluation report and the first evaluation data in a preset database.

5. The method of claim 4, wherein, further comprising: in a case where a data amount in the preset database is greater than a threshold value or a preset model update period is reached, updating the large language model using the data in the preset database to acquire an updated language model.

6. The method of claim 5, wherein, The method comprises: a first acquisition module, configured to acquire first evaluation data output by a first automatic driving algorithm; 7. An apparatus for generating an automatic driving algorithm evaluation report, the apparatus comprising: a report generation unit configured to generate the automatic driving algorithm evaluation report based on a result of an evaluation of an automatic driving algorithm. a determination module, configured to determine a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm; ​ ​ The second acquisition module is configured to input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model. The determination module is further configured to: determine a first prompt word associated with the first evaluation data and a second prompt word associated with the first automatic driving algorithm; in a case where the first prompt word is different from the second prompt word, fuse the first prompt word and the second prompt word based on confidence degrees corresponding to the first prompt word and the second prompt word respectively to obtain the target prompt word.

8. An electronic device, comprising: A computer program product includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the method for generating an automatic driving algorithm evaluation report according to any one of claims 1-6 when executing the program.

9. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the method for generating an automatic driving algorithm evaluation report according to any one of claims 1-6.

Images