Method and device for generating automatic driving algorithm evaluation report

Abstract

The invention provides a method and device for generating an automatic driving algorithm evaluation report, and relates to the technical field of automatic driving and artificial intelligence. Comprising the steps of obtaining first evaluation data output by a first automatic driving algorithm; determining a target prompt word according to the first evaluation data and / or the type of the first automatic driving algorithm; and inputting the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model. Therefore, the target cue word is firstly determined based on different evaluation data and / or algorithm types, and then the evaluation report is automatically generated by utilizing the large language model based on the target cue word, so that the difficulty and the cost of generating the evaluation report are reduced, and the generation efficiency and the report quality of the evaluation report are improved.

Description

Technical Field

[0001] The present disclosure relates to the fields of autonomous driving and artificial intelligence technology, and in particular to a method and device for generating an autonomous driving algorithm evaluation report. Background Art

[0002] There are many things that need to be tested in autonomous driving software, such as testing of modules including perception, planning, control, etc., as well as testing of software package delivery. After testing, it is often necessary to generate evaluation reports based on a large amount of test data according to different evaluation indicators, and further statistically analyze the test data to determine the problems existing in the current autonomous driving algorithm. In the current report generation method, the cognitive level of the reporter determines the upper limit of the report, so a more efficient and high-quality evaluation report generation method is needed to complete the most time-consuming and labor-intensive report writing task. Summary of the invention

[0003] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0004] The first embodiment of the present disclosure provides a method for generating an autonomous driving algorithm evaluation report, comprising:

[0005] Obtaining first evaluation data output by a first autonomous driving algorithm;

[0006] Determining a target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm;

[0007] The target prompt word and the first evaluation data are input into a preset large language model to obtain a first evaluation report output by the model.

[0008] The second embodiment of the present disclosure provides a device for generating an autonomous driving algorithm evaluation report, including:

[0009] A first acquisition module, used to acquire first evaluation data output by a first autonomous driving algorithm;

[0010] a determination module, configured to determine a target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm;

[0011] The second acquisition module is used to input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0012] The third aspect embodiment of the present disclosure proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for generating an autonomous driving algorithm evaluation report as proposed in the embodiment of the present disclosure.

[0013] The fourth aspect embodiment of the present disclosure proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for generating an autonomous driving algorithm evaluation report as proposed in the embodiment of the present disclosure is implemented.

[0014] The fifth aspect embodiment of the present disclosure proposes a computer program product, including a computer program, which, when executed by a processor, implements the method for generating an autonomous driving algorithm evaluation report as proposed in the embodiment of the present disclosure.

[0015] The method and device for generating an autonomous driving algorithm evaluation report provided by the present disclosure have the following beneficial effects:

[0016] In this embodiment, firstly, the first evaluation data output by the first autonomous driving algorithm is obtained, and then the target prompt word is determined according to the first evaluation data and / or the type of the first autonomous driving algorithm, and then the target prompt word and the first evaluation data are input into the preset large language model to obtain the first evaluation report output by the model. Thus, the target prompt word is first determined based on different evaluation data and / or algorithm types, and then the evaluation report is automatically generated based on the target prompt word and using the large language model, which reduces the difficulty and cost of generating the evaluation report and improves the generation efficiency and report quality of the evaluation report.

[0017] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a method for generating an autonomous driving algorithm evaluation report provided by an embodiment of the present disclosure;

[0020] Figure 2 A flowchart of a method for generating an autonomous driving algorithm evaluation report provided by another embodiment of the present disclosure;

[0021] Figure 3 A flowchart of a method for generating an autonomous driving algorithm evaluation report provided by another embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram of the structure of a device for generating an autonomous driving algorithm evaluation report provided by an embodiment of the present disclosure;

[0023] Figure 5A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0025] The method for generating an autonomous driving algorithm evaluation report in the embodiments of the present disclosure may be executed by a device for generating an autonomous driving algorithm evaluation report in the embodiments of the present disclosure, and the device may be configured in an electronic device, which is not limited in the present disclosure. The electronic device may be any device with computing capabilities, such as a mobile phone, tablet computer, personal computer, personal digital assistant, wearable device, or other hardware device with various operating systems, touch screens, and / or display screens. The following embodiments of the present disclosure take the algorithm evaluation system to perform the method for generating an autonomous driving algorithm evaluation report as an example for explanation.

[0026] The following is a detailed introduction to a method and device for generating an autonomous driving algorithm evaluation report provided by the present invention in conjunction with the accompanying drawings.

[0027] Figure 1 A flowchart of a method for generating an autonomous driving algorithm evaluation report provided in one embodiment of the present disclosure.

[0028] like Figure 1 As shown, the method for generating the autonomous driving algorithm evaluation report may include:

[0029] Step 101: Obtain first evaluation data output by a first autonomous driving algorithm.

[0030] The first autonomous driving algorithm refers to a computer program used to perceive and analyze the vehicle's surrounding environment, or to control the vehicle's driving state, such as a path planning algorithm, etc. The first evaluation data refers to test data obtained after testing the first autonomous driving algorithm, which may be sensor data, vehicle state data, algorithm operation log, etc., and the present disclosure does not limit this.

[0031] In the disclosed embodiment, the algorithm evaluation system may test the first autonomous driving algorithm by deploying it to a real autonomous driving vehicle system, or may use a simulator to simulate an autonomous driving scenario for testing, etc. The disclosed embodiment is not limited to this, and then the data output by the first autonomous driving algorithm in the test is used as the first evaluation data.

[0032] Step 102: Determine a target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm.

[0033] The type of the first autonomous driving algorithm may be a perception algorithm, a positioning and mapping algorithm, a planning and decision algorithm, or a control algorithm, etc., which is not limited in this disclosure. The target prompt word is one or more evaluation indicators used to generate a target evaluation report, which may be safety, accuracy and stability, multi-scenario adaptability, energy consumption and efficiency, etc., which is not limited in this disclosure.

[0034] In an embodiment of the present disclosure, a prompt word library can be preset in the algorithm evaluation system, and then the algorithm evaluation system can select one or more suitable prompt words as target prompt words in the prompt word library according to the first evaluation data. For example, when the first evaluation data is sensor data, the target prompt words may be "distance accuracy", "multi-scene perception adaptability", etc. Alternatively, the target prompt words may be selected according to the type of the first autonomous driving algorithm. For example, when the first autonomous driving algorithm is a control algorithm, the determined target prompt words may be "vehicle handling accuracy", "directional control stability", etc. The present disclosure does not limit this. Alternatively, the target prompt words may also be determined jointly according to the type of the first evaluation data and the type of the first autonomous driving algorithm.

[0035] Optionally, the algorithm evaluation system can determine the target prompt word associated with the first evaluation data based on the association between the evaluation data and the prompt word, and / or determine the target prompt word associated with the first autonomous driving algorithm based on the association between the algorithm and the prompt word.

[0036] In the embodiment of the present disclosure, the algorithm evaluation system can determine whether the evaluation data is associated with each prompt word, and whether the algorithm is associated with each prompt word by manual annotation or semantic analysis. In some cases, the algorithm evaluation system can only obtain the prompt words associated with the first evaluation data as the target prompt words, or only obtain the prompt words associated with the first autonomous driving algorithm as the target prompt words, or can also use the prompt words associated with both as the target prompt words, which is not limited in the present disclosure.

[0037] Optionally, the algorithm evaluation system may first determine a first prompt word associated with the first evaluation data and a second prompt word associated with the first autonomous driving algorithm, and then, when the first prompt word is different from the second prompt word, based on the confidence levels corresponding to the first prompt word and the second prompt word respectively, fuse the first prompt word and the second prompt word to obtain the target prompt word.

[0038] Among them, the confidence of the prompt word can be determined based on the feasibility and rationality of the prompt word in the actual application scenario, or the frequency of occurrence in different scenarios or different data sets. For example, the higher the frequency, the higher the confidence corresponding to the prompt word, and so on. The present disclosure does not limit this.

[0039] In the embodiment of the present disclosure, when the first prompt word is different from the second prompt word, the algorithm evaluation system can obtain the confidence corresponding to the first prompt word and the second prompt word from the prompt word library, and weightedly fuse the first prompt word and the second prompt word according to the size of the confidence. Therefore, the target prompt word finally determined may be the first prompt word, or the second prompt word, or it may be a hypernym that contains the meanings of the two prompt words, or the combination of the meanings of the two prompt words, etc., and the present disclosure does not limit this.

[0040] Step 103: input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0041] Among them, the preset large language model can be a generative model generated by training based on a large amount of historical report data, or it can also be a model obtained after fine-tuning based on the natural language processing tool ChatGPT (Chat Generative Pre-trained Transformer).

[0042] In the disclosed embodiment, when the input data may contain data such as version, date, tester, optimization points, etc., the preset large language model can generate a report in a corresponding specific format, such as a comparison table, a bar chart, a curve chart, a summary of different version formats, etc., and then combine the first evaluation data to summarize, aggregate and format the information to output a first evaluation report.

[0043] In this embodiment, the algorithm evaluation system first obtains the first evaluation data output by the first autonomous driving algorithm, and then determines the target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm, and then inputs the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model. Thus, the target prompt word is first determined based on different evaluation data and / or algorithm types, and then the evaluation report is automatically generated based on the target prompt word and the large language model, which reduces the difficulty and cost of generating the evaluation report and improves the generation efficiency and report quality of the evaluation report.

[0044] Figure 2 A flowchart of a method for generating an autonomous driving algorithm evaluation report provided in another embodiment of the present disclosure.

[0045] like Figure 2As shown, the method for generating the autonomous driving algorithm evaluation report may include:

[0046] Step 201: Obtain first evaluation data output by a first autonomous driving algorithm.

[0047] The specific implementation of the above step 201 can refer to the above embodiment and will not be described again here.

[0048] Step 202: Determine reference evaluation data associated with the type of the first autonomous driving algorithm.

[0049] Among them, the reference evaluation data refers to the data indicators that should be included when evaluating different types of first autonomous driving algorithms.

[0050] In the disclosed embodiment, the algorithm evaluation system can determine corresponding reference evaluation data for different types of first autonomous driving algorithms. For example, when the first autonomous driving algorithm is a perception algorithm, its associated reference evaluation data may include data on target detection, target tracking, semantic segmentation, etc.; or when the first autonomous driving algorithm is a control algorithm, its associated reference evaluation data may include data on vehicle dynamics control, steering control, braking control, etc.

[0051] Step 203: verify the first evaluation data based on the reference evaluation data.

[0052] In the disclosed embodiment, the algorithm evaluation data may compare the vectors in the acquired first evaluation data with the vectors in the reference evaluation data one by one to determine whether the information contained in the first evaluation data is complete.

[0053] Step 204, when the amount of first data included in the first evaluation data is less than the amount of second data included in the reference evaluation data, and / or when the first data type included in the first evaluation data is different from the second data type included in the reference evaluation data, output first evaluation data missing warning information.

[0054] In the embodiment of the present disclosure, when the first evaluation data satisfies the condition that the first data amount contained is smaller than the second data amount contained in the reference evaluation data, and / or the first data type contained in the first evaluation data is different from the second data type contained in the reference evaluation data, the algorithm evaluation system can determine that there is missing data in the first evaluation data, and thereby send a first evaluation data missing warning message to prompt the tester to supplement the missing part.

[0055] Alternatively, no data missing warning reminder may be issued, and the first evaluation data with missing information may be directly input into the model to generate a report containing incomplete indicators.

[0056] In the disclosed embodiment, after receiving the missing warning information output by the algorithm evaluation system, the tester may choose to supplement the corresponding missing data, so that the algorithm evaluation system can receive the first evaluation data update instruction sent by the tester and obtain new first evaluation data. Alternatively, the tester may choose to ignore the missing warning information, so that the algorithm testing system can perform subsequent processing based on the first evaluation data.

[0057] Step 205 : upon receiving the updated first evaluation data, determining a target prompt word according to the updated first evaluation data and / or the type of the first autonomous driving algorithm.

[0058] Step 206 , input the target prompt word and the updated first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0059] The specific implementation of the above steps 205 and 206 can refer to the above embodiments and will not be described again here.

[0060] In this embodiment, after obtaining the first evaluation data output by the first autonomous driving algorithm, the algorithm evaluation system verifies the first evaluation data according to the reference evaluation data associated with the type of the first autonomous driving algorithm, and outputs first evaluation data missing warning information when the first data amount contained in the first evaluation data is less than the second data amount contained in the reference evaluation data, and / or when the first data type contained in the first evaluation data is different from the second data type contained in the reference evaluation data, prompting the tester to supplement the missing information, thereby ensuring the integrity of the first evaluation data and providing conditions for improving the efficiency and quality of the generated evaluation report. Then, based on the updated first evaluation data and the target prompt word, the evaluation report is obtained through the preset large language model, thereby further improving the quality of the generated evaluation report.

[0061] Figure 3 A flowchart of a method for generating an autonomous driving algorithm evaluation report provided in another embodiment of the present disclosure.

[0062] like Figure 3 As shown, the method for generating the autonomous driving algorithm evaluation report may include:

[0063] Step 301: Obtain first evaluation data output by a first autonomous driving algorithm.

[0064] Step 302: Determine a target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm.

[0065] Step 303: input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0066] The specific implementation of the above steps 301 to 303 can refer to the above embodiments and will not be described again here.

[0067] Step 304: Determine a reference report associated with the target prompt word.

[0068] Among them, the reference report can be a driving dynamic target report, a driving and parking integrated report, a parking space detection report, etc., and the present disclosure does not limit this.

[0069] In the disclosed embodiment, multiple types of reference reports and prompt words related to each reference report can be pre-stored in the algorithm evaluation system, so after obtaining the target prompt word, the algorithm evaluation system can search for related reference reports according to the target prompt word.

[0070] Step 305: verify the first evaluation report based on the reference report.

[0071] Step 306, when the evaluation report does not contain at least one item of information indicated in the reference report, update the first evaluation report based on at least one item of information indicated in the reference report to obtain a second evaluation report, wherein the display mode of the item of information indicated in the second evaluation report is different from the display mode of the remaining information.

[0072] In an embodiment of the present disclosure, when the evaluation report does not contain at least one item of information indicated in the reference report, the algorithm evaluation system can use the at least one item of information indicated in the reference report to supplement the first evaluation report to obtain an updated second evaluation report.

[0073] It should be noted that the display mode of the content supplemented based on the reference report in the second evaluation report is different from the display mode of other information. For example, the font of the supplementary content can be displayed in red, and the remaining information can be displayed in black, etc. The present disclosure does not limit this. The two can be distinguished by the different display modes, thereby improving the readability of the generated evaluation report and facilitating testers to determine the source of the data contained in the evaluation report.

[0074] Step 307: display the second evaluation report.

[0075] In the disclosed embodiment, the algorithm evaluation system can display the second evaluation report, so that the tester can intuitively confirm the problems exposed by the corresponding first autonomous driving algorithm, and can also make adaptive modifications in the second evaluation report.

[0076] Optionally, upon receiving an update instruction for the second evaluation report, the algorithm evaluation system may associate the updated third evaluation report with the first evaluation data and store them in a preset database.

[0077] In the disclosed embodiment, the tester can click on the second evaluation report on the display interface, or the update control in the display interface of the second evaluation report, to send an update instruction for the second evaluation report to the algorithm evaluation system. The algorithm evaluation system can then obtain the information modified by the tester on the second evaluation report, generate an updated third evaluation report, and associate it with the first evaluation data and store it in a preset database.

[0078] Then, when the amount of data in the preset database is greater than a threshold, or when a preset model update cycle is reached, the large language model is updated using the data in the preset database to obtain an updated language model.

[0079] The threshold value may be a fixed value or a variable value determined according to the actual situation. For example, when the requirements for the generated evaluation report are high, the preset threshold value may be smaller. The update cycle may be a preset fixed time period, such as 1 day, 1 week, etc., which is not limited in the present disclosure, or it may be a variable value determined according to the frequency of algorithm evaluation, etc.

[0080] In the disclosed embodiment, when the amount of data in the preset database is greater than the threshold, or any one of the conditions of the preset model update cycle is reached, it can be considered that the accuracy and timeliness of the current large language model are reduced. Therefore, the algorithm evaluation system can use the data in the preset database to update the large language model, improve the accuracy of the model, and ensure the quality and reliability of the generated evaluation report.

[0081] In the disclosed embodiment, the tester may also score the generated evaluation report, and then the algorithm evaluation system may use the report with a higher score to adjust the large language model in real time.

[0082] In this embodiment, after obtaining the first evaluation report output by the model, the algorithm evaluation system verifies it according to the reference report associated with the target prompt word, and then, if the evaluation report does not contain at least one piece of information indicated in the reference report, the first evaluation report is updated based on at least one piece of information indicated in the reference report to obtain a second evaluation report, and then the second evaluation report is displayed. In this way, the missing information in the generated first evaluation report can be supplemented and updated, further improving the integrity and quality of the evaluation report, and providing conditions for updating the generation model of the evaluation report and ensuring the accuracy and reliability of the model.

[0083] In order to implement the above-mentioned embodiment, the present disclosure also proposes a device for generating an autonomous driving algorithm evaluation report.

[0084] like Figure 4 As shown, Figure 4This is a schematic diagram of a structure of a device for generating an autonomous driving algorithm evaluation report provided by an embodiment of the present disclosure. The device 400 for generating an autonomous driving algorithm evaluation report may include:

[0085] A first acquisition module 401 is used to acquire first evaluation data output by a first autonomous driving algorithm;

[0086] A determination module 402 is used to determine a target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm;

[0087] The second acquisition module 403 is used to input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

[0088] In some embodiments, the determination module 402 may also be used to:

[0089] Determining a target prompt word associated with the first evaluation data according to the association relationship between the evaluation data and the prompt word; and / or,

[0090] According to the association relationship between the algorithm and the prompt word, a target prompt word associated with the first autonomous driving algorithm is determined.

[0091] In some embodiments, the determination module 402 may also be used to:

[0092] Determine a first prompt word associated with the first evaluation data and a second prompt word associated with the first autonomous driving algorithm;

[0093] When the first prompt word is different from the second prompt word, the first prompt word and the second prompt word are fused based on the confidences corresponding to the first prompt word and the second prompt word respectively to obtain the target prompt word.

[0094] In some embodiments, the first acquisition module 401 may also be used to:

[0095] Determining reference evaluation data associated with a type of the first autonomous driving algorithm;

[0096] Verifying the first evaluation data based on the reference evaluation data;

[0097] When the amount of first data included in the first evaluation data is smaller than the amount of second data included in the reference evaluation data, and / or when the type of first data included in the first evaluation data is different from the type of second data included in the reference evaluation data, first evaluation data missing warning information is output.

[0098] In some embodiments, the second acquisition module 403 may also be used to:

[0099] Determine the reference report associated with the target cue word;

[0100] Verify the first evaluation report based on the reference report;

[0101] In the case where the evaluation report does not include at least one item of information indicated in the reference report, updating the first evaluation report based on at least one item of information indicated in the reference report to obtain a second evaluation report, wherein a display mode of the item of information indicated in the second evaluation report is different from a display mode of the remaining information;

[0102] Display the second evaluation report.

[0103] In some embodiments, the second acquisition module 403 may also be used to:

[0104] When an update instruction for the second evaluation report is received, the updated third evaluation report and the first evaluation data are associated and stored in a preset database.

[0105] In some embodiments, the second acquisition module 403 may also be used to:

[0106] When the amount of data in the preset database is greater than a threshold, or when a preset model update cycle is reached, the large language model is updated using the data in the preset database to obtain an updated language model.

[0107] The functions and specific implementation principles of any of the above modules in this implementation can be referred to the above method embodiments of the present disclosure, and will not be repeated here.

[0108] In this embodiment, the algorithm evaluation system first obtains the first evaluation data output by the first autonomous driving algorithm, and then determines the target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm, and then inputs the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model. Thus, the target prompt word is first determined based on different evaluation data and / or algorithm types, and then the evaluation report is automatically generated based on the target prompt word using the large language model, which reduces the difficulty and cost of generating the evaluation report and improves the generation efficiency and report quality of the evaluation report.

[0109] In order to implement the above embodiments, the present disclosure also proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for generating an autonomous driving algorithm evaluation report as proposed in the above embodiments of the present disclosure.

[0110] In order to implement the above embodiments, the present disclosure also proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for generating an autonomous driving algorithm evaluation report proposed in the above embodiments of the present disclosure is implemented.

[0111] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method for generating an autonomous driving algorithm evaluation report proposed in the aforementioned embodiments of the present disclosure.

[0112] Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 5 The electronic device 500 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0113] like Figure 5 As shown, the electronic device 500 is in the form of a general purpose computing device. The components of the electronic device 500 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0114] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnection (PCI) bus.

[0115] The electronic device 500 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 500, including volatile and non-volatile media, removable and non-removable media.

[0116] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 500 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disk read only memory (hereinafter referred to as: CD-ROM), a digital versatile disk read only memory (hereinafter referred to as: DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0117] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0118] The electronic device 500 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 500, and / or communicate with any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the electronic device 500 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 500 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0119] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the above embodiments.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0122] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0123] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0124] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0125] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0126] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0127] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A method for generating an autonomous driving algorithm evaluation report, characterized in that: include: Obtaining first evaluation data output by a first autonomous driving algorithm; Determining a target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm; The target prompt word and the first evaluation data are input into a preset large language model to obtain a first evaluation report output by the model.

2. The method according to claim 1, characterized in that The determining of the target prompt word according to the evaluation data and / or the type of the first autonomous driving algorithm includes: Determining a target prompt word associated with the first evaluation data according to the association relationship between the evaluation data and the prompt word; and / or, According to the association relationship between the algorithm and the prompt word, determine the target prompt word associated with the first autonomous driving algorithm.

3. The method according to claim 1, characterized in that The determining of the target prompt word according to the evaluation data and / or the type of the first autonomous driving algorithm includes: Determining a first prompt word associated with the first evaluation data and a second prompt word associated with the first autonomous driving algorithm; When the first prompt word is different from the second prompt word, the first prompt word and the second prompt word are fused based on the confidences respectively corresponding to the first prompt word and the second prompt word to obtain the target prompt word.

4. The method according to claim 1, characterized in that After obtaining the first evaluation data output by the first autonomous driving algorithm, the method further includes: Determining reference evaluation data associated with the type of the first autonomous driving algorithm; Verifying the first evaluation data based on the reference evaluation data; When the amount of first data included in the first evaluation data is smaller than the amount of second data included in the reference evaluation data, and / or when the type of first data included in the first evaluation data is different from the type of second data included in the reference evaluation data, output first evaluation data missing warning information.

5. The method according to claim 1, characterized in that After obtaining the first evaluation report output by the model, the method further includes: Determining a reference report associated with the target cue word; Based on the reference report, verifying the first evaluation report; In a case where the evaluation report does not include at least one item of information indicated in the reference report, updating the first evaluation report based on the at least one item of information indicated in the reference report to obtain a second evaluation report, wherein a display mode of the item of information indicated in the second evaluation report is different from a display mode of other information; The second evaluation report is displayed.

6. The method according to claim 5, characterized in that After displaying the second evaluation report, the method further includes: When an update instruction for the second evaluation report is received, the updated third evaluation report and the first evaluation data are associated and stored in a preset database.

7. The method according to claim 6, characterized in that Also includes: When the amount of data in the preset database is greater than a threshold, or when a preset model update cycle is reached, the large language model is updated using the data in the preset database to obtain an updated language model.

8. A device for generating an autonomous driving algorithm evaluation report, characterized in that: include: A first acquisition module, used to acquire first evaluation data output by a first autonomous driving algorithm; a determination module, configured to determine a target prompt word according to the first evaluation data and / or the type of the first autonomous driving algorithm; The second acquisition module is used to input the target prompt word and the first evaluation data into a preset large language model to obtain a first evaluation report output by the model.

9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for generating an autonomous driving algorithm evaluation report as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the method for generating an autonomous driving algorithm evaluation report as described in any one of claims 1 to 7.

Images