Method, device and equipment for diagnosing automatic driving test problem and storage medium

Abstract

The application discloses a kind of automatic driving test problem diagnosis method, device, equipment and storage medium, method includes: from vector database inquires target document relevant to the description of the test problem of driving system, vector database includes the description document of different functional modules in driving system;According to test problem description and target document, using preset large language model carries out multi-round analysis and obtains diagnostic result.Through inquiring target document relevant to current problem, to obtain necessary effective information while reducing the text amount that large language model needs to process.Because large language model can automatically understand and analyze complex data through natural language processing capability, without relying on predefined rules, this enables large language model to flexibly and intelligently cope with various test problems, and large language model is analyzed through multi-round thinking, the next analysis depends on the output of last time, to obtain accurate diagnostic result, improves the comprehensive diagnosis of test problem.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and specifically to a diagnostic method, apparatus, device, and storage medium for autonomous driving testing problems. Background Technology

[0002] Currently, the testing and verification of autonomous driving technology mainly relies on two methods for problem diagnosis and handling: manual analysis and automated methods. Manual analysis typically involves experts manually reviewing system log data, sensor information, and test results to identify and classify problems. Automated methods, on the other hand, often employ rule-based approaches to set specific rules for problem identification or machine learning methods for problem diagnosis.

[0003] However, manual analysis struggles to handle large-scale data and cannot respond in real time. Rule-based methods are insufficient to fully cover the problem, while machine learning classification algorithms rely on high-quality data and have limited generalization capabilities. Summary of the Invention

[0004] The purpose of this application is to provide a diagnostic method, apparatus, device, and storage medium for autonomous driving testing problems, addressing the shortcomings of the prior art. This purpose is achieved through the following technical solutions.

[0005] The first aspect of this application proposes a diagnostic method for autonomous driving test problems, the method comprising:

[0006] Retrieve target documents related to the test problem description of the driving system from a vector database, which includes description documents of different functional modules in the driving system;

[0007] Based on the test problem description and the target document, a multi-round analysis is performed using a preset large language model to obtain diagnostic results.

[0008] A second aspect of this application provides a diagnostic apparatus for autonomous driving testing problems, the apparatus comprising:

[0009] The query module is used to query target documents related to the test problem description of the driving system from a vector database, which includes description documents of different functional modules in the driving system;

[0010] The analysis module is used to perform multiple rounds of analysis using a preset large language model based on the test problem description and the target document to obtain diagnostic results.

[0011] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method described in the first aspect above.

[0012] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0013] Based on the diagnostic methods, apparatus, equipment, and storage media for autonomous driving testing problems described above, this application has at least the following beneficial effects or advantages:

[0014] By querying target documents related to the current problem from a vector database based on the test problem description, necessary and effective information is obtained while reducing the amount of text that the large language model needs to process. Then, based on the test problem description and the retrieved target documents, a pre-defined large language model performs multiple rounds of analysis to obtain diagnostic results for the test problem. Because the large language model utilizes natural language processing capabilities, it can automatically understand and analyze complex data without relying on predefined rules. This allows it to flexibly and intelligently handle various test problems, especially unknown problems or complex faults. Furthermore, through multiple rounds of analysis, each subsequent analysis relies on the output of the previous one to obtain accurate diagnostic results, improving the comprehensiveness of test problem diagnosis. Using the large language model allows for real-time data analysis and timely feedback during the testing of driving systems, ensuring that problems are quickly identified and addressed, thereby improving the testing efficiency and reliability of driving systems.

[0015] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a flowchart illustrating an embodiment of a diagnostic method for autonomous driving testing problems, according to an exemplary embodiment.

[0018] Figure 2 This is a flowchart illustrating a multi-turn thinking process using a large language model, according to an exemplary embodiment.

[0019] Figure 3 This is a schematic diagram illustrating a diagnostic framework for an autonomous driving test problem according to an exemplary embodiment;

[0020] Figure 4This is a schematic diagram illustrating the structure of a diagnostic device for autonomous driving testing problems according to an exemplary embodiment;

[0021] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment;

[0022] Figure 6 This is a schematic diagram illustrating the structure of a storage medium according to an exemplary embodiment. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0024] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0025] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0026] Currently, both manual and automated methods used in testing autonomous driving systems have certain technical limitations. The limitations of manual analysis include: the sheer volume of data generated by autonomous driving systems makes it difficult for manual analysis to effectively handle large-scale, high-dimensional sensor and log data, easily overlooking potential problems; and manual analysis is typically conducted offline, failing to quickly identify issues during real-time testing. The limitations of rule-based automated methods include: they can only identify problems matching predefined rules and cannot handle unknown problems outside the rule scope. The limitations of machine learning-based classification algorithms used in automated methods include: the performance of machine learning methods is highly dependent on the quality and coverage of the training data; if the training data is insufficient, the machine learning model may fail to accurately identify problems in the test scenario; and when dealing with test scenarios with significantly different distributions from the training data, the machine learning model may experience performance degradation or failure.

[0027] Based on this, this application abandons the rule-based or machine learning methods used in manual analysis and automation methods, and proposes a diagnostic method for autonomous driving test problems. By using the natural language processing capabilities of the Large Language Model (LLM), it can automatically understand and analyze complex data to achieve accurate diagnosis of driving system test problems.

[0028] By querying target documents related to the current problem from a vector database based on the test problem description of the driving system, necessary and effective information is obtained while reducing the amount of text that the large language model needs to process. Then, based on the test problem description and the retrieved target documents, a pre-defined large language model is used for multiple rounds of analysis to obtain diagnostic results for the test problem. Because the large language model can automatically understand and analyze complex data without relying on predefined rules, it can flexibly and intelligently handle various test problems, especially unknown problems or complex faults. Furthermore, through multiple rounds of analysis, each subsequent analysis relies on the output of the previous one to obtain accurate diagnostic results, improving the comprehensiveness of test problem diagnosis. Using the large language model allows for real-time data analysis and timely feedback during the testing process of the driving system, ensuring that problems can be quickly identified and addressed, thereby improving the testing efficiency and reliability of the driving system.

[0029] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1The flowchart illustrates an embodiment of a diagnostic method for autonomous driving testing problems according to an exemplary embodiment, including the following steps 101 to 102:

[0031] Step 101: Query the target documents related to the test problem description of the driving system from the vector database. The vector database includes description documents of different functional modules in the driving system.

[0032] Step 102: Based on the test problem description and target document, perform multiple rounds of analysis using a preset large language model to obtain diagnostic results.

[0033] A vector database can be understood as a system that records text vectors. This vector database can be regarded as a data augmentation module for a large language model. Although the large language model has a certain knowledge reserve, there is a lot of non-public information for a specific driving system. Therefore, description documents of each functional module can be prepared for the driving system to be analyzed, and then stored in the vector database after processing, so that the large language model can understand the driving system more accurately.

[0034] The specific steps for establishing the vector database are as follows: Obtain the description documents for each functional module included in the driving system. If the description document for a single functional module exceeds a preset word count (e.g., 1000 words), it is divided into document blocks shorter than the preset word count. If the description document for a single functional module does not exceed the preset word count, it is treated as a single document block. Each document block is then converted into a vector and stored in the vector database. It is worth noting that the description documents for each functional module record various information, including functional information, dependency information, and output field names. Furthermore, the responsible personnel information for each functional module can also be stored in the vector database for easy subsequent querying.

[0035] The test problem description can be understood as the content of the problem work order generated by the driving system during the testing process. When querying relevant documents, this content needs to be converted into a vector representation. Therefore, the data representation form of the test problem description is a text vector.

[0036] The target document can be understood as a document block in the vector database that is related to the description of the test problem. Considering the high information density of the vector database and the limited amount of text that the large language model can process, it is necessary to achieve both retrieving the necessary effective information and maintaining a relatively small amount of text in the actual query. Therefore, the number of target documents retrieved can be one or more.

[0037] The diagnostic results are a detailed analysis of the causes of the test problems.

[0038] The pre-defined large language model can be understood as an artificial intelligence model based on deep learning technology, used to understand, generate, and process natural language, capable of generating text with relatively natural grammar and semantics. In multi-round analysis using the pre-defined large language model, each round of analysis relies on the output of the previous round; therefore, they need to be executed sequentially to obtain accurate diagnostic results.

[0039] This completes the above. Figure 1 The diagnostic process for autonomous driving test problems, as shown, involves querying target documents related to the current problem from a vector database based on the test problem description. This obtains necessary and effective information while reducing the amount of text that the large language model needs to process. Then, based on the test problem description and the retrieved target documents, a pre-defined large language model performs multiple rounds of analysis to obtain the diagnostic results. Because the large language model utilizes natural language processing capabilities, it can automatically understand and analyze complex data without relying on predefined rules. This allows it to flexibly and intelligently handle various test problems, especially unknown problems or complex faults. Furthermore, through multiple rounds of analysis, each subsequent analysis relies on the output of the previous one to obtain accurate diagnostic results, improving the comprehensiveness of test problem diagnosis. Using the large language model allows for real-time data analysis and timely feedback during the testing process of the driving system, ensuring that problems are quickly identified and addressed, thereby improving the testing efficiency and reliability of the driving system.

[0040] In some embodiments of this application, the step 101 above, querying the target document related to the test problem description of the driving system from the vector database, may include:

[0041] Obtain the target number by retrieving the target number of target documents related to the test question description from the vector database.

[0042] The target number refers to the number of documents that need to be retrieved from the vector database. It reflects the optimal number of documents containing sufficient valid information that the query needs to return in the current scenario. This target number is determined by the failure probability of the preset large language model. Specifically, initially, the target number uses a default value. After using the preset large language model for task analysis, the target number needs to be dynamically adjusted periodically based on the failure probability of the preset large language model.

[0043] The target document can be a document block in the vector database that is most similar to the test question description. Specifically, the similarity between the test question description and each document block is calculated (e.g., cosine similarity, Euclidean distance, etc.), and then the top target number of document blocks with the highest similarity are selected as the target document.

[0044] In this embodiment, by obtaining the current required target number, the target number of target documents related to the test problem description are queried from the vector database. Since the target number is determined by the failure probability of the preset large language model, the target number will be dynamically adjusted according to the failure probability of the preset large language model as the task analysis is continuously performed using the preset large language model, so as to control the number of returned documents and information density, and ensure that the returned documents can provide effective information without bringing too much redundancy or information noise.

[0045] It should be noted that the dynamic adjustment process of the target quantity may include:

[0046] Every preset period, the failure probability of task analysis using the preset large language model is statistically analyzed. If the failure probability exceeds the preset threshold, a first quantity and a second quantity are obtained based on the target quantity. The first quantity is less than the target quantity, and the second quantity is greater than the target quantity. Then, the task analysis of the preset large language model is tested using the first quantity and the second quantity respectively. The target quantity is adjusted by selecting the first quantity or the second quantity based on the test results.

[0047] Analysis failure probability can be understood as the ratio of the number of times the preset large language model failed to analyze the target number to the total number of analyses. This analysis failure probability may be affected by the number of documents. If the analysis failure probability increases, it is usually because the number of returned documents contains too much redundant information or does not contain enough effective information, making it difficult for the preset large language model to analyze successfully.

[0048] The first quantity refers to the quantity after the target quantity is reduced, and the second quantity refers to the quantity after the target quantity is increased.

[0049] The task analysis performance of the preset large language model was tested using both the first and second sets of data. Specifically, a certain number of question tasks were prepared. For each question task, a first set of documents similar to the question task were retrieved from the vector database. The preset large language model was then used to analyze both the question task and the retrieved documents, obtaining either successful or unsuccessful analysis results. The proportion of unsuccessful analysis results to the total number of question tasks was then calculated as the analysis failure probability after using the first set of data. The testing procedure using the second set of data was similar to that using the first set of data and will not be described further.

[0050] The test results represent the probability of analysis failure using the first quantity and the probability of analysis failure using the second quantity. When adjusting the target quantity based on the test results, the probability of analysis failure using the first quantity and the probability of analysis failure using the second quantity can be compared with the probability of analysis failure using the target quantity. If the probability of analysis failure using the first quantity is less than the probability of analysis failure using the target quantity, then the first quantity is selected to adjust the target quantity. If the probability of analysis failure using the second quantity is less than the probability of analysis failure using the target quantity, then the second quantity is selected to adjust the target quantity. Alternatively, the quantity with the greater reduction in the probability of analysis failure can be selected from the first quantity and the second quantity to adjust the target quantity.

[0051] In this embodiment, the failure probability of task analysis using a target quantity is statistically analyzed periodically. When the statistical failure probability is relatively high, a first quantity and a second quantity are obtained based on the target quantity in two directions (one increasing and one decreasing). Then, the task analysis of a preset large language model is tested using the first and second quantities respectively. Based on the test results, one of the quantities is selected to adjust the target quantity. By comparing the actual adjustment effect, the target quantity is adjusted to ensure that the documents returned by querying using the adjusted target quantity provide effective information without introducing excessive redundancy or information noise.

[0052] In some embodiments of this application, step 102 above, which involves performing multiple rounds of analysis using a preset large language model based on the test problem description and the target document to obtain diagnostic results, may include:

[0053] The first input text is generated based on the tool description, test problem description, and target document of the tool to be invoked. A preset large language model is used to output a first tool invocation description based on the first input text. The first tool invocation description is used to invoke the first tool to perform jump time point analysis. The jump time points returned by the first tool are appended to the first input text to obtain the second input text. The preset large language model is used to output a second tool invocation description based on the second input text. The second tool invocation description is used to invoke the second tool to perform field content queries. The field content returned by the second tool is appended to the second input text to obtain the third input text. The preset large language model is used to output a third tool invocation description based on the third input text. The third tool invocation description is used to invoke the third tool to perform structured processing of the diagnostic results. The processing results returned by the third tool are used as the diagnostic results.

[0054] The tool description can be understood as a description of the function and input parameters of the tool to be called. The preset large language model can generate a corresponding tool call description based on the tool description. This tool call description is parsed into a tool call and the tool is called.

[0055] In this embodiment, the tools to be invoked include a first tool, a second tool, and a third tool. The first tool is used to find the time point when the data changes, i.e., the change point. The second tool is used to query the detailed field content of a specified field at a specified time point. The third tool is used to output the final diagnostic result in a structured format.

[0056] Specifically, considering that the log data generated by the test driving system records the full system state and is extremely large, providing a large language model with log data from a long period would drastically reduce its understanding capabilities. The first tool identifies the points in time when data abrupt changes, reducing the amount of data that needs to be analyzed by the large language model. The second tool queries detailed information for specific fields at a specific time point, further reducing the amount of data required for analysis. The third tool structures the diagnostic results from the large language model analysis, facilitating downstream system recognition and processing.

[0057] The large language model continuously increases the amount of text data input during each round of analysis and consideration. Specifically, the first input text in the first round of analysis and consideration includes three items: the test question description, the target document, and the tool description. The second input text in the second round of analysis and consideration adds the jump time points to the first input text. The third input text in the third round of analysis and consideration adds field content to the second input text. This demonstrates that the large language model references different input information for each round of analysis and consideration, and that each round of analysis and consideration must be executed sequentially, with the next round depending on the output of the previous round.

[0058] In addition, the large language model outputs different tool call descriptions after each round of analysis and thinking. That is, the first round of analysis and thinking outputs the first tool call description, the second round of analysis and thinking outputs the second tool call description, and the third round of analysis and thinking outputs the third tool call description.

[0059] Each tool call description contains the input parameters that need to be entered into the tool and other information required to call the tool.

[0060] It is worth noting that the first, second, and third tools are pre-developed software programs. Among them, the first and second tools can access the log dataset generated by the driving system.

[0061] like Figure 2The flowchart shown illustrates the multi-round thinking process using a large language model. First, based on the test question description, relevant target documents are queried from a vector database. The retrieved target documents, the test question description, and the tool descriptions of the three tools are combined to form the first input text, which is then provided to the large language model. The large language model performs a first round of analysis and outputs a first tool call description. Using this description, the first tool is invoked to analyze jump time points. The jump time points returned by the first tool are appended to the first input text, resulting in the second input text, which is then provided to the large language model. The large language model performs a second round of analysis and outputs a second tool call description. Using this description, the second tool is invoked to query field content. The field content returned by the second tool is appended to the second input text, resulting in the third input text, which is then provided to the large language model. The large language model performs a third round of analysis and outputs a third tool call description. Using this description, the third tool is invoked to perform structured processing of the diagnostic results. The processing results returned by the third tool are used as the diagnostic results.

[0062] In this embodiment, a large language model is used for multi-round thinking and analysis. Since the input information referenced by the large language model in each round of analysis and thinking is different, the input of the large language model in the next round of analysis and thinking depends on the output of the large language model in the previous round of analysis and thinking and calls the corresponding tools to obtain further information. Thus, the large language model is a progressive analysis to obtain the final accurate diagnostic result.

[0063] In some embodiments of this application, the input parameters in the first tool invocation description include the time point when the test problem occurred and the name of a field to be queried, which is used to characterize the direct cause field of the problem. The time point can be provided by the test problem description, and the name of the field to be queried is obtained by the large language model based on the analysis and consideration of the first input text.

[0064] Based on this, the process of performing jump time point analysis using the first tool may include:

[0065] From the log dataset generated by the driving system, retrieve two log entries containing the specified field name that are less than the time of the problem. These two log entries are the closest to the time of the problem. Determine if the field values ​​of the specified field name in the two log entries in the target log group are equal. If they are equal, discard the log entry with the largest timestamp in the target log group. Query the log dataset for log entries containing the specified field name that are closest to and less than the timestamps of the remaining log entries in the target log group. Add the retrieved log entries to the target log group. Repeat the process of determining if the field values ​​of the specified field name in the two log entries in the target log group are equal. If they are not equal, use the smallest timestamp among the timestamps of the two log entries in the target log group as the jump time point.

[0066] In the log dataset, all log data is stored one by one in chronological order. The jump point can be understood as the point in time when the value of the field name to be queried changes.

[0067] The target log group can be implemented as a queue. The specific implementation process is as follows: First, find the two log data that contain the field name and are less than or closest to the problem time and put them into the queue. Then, check whether the field value of the field name in the two log data in the queue is equal. If they are equal, it means that the data has not changed. Discard the log data with the later time from the queue and read a new log data that is closest to the problem time and put it into the queue to continue the comparison. Continue until the field value of the field name in the two log data in the queue is not equal, which means that the data has changed. Output the timestamp of the log data with the earlier time in the queue.

[0068] In this embodiment, the first tool queries the log data in the log dataset backward from the problem time point where the field value of the input field name changes, and then uses the timestamp of the queried log data as the change time point.

[0069] In some embodiments of this application, the input parameters in the second tool invocation description include a jump time point and multiple field names to be queried, which are used to characterize all possible fields that caused the problem. The jump time point is derived from the output of the first tool, and the multiple field names to be queried are obtained by the large language model during its analysis and consideration of the second input text.

[0070] Based on this, the process of using a second tool to query field content can include:

[0071] In the log dataset generated by the driving system, starting from the timestamp corresponding to the jump time point, the field value of each field name is searched forward in each log data. The log data to which the field value of each field name belongs is the log data closest to the jump time point. Then, each field name and its field value are used as the field content.

[0072] In other words, for multiple field names to be queried, but each field name appears at a different frequency in the log data, meaning that the log data corresponding to the jump time point may not contain the field values ​​of all the field names to be queried, the system will start from the jump time point and search forward one by one in the log data. When it encounters any of the multiple field names, it will first cache the field value of that field name and determine whether all field names have been found. If not all field names are found, it will continue to search forward until all field names are found.

[0073] Therefore, it can be seen that the log data to which the field values ​​of each field name in the final query belong are all log data that are less than the jump time point.

[0074] In this embodiment, the second tool provides the value of a specified field name in the log dataset at a specified time point as input to the large language model, instead of providing the entire log data containing the specified field name to the large language model. This allows the large language model to analyze the provided valid and necessary information and obtain accurate diagnostic results.

[0075] In some embodiments of this application, the process of using a preset large language model to output a third tool call description based on a third input text may include: generating analysis results by using a preset large language model based on the field content, jump time point, target document, and test question description contained in the third input text; querying the person in charge information of the functional module to which the target field name contained in the analysis results belongs from the vector database; and generating a third tool call description based on the person in charge information, the analysis results, and the tool description contained in the third input text.

[0076] The target field name is used to identify the exact field that caused the problem. Since the description documents of each functional module stored in the vector database record the field names that will be output during the operation of the functional module, the corresponding person in charge information can be retrieved from the vector database by using the target field name.

[0077] In this embodiment, since the large language model can obtain the dependencies of different fields through the target document, and the large language model can obtain the field content of all possible fields that may cause the problem, the large language model can analyze which field the problem occurs in, that is, the target field name, and further query the vector database to obtain the corresponding module person in charge information, and obtain the diagnosis result that the problem needs to be assigned to a certain person.

[0078] Based on the description of the above embodiments, Figure 3 This is a schematic diagram illustrating a diagnostic framework for autonomous driving testing problems according to an exemplary embodiment. External inputs include: description documents of each functional module in the driving system and work orders generated by the test driving system. These work orders describe the performance of the test problems. The description documents are vectorized and stored in a vector database. Three tools to be invoked are prepared, and tool descriptions for these three tools are added.

[0079] The specific diagnostic process for the input work order is as follows: First, based on the test problem description of the work order, relevant target documents are queried from the vector database. The queried target documents, test problem description, and tool descriptions of the three tools are combined to form the first input text, which is provided to the large language model. The large language model performs a first round of analysis and outputs a first tool call description. Using the first tool call description, the first tool is invoked to perform jump time point analysis. The jump time points returned by the first tool are appended to the first input text to obtain the second input text, which is then provided to the large language model. The large language model performs a second round of analysis and outputs a second tool call description. Using the second tool call description, the second tool is invoked to query field content. The field content returned by the second tool is appended to the second input text to obtain the third input text, which is then provided to the large language model. The large language model performs a third round of analysis and outputs a third tool call description. Using the third tool call description, the third tool is invoked to perform structured processing of the diagnostic results. The processing results returned by the third tool are used as the final diagnostic results.

[0080] Corresponding to the embodiments of the aforementioned diagnostic methods for autonomous driving test problems, this application also provides embodiments of diagnostic devices for autonomous driving test problems.

[0081] Figure 4 This is a schematic diagram illustrating the structure of a diagnostic apparatus for autonomous driving test problems according to an exemplary embodiment. The apparatus is used to execute the diagnostic method for autonomous driving test problems provided in any of the above embodiments, such as... Figure 4 As shown, the diagnostic device for this autonomous driving test problem includes:

[0082] The query module 410 is used to query target documents related to the test problem description of the driving system from the vector database, the vector database including description documents of different functional modules in the driving system;

[0083] The analysis module 420 is used to perform multiple rounds of analysis using a preset large language model based on the test problem description and the target document to obtain diagnostic results.

[0084] In an optional implementation, the query module 410 is specifically used to obtain a target number, which is determined by the analysis failure probability of the preset large language model; and to obtain the target number of target documents related to the test question description from the vector database.

[0085] In an alternative implementation, the apparatus further includes ( Figure 4 (Not shown in the image):

[0086] The quantity adjustment module is used to calculate the failure probability of task analysis using the target quantity at preset intervals; when the failure probability exceeds a preset threshold, a first quantity and a second quantity are obtained based on the target quantity; the first quantity is less than the target quantity, and the second quantity is greater than the target quantity; the task analysis of the preset large language model is tested using the first quantity and the second quantity respectively; and the target quantity is adjusted by selecting the first quantity or the second quantity based on the test results.

[0087] In an optional implementation, the analysis module 420 is specifically configured to: generate a first input text based on the tool description of the tool to be invoked, the test problem description, and the target document; output a first tool invocation description based on the first input text using the preset large language model; invoke the first tool using the first tool invocation description to perform jump time point analysis, append the jump time points returned by the first tool to the first input text to obtain a second input text; output a second tool invocation description based on the second input text using the preset large language model; invoke the second tool using the second tool invocation description to perform field content query, append the field content returned by the second tool to the second input text to obtain a third input text; output a third tool invocation description based on the third input text using the preset large language model; invoke the third tool using the third tool invocation description to perform diagnostic result structure processing, and use the processing result returned by the third tool as the diagnostic result.

[0088] In an optional implementation, the input parameters in the first tool invocation description include the time point when the test problem occurred and a field name to be queried, wherein the field name is used to characterize the direct cause field of the problem; the device further includes ( Figure 4 (Not shown in the image):

[0089] The first tool execution module is used to query two log data sets containing the field name that are less than the time point of the problem from the log dataset generated by the driving system, and to select them as the target log group; the two log data sets are the log data sets closest to the time point of the problem; to determine whether the field values ​​of the field name contained in the two log data sets in the target log group are equal; if they are equal, the log data corresponding to the largest timestamp in the target log group is discarded, and log data containing the field name that is closest to and less than the timestamp of the remaining log data in the target log group is queried from the log dataset, and the queried log data is added to the target log group, and the step of determining whether the field values ​​of the field name contained in the two log data sets in the target log group are equal is repeated; if they are not equal, the smallest timestamp among the timestamps corresponding to the two log data sets in the target log group is selected as the jump time point.

[0090] In one optional implementation, the input parameters in the second tool invocation description include the jump time point and multiple field names to be queried; the multiple field names are used to characterize all possible fields causing the problem; the device further includes ( Figure 4 (Not shown in the image):

[0091] The second tool execution module is used to search forward through the log dataset generated by the driving system, starting from the timestamp corresponding to the jump time point, for the field value of each field name in each log data; the log data to which the field value of each field name belongs is the log data closest to the jump time point; and each field name and its field value are used as the field content.

[0092] In an optional implementation, the analysis module 420 is specifically used to generate analysis results during the process of outputting a third tool call description based on the third input text using the preset large language model. This analysis results are generated based on the field content, jump time point, target document, and test problem description contained in the third input text. The module also queries the vector database for the person in charge information of the functional module to which the target field name contained in the analysis results belongs. Based on the person in charge information, the analysis results, and the tool description contained in the third input text, the module generates the third tool call description. The target field name is used to characterize the exact field that caused the problem.

[0093] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0094] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0095] This application also provides an electronic device corresponding to the diagnostic method for autonomous driving test problems provided in the foregoing embodiments, for executing the aforementioned diagnostic method for autonomous driving test problems.

[0096] Figure 5 The present invention illustrates a hardware structure diagram of an electronic device according to an exemplary embodiment. The electronic device includes a communication interface 601, a processor 602, a memory 603, and a bus 604. The communication interface 601, processor 602, and memory 603 communicate with each other via the bus 604. The processor 602 can execute the diagnostic method for the autonomous driving test problem described above by reading and executing machine-executable instructions corresponding to the control logic of the diagnostic method for the autonomous driving test problem stored in the memory 603. The specific content of this method is described in the above embodiment and will not be repeated here.

[0097] The memory 603 mentioned in this application can be any electronic, magnetic, optical, or other physical storage device, and can contain stored information such as executable instructions, data, etc. Specifically, the memory 603 can be RAM (Random Access Memory), flash memory, storage drive (such as hard disk drive), any type of storage disk (such as optical disc, DVD, etc.), or similar storage media, or combinations thereof. Communication between this system network element and at least one other network element is achieved through at least one communication interface 601 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc., can be used.

[0098] Bus 604 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 603 is used to store programs, and the processor 602 executes the programs after receiving execution instructions.

[0099] Processor 602 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 602 or by instructions in software form. The processor 602 can be a general-purpose processor, including a network processor (NP), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor.

[0100] The electronic device provided in this application embodiment and the diagnostic method for autonomous driving test problems provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0101] This application also provides a computer-readable storage medium corresponding to the diagnostic method for autonomous driving test problems provided in the foregoing embodiments. Please refer to... Figure 6 As shown, the computer-readable storage medium is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the diagnostic method for autonomous driving test problems provided in any of the aforementioned embodiments.

[0102] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0103] The computer-readable storage medium provided in the above embodiments of this application and the diagnostic method for autonomous driving test problems provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0104] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0105] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0106] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A diagnostic method for problems in autonomous driving testing, characterized in that, The method includes: Retrieve target documents related to the test problem description of the driving system from a vector database, which includes description documents of different functional modules in the driving system; Based on the test problem description and the target document, a multi-round analysis is performed using a preset large language model to obtain diagnostic results.

2. The method according to claim 1, characterized in that, The querying of target documents related to the test problem description of the driving system from the vector database includes: Obtain the target number, which is determined by the failure probability of the analysis of the preset large language model; Retrieve the target number of target documents related to the test problem description from the vector database.

3. The method according to claim 2, characterized in that, The method further includes: Every preset period, the probability of analysis failure when using the target quantity for task analysis is statistically analyzed. If the probability of analysis failure exceeds a preset threshold, a first quantity and a second quantity are obtained based on the target quantity; the first quantity is less than the target quantity, and the second quantity is greater than the target quantity. The task analysis performance of the preset large language model was tested using the first quantity and the second quantity, respectively. The target quantity is adjusted by selecting either the first or second quantity based on the test results.

4. The method according to claim 1, characterized in that, The step involves performing multiple rounds of analysis using a pre-defined large language model based on the test problem description and the target document to obtain diagnostic results, including: The first input text is generated based on the tool description of the tool to be invoked, the test problem description, and the target document; The preset large language model is used to output a first tool call description based on the first input text; The first tool is used to call the description to perform jump time point analysis, and the jump time points returned by the first tool are appended to the first input text to obtain the second input text; The preset large language model is used to output a second tool call description based on the second input text; The second tool is used to call the description to query the field content, and the field content returned by the second tool is appended to the second input text to obtain the third input text; The preset large language model is used to output a third tool call description based on the third input text; The third tool is invoked to perform structured processing of the diagnostic results, and the processing result returned by the third tool is used as the diagnostic result.

5. The method according to claim 4, characterized in that, The input parameters in the first tool call description include the time point when the test problem occurred and the name of a field to be queried, wherein the field name is used to characterize the direct cause field of the problem. The process of the first tool performing jump time point analysis includes: From the log dataset generated by the driving system, two log entries containing the field name that are less than the time of the problem are retrieved as the target log group; the two log entries are the log entries closest to the time of the problem. Determine whether the field values ​​of the field names contained in two log data entries in the target log group are equal; If the values ​​are equal, discard the log data corresponding to the largest timestamp in the target log group, query the log dataset for log data that contains the field name and whose timestamp is closest to and smaller than the timestamp of the remaining log data in the target log group, add the queried log data to the target log group, and repeat the step of judging whether the field values ​​of the field name contained in two log data in the target log group are equal. If the two log data are not equal, the minimum timestamp among the timestamps corresponding to the two log data in the target log group is taken as the jump time point.

6. The method according to claim 4, characterized in that, The input parameters in the second tool call description include the jump time point and the names of multiple fields to be queried; the multiple field names are used to characterize all possible fields that caused the problem; The process of querying field content using the second tool includes: In the log dataset generated by the driving system, starting from the timestamp corresponding to the jump time point, the field value of each field name is searched forward for each log data item; The log data to which the field value of each field name belongs is the log data most recent to the jump time point; The field content is defined as the names of each field and the values ​​of each field name.

7. The method according to claim 4, characterized in that, The step of using the preset large language model to output a third tool call description based on the third input text includes: The preset large language model generates analysis results based on the field content, jump time points, target documents, and test question descriptions contained in the third input text. It also queries the vector database for the person in charge of the functional module to which the target field name in the analysis results belongs, and generates the third tool call description based on the person in charge information, the analysis results, and the tool descriptions contained in the third input text. The target field name is used to characterize the exact field that caused the problem.

8. A diagnostic device for autonomous driving test problems, characterized in that, The device includes: The query module is used to query target documents related to the test problem description of the driving system from a vector database, which includes description documents of different functional modules in the driving system; The analysis module is used to perform multiple rounds of analysis using a preset large language model based on the test problem description and the target document to obtain diagnostic results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-7.

Images