Data processing method, big language model training method, related equipment and storage medium

Abstract

The invention discloses a data processing method, a big language model training method, related equipment and a storage medium, and relates to the technical field of data processing.The data processing method comprises the steps that a simulation log generated by testing a test object is obtained; error reporting logs are screened out from the simulation logs; obtaining a first problem and first prompt information generated for the error report log; obtaining first valid data related to the first problem in the error report log; inputting the first effective data, the first question and the first prompt information into a first large language model to obtain output data of the first large language model; wherein the output data is data which is output according to the prompt of the first prompt information and is used for replying the first problem, and the output data at least comprises error information and / or error reasons of the error report log. According to the invention, automatic and intelligent analysis of error problems and / or error reasons in a chip verification scene can be realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device, a large language model training method and device, an electronic device, and a storage medium. Background Art

[0002] With the development of hardware technology, chip simulation and verification have posed certain challenges. In related technologies, chip simulation and verification can be used to determine whether there are problems with the chip design, ensuring that the chip design and production processes are error-free, thereby achieving efficient chip production. Currently, chip simulation and verification still require a large amount of human participation, resulting in low efficiency. Therefore, how to improve the efficiency of chip simulation and verification has become a technical problem that needs to be solved urgently. Summary of the Invention

[0003] The present application provides a data processing method and device, a large language model training method and device, an electronic device and a storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present application, a data processing method is provided, the method comprising: Obtaining simulation logs generated by testing the test object; Filter out error logs from simulation logs; Obtaining a first question and a first prompt message generated in response to the error log; Obtaining first valid data related to the first problem in the error log; The first valid data, the first question and the first prompt information are input into a first large language model to obtain output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information for answering the first question, and the output data includes at least the error information and / or error cause of the error log.

[0005] In one possible implementation, the simulation log is unstructured data; The step of filtering out error logs from simulation logs includes: Preprocess the unstructured simulation log to obtain a structured simulation log; Based on the structural information of the structured simulation log, the error log is filtered out from the simulation log.

[0006] In one possible implementation manner, the first large language model is obtained by training the first large language model to be trained using historical error logs generated by historical tests of the test object and reference data of the historical error logs.

[0007] In one embodiment, the method includes: Obtaining first vector information based on error information, error type, and a test function of a test object corresponding to the error information in a historical error log; Obtaining second vector information based on key information of reference data in historical error logs; The first large language model to be trained is trained using the first vector information and the second vector information to obtain a first large language model.

[0008] In one possible implementation, the training of the first large language model to be trained using the first vector information and the second vector information includes: Obtaining a first historical problem and a first historical prompt message generated in a historical error log of the test object; Processing the first historical question to obtain a first historical question vector; Obtaining knowledge information related to the first historical question vector from the first vector information and the second vector information; The knowledge information related to the first historical question vector, the first historical question vector and the first historical prompt information are input into a first large language model to be trained to train the first large language model to be trained.

[0009] In one embodiment, the method further comprises: obtaining verification data generated by testing the test object; Obtaining a second question and second prompt information generated for the verification data; Obtaining second valid data related to the second question from the verification data; The second valid data, the second question and the second prompt information are input into the second largest language model to obtain output data of the second largest language model; wherein, the output data is data output according to the prompt of the second prompt information for answering the second question, and the output data at least includes a verification report generated based on the verification data.

[0010] In one possible implementation manner, the second large language model is obtained by training the second large language model to be trained using historical verification data generated by historical tests of the test subject and reference data of the historical verification data.

[0011] In one embodiment, the method further comprises: Format historical verification data; Obtaining third vector information based on the formatted historical verification data; Based on the key information of the reference data of the historical verification data, the fourth vector information is obtained; The second large language model to be trained is trained using the third vector information and the fourth vector information to obtain a second large language model.

[0012] In one possible implementation manner, the training of the second large language model to be trained using the third vector information and the fourth vector information includes: Obtaining a second historical question and second historical prompt information generated based on the historical verification data of the test subject; Processing the second historical question to obtain a second historical question vector; Obtaining knowledge information related to the second historical question vector from the third vector information and the fourth vector information; The knowledge information related to the second historical question vector, the second historical question vector and the second historical prompt information are input into the second large language model to be trained to train the second large language model to be trained.

[0013] According to a second aspect of the present application, a method for training a large language model is provided, comprising: Obtain historical error logs generated by historical tests on the test object and reference data of the historical error logs; Based on historical error logs and reference data of historical error logs, a first large language model to be trained is trained to obtain a first large language model; wherein, the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

[0014] In one possible implementation manner, obtaining historical error logs generated by historical tests on the test object includes: Obtaining historical simulation logs generated by historical testing of the test object; In response to the historical simulation log being unstructured data, preprocessing the historical simulation log to obtain a structured historical simulation log; Based on the structural information of the structured historical simulation log, the historical error log is filtered out from the historical simulation log.

[0015] According to a third aspect of the present application, there is provided a data processing device, comprising: A first obtaining unit is used to obtain a simulation log generated by testing the test object; A screening unit, used for screening out error logs from simulation logs; A second obtaining unit is configured to obtain a first question and a first prompt message generated in response to the error log; The third obtaining unit is used to obtain first valid data related to the first question in the error log; the fourth obtaining unit is used to input the first valid data, the first question and the first prompt information into the first large language model to obtain output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information for answering the first question, and the output data at least includes the error information and / or error cause of the error log.

[0016] According to a fourth aspect of the present application, a large language model training device is provided, comprising: A data acquisition unit, configured to acquire historical error logs generated by historical tests on a test object and reference data of the historical error logs; A model training unit is used to train a first large language model to be trained based on historical error logs and reference data of historical error logs to obtain a first large language model; wherein the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

[0017] According to a fifth aspect of the present application, an electronic device is provided, including: at least one processor; a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0018] According to a sixth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0019] According to a seventh aspect of the present application, a computer program product is provided, comprising a computer program or instructions, which implement the method described in the present application when executed by a processor.

[0020] The technical solution of this application can use a large language model to achieve automatic and intelligent analysis of errors and / or error causes in chip verification scenarios, avoiding excessive manual intervention and improving the efficiency of chip simulation verification. In addition, the large language model can also be used to automatically generate verification reports in chip verification scenarios, avoiding excessive manual intervention and improving the automation, intelligence, and efficiency of verification report generation in chip verification scenarios.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which: In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0023] Figure 1 The implementation diagram of the data processing method in the embodiment of the present application is shown Figure 1 ; Figure 2 The implementation diagram of the data processing method in the embodiment of the present application is shown Figure 2 ; Figure 3 The following diagram shows the implementation of the training method of the first language model in the embodiment of the present application. Figure 1 ; Figure 4 The following diagram shows the implementation of the training method of the first language model in the embodiment of the present application. Figure 2 ; Figure 5 The implementation diagram of the data processing method in the embodiment of the present application is shown Figure 3 ; Figure 6 The implementation diagram of the data processing method in the embodiment of the present application is shown Figure 4 ; Figure 7 The following diagram shows the implementation of the training method of the second language model in the embodiment of the present application. Figure 1 ; Figure 8 The following diagram shows the implementation of the training method of the second language model in the embodiment of the present application. Figure 2 ; Figure 9 A schematic diagram of the structure of a data processing device in an embodiment of the present application is shown; Figure 10 A schematic diagram of the structure of a large language model training device according to an embodiment of the present application is shown; Figure 11 A schematic diagram of the structure of an electronic device in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0025] In the following description, the terms "first" and "second" are used solely to distinguish similar objects and do not represent a specific ordering of the objects. In the various embodiments of this application, the order of the sequence numbers of the implementation processes does not imply a specific order of execution. The execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0026] The data processing method of this application can be applied to servers or terminal devices. The server can be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be, but is not limited to, a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc. Preferably, the data processing method of this application is applied to a server. The technical solution of this application is described in detail below.

[0027] Figure 1 The implementation diagram of the data processing method in the embodiment of the present application is shown Figure 1 .like Figure 1 As shown, the method includes: S (Step) 101: Obtain a simulation log generated by testing the test object.

[0028] In this application, the test object can be an object that needs to be tested in the design, verification, and production links. For example, in the chip verification scenario, in a broad sense, the test object can be a chip. In a narrow sense, the test object can refer to chips of the same type, such as chips of the same model or the same chip. For designed chips, various test cases can be used to test the various functions of the test object. For example, test case 1 is used to test function 1 of chip A, and test case 2 is used to test function 2 of chip A. Each test will generate a corresponding simulation log. The simulation log can be obtained by monitoring the simulation log generated by testing the various functions of the chip.

[0029] S102: Filter out error logs from the simulation logs.

[0030] In this application, the simulation logs generated by testing the test object include logs that indicate test errors (error logs) and logs that indicate successful tests (correct logs). For the logs with test errors, it is necessary to determine the error problem and / or cause based on the logs to avoid chip function errors.

[0031] In actual applications, chips have diverse functions, requiring a large number of functions to be verified, and employing a large number of test cases. Consequently, a large number of simulation logs are generated when testing the chip. In some embodiments, the generated simulation logs may be structured data, which may include identifiers indicating which logs are correct and which are error logs. This identifier can be used to identify error logs from a large number of simulation logs, thereby enabling the screening of error logs.

[0032] In some embodiments, the generated simulation log may be unstructured data. For the simulation log of the generated unstructured data, in the present application, a regular expression may be used to pre-process the unstructured simulation log to obtain a structured simulation log. For the simulation log, in addition to the log content itself, two parts for the log may be obtained after pre-processing. The two parts include a constant part and a variable part. Among them, the constant part includes the error type of the error log, the simulation stage (such as whether the error occurs in the data sending stage or the data processing stage), the error flag, the number of errors, etc. The variable part includes the specific error information and the chip function and the module corresponding to the error information. Furthermore, it may also include the context content related to the error information in the log.

[0033] It is understandable that chips usually have many functions. During testing, the functions of the chip can be divided into different modules. With the modules as the test units, each function in each module can be tested one by one to avoid test confusion and incomplete testing caused by too many functions to be tested.

[0034] The aforementioned constant part and variable part can be used as the structural information of the structured simulation log. For the simulation log, if the aforementioned two parts can be extracted or obtained through the preprocessing of the regular expression, it means that the simulation log is an error log. If the aforementioned two parts cannot be extracted or obtained, it means that the simulation log is a correct log. In this way, based on the structural information of the structured simulation log, the error log can be filtered out from the simulation log, achieving efficient screening of the error log.

[0035] Moreover, due to the accuracy of regular expression preprocessing, the use of regular expressions can ensure accurate screening of error logs, providing a guarantee for the large language model (LLM) to accurately locate the error information and / or error causes in the error logs.

[0036] S103: Obtain a first question and a first prompt message generated in response to the error log.

[0037] In this step, questions generated by the user for the error log that require large language model reasoning are received. The first question refers to the question that the user inputs to the large language model that requires large language model reasoning. For example, for a simulation log, it includes log content for functional verification of the chip from various aspects. For example, it includes log content for verifying whether the handshake timing of each signal of the chip is normal, log content for verifying whether the register read and write function of the chip is correct, and log content for verifying whether the logical function of the chip such as A+B=C is normal. For this simulation log, the user will generate at least one of the following questions, such as "analyzing the abnormality of the signal handshake timing", "analyzing the abnormality of the register read and write function verification", "analyzing the abnormality of the verification of the logical function A+B=C", etc., and use this question as the first question.

[0038] In actual applications, for the same simulation log, in a solution that requires a large language model for reasoning, the question generated by the user can be one question or multiple questions, without specific limitation. The first prompt information is the Prompt in the chip verification scenario. The Prompt can be a pre-built prompt information for the large language model based on the terminology and corpus in the chip verification scenario. For example, the Prompt can serve as a guide for the large language model to reason in which directions based on which existing standards or data are combined with the first question generated by the user. In addition, the Prompt can also guide the large language model in how to output the reasoning results, such as in what text format to output the reasoning results.

[0039] S104: Obtain first valid data related to the first problem in the error log.

[0040] Considering that error logs may verify chip functionality from multiple aspects, the first issue may be specific to a particular aspect. For example, issues may arise from verifying only timing signals or register read / write functionality. From the error logs, extract the log content related to the issue the user is attempting to infer as the (first) valid data.

[0041] For example, when the first question is "analyzing the abnormality of the verification of the logic function A+B=C", the log content related to the read and write verification of the register is extracted from the error log as valid data. When the first question is "analyzing the abnormality of the verification of the read and write function of the register", the log content related to the verification of the logic function A+B=C is extracted from the error log as valid data. Figure 2 As shown, a retrieval-augmented generation framework (RAG) can be used to extract log content related to the first problem from the error log.

[0042] In this application, RAG extracts log content related to the first problem from the error log, creating the context for the problem requiring reasoning using a large language model. This provides enhanced prompts for reasoning using the large language model. This enhanced prompt ensures accurate reasoning using the large language model.

[0043] S105: Input the first valid data, the first question and the first prompt information into the first large language model to obtain output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information for answering the first question, and the output data at least includes the error information and / or error cause of the error log.

[0044] In this step, the log content related to the first problem, the first problem, and the first prompt information are extracted from the error log and input into the (first) large language model. The large language model performs model inference based on the above three input information, obtains the inference result, and outputs it.

[0045] For example, if the first question is "analyzing an anomaly in register read / write function verification," the large language model uses the log content related to register read / write function verification in the error log to perform model reasoning on the register read / write function anomaly, locating the register read / write error information and the cause of the error. For example, in the logic verification of A+B=C, the logic function is reasoned to determine whether an error occurred in the values of A and B, or in the logical operation between A and B, such as addition, which caused the error in the logic verification of A+B=C.

[0046] The scheme shown in S101-S105 obtains a first question and a first prompt message generated in the error log generated by testing the test object, obtains first valid data related to the first question in the error log, inputs the first valid data, the first question, and the first prompt message into a first large language model, and obtains error information and / or error causes inferred by the first large language model. The technical solution of this application utilizes a large language model to achieve automatic and intelligent analysis of error problems and / or error causes in chip verification scenarios, avoiding excessive manual intervention and improving the efficiency of chip simulation verification.

[0047] This application's technical solution provides technical support for intelligent analysis of chip simulation verification. Furthermore, in this application, the log content related to the first issue extracted from the error log is equivalent to constructing an inference context, providing enhanced prompts for the inference of the (first) large language model and ensuring the accurate inference of the (first) large language model.

[0048] The first large language model is a trained large language model. The solution shown in S101-S105 utilizes the trained large language model to intelligently analyze simulation logs in a chip verification scenario. This application also provides a solution for obtaining a trained first large language model by training the first large language model to be trained.

[0049] Figure 3 The following diagram shows the implementation of the training method of the first language model in the embodiment of the present application. Figure 1 .like Figure 3 As shown, the training method includes: S301: Obtain historical error logs generated by historical tests on a test object and reference data of the historical error logs.

[0050] In this step, "history" refers to the time on the timeline prior to the test chip test in S101. Historical testing refers to historical testing of the test object. Historical error logs refer to error logs generated by historical testing of the test object. Historical error logs are generated by monitoring historical testing of the test object.

[0051] The reference data of the historical error log can be used to indicate the standards or specifications for testing various functions of the chip. For example, for testing whether the register reading and writing are normal, the reference data can be used to indicate that when the register is written with 1, it can read 1, indicating that the register reading and writing are normal, otherwise it is considered abnormal. For testing the logic function of the chip, the reference data can be used to indicate that the numerical value A and the numerical value B and the numerical value C should be the operation logic of A+B=C. If this logic is met, the logic function is normal, otherwise it is considered that the logic function is abnormal. Taking into account that technical documents, standard documents, and expert reports can be used to record standards or specifications, the reference data of the historical error log can be pre-recorded in the technical documents, standard documents, and expert reports. The reference data of the historical error log is obtained by reading the data recorded in the technical documents, standard documents, and expert reports.

[0052] S302: Based on historical error logs and reference data of historical error logs, a first large language model to be trained is trained to obtain a first large language model; wherein the first large language model is used to infer and output error information and / or error causes of the error logs generated by the test object.

[0053] In this step, it can be considered that the first large language model is obtained by training the first large language model to be trained using historical error logs generated by historical tests of the test object and reference data of the historical error logs.

[0054] In solutions S301-S302, the (first) large language model to be trained is trained based on historical error logs and reference data generated in chip verification scenarios. This results in a large language model capable of inferring error messages and / or error causes from error logs. This solution provides technical support for training large language models capable of automatically locating error messages and causes in chip verification scenarios.

[0055] In some embodiments, the scheme for obtaining historical error logs generated by historical tests on the test object in the aforementioned S301 includes: obtaining historical simulation logs generated by historical tests on the test object; in response to the historical simulation logs being unstructured data, preprocessing the historical simulation logs to obtain structured historical simulation logs; and filtering out historical error logs from the historical simulation logs based on the structural information of the structured historical simulation logs.

[0056] Historical simulation logs refer to simulation logs generated by historical tests on test objects. Historical simulation logs can be structured data or unstructured data. If it is structured data, historical error logs can be identified from historical simulation logs by identifying whether the log is an error log or a correct log. If it is unstructured data, regular expressions can be used to preprocess the unstructured simulation logs to obtain structured simulation logs. If the constants and variables of the historical simulation logs can be extracted or obtained through preprocessing with regular expressions, it means that the historical simulation log is a historical error log. For details about this, please refer to the relevant description above. The use of regular expressions can ensure the accurate screening of historical error logs, which provides a guarantee for the accurate training of the (first) large language model.

[0057] It can be understood that the historical error logs filtered out based on the structural information of the historical simulation logs are a type of structural data, and each error message appearing in the historical error logs can be recorded in the (first) relational database. Figure 4 As shown, the data can be recorded in the (first) relational database in the form of a triple of "Entity-Relationship-Entity". An example triple may be "uvm_error" (error type) - "DEBUG_AXI" (chip bus protocol AXI function) - "The handshake from the ready and valid signals is incorrect".

[0058] Among them, "The handshake from the ready and valid signals is incorrect" can be used as an error message. "DEBUG_AXI" can be used as a test function of the test object corresponding to the error message. During implementation, the first vector information can be obtained based on the error message, error type and test function of the test object corresponding to the error message in the historical error log. For example, the word vector model in RAG can be used to vectorize the triples in the (first) relational database to obtain vector representation data of the error message. The vector representation data of the error message can be recorded in the (first) verification database. Each vector data in the (first) verification database corresponds to a vector expression between an error message, the test function of the test object corresponding to the error message and the error type. Each vector data in the (first) verification database can be regarded as the first vector information, which is a vector expression of a triple.

[0059] Unlike the relational data recorded in the (first) relational database, this application also provides a (first) non-relational database for recording reference data from historical error logs. Since technical documents and standard documents may have different formats, they can be considered unstructured data. During implementation, a Python file processing library can be called to extract text content from documents in various formats and convert the extracted text into a unified format. Considering that technical documents and standard documents, in addition to recording reference data, also record other data, namely, information unrelated to the reference data, such as instructions for using the technical document. During implementation, irrelevant information is removed from the text content, and the text content is hashed to remove duplicates. The remaining text content is then normalized. Information is extracted from the normalized text content, such as text related to register reads and writes, text related to the chip's A+B=C logic function, and text related to AXI handshake timing. A dictionary structure is created from the extracted text content based on information such as title, content, and keywords, and stored in the (first) non-relational database. The dictionary structure stores reference data in the form of title, content, keywords, etc. For example, the title is "AXI", the content is "timing content of AXI handshake signal", and the keyword is "handshake".

[0060] In this application, information such as title, content, and keywords can be used as key information for reference data. Figure 4 A solution for preprocessing technical documents, standard documents, etc. is described in [1]. During implementation, second vector information is obtained based on the key information of the reference data in historical error logs. For example, the word vector model in RAG can be used to vectorize the key information of the reference data, obtaining vector representation data of the key information of the reference data. This vector representation data is recorded in the (first) verification database.

[0061] The word embedding model in RAG can vectorize the relationship between error information, the test function of the test object corresponding to the error information, and the error type. It can also vectorize key information in reference data and mine relationships between data recorded in the (first) relational database and the (first) non-relational database. For example, the (first) relational database records the values of A, B, and C in a chip logic function A+B=C during a chip verification. The (first) non-relational database records the logical relationship between A, B, and C, which should be A+B=C. The values of A, B, and C recorded in the (first) relational database are associated with the relationship between the three recorded in the (first) non-relational database. Only by combining the two can the chip logic function A+B=C be verified. The relationship between the two database records can guide the training of the first large language model based on this data relationship, providing a certain guarantee for the accurate training of the (first) large language model.

[0062] In this application, the first large language model to be trained can be trained using the first vector information and the second vector information to obtain the first large language model. This method of using two types of vector information to train a large language model is easy to implement in engineering and has high reliability.

[0063] During implementation, it is possible to obtain a first historical problem and a first historical prompt information generated for the historical error log of the test object; process the first historical problem to obtain a first historical problem vector; obtain knowledge information related to the first historical problem vector from the first vector information and the second vector information; and input the knowledge information related to the first historical problem vector, the first historical problem vector and the first historical prompt information into the first large language model to be trained to train the first large language model to be trained.

[0064] The first historical prompt information refers to the prompt in the chip verification scenario constructed in history for the training of the first large language model to be trained. In the chip verification scenario of the present application, the prompt used when training the large language model may include roles, analysis steps, output requirements, etc. Exemplarily, the role of constructing the prompt is "senior chip verification expert, who needs to analyze the problem in combination with the following knowledge". The analysis steps are "error type identification", "knowledge protocol material matching (protocol material that complies with IEEE / AMBA standards)", "defect case reference (defect discovery information and solutions)", etc. Output requirements include: "specified text format", "labeled knowledge source", etc. According to this exemplary prompt, the large language model can be guided to infer the error information and error cause by identifying the error type in the error log and combining the knowledge protocol material and defect case reference information to perform error reasoning. And guide the large language model to output the error information and error cause in the specified text format to achieve the training of the large language model.

[0065] Combine Figure 4 As shown, in the present application, questions generated by users regarding historical error logs are received, and the questions are used as the first historical questions. For example, regarding historical error log 1, the user generated the historical question "Analysis of the abnormality of chip logic function A+B=C". The historical question is converted into a vector expression of the historical question using RAG's question vector model. The vector expression of the historical question can be used as the first historical question vector. From the (first) verification database, RAG is used to read the vector expression of the values of A, B, and C in historical error log 1, as well as the vector expression of the logical relationship (A+B=C) between the three, and the information read is used as knowledge information related to the first historical question vector. The knowledge information, the vector expression of the historical question, and the Prompt generated for the historical error log are input into the first large language model to be trained to train the first large language model to be trained.

[0066] It can be understood that the (first) verification database is used to record the vector expression of the data stored in the (first) relational database and the (first) non-relational database. Among them, the values of A, B, and C come from the (first) relational database. The logical relationship between A, B, and C comes from the (first) non-relational database. The knowledge information obtained from the (first) verification database is equivalent to the information obtained from the (first) relational database and the (first) non-relational database respectively, and by combining the information obtained from the two databases, an enhanced construction of the data input into the first large language model to be trained is realized. In the present application, this enhanced construction can improve the training accuracy of the (first) large language model.

[0067] In this application, pre-trained large language models include but are not limited to DeepSeek-R1 and ChatGLM-4. Because this application involves chip verification scenarios, the corpus and terminology used to train the large language model in this scenario are all derived from data in the chip verification scenario, so the large language model can be continuously trained and fine-tuned using the corpus and terminology in this scenario.

[0068] In this application, the data stored in the (first) relational database and the (first) non-relational database are derived from data related to error logs and reference data in chip verification scenarios. This technical solution provides technical support for training a large language model capable of automatically analyzing error messages and error causes in chip verification scenarios. Furthermore, combining the (first) relational database and the (first) non-relational database to train this large language model can improve training accuracy.

[0069] In the training scheme for the first language model, for the selected historical error logs, the following can also be done: Using a cluster analysis algorithm, analyze the error messages in the historical error logs to obtain multiple error messages with common characteristics. Multiple error messages with the same or similar error types can be considered to have common characteristics. Error messages with shared characteristics in the historical error logs can be clustered. For example, error messages related to reset logic errors can be clustered and merged into a single cluster. Error messages related to register read and write errors can also be clustered and merged into a single cluster.

[0070] It can be understood that the error information merged into a cluster appears in the (first) relational database in the form of similar triples. Since similar triples have common characteristics, using similar triples to train the first large language model to be trained can facilitate rapid reasoning and output of error information and error causes, making the reasoning of the large language model more accurate.

[0071] In practical applications, if the trained (first) large language model is used to infer and output the error message and error cause in the error log, and if the error message is found to be a new error type, the log content of the error message, the error type, the test function of the chip corresponding to the error message, etc. in the error log can be converted into a vector representation and stored in the (first) verification database. In this way, the new data stored in the (first) verification database can be used to retrain the first large language model to locate more comprehensive error information. This solution can be considered a dynamic feedback mechanism, which can use new data (new error information, the test function of the chip corresponding to the new error information, new error type, etc.) to improve the training of the large language model and enhance the training accuracy of the large language model.

[0072] In practical applications, the error information and error causes in the error log inferred using the trained (first) large language model can be output to relevant personnel. For example, the error information and error causes in the error log can be output to engineers, who can then refer to the output information to resolve the error.

[0073] Figure 5 The implementation diagram of the data processing method in the embodiment of the present application is shown Figure 3 .like Figure 5 As shown, the method includes: S501: Obtain verification data generated by testing a test object.

[0074] In this step, the verification data can be any reasonable verification data generated by testing the test object in the chip verification scenario. For example, verification test points, chip verification plans, bug analysis tables, and coverage analysis tables. Verification test points indicate the verification of each chip test point. Verification plans indicate the chip verification plan. Bug analysis tables indicate the analysis of generated error information. Coverage analysis tables indicate the analysis of test coverage for test cases.

[0075] Verification data can be text. For example, using verification data as a verification test point, the verification data could be the test results for the chip's clock signal, clk. For example, clk is tested for frequency jitter, phase randomness, and duty cycle, including the verification methods and results for each of these three aspects. For example, calling clk macro 1 for random frequency jitter (test method) reveals that the chip's encoding and decoding is normal (test result). Calling clk macro 2 for phase jitter reveals that the chip's encoding and decoding is normal. Calling clk macro 3 for random duty cycle jitter reveals that the chip's encoding and decoding is normal.

[0076] S502: Obtain a second question and second prompt information generated for the verification data.

[0077] In this step, questions generated by the user regarding the verification data are received. For example, the user raises a question regarding the test condition of the clk test point: "How is the clk signal verified? Is the verification comprehensive?" The second prompt information is the prompt in the chip verification scenario, which can be pre-built prompt information for the large language model based on the terminology and corpus in the chip verification scenario. For example, the prompt can serve as a second question generated by the user, and the large language model can use which existing standards or data to reason in which direction. In addition, the prompt can also guide the large language model in how to output the reasoning results, such as in what text format to output the reasoning results.

[0078] S503: Obtain second valid data related to the second question in the verification data.

[0079] Considering that verification data may be data used to verify chip functionality from multiple aspects, such as data used to verify clk, data used to verify the normal reading and writing of registers, and data used to verify the normal handshake timing of AXI, the first question may be specific to a specific aspect. For example, the second question may be "How is the clk signal verified? Is the verification comprehensive?" or "How is the register reading and writing verified? Is the verification comprehensive?" Therefore, the text content related to clk signal verification and register reading and writing verification in the verification data can be used as (second) valid data.

[0080] When implementing, you can combine Figure 6 As shown, RAG can be used to extract text content related to the second question from the verification data. In this application, using RAG to extract text content related to the second question from the verification data is equivalent to constructing the context of the question that requires reasoning with a large language model, thereby enhancing the reasoning of the (second) large language model. This enhanced prompt ensures accurate reasoning of the (second) large language model.

[0081] S504: Input the second valid data, the second question and the second prompt information into the second largest language model to obtain output data of the second largest language model; wherein, the output data is data output according to the prompt of the second prompt information for answering the second question, and the output data at least includes a verification report generated based on the verification data.

[0082] In this step, the text content related to the second question, the second question, and the second prompt information are extracted from the verification data and input into the (second) large language model. The (second) large language model performs model inference based on the three aforementioned inputs, obtains the inference result, and outputs it.

[0083] For example, if the second question is "How is the clk signal verified? Is the verification comprehensive?", the (second) large language model will combine the text content and prompt related to the question to perform model reasoning on the question and output the reasoning results. For example, the (second) large language model will summarize the aspects of the clk signal verification, whether the verification is comprehensive or incomplete, and the test cases used for these aspects in the verification report and output it. The verification report may also include information such as verification errors for the clk signal and the coverage of the test cases used to verify the clk signal.

[0084] The verification report can be output according to a pre-defined template. The template specifies the information to be recorded in each section of the verification report. For example, Section 1 might record the verification aspects, while Section 2 might record the test cases used during verification. During implementation, the verification report can be output according to the template.

[0085] The solution shown in S501-S504 is to obtain a second question and second prompt information generated for the verification data generated by testing the test object, obtain second valid data related to the second question in the verification data, input the second valid data, the second question, and the second prompt information into the second large language model, and obtain a verification report output by the second large language model. The technical solution of this application utilizes a large language model to automatically generate verification reports in chip verification scenarios, avoiding excessive manual intervention and improving the automation, intelligence, and efficiency of verification report generation in chip verification scenarios.

[0086] This application's technical solution provides technical support for the automatic and intelligent generation of verification reports in chip verification scenarios. Furthermore, the text content related to the second question extracted from the verification data in this application serves as the inference context, providing enhanced prompts for the inference of the (second) largest language model and ensuring its accurate inference.

[0087] The second large language model is a trained large language model. The solution shown in S501-S504 utilizes the trained large language model to intelligently and automatically generate verification reports for chip verification scenarios. This application also provides a solution for obtaining a trained second large language model by training the second large language model to be trained.

[0088] Figure 7 FIG. 1 shows a schematic diagram of the implementation of the training method of the second largest language model in the embodiment of the present application. Figure 7 As shown, the training method includes: S701: Obtain historical verification data generated by historical tests on a test object and reference data of the historical verification data.

[0089] In this step, history refers to the time on the timeline prior to the test chip test in S501. Historical verification data refers to verification data such as historical verification test points and historical verification plans generated by historical tests of the test object. Historical verification data is obtained by monitoring historical tests of the test object.

[0090] Reference data for historical verification data can be used to indicate the standards or specifications for various chip functional tests. For example, for verification testing of the CLK signal, the reference data includes its definition (single clock signal) and frequency specification (designed for a maximum frequency of 800 MHz). Given that design standards and verification strategy documents can be used to document standards or specifications, reference data for historical verification data can be pre-recorded in these documents. The reference data for historical verification data can be obtained by reading the data recorded in these documents.

[0091] S702: Using the historical verification data and reference data of the historical verification data, train the second large language model to be trained to obtain a second large language model; wherein the second large language model is used to infer and output a verification report generated based on the verification data.

[0092] It can be considered that the second large language model in the present application is obtained by training the second large language model to be trained using historical verification data generated by historical tests of the test object and reference data of the historical verification data.

[0093] As can be seen from S701 to S702, the (second) large language model to be trained is trained based on historical verification data generated in the chip verification scenario and reference data from the historical verification data, resulting in a large language model capable of generating verification reports. This solution provides technical support for training large language models capable of automatically generating verification reports in chip verification scenarios.

[0094] In practical applications, historical verification data can be text data. Historical verification data such as historical verification test points and historical verification plans generated by historical chip testing may not be structured text data. In this application, Python's pandas library is used to parse documents, and knowledge graph technology is used to preprocess unstructured text data into structured text data. That is, the historical verification data is formatted. For example, after formatting the historical verification test points, a historical test point master table is obtained. The master table includes at least the following structured fields: test point ID (identification) and feature description.

[0095] Taking the verification test of clk as an example, the test point IDs include test point ID1, test point ID2, and test point ID3. Test point ID 1 can indicate a frequency jitter test on clk; test point ID 2 can indicate a phase random test on clk; and test point ID 3 can indicate a duty cycle test on clk. Feature descriptions can be used to indicate whether the codec behaves normally when using different clk macros to perform random jitter on frequency, phase, and duty cycle.

[0096] In the present application, when the historical verification data is formatted, the third vector information is obtained based on the formatted historical verification data. For example, taking the test of clk as an example, the correlation information between the random phase test of clk, the random jittering of the phase of clk using macro 1, and the normal encoding and decoding behavior during jittering, as well as the correlation information between the random frequency test of clk, the random jittering of the frequency of clk using macro 2, and the normal encoding and decoding behavior during jittering, etc., are all recorded in the form of tuples in the (second) relational database. The tuples in the (second) relational database are vectorized using the RAG word vector model. The vector expression of the tuple in the (second) relational database is recorded in the (second) verification database. Each vector data in the (second) verification database can be regarded as third vector information. The vector expression of the three verification aspects, verification results and other related information about the clk test point can be used as a third vector information and recorded in the (second) verification database.

[0097] This application also provides a (second) non-relational database for recording reference data for historical verification data. Since design standard documents and verification strategy documents may have different formats, they can be considered unstructured data. During implementation, a Python file processing library can be called to extract text content from documents in various formats and convert the extracted text into a unified format. Considering that design standard documents and verification strategy documents, in addition to recording reference data, also record other data, namely, information unrelated to the reference data, such as instructions for using the document. During implementation, irrelevant information is removed from the text content, and the text content is hashed to remove duplicates. The remaining text content is normalized. Information is extracted from the normalized text content, such as text content related to CLK signal verification or text content related to the reset signal. A dictionary structure is created from the extracted text content based on information such as title, content, and keywords, and stored in the (second) non-relational database. The dictionary structure stores reference data in the form of title, content, and keywords.

[0098] For example, the title is "clock signal clk", the content is "the frequency or frequency range of the clock signal, the three verification aspects of the clock signal and the verification method of each verification aspect (random jitter), the verification result of each verification aspect (normal or abnormal encoding and decoding)", and the keyword is "clock frequency".

[0099] In this application, information such as title, content, and keywords can be used as key information for reference data of historical verification data. Figure 8 A solution for processing unstructured data such as design standards documents and verification strategy documents is provided. Based on the key information of the historical verification data's reference data, fourth vector information is obtained. For example, the word vector model in RAG can be used to vectorize the key information of the historical verification data's reference data, obtaining vector representation data of the key information of the reference data. This vector representation data is recorded in the (second) verification database.

[0100] The word embedding model in RAG can vectorize data recorded in both the (second) relational database and the (second) non-relational database. It can also mine relationships between the data recorded in the (second) relational and non-relational databases. For example, the (second) relational database records the three verification aspects of CLK, the verification method used for each verification aspect, and the verification results for each verification aspect. The (second) non-relational database records information such as the definition and frequency of CLK. Combining these two databases allows the generation of a verification report for the CLK signal. Mining the relationships between the data recorded in the two databases ensures the accurate training of the (second) large language model.

[0101] In this application, the third vector information and the fourth vector information can be used to train the second large language model to obtain the second large language model. This method of using two types of vector information to train a large language model is easy to implement in engineering and has high reliability.

[0102] During implementation, it is possible to obtain a second historical question and a second historical prompt information generated for the historical verification data of the test object; process the second historical question to obtain a second historical question vector; obtain knowledge information related to the second historical question vector from the third vector information and the fourth vector information; and input the knowledge information related to the second historical question vector, the second historical question vector, and the second historical prompt information into the second large language model to be trained to train the second large language model to be trained.

[0103] The second historical prompt information can be a prompt in a chip verification scenario constructed historically for training the second large language model to be trained. The understanding of this prompt refers to the understanding of the first historical prompt information mentioned above and will not be repeated here.

[0104] Combine Figure 8 As shown, in the present application, questions generated by users for historical verification data are received, and the questions are used as the second historical questions. For example, for historical verification data 1, the user generated the historical question "How is the clk signal verified? Is the verification comprehensive?" The historical question is converted into a vector expression of the historical question using the RAG question vector model. The vector expression of the historical question can be used as the second historical question vector. From the (second) verification database, RAG is used to read the three verification aspects, verification aspects and verification results of the clk signal in the historical verification data 1, as well as the vector expression of the definition and frequency specification of clk, and the read information is used as knowledge information related to the second historical question vector. The knowledge information, the vector expression of the historical question and the prompt generated for the historical verification data are input into the second large language model to be trained to train the second large language model to be trained.

[0105] The above is an explanation of the training of the second large language model to be trained using the historical question "How is the clk signal verified? Is the verification comprehensive?" In actual applications, various historical questions can be used to train the large language model to ensure the improvement of the reasoning ability of the large language model.

[0106] In the aforementioned solution, the knowledge information obtained from the (second) verification database is equivalent to the information obtained from the (second) relational database and the (second) non-relational database, respectively. By combining the information obtained from the two databases, the data input to the second large language model to be trained is enhanced. In the present application, this enhanced construction can improve the training accuracy of the (second) large language model.

[0107] In this application, the data stored in the (second) relational database and the (second) non-relational database are derived from data related to historical verification data and reference data for historical verification data in chip verification scenarios. This technical solution provides technical support for training large language models capable of automatically generating verification reports in chip verification scenarios. Furthermore, combining the (second) relational database and the (second) non-relational database to train this large language model can improve training accuracy.

[0108] In practical applications, the verification report generated using the trained (second) largest language model can be output to relevant personnel. For example, the verification report can be output to engineers to help them understand more comprehensive verification data.

[0109] It should be noted that in this application, the first large language model to be trained and the second large language model to be trained can be the same model. Naturally, the first large language model and the second large language model can be the same model. The first relational database and the second relational database can be the same relational database. The first non-relational database and the second non-relational database can be the same non-relational database. The first verification database and the second verification database can be the same database.

[0110] It should be noted that in this application, Figure 1 The scheme shown can be Figure 5 The solutions shown can be executed separately. Alternatively, they can be executed first in a certain order. Figure 1 The scheme shown and Figure 5 Try one of the options shown, then proceed to the other option.

[0111] This application provides a data processing device embodiment, such as Figure 9 As shown, the device includes: A first obtaining unit 901 is configured to obtain a simulation log generated by testing a test object; A screening unit 902 is used to screen out error logs from simulation logs; The second obtaining unit 903 is configured to obtain a first question and a first prompt message generated in response to the error log; A third obtaining unit 904 is configured to obtain first valid data related to the first problem in the error log; The fourth obtaining unit 905 is used to input the first valid data, the first question and the first prompt information into the first large language model to obtain output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information for answering the first question, and the output data at least includes the error information and / or error cause of the error log.

[0112] In some embodiments, the simulation log is unstructured data; The screening unit 902 is used to pre-process the unstructured simulation log to obtain a structured simulation log; Based on the structural information of the structured simulation log, the error log is filtered out from the simulation log.

[0113] In some embodiments, the first large language model is obtained by training the first large language model to be trained using historical error logs generated by historical tests of the test object and reference data of the historical error logs.

[0114] In some embodiments, the device further comprises a first training unit configured to: Obtaining first vector information based on error information, error type, and a test function of a test object corresponding to the error information in a historical error log; Obtaining second vector information based on key information of reference data in historical error logs; The first large language model to be trained is trained using the first vector information and the second vector information to obtain a first large language model.

[0115] In some embodiments, the first training unit is configured to: Obtaining a first historical problem and a first historical prompt message generated in a historical error log of the test object; Processing the first historical question to obtain a first historical question vector; Obtaining knowledge information related to the first historical question vector from the first vector information and the second vector information; The knowledge information related to the first historical question vector, the first historical question vector and the first historical prompt information are input into a first large language model to be trained to train the first large language model to be trained.

[0116] In some embodiments, the apparatus further comprises: A first obtaining module is used to obtain verification data generated by testing the test object; A second obtaining module, configured to obtain a second question and second prompt information generated for the verification data; A third obtaining module is used to obtain second valid data related to the second question in the verification data; A fourth acquisition module is used to input the second valid data, the second question and the second prompt information into the second largest language model to obtain output data of the second largest language model; wherein, the output data is data output according to the prompt of the second prompt information for answering the second question, and the output data at least includes a verification report generated based on the verification data.

[0117] In some embodiments, the second large language model is obtained by training the second large language model to be trained using historical verification data generated by historical tests of test subjects and reference data of the historical verification data.

[0118] In some embodiments, the device further comprises a first training module for: Format historical verification data; Obtaining third vector information based on the formatted historical verification data; Based on the key information of the reference data of the historical verification data, the fourth vector information is obtained; The second large language model to be trained is trained using the third vector information and the fourth vector information to obtain a second large language model.

[0119] In some embodiments, the first training module is used to: Obtaining a second historical question and second historical prompt information generated based on the historical verification data of the test subject; Processing the second historical question to obtain a second historical question vector; Obtaining knowledge information related to the second historical question vector from the third vector information and the fourth vector information; The knowledge information related to the second historical question vector, the second historical question vector and the second historical prompt information are input into the second large language model to be trained to train the second large language model to be trained.

[0120] This application also provides an embodiment of a large language model training device, which is a first large language model training device. Figure 10 As shown, the device includes: The data acquisition unit 1001 is used to obtain historical error logs generated by historical tests on the test object and reference data of the historical error logs; The model training unit 1002 is used to train the first large language model to be trained based on historical error logs and reference data of historical error logs to obtain a first large language model; wherein, the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

[0121] In some embodiments, the data obtaining unit 1001 is configured to: Obtaining historical simulation logs generated by historical testing of the test object; In response to the historical simulation log being unstructured data, preprocessing the historical simulation log to obtain a structured historical simulation log; Based on the structural information of the structured historical simulation log, the historical error log is filtered out from the historical simulation log.

[0122] It should be noted that the data processing device and the training device of the large language model in the embodiments of the present application have similar principles for solving problems as the aforementioned data processing method and the training method of the large language model. Therefore, the implementation process and implementation principles of the data processing device and the training device of the large language model can be referred to the description of the implementation process and implementation principles of the aforementioned method, and the repeated parts will not be repeated.

[0123] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0124] The electronic device includes at least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data processing method and the large language model training method described in this application. The computer instructions are used to cause the computer to perform the data processing method and the large language model training method described in this application.

[0125] The present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the data processing method and large language model training method of the present application.

[0126] Figure 11 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0127] like Figure 11 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. Computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to bus 804.

[0128] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0129] The computing unit 801 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the data processing method and the large language model training method. For example, in some embodiments, the data processing method and the large language model training method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the data processing method and the large language model training method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the data processing method and the large language model training method in any other appropriate manner (for example, by means of firmware).

[0130] In the context of this application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: Obtaining simulation logs generated by testing the test object; Filter out error logs from simulation logs; Obtaining a first question and a first prompt message generated in response to the error log; Obtaining first valid data related to the first problem in the error log; The first valid data, the first question and the first prompt information are input into a first large language model to obtain output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information for answering the first question, and the output data includes at least the error information and / or error cause of the error log.

2. The method according to claim 1, characterized in that The simulation log is unstructured data; The step of filtering out error logs from simulation logs includes: Preprocess the unstructured simulation log to obtain a structured simulation log; Based on the structural information of the structured simulation log, the error log is filtered out from the simulation log.

3. The method according to claim 1, characterized in that The first large language model is obtained by training the first large language model to be trained using historical error logs generated by historical tests of the test object and reference data of the historical error logs.

4. The method according to claim 3, characterized in that The method comprises: Obtaining first vector information based on error information, error type, and a test function of a test object corresponding to the error information in a historical error log; Obtaining second vector information based on key information of reference data in historical error logs; The first large language model to be trained is trained using the first vector information and the second vector information to obtain a first large language model.

5. The method according to claim 4, characterized in that The step of training the first large language model to be trained by using the first vector information and the second vector information includes: Obtaining a first historical problem and a first historical prompt message generated in a historical error log of the test object; Processing the first historical question to obtain a first historical question vector; Obtaining knowledge information related to the first historical question vector from the first vector information and the second vector information; The knowledge information related to the first historical question vector, the first historical question vector and the first historical prompt information are input into a first large language model to be trained to train the first large language model to be trained.

6. The method according to claim 1, characterized in that The method further comprises: obtaining verification data generated by testing the test object; Obtaining a second question and second prompt information generated for the verification data; Obtaining second valid data related to the second question from the verification data; The second valid data, the second question and the second prompt information are input into the second largest language model to obtain output data of the second largest language model; wherein, the output data is data output according to the prompt of the second prompt information for answering the second question, and the output data at least includes a verification report generated based on the verification data.

7. The method according to claim 6, characterized in that The second large language model is obtained by training the second large language model to be trained using historical verification data generated by historical tests of the test subject and reference data of the historical verification data.

8. The method according to claim 7, characterized in that The method further comprises: Format historical verification data; Obtaining third vector information based on the formatted historical verification data; Based on the key information of the reference data of the historical verification data, the fourth vector information is obtained; The second large language model to be trained is trained using the third vector information and the fourth vector information to obtain a second large language model.

9. The method according to claim 8, characterized in that The training of the second large language model to be trained by using the third vector information and the fourth vector information includes: Obtaining a second historical question and second historical prompt information generated based on the historical verification data of the test subject; Processing the second historical question to obtain a second historical question vector; Obtaining knowledge information related to the second historical question vector from the third vector information and the fourth vector information; The knowledge information related to the second historical question vector, the second historical question vector and the second historical prompt information are input into the second large language model to be trained to train the second large language model to be trained.

10. A method for training a large language model, characterized in that: include: Obtain historical error logs generated by historical tests on the test object and reference data of the historical error logs; Based on historical error logs and reference data of historical error logs, a first large language model to be trained is trained to obtain a first large language model; wherein, the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

11. The method according to claim 10, characterized in that The obtaining of historical error logs generated by historical tests on the test object includes: Obtaining historical simulation logs generated by historical testing of the test object; In response to the historical simulation log being unstructured data, preprocessing the historical simulation log to obtain a structured historical simulation log; Based on the structural information of the structured historical simulation log, the historical error log is filtered out from the historical simulation log.

12. A data processing device, characterized in that: include: A first obtaining unit is used to obtain a simulation log generated by testing the test object; A screening unit, used for screening out error logs from simulation logs; A second obtaining unit is configured to obtain a first question and a first prompt message generated in response to the error log; A third obtaining unit is configured to obtain first valid data related to the first problem in the error log; The fourth obtaining unit is used to input the first valid data, the first question and the first prompt information into the first large language model to obtain output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information for answering the first question, and the output data at least includes the error information and / or error cause of the error log.

13. The device according to claim 12, characterized in that The simulation log is unstructured data; The screening unit is used to pre-process the unstructured simulation log to obtain a structured simulation log; Based on the structural information of the structured simulation log, the error log is filtered out from the simulation log.

14. The device according to claim 12, characterized in that The first large language model is obtained by training the first large language model to be trained using historical error logs generated by historical tests of the test object and reference data of the historical error logs.

15. The device according to claim 14, characterized in that The device further comprises a first training unit, configured to: Obtaining first vector information based on error information, error type, and a test function of a test object corresponding to the error information in a historical error log; Obtaining second vector information based on key information of reference data in historical error logs; The first large language model to be trained is trained using the first vector information and the second vector information to obtain a first large language model.

16. The device according to claim 15, characterized in that The first training unit is configured to: Obtaining a first historical problem and a first historical prompt message generated in a historical error log of the test object; Processing the first historical question to obtain a first historical question vector; Obtaining knowledge information related to the first historical question vector from the first vector information and the second vector information; The knowledge information related to the first historical question vector, the first historical question vector and the first historical prompt information are input into a first large language model to be trained to train the first large language model to be trained.

17. The device according to claim 12, characterized in that The device further comprises: A first obtaining module is used to obtain verification data generated by testing the test object; A second obtaining module, configured to obtain a second question and second prompt information generated for the verification data; A third obtaining module is used to obtain second valid data related to the second question in the verification data; A fourth acquisition module is used to input the second valid data, the second question and the second prompt information into the second largest language model to obtain output data of the second largest language model; wherein, the output data is data output according to the prompt of the second prompt information for answering the second question, and the output data at least includes a verification report generated based on the verification data.

18. The device according to claim 17, characterized in that The second large language model is obtained by training the second large language model to be trained using historical verification data generated by historical tests of the test subject and reference data of the historical verification data.

19. The device according to claim 18, characterized in that The device further includes a first training module for: Format historical verification data; Obtaining third vector information based on the formatted historical verification data; Based on the key information of the reference data of the historical verification data, the fourth vector information is obtained; The second large language model to be trained is trained using the third vector information and the fourth vector information to obtain a second large language model.

20. The device according to claim 19, characterized in that The first training module is used to: Obtaining a second historical question and second historical prompt information generated based on the historical verification data of the test subject; Processing the second historical question to obtain a second historical question vector; Obtaining knowledge information related to the second historical question vector from the third vector information and the fourth vector information; The knowledge information related to the second historical question vector, the second historical question vector and the second historical prompt information are input into the second large language model to be trained to train the second large language model to be trained.

21. A large language model training device, characterized in that: include: A data acquisition unit, configured to acquire historical error logs generated by historical tests on a test object and reference data of the historical error logs; A model training unit is used to train a first large language model to be trained based on historical error logs and reference data of historical error logs to obtain a first large language model; wherein the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

22. The device according to claim 21, characterized in that The data acquisition unit is used to: Obtaining historical simulation logs generated by historical testing of the test object; In response to the historical simulation log being unstructured data, preprocessing the historical simulation log to obtain a structured historical simulation log; Based on the structural information of the structured historical simulation log, the historical error log is filtered out from the historical simulation log.

23. An electronic device, characterized in that: The electronic device comprises: at least one processor; a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9 and / or the method of any one of claims 10 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 9 and / or the method according to any one of claims 10 to 11.

Images