Data processing method, training method of large language model, related device and storage medium

Abstract

The application discloses a data processing method, a large language model training method, related equipment and a storage medium, and relates to the technical field of data processing. The data processing method comprises the following steps: obtaining a simulation log generated by testing a test object; screening an error log from the simulation log; obtaining a first problem and first prompt information generated for the error log; obtaining first effective data related to the first problem in the error log; inputting the first effective data, the first problem and the first prompt information into a first large language model to obtain output data of the first large language model; wherein the output data is data for replying to the first problem output according to the prompt of the first prompt information, and the output data at least comprises error information and / or an error cause of the error log. The automatic and intelligent analysis of error problems and / or error causes in a chip verification scene can be realized.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method and apparatus, a training method and apparatus for large language models, electronic equipment, and storage medium. Background Technology

[0002] With the development of hardware technology, chip simulation and verification have presented certain challenges. Related technologies allow for the assessment of chip design flaws through simulation and verification, ensuring error-free design and manufacturing processes and ultimately achieving efficient chip production. Currently, chip simulation and verification still requires significant human intervention, resulting in low efficiency. Therefore, improving the efficiency of chip simulation and verification has become an urgent technical problem to be solved. Summary of the Invention

[0003] This application provides a data processing method and apparatus, a training method and apparatus for large language models, an electronic device, and a storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this application, a data processing method is provided, the method comprising:

[0005] Obtain simulation logs generated during testing of the test object;

[0006] Filter out error logs from the simulation logs;

[0007] Obtain the first problem and first prompt information generated in response to the error log;

[0008] Obtain the first valid data related to the first problem from the error log;

[0009] The first valid data, the first question, and the first prompt information are input into the first large language model to obtain the output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information to reply to the first question, and the output data includes at least the error information and / or error reason of the error log.

[0010] In one possible implementation, the simulation log is unstructured data;

[0011] The step of filtering error logs from simulation logs includes:

[0012] Unstructured simulation logs are preprocessed to obtain structured simulation logs;

[0013] Error logs are filtered out from the structured simulation logs based on their structural information.

[0014] In one possible implementation, the first large language model is obtained by training the first large language model to be trained using historical error logs generated from the historical tests of the test subjects and reference data of the historical error logs.

[0015] In one possible implementation, the method includes:

[0016] Based on the error information, error type, and test function of the test object corresponding to the error information in the historical error log, the first vector information is obtained;

[0017] Based on key information from historical error log reference data, obtain the second vector information;

[0018] The first large language model to be trained is trained using the first vector information and the second vector information to obtain the first large language model.

[0019] In one possible implementation, training the first large language model to be trained using the first vector information and the second vector information includes:

[0020] Obtain the first historical problem and the first historical prompt information generated from the historical error logs of the test object;

[0021] The first historical problem is processed to obtain the first historical problem vector;

[0022] From the first vector information and the second vector information, obtain knowledge information related to the first historical problem vector;

[0023] The knowledge information related to the first historical question vector, the first historical question vector, and the first historical prompt information are input into the first large language model to be trained, so as to train the first large language model to be trained.

[0024] In one possible implementation, the method further includes:

[0025] Obtain verification data generated from testing the test object;

[0026] Obtain a second question and a second prompt message generated in response to the verification data;

[0027] Obtain the second valid data from the verification data that is relevant to the second question;

[0028] The second valid data, the second question, and the second prompt information are input into the second large language model to obtain the output data of the second large language model; wherein, the output data is data output according to the prompt of the second prompt information to answer the second question, and the output data includes at least a verification report generated based on the verification data.

[0029] In one possible implementation, the second large language model is obtained by training the second large language model to be trained using historical verification data generated from the historical tests of the test subjects and reference data of the historical verification data.

[0030] In one possible implementation, the method further includes:

[0031] Format the historical verification data;

[0032] The third vector information is obtained based on formatted historical verification data.

[0033] Based on key information from historical verification data, the fourth vector information is obtained;

[0034] The second large language model is trained using the third and fourth vector information to obtain the second large language model.

[0035] In one possible implementation, training the second large language model to be trained using the third vector information and the fourth vector information includes:

[0036] Obtain a second historical question and a second historical prompt message generated from the historical verification data of the test object;

[0037] The second historical problem is processed to obtain the second historical problem vector;

[0038] From the third and fourth vector information, obtain knowledge information related to the second historical problem vector;

[0039] The knowledge information related to the second historical question vector, the second historical question vector, and the second historical prompt information are input into the second large language model to be trained, so as to train the second large language model.

[0040] According to a second aspect of this application, a method for training a large language model is provided, comprising:

[0041] Obtain historical error logs generated from previous tests on the test object, as well as reference data for the historical error logs;

[0042] Based on historical error logs and reference data of historical error logs, the first large language model to be trained is trained to obtain the first large language model; wherein, the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

[0043] In one possible implementation, obtaining the historical error logs generated from historical tests on the test object includes:

[0044] Obtain historical simulation logs generated from historical tests on the test object;

[0045] Since the historical simulation logs are unstructured data, they are preprocessed to obtain structured historical simulation logs.

[0046] Based on the structural information of the historical simulation logs, historical error logs are filtered out from the historical simulation logs.

[0047] According to a third aspect of this application, a data processing apparatus is provided, comprising:

[0048] The first acquisition unit is used to acquire simulation logs generated during the testing of the test object;

[0049] The filtering unit is used to filter out error logs from the simulation logs;

[0050] The second obtaining unit is used to obtain the first problem and the first prompt information generated in response to the error log;

[0051] The third obtaining unit is used to obtain the first valid data related to the first problem in the error log; the fourth obtaining unit is used to input the first valid data, the first problem and the first prompt information into the first large language model to obtain the output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information to reply to the first problem, and the output data includes at least the error information and / or error reason of the error log.

[0052] According to a fourth aspect of this application, a training device for a large language model is provided, comprising:

[0053] The data acquisition unit is used to acquire historical error logs generated from historical tests of the test object, as well as reference data for the historical error logs.

[0054] The model training unit is used to train the first large language model to be trained based on historical error logs and reference data of historical error logs, so as to obtain the first large language model; wherein, the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

[0055] According to a fifth aspect of this application, an electronic device is provided, comprising:

[0056] At least one processor;

[0057] A memory communicatively connected to the at least one processor; wherein,

[0058] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0059] According to a sixth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0060] According to a seventh aspect of this application, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in this application.

[0061] The technical solution of this application can utilize a large language model to achieve automatic and intelligent analysis of errors and / or causes in chip verification scenarios, avoiding excessive manual intervention and improving the efficiency of chip simulation verification. Furthermore, the large language model can also be used to automatically generate verification reports in chip verification scenarios, avoiding excessive manual intervention and improving the automation, intelligence, and efficiency of verification report generation in chip verification scenarios.

[0062] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0063] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which:

[0064] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0065] Figure 1 This document illustrates the implementation of the data processing method in an embodiment of this application. Figure 1 ;

[0066] Figure 2 This document illustrates the implementation of the data processing method in an embodiment of this application. Figure 2 ;

[0067] Figure 3 This illustration shows the implementation of the training method for the first large language model in the embodiments of this application. Figure 1 ;

[0068] Figure 4 This illustration shows the implementation of the training method for the first large language model in the embodiments of this application. Figure 2 ;

[0069] Figure 5 This document illustrates the implementation of the data processing method in an embodiment of this application. Figure 3 ;

[0070] Figure 6 This document illustrates the implementation of the data processing method in an embodiment of this application. Figure 4 ;

[0071] Figure 7 This illustration shows the implementation of the training method for the second language model in an embodiment of this application. Figure 1 ;

[0072] Figure 8 This illustration shows the implementation of the training method for the second language model in an embodiment of this application. Figure 2 ;

[0073] Figure 9 A schematic diagram of the composition structure of the data processing device in an embodiment of this application is shown;

[0074] Figure 10 This illustration shows a schematic diagram of the composition structure of the training device for the large language model in an embodiment of this application;

[0075] Figure 11 A schematic diagram of the composition structure of the electronic device in an embodiment of this application is shown. Detailed Implementation

[0076] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0077] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. In the various embodiments of this application, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0078] The data processing method in this application can be applied to servers or terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Preferably, the data processing method of this application is applied to a server. The technical solution of this application will be described in detail below.

[0079] Figure 1 This document illustrates the implementation of the data processing method in an embodiment of this application. Figure 1 .like Figure 1 As shown, the method includes:

[0080] S (Step) 101: Obtain the simulation log generated from testing the test object.

[0081] In this application, the test object can be any object that needs to be tested during the design, verification, and production stages. For example, in a chip verification scenario, broadly speaking, the test object can be a chip. Narrowly speaking, the test object can refer to chips of the same type, such as chips with the same model or the same chip. For a designed chip, various test cases can be used to test each function of the test object. For example, test case 1 can be used to test function 1 of chip A, and test case 2 can be used to test function 2 of chip A. Each test generates a corresponding simulation log. The simulation log can be obtained by monitoring the simulation logs generated from testing each function of the chip.

[0082] S102: Filter out error logs from the simulation logs.

[0083] In this application, the simulation logs generated during testing of the test object include logs characterizing test errors (error logs) and logs indicating successful tests (success logs). For the error logs, it is necessary to determine the error and / or cause based on these logs to prevent malfunctions in the chip's functionality.

[0084] In practical applications, chips have diverse functions, requiring verification of a large number of functions and employing numerous test cases. Consequently, the number of simulation logs generated during chip testing is substantial. In some embodiments, the generated simulation logs can be structured data, which includes identifiers indicating which logs are correct and which are incorrect. These identifiers can be used to identify incorrect logs from a large volume of simulation logs, thereby enabling error log filtering.

[0085] In some embodiments, the generated simulation logs can be unstructured data. For the generated unstructured simulation logs, regular expressions can be used in this application to preprocess the unstructured simulation logs to obtain structured simulation logs. For the simulation logs, in addition to the log content itself, after preprocessing, two parts can be obtained for the log. These two parts include a constant part and a variable part. The constant part includes the error type of the error log, the simulation stage (e.g., whether the error occurred during the data transmission stage or the data processing stage), the error flag, the number of errors, etc. The variable part includes the specific error information and the chip function and module corresponding to the error information. Furthermore, it can also include contextual content related to the error information in the log.

[0086] It is understandable that chips typically have many functions. During testing, the functions of the chip can be divided into different modules, and each module can be used as a test unit to test each function in each module one by one. This can avoid the problems of test confusion and incomplete testing caused by too many functions to be tested.

[0087] The aforementioned constant and variable parts can be used as structural information for the simulation log. For the simulation log, if the aforementioned two parts can be extracted or obtained after preprocessing with regular expressions, it indicates that the simulation log is an error log. If the aforementioned two parts cannot be extracted or obtained, it indicates that the simulation log is a correct log. Thus, based on the structural information of the simulation log, error logs can be filtered out from the simulation log, achieving efficient filtering of error logs.

[0088] Furthermore, due to the accuracy of regular expression preprocessing, the use of regular expressions can ensure accurate filtering of error logs, providing a guarantee for the Large Language Model (LLM) to accurately locate error information and / or error causes in the error logs.

[0089] S103: Obtain the first problem and first prompt information generated in response to the error log.

[0090] In this step, we receive user-generated questions from the error log that require inference from the large language model. The first question refers to the user-input question that requires inference from the large language model. For example, a simulation log might include log content verifying the chip's functionality from various aspects. This could include log content verifying the correctness of the chip's signal handshake timings, the correctness of the chip's register read / write functions, and the correctness of the chip's logical functions such as A+B=C. Based on this simulation log, the user might generate at least one of the following questions: "Analyze abnormal signal handshake timings," "Analyze abnormalities in register read / write function verification," or "Analyze anomalies in the verification of the A+B=C logical function." This question will be used as the first question.

[0091] In practical applications, for the same simulation log, in a scenario requiring large language models for inference, the user may raise one or more questions, without specific limitations. The first prompt is a pre-built prompt for the chip verification scenario. This prompt can be based on the terminology and corpus of the chip verification scenario, providing the large language model with pre-constructed prompts. For example, the prompt can guide the large language model on which existing standards or materials to combine and in which direction to reason based on the user's first question. Furthermore, the prompt can also guide the large language model on how to output the inference results, such as the text format used.

[0092] S104: Obtain the first valid data related to the first problem in the error log.

[0093] Considering that error logs may be logs from multiple aspects of chip functional verification, but the first problem encountered may be related to only certain aspects. For example, a problem arising only from verification of timing signals, or a problem arising only from verification of register read / write functions. From the error logs, log content related to the problem the user expects to infer is extracted as (first) valid data.

[0094] For example, in the case of the first problem being "analyzing the verification anomaly of the logical function A+B=C", log content related to register read / write verification is extracted from the error log as valid data. Similarly, in the case of the first problem being "analyzing the abnormality of register read / write function verification", log content related to the verification of the logical function A+B=C is extracted from the error log as valid data. In implementation, this can be combined with... Figure 2As shown, a Retrieval-Augmented Generation (RAG) framework can be used to extract log content related to the first problem from the error log.

[0095] In this application, log content related to the first question extracted from the error log using RAG is used. This effectively constructs the context of the question requiring reasoning by the large language model, thus providing enhanced hints for the large language model's reasoning. This enhanced hint ensures accurate reasoning by the large language model.

[0096] S105: Input the first valid data, the first question, and the first prompt information into the first large language model to obtain the output data of the first large language model; wherein, the output data is the data output according to the prompt of the first prompt information to reply to the first question, and the output data includes at least the error information and / or error reason from the error log.

[0097] In this step, the log content related to the first problem, the first problem, and the first prompt information will be extracted from the error log and input into the (first) large language model. The large language model will perform model inference based on the three input information, obtain the inference result, and output it.

[0098] For example, if the first problem is "analyzing the abnormality of register read / write function verification", the large language model uses the log content related to register-related read / write function verification in the error log to perform model inference on the abnormality of register read / write function, and locate the register read / write error information and the cause of the error. For example, in the logical verification of A+B=C, the model infers whether the values ​​of A and B are incorrect, or whether the logical operation between A and B, such as addition, is incorrect, thus causing the logical verification of A+B=C to fail.

[0099] The solutions shown in S101-S105, based on the error logs generated during testing of the test object, obtain the first problem and the first prompt information generated by the error logs, obtain the first valid data related to the first problem in the error logs, and input the first valid data, the first problem, and the first prompt information into a first large language model to obtain the error information and / or error cause inferred by the first large language model. This technical solution utilizes a large language model to achieve automatic and intelligent analysis of error problems and / or error causes in chip verification scenarios, avoiding excessive manual intervention and improving the efficiency of chip simulation verification.

[0100] The technical solution of this application provides technical support for intelligent analysis of chip simulation verification. Furthermore, in this application, the log content extracted from the error log that is related to the first problem is equivalent to constructing a reasoning context, thereby enhancing the reasoning of the (first) large language model and ensuring accurate reasoning of the (first) large language model.

[0101] The first large language model is a pre-trained large language model. The schemes shown in S101-S105 utilize this pre-trained large language model to intelligently analyze simulation logs in a chip verification scenario. This application also provides a scheme for obtaining a pre-trained first large language model by training the first large language model to be trained.

[0102] Figure 3 This illustration shows the implementation of the training method for the first large language model in the embodiments of this application. Figure 1 .like Figure 3 As shown, the training method includes:

[0103] S301: Obtain historical error logs and reference data of historical error logs generated from historical tests on the test object.

[0104] In this step, "history" refers to a timeline earlier than the test time of the test chip in S101. Historical testing refers to testing the test object in the past. Historical error logs refer to the error logs generated from historical tests of the test object. Historical error logs are obtained by monitoring the historical tests of the test object.

[0105] Historical error log reference data can be used to indicate standards or specifications for testing various chip functions. For example, for testing the normality of register read / write operations, reference data can indicate that if a 1 is written to a register and a 1 is read, the register read / write is normal; otherwise, it is considered abnormal. For testing chip logic functions, reference data can indicate that the operation logic of A, B, and C should be A+B=C. If this logic is satisfied, the logic function is normal; otherwise, it is considered abnormal. Considering that technical documents, standard documents, and expert reports can be used to record standards or specifications, the historical error log reference data can be pre-recorded in these documents. The historical error log reference data is obtained by reading the data recorded in these documents.

[0106] S302: Based on historical error logs and reference data of historical error logs, train the first large language model to be trained to obtain the first large language model; wherein, the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

[0107] In this step, the first large language model can be considered to be obtained by training the first large language model to be trained using historical error logs generated from the historical tests of the test subjects and reference data of the historical error logs.

[0108] In schemes S301-S302, the (first) large language model to be trained is trained based on historical error logs and reference data generated in the chip verification scenario, thereby obtaining a large language model capable of inferring error information and / or error causes from the error logs. This scheme provides technical support for training a large language model capable of automatically locating error information and error causes in the chip verification scenario.

[0109] In some embodiments, the scheme for obtaining historical error logs generated from historical testing of the test object in the aforementioned S301 includes: obtaining historical simulation logs generated from historical testing of the test object; in response to the fact that the historical simulation logs are unstructured data, preprocessing the historical simulation logs to obtain structured historical simulation logs; and filtering out historical error logs from the historical simulation logs based on the structure information of the structured historical simulation logs.

[0110] Historical simulation logs refer to simulation logs generated from historical tests on the test object. Historical simulation logs can be structured or unstructured data. If it's structured data, historical error logs can be identified by distinguishing between error and correct logs. If it's unstructured data, regular expressions can be used to preprocess the unstructured simulation logs to obtain structured simulation logs. If, after preprocessing with regular expressions, the constants and variables of the historical simulation logs can be extracted, then the historical simulation log is a historical error log. For details, please refer to the aforementioned descriptions. Using regular expressions ensures accurate filtering of historical error logs, providing a guarantee for the accurate training of the (first) large language model.

[0111] It is understandable that the historical error logs, filtered based on the structural information of historical simulation logs, are a type of structured data. Each error message appearing in the historical error logs can be recorded in the (first) relational database. Combined with... Figure 4As shown, for example, it can be recorded in the (first) relational database in the form of a triple of "Entity-Relationship-Entity". An example of a triple could be "uvm_error" (error type) – "DEBUG_AXI" (chip bus protocol AXI function) – "The handshake from the ready and valid signals is incorrect".

[0112] The error message "The handshake from the ready and valid signals is incorrect" can be used as an error message. "DEBUG_AXI" can be used as a test function of the test object corresponding to the error message. During implementation, the first vector information can be obtained based on the error message, error type, and test function of the test object corresponding to the error message in the historical error log. For example, the word vector model in RAG can be used to vectorize the triples in the (first) relational database to obtain the vector representation data of the error message. The vector representation data of the error message can be recorded in the (first) verification database. Each vector data in the (first) verification database corresponds to a vector expression of the error message, the test function of the test object corresponding to the error message, and the error type. Each vector data in the (first) verification database can be regarded as the first vector information, which is a vector expression of a triple.

[0113] Unlike the relational data recorded in the (first) relational database, this application also provides a (first) non-relational database for recording reference data of historical error logs. Since technical documents and standard documents may have different formats, they can be considered unstructured data. In implementation, a Python file processing library can be called to extract text content from documents in various formats and convert the extracted text into a unified format. Considering that technical documents and standard documents, in addition to recording reference data, may also record other data, i.e., information unrelated to the reference data, such as usage instructions for the technical document, in implementation, irrelevant information in the text content is removed, the text content is hashed, and duplicate irrelevant information is removed. The remaining text content is then standardized. Information is extracted from the standardized text content, such as text content related to register read / write, text content related to the logical function of chip A+B=C, and text content related to the AXI handshake timing. From the extracted text content, a dictionary structure is created and stored according to information such as title, content, and keywords, and then stored in the (first) non-relational database. The dictionary structure stores reference data in the form of titles, content, and keywords. For example, the title is "AXI", the content is "timing content of AXI handshake signals", and the keyword is "handshake".

[0114] In this application, information such as title, content, and keywords can be used as key information in the reference data. The aforementioned scheme can be considered as... Figure 4 The solution involves preprocessing technical documents and standard documents. During implementation, second vector information is obtained based on key information from historical error log reference data. For example, the word vector model in RAG can be used to vectorize the key information of the reference data, resulting in vector representation data of the key information. This vector representation data is then recorded in the (first) verification database.

[0115] The word vector model in RAG can vectorize the relationship between error information, the test function of the corresponding test object, and the error type. It can also vectorize key information from reference data and mine the relationships between data recorded in a (first) relational database and a (first) non-relational database. For example, the (first) relational database records the values ​​of A, B, and C in a chip logic function A+B=C during a chip verification. The (first) non-relational database records the logical relationship between A, B, and C as A+B=C. The values ​​of A, B, and C recorded in the (first) relational database are related to the relationship between these three values ​​recorded in the (first) non-relational database; only by combining the two can the chip logic function A+B=C be verified. The relationship between the data recorded in the two databases can guide the training of the first large language model to be trained using this data relationship, providing a certain guarantee for the accurate training of the (first) large language model.

[0116] In this application, a first large language model to be trained can be obtained by using first vector information and second vector information. This method of training a large language model using two types of vector information is easy to implement in engineering and has high reliability.

[0117] In implementation, the following steps can be taken: first historical problem and first historical prompt information generated from the historical error logs of the test object; first historical problem can be processed to obtain first historical problem vector; knowledge information related to the first historical problem vector can be obtained from the first vector information and second vector information; and the knowledge information related to the first historical problem vector, the first historical problem vector and the first historical prompt information can be input into the first large language model to be trained to train the first large language model.

[0118] The first historical prompt information refers to the Prompt constructed historically for the training of the first large language model to be trained in the chip verification scenario. In the chip verification scenario of this application, the Prompt used when training the large language model can include roles, analysis steps, output requirements, etc. For example, the role in constructing the Prompt is "Senior Chip Verification Expert, who needs to analyze the problem using the following knowledge." The analysis steps include "Error Type Identification," "Knowledge Protocol Document Matching (Protocol Documents conforming to IEEE / AMBA standards)," and "Defect Case Reference (Defect Discovery Information and Solutions)," etc. Output requirements include: "Specified Text Format," "Identification of Knowledge Sources," etc. Following this exemplary Prompt, the large language model can be guided to infer error information and causes by identifying error types in error logs and combining information such as knowledge protocol documents and defect case references. The large language model is also guided to output error information and causes in a specified text format to achieve the training of the large language model.

[0119] Combination Figure 4 As shown, in this application, a question generated by a user regarding a historical error log is received and used as a first historical question. For example, regarding historical error log 1, the user generated the historical question "Analyze the anomaly in the chip logic function A+B=C". The RAG question vector model is used to convert this historical question into a vector representation of the historical question. This vector representation of the historical question can be used as the first historical question vector. From the (first) verification database, the vector representations of the values ​​of A, B, and C in historical error log 1 are read using RAG, as well as the vector representation of the logical relationship (A+B=C) between the three. This information is used as knowledge information related to the first historical question vector. This knowledge information, the vector representation of the historical question, and the prompt generated for the historical error log are input into the first large language model to be trained to train the first large language model.

[0120] It can be understood that the (first) verification database is used to record the vector representations of data stored in the (first) relational database and the (first) non-relational database. The values ​​of A, B, and C originate from the (first) relational database. The logical relationships between A, B, and C originate from the (first) non-relational database. The knowledge information obtained from the (first) verification database is equivalent to information obtained from both the (first) relational database and the (first) non-relational database. By combining the information obtained from the two databases, the data input to the first large language model to be trained is enhanced. In this application, this enhanced construction can improve the training accuracy of the (first) large language model.

[0121] In this application, the pre-trained large language models include, but are not limited to, DeepSeek-R1 and ChatGLM-4. Because this application involves a chip verification scenario, the corpus and terms used to train the large language models in this scenario are all derived from data in the chip verification scenario, so as to continuously train and fine-tune the large language models using the corpus and terms in this scenario.

[0122] In this application, the data stored in both the (first) relational database and the (first) non-relational database originates from data related to error logs and their reference data in a chip verification scenario. This application's technical solution provides technical support for training a large language model capable of automatically analyzing error information and causes in a chip verification scenario. Furthermore, combining the (first) relational database and the (first) non-relational database to train this large language model can improve training accuracy.

[0123] In the training scheme of the first major language model, for the selected historical error logs, the following can also be done: Based on clustering analysis algorithms, analyze the error information in the historical error logs to obtain multiple error messages with common characteristics. Multiple error messages with the same or similar error types can be considered as error messages with common characteristics. Cluster the error messages with shared characteristics in the historical error logs. For example, cluster the error messages of reset logic errors and merge them into one cluster. Cluster the error messages of register read / write errors and merge them into one cluster.

[0124] It is understandable that error messages merged into a cluster appear in the (first) relational database as similar triples. Because similar triples have common features, using similar triples to train the first large language model can facilitate rapid reasoning and output of error messages and error causes, making the reasoning of the large language model more accurate.

[0125] In practical applications, if the trained (first) large language model is used to infer and output the error information and causes from the error log, and it is found that the error information is a new type of error, then the log content related to this error information, the error type of this error information, and the test function of the chip corresponding to this error information in the error log can be converted into vector representations and stored in the (first) verification database. In this way, the new data stored in the (first) verification database can be used to retrain the first large language model to be trained, so as to be able to locate more comprehensive error information. This scheme can be regarded as a dynamic feedback mechanism, which can use the new data (new error information, the test function of the chip corresponding to the new error information, new error types, etc.) to improve the training of the large language model and enhance the training accuracy of the large language model.

[0126] In practical applications, the error information and causes of errors in the error logs inferred using the trained (first) large language model can be output to relevant personnel. For example, the error information and causes of errors in the error logs can be output to engineers so that they can refer to the output information to resolve the error.

[0127] Figure 5 This document illustrates the implementation of the data processing method in an embodiment of this application. Figure 3 .like Figure 5 As shown, the method includes:

[0128] S501: Obtain verification data generated from testing the test object.

[0129] In this step, the verification data can be any reasonable verification data generated from testing the test object in a chip verification scenario. Examples include verification test points, chip verification plans, bug analysis tables, and coverage analysis tables. Verification test points indicate the verification of each test point on the chip. Verification plans indicate the verification scheme for the chip. Bug analysis tables indicate the analysis of generated error messages. Coverage analysis tables indicate the analysis of test case coverage.

[0130] Verification data can be text. Taking verification data as a test point, the verification data can be the test results of the chip's clock signal clk. For example, it can include verification tests on clk in three aspects: frequency jitter, phase randomness, and duty cycle, as well as the verification methods and results for these three aspects. For instance, calling clk macro1 to perform random frequency jitter (test method) shows that the chip's encoding and decoding behavior is normal (test result). Calling clk macro2 to perform phase jitter shows that the chip's encoding and decoding behavior is normal. Calling clk macro3 to perform random duty cycle jitter shows that the chip's encoding and decoding behavior is normal.

[0131] S502: Obtain a second question and a second prompt message generated in response to the verification data.

[0132] In this step, the system receives user questions regarding the verification data. For example, a user might ask, "How was the clk signal verified? Was the verification comprehensive?" The second prompt is a pre-built prompt for the large language model, based on the terminology and corpus specific to the chip verification scenario. For instance, the prompt can address the user's second question by indicating which existing standards or data the large language model should incorporate for reasoning. Furthermore, the prompt can guide the large language model on how to output the inference results, such as the appropriate text format.

[0133] S503: Obtain the second valid data in the verification data that is relevant to the second question.

[0134] Considering that the verification data may be used to verify the chip's functionality from multiple aspects—such as data verifying the clock signal, data verifying the normality of register read / write operations, and data verifying the normality of the AXI handshake timing—the first question might be about specific aspects. For example, the second question might be "How is the clock signal verified? Is the verification comprehensive?" and "How is register read / write verified? Is the verification comprehensive?"—then the text content related to clock signal verification and register read / write verification in the verification data can be considered as (second) valid data.

[0135] When implementing, it can be combined with Figure 6 As shown, RAG can be used to extract text content related to the second question from the validation data. In this application, the text content related to the second question extracted from the validation data using RAG is equivalent to constructing the context of the question requiring reasoning by the large language model, thus providing enhanced hints for the (second) large language model's reasoning. This enhanced hint provides a guarantee for the accurate reasoning of the (second) large language model.

[0136] S504: Input the second valid data, the second question, and the second prompt information into the second large language model to obtain the output data of the second large language model; wherein, the output data is data output according to the prompt of the second prompt information to answer the second question, and the output data includes at least a verification report generated based on the verification data.

[0137] In this step, the text content related to the second question, the second question, and the second prompt information will be extracted from the verification data and input into the (second) large language model. The (second) large language model will perform model inference based on the three inputs mentioned above, obtain the inference result, and output it.

[0138] For example, if the second question is "How is the clk signal verified? Is the verification comprehensive?", then the (second) large language model will combine the text content and prompt related to the question to perform model inference on the question and output the inference result. For instance, the (second) large language model will summarize information such as the aspects from which the clk signal was verified, whether the verification was comprehensive or incomplete, and the test cases used in these aspects of verification into a verification report and output it. In addition, the verification report may also include outputs on the verification errors of the clk signal, the coverage of the test cases used in verifying the clk signal, etc.

[0139] The verification report can be generated according to a pre-defined template. The template specifies the information to be recorded in each section of the verification report. For example, section 1 records the aspects verified, and section 2 records the test cases used during verification. During implementation, the verification report can be generated according to the template.

[0140] The solutions shown in S501-S504, based on the verification data generated from testing the test object, obtain a second question and a second prompt message related to the verification data, obtain second valid data related to the second question from the verification data, and input the second valid data, the second question, and the second prompt message into a second large language model to obtain a verification report output by the second large language model. This technical solution utilizes a large language model to automatically generate verification reports in chip verification scenarios, avoiding excessive manual intervention and improving the automation, intelligence, and efficiency of verification report generation in chip verification scenarios.

[0141] The technical solution of this application provides technical support for the automatic and intelligent generation of verification reports in chip verification scenarios. Furthermore, in this application, the text content related to the second question extracted from the verification data is equivalent to constructing a reasoning context, thereby enhancing the reasoning of the (second) large language model and ensuring its accurate reasoning.

[0142] The second large language model is a pre-trained large language model. The solutions shown in S501-S504 utilize this pre-trained large language model to intelligently and automatically generate verification reports in chip verification scenarios. This application also provides a solution for obtaining a pre-trained second large language model by training the second large language model to be trained.

[0143] Figure 7 A schematic diagram illustrating the implementation of the training method for the second largest language model in an embodiment of this application is shown. For example... Figure 7 As shown, the training method includes:

[0144] S701: Obtain historical verification data and reference data of historical verification data generated from historical testing of the test object.

[0145] In this step, "historical" refers to times earlier than the testing time of the test chip in S501 on the timeline. Historical verification data refers to verification data such as historical verification test points and historical verification plans generated from historical testing of the test object. Historical verification data is obtained by monitoring the historical testing of the test object.

[0146] Historical verification data reference data can be used to indicate standards or specifications for testing various chip functions. For example, for verification testing of the clk signal, the reference data includes the definition of the clk signal (single clock signal) and the frequency specification (expected design frequency of up to 800 MHz). Considering that design standard documents and verification strategy documents can be used to record standards or specifications, the historical verification data reference data can be pre-recorded in these documents. The historical verification data reference data is obtained by reading the data recorded in these documents.

[0147] S702: Using historical validation data and reference data of historical validation data, train the second large language model to be trained to obtain the second large language model; wherein the second large language model is used for reasoning and outputting a validation report based on the validation data.

[0148] It can be assumed that the second large language model in this application is obtained by training the second large language model to be trained using historical verification data generated from the historical tests of the test subjects and reference data of the historical verification data.

[0149] As can be seen from S701-S702, by training the (second) large language model to be trained based on historical verification data and reference data generated in the chip verification scenario, a large language model capable of generating verification reports is obtained. This scheme provides technical support for training a large language model capable of automatically generating verification reports in the chip verification scenario.

[0150] In practical applications, historical verification data can be text data. Historical verification data, such as historical verification test points and historical verification plans generated from historical chip testing, may not be structured text data. This application utilizes Python's pandas library for document parsing and employs knowledge graph technology to preprocess unstructured text data into structured text data. That is, the historical verification data is formatted. For example, after formatting the historical verification test points, a main table of historical test points is obtained. This main table includes at least the following structured fields: test point ID (identifier) ​​and feature description.

[0151] Taking the verification test of clk as an example, the test point IDs include test point ID1, test point ID2, and test point ID3. Test point ID1 can represent the frequency jitter test of clk; test point ID2 can represent the phase randomness test of clk; and test point ID3 can represent the duty cycle test of clk. The feature description can be used to indicate whether the encoding and decoding behavior is normal when different clk macros are called to randomly jitter the frequency, phase jitter, and duty cycle.

[0152] In this application, when historical verification data is formatted, third vector information is obtained based on the formatted historical verification data. For example, taking the test of clk as an example, the correlation information between random phase testing of clk, random phase jittering of clk using macro 1, and normal encoding / decoding behavior during jittering, as well as the correlation information between random frequency testing of clk, random frequency jittering of clk using macro 2, and normal encoding / decoding behavior during jittering, are all recorded in the form of tuples in the (second) relational database. The word vector model of RAG is used to express the tuples in the (second) relational database as vectors. The vector expression of the tuples in the (second) relational database is recorded in the (second) verification database. Each vector data in the (second) verification database can be regarded as third vector information. The vector expression of the three verification aspects and verification results of the clk test point can be recorded as a kind of third vector information in the (second) verification database.

[0153] This application also provides a (second) non-relational database for recording reference data for historical verification data. Since the formats of design standard documents and verification strategy documents may differ, they can be considered unstructured data. In implementation, a Python file processing library can be called to extract text content from documents in various formats and convert the extracted text into a unified format. Considering that design standard documents and verification strategy documents, in addition to recording reference data, may also record other data, i.e., information unrelated to the reference data, such as usage instructions, in implementation, irrelevant information in the text content is removed, and the text content is hashed to remove duplicates of irrelevant information. The remaining text content is then standardized. Information is extracted from the standardized text content, such as extracting text content related to clk signal verification or text content related to reset signal. From the extracted text content, a dictionary structure is created and stored according to information such as title, content, and keywords, and stored in the (second) non-relational database. The dictionary structure stores reference data in the form of title, content, and keywords.

[0154] For example, the title is "Clock signal clk", the content is "the frequency or frequency range of the clock signal, the three verification aspects of the clock signal and the verification method of each verification aspect (random jitter), the verification result of each verification aspect (normal or abnormal encoding / decoding)", and the keyword is "clock frequency".

[0155] In this application, information such as title, content, and keywords can be used as key information for reference data in historical verification data. The aforementioned scheme can be considered as... Figure 8 This paper describes a scheme for processing unstructured data such as design standard documents and verification strategy documents. Based on key information from reference data of historical verification data, a fourth vector information is obtained. For example, the word vector model in RAG can be used to vectorize the key information of the reference data of historical verification data, resulting in vector representation data of the key information of the reference data. This vector representation data is then recorded in the (second) verification database.

[0156] The word vector model in RAG can vectorize data recorded in both the (second) relational database and the (second) non-relational database, and can also mine the relationships between the data recorded in the (second) relational database and the (second) non-relational database. For example, the (second) relational database records three verification aspects of clk, the verification method used for each verification aspect, and the verification results for each verification aspect. The (second) non-relational database records information such as the definition and frequency specification of clk. Only by combining the two can a report on the verification of clk signals be generated. Mining the relationships between the data recorded in the two databases can provide a certain guarantee for the accurate training of the (second) large language model.

[0157] In this application, the second large language model to be trained can be obtained by utilizing the third and fourth vector information. This training scheme for the large language model using two types of vector information is easy to implement in engineering and has high reliability.

[0158] In implementation, the following steps can be taken: obtaining a second historical question and a second historical prompt information generated from the historical verification data of the test object; processing the second historical question to obtain a second historical question vector; obtaining knowledge information related to the second historical question vector from the third and fourth vector information; and inputting the knowledge information related to the second historical question vector, the second historical question vector, and the second historical prompt information into the second large language model to be trained to train the second large language model.

[0159] The second historical prompt information can be a Prompt constructed historically for the training of the second large language model to be trained, within a chip verification scenario. For an understanding of this Prompt, please refer to the aforementioned understanding of the first historical prompt information, which will not be repeated here.

[0160] Combination Figure 8 As shown, in this application, questions raised by users regarding historical verification data are received and used as second historical questions. For example, regarding historical verification data 1, a user raised the historical question, "How is the clk signal verified? Is the verification comprehensive?" The RAG question vector model is used to convert this historical question into a vector representation. This vector representation of the historical question can serve as the second historical question vector. From the (second) verification database, the RAG is used to read the vector representations of the three verification aspects, verification aspects, and verification results of the clk signal in historical verification data 1, as well as the vector representations of the definition and frequency specification of clk. This information is used as knowledge information related to the second historical question vector. This knowledge information, the vector representation of the historical question, and the prompt generated for the historical verification data are input into the second large language model to be trained to train the second large language model.

[0161] The above explanation uses the historical question of "how is the clk signal verified? Is the verification comprehensive?" as an example to illustrate the training of the second large language model to be trained. In practical applications, various historical questions can be used to train the large language model to ensure the improvement of the large language model's reasoning ability.

[0162] In the aforementioned scheme, the knowledge information obtained from the (second) verification database is equivalent to the information obtained from the (second) relational database and the (second) non-relational database, respectively. By combining the information obtained from the two databases, the data input to the second large language model to be trained is enhanced. In this application, this enhancement can improve the training accuracy of the (second) large language model.

[0163] In this application, the data stored in both the (second) relational database and the (second) non-relational database originates from data related to historical verification data and reference data of historical verification data in the chip verification scenario. The technical solution of this application provides technical support for training a large language model capable of automatically generating verification reports in the chip verification scenario. Furthermore, combining the (second) relational database and the (second) non-relational database to train this large language model can improve training accuracy.

[0164] In practical applications, the verification report generated using the trained (second) largest language model can be output to relevant personnel. For example, the verification report can be output to engineers so that they can understand the verification data more comprehensively.

[0165] It should be noted that in this application, the first large language model to be trained and the second large language model to be trained can be the same model; naturally, the first large language model and the second large language model can be the same model. The first relational database and the second relational database can be the same relational database. The first non-relational database and the second non-relational database can be the same non-relational database. The first validation database and the second validation database can be the same database.

[0166] It should be noted that in this application, Figure 1 The proposed solution can be combined with... Figure 5 The solutions shown can be implemented individually. Alternatively, they can be executed in a specific order, starting with the first solution. Figure 1 The scheme shown and Figure 5 One of the proposed solutions is shown, and then another solution is proposed.

[0167] This application provides an embodiment of a data processing device, such as... Figure 9 As shown, the device includes:

[0168] The first acquisition unit 901 is used to acquire simulation logs generated during the testing of the test object;

[0169] The filtering unit 902 is used to filter out error logs from the simulation logs;

[0170] The second obtaining unit 903 is used to obtain the first problem and the first prompt information generated in response to the error log;

[0171] The third obtaining unit 904 is used to obtain the first valid data related to the first problem in the error log;

[0172] The fourth obtaining unit 905 is used to input the first valid data, the first question, and the first prompt information into the first large language model to obtain the output data of the first large language model; wherein, the output data is data output according to the prompt of the first prompt information to reply to the first question, and the output data includes at least the error information and / or error reason of the error log.

[0173] In some embodiments, the simulation log is unstructured data;

[0174] The filtering unit 902 is used to preprocess the unstructured simulation logs to obtain structured simulation logs.

[0175] Error logs are filtered out from the structured simulation logs based on their structural information.

[0176] In some embodiments, the first large language model is obtained by training the first large language model to be trained using historical error logs generated from the historical tests of the test object and reference data of the historical error logs.

[0177] In some embodiments, the device further includes a first training unit for:

[0178] Based on the error information, error type, and test function of the test object corresponding to the error information in the historical error log, the first vector information is obtained;

[0179] Based on key information from historical error log reference data, obtain the second vector information;

[0180] The first large language model to be trained is trained using the first vector information and the second vector information to obtain the first large language model.

[0181] In some embodiments, the first training unit is configured to:

[0182] Obtain the first historical problem and the first historical prompt information generated from the historical error logs of the test object;

[0183] The first historical problem is processed to obtain the first historical problem vector;

[0184] From the first vector information and the second vector information, obtain knowledge information related to the first historical problem vector;

[0185] The knowledge information related to the first historical question vector, the first historical question vector, and the first historical prompt information are input into the first large language model to be trained, so as to train the first large language model to be trained.

[0186] In some embodiments, the device further includes:

[0187] The first obtaining module is used to obtain verification data generated from testing the test object;

[0188] The second obtaining module is used to obtain a second question and a second prompt message generated in response to the verification data;

[0189] The third obtaining module is used to obtain second valid data related to the second question from the verification data;

[0190] The fourth obtaining module is used to input the second valid data, the second question, and the second prompt information into the second large language model to obtain the output data of the second large language model; wherein, the output data is data output according to the prompt of the second prompt information to answer the second question, and the output data includes at least a verification report generated based on the verification data.

[0191] In some embodiments, the second large language model is obtained by training the second large language model to be trained using historical verification data generated from the historical tests of the test subjects and reference data of the historical verification data.

[0192] In some embodiments, the device further includes a first training module, configured to:

[0193] Format the historical verification data;

[0194] The third vector information is obtained based on formatted historical verification data.

[0195] Based on key information from historical verification data, the fourth vector information is obtained;

[0196] The second large language model is trained using the third and fourth vector information to obtain the second large language model.

[0197] In some embodiments, the first training module is configured to:

[0198] Obtain a second historical question and a second historical prompt message generated from the historical verification data of the test object;

[0199] The second historical problem is processed to obtain the second historical problem vector;

[0200] From the third and fourth vector information, obtain knowledge information related to the second historical problem vector;

[0201] The knowledge information related to the second historical question vector, the second historical question vector, and the second historical prompt information are input into the second large language model to be trained, so as to train the second large language model.

[0202] This application also provides an embodiment of a training device for a large language model, which is a training device for a first large language model. For example... Figure 10 As shown, the device includes:

[0203] The data acquisition unit 1001 is used to acquire historical error logs generated from historical tests of the test object and reference data of the historical error logs;

[0204] The model training unit 1002 is used to train the first large language model to be trained based on historical error logs and reference data of historical error logs, so as to obtain the first large language model; wherein, the first large language model is used to infer and output the error information and / or error cause of the error log generated by the test object.

[0205] In some embodiments, the data acquisition unit 1001 is configured to:

[0206] Obtain historical simulation logs generated from historical tests on the test object;

[0207] Since the historical simulation logs are unstructured data, they are preprocessed to obtain structured historical simulation logs.

[0208] Based on the structural information of the historical simulation logs, historical error logs are filtered out from the historical simulation logs.

[0209] It should be noted that the data processing device and the large language model training device in this application embodiment are similar in principle to the aforementioned data processing method and large language model training method in solving the problem. Therefore, the implementation process and implementation principle of the data processing device and the large language model training device can be referred to the aforementioned method implementation process and implementation principle description, and the repeated parts will not be repeated.

[0210] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0211] The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to perform the data processing method and the large language model training method described in this application. The computer instructions are used to cause the computer to perform the data processing method and the large language model training method described in this application.

[0212] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the data processing method and the large language model training method of this application.

[0213] Figure 11A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0214] like Figure 11 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0215] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0216] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods and large language model training methods. For example, in some embodiments, the data processing methods and large language model training methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing methods and large language model training methods described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform data processing methods or training methods for large language models.

[0217] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0218] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized by, The method comprises: obtaining a simulation log generated by testing a test object; screening an error log from the simulation log; obtaining a first problem and a first prompt information generated for the error log; obtaining first valid data related to the first problem in the error log; inputting the first valid data, the first problem and the first prompt information into a first large language model to obtain output data of the first large language model; wherein the output data is data output for replying to the first problem according to the prompt of the first prompt information, and the output data at least includes error information and / or error reason of the error log; the first large language model is obtained by training a first to-be-trained large language model using historical error logs and reference data of the historical error logs generated by historical testing of the test object; wherein the training of the first to-be-trained large language model using the historical error logs and the reference data of the historical error logs generated by the historical testing of the test object comprises: obtaining first vector information based on error information, error type and test function of the test object corresponding to the error information in the historical error log; obtaining second vector information based on key information of the reference data of the historical error log, wherein the reference data of the historical error log is used to indicate standards or specifications for testing each function of the test object; obtaining a first historical problem and a first historical prompt information generated for the historical error log of the test object; processing the first historical problem to obtain a first historical problem vector; obtaining knowledge information related to the first historical problem vector from the first vector information and the second vector information; inputting the knowledge information related to the first historical problem vector, the first historical problem vector and the first historical prompt information into the first to-be-trained large language model to train the first to-be-trained large language model.

2. The method of claim 1, wherein, The simulation log is unstructured data; the screening of the error log from the simulation log comprises: preprocessing the unstructured simulation log to obtain a structured simulation log; screening the error log from the simulation log based on the structure information of the structured simulation log.

3. The method of claim 1, wherein, The method further comprises: obtaining verification data generated by testing the test object; obtaining a second problem and a second prompt information generated for the verification data; obtaining second valid data related to the second problem in the verification data; inputting the second valid data, the second problem and the second prompt information into a second large language model to obtain output data of the second large language model; wherein the output data is data output for replying to the second problem according to the prompt of the second prompt information, and the output data at least includes a verification report generated based on the verification data.

4. The method of claim 3, wherein the second large language model is obtained by training a second to-be-trained large language model using historical verification data and reference data of the historical verification data generated by historical testing of the test object.

5. The method of claim 4, wherein, The method further comprises: formatting historical verification data; obtaining third vector information based on the formatted historical verification data; obtaining fourth vector information based on key information of reference data of the historical verification data; training the second large language model to be trained using the third vector information and the fourth vector information to obtain a second large language model.

6. The method of claim 5, wherein, The training of the second large language model to be trained using the third vector information and the fourth vector information comprises: obtaining second historical questions and second historical prompt information generated for historical verification data of the test object; processing the second historical questions to obtain second historical question vectors; obtaining knowledge information related to the second historical question vectors from the third vector information and the fourth vector information; inputting the knowledge information related to the second historical question vectors, the second historical question vectors and the second historical prompt information into the second large language model to be trained to train the second large language model to be trained.

7. A method for training a large language model, comprising: comprises: obtaining historical error logs and reference data of the historical error logs generated for historical testing of the test object; obtaining first vector information based on error information, error types and test functions of the test object corresponding to the error information in the historical error logs; obtaining second vector information based on key information of the reference data of the historical error logs, wherein the reference data of the historical error logs is used to indicate standards or specifications for testing each function of the test object; obtaining first historical questions and first historical prompt information generated for the historical error logs of the test object; processing the first historical questions to obtain first historical question vectors; obtaining knowledge information related to the first historical question vectors from the first vector information and the second vector information; inputting the knowledge information related to the first historical question vectors, the first historical question vectors and the first historical prompt information into the first large language model to be trained to train the first large language model to be trained to obtain a first large language model; wherein the first large language model is used to infer and output error information and / or error causes of error logs generated by the test object.

8. The method of claim 7, wherein, The obtaining of the historical error logs generated for the historical testing of the test object comprises: obtaining historical simulation logs generated for the historical testing of the test object; in response to the historical simulation logs being unstructured data, preprocessing the historical simulation logs to obtain structured historical simulation logs; based on the structure information of the structured historical simulation logs, screening the historical error logs from the historical simulation logs.

9. A data processing device, characterized by comprises: a first obtaining unit configured to obtain simulation logs generated for testing of a test object; a screening unit configured to screen error logs from the simulation logs; a second obtaining unit configured to obtain first questions and first prompt information generated for the error logs; a third obtaining unit configured to obtain first valid data related to the first questions in the error logs; The fourth obtaining unit is configured to input the first valid data, the first question, and the first prompt information into a first large language model to obtain output data of the first large language model; the output data is data output for replying to the first question according to the prompt of the first prompt information, and the output data at least includes error information and / or an error cause of the error log; The device further includes a first training unit configured to: obtain a historical error log generated by historical testing of a test object and reference data of the historical error log; obtain first vector information based on error information, an error type, and a test function of the test object corresponding to the error information in the historical error log; obtain second vector information based on key information of the reference data of the historical error log, wherein the reference data of the historical error log is used to indicate a standard or specification for testing each function of the test object; obtain a first historical question and first historical prompt information generated for the historical error log of the test object; process the first historical question to obtain a first historical question vector; obtain knowledge information related to the first historical question vector from the first vector information and the second vector information; input the knowledge information related to the first historical question vector, the first historical question vector, and the first historical prompt information into a first large language model to be trained to train the first large language model to be trained, and obtain the first large language model.

10. The apparatus of claim 9, wherein, The simulation log is unstructured data; The screening unit is configured to preprocess the unstructured simulation log to obtain a structured simulation log; The error log is screened from the simulation log based on structure information of the structured simulation log.

11. The apparatus of claim 9, wherein, The device further includes: a first obtaining module configured to obtain verification data generated by testing the test object; a second obtaining module configured to obtain a second question and second prompt information generated for the verification data; a third obtaining module configured to obtain second valid data related to the second question in the verification data; a fourth obtaining module configured to input the second valid data, the second question, and the second prompt information into a second large language model to obtain output data of the second large language model; the output data is data output for replying to the second question according to the prompt of the second prompt information, and the output data at least includes a verification report generated based on the verification data.

12. The apparatus of claim 11, wherein, The second large language model is obtained by training a second large language model to be trained using historical verification data generated by historical testing of the test object and reference data of the historical verification data.

13. The apparatus of claim 12, wherein, The device further includes a first training module configured to: format the historical verification data; obtain third vector information based on the formatted historical verification data; obtain fourth vector information based on key information of the reference data of the historical verification data; train the second large language model to be trained using the third vector information and the fourth vector information to obtain the second large language model.

14. The apparatus of claim 13, wherein, The first training module is configured to: obtain second historical questions and second historical prompt information generated for historical verification data of the test object; process the second historical questions to obtain second historical question vectors; obtain knowledge information related to the second historical question vectors from the third vector information and the fourth vector information; input the knowledge information related to the second historical question vectors, the second historical question vectors, and the second historical prompt information into a second to-be-trained large language model to train the second to-be-trained large language model.

15. A training device of a large language model, comprising: comprise: a data obtaining unit configured to obtain historical error logs and reference data of the historical error logs generated for historical testing of a test object; a model training unit configured to obtain first vector information based on error information, error types, and test functions of the test object corresponding to the error information in the historical error logs; obtain second vector information based on key information of the reference data of the historical error logs, wherein the reference data of the historical error logs is used to indicate standards or specifications for testing each function of the test object; obtain first historical questions and first historical prompt information generated for the historical error logs of the test object; process the first historical questions to obtain first historical question vectors; obtain knowledge information related to the first historical question vectors from the first vector information and the second vector information; input the knowledge information related to the first historical question vectors, the first historical question vectors, and the first historical prompt information into a first to-be-trained large language model to train the first to-be-trained large language model to obtain a first large language model; wherein the first large language model is used to reason and output error information and / or error causes of error logs generated by the test object.

16. The apparatus of claim 15, wherein, The data obtaining unit is configured to: obtain historical simulation logs generated for historical testing of a test object; in response to the historical simulation logs being unstructured data, pre-process the historical simulation logs to obtain structured historical simulation logs; based on structure information of the structured historical simulation logs, filter historical error logs from the historical simulation logs.

17. An electronic device, comprising: The electronic device comprises: at least one processor; a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1-6 and / or the method of any one of claims 7-8.

18. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method of any one of claims 1-6 and / or the method of any one of claims 7-8.

Images