Large model training and evaluation method

Abstract

The invention provides a large model training and evaluation method, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining intelligent question and answer data and a training data set, and the intelligent question and answer data is generated by a user side in response to the interaction operation between a user and a pre-trained large model in a dynamic changeable environment; performing online annotation processing on the intelligent question and answer data to obtain online annotation data; performing data integration processing on the online annotation data and the training data set to obtain an integrated training data set; performing model training on the pre-trained large model by adopting the integrated training data set to obtain an intermediate large model; performing dynamic simulation evaluation processing on the intermediate large model to obtain a first model performance evaluation result, and performing model parameter adjustment processing according to the model performance evaluation result to obtain a target large model; wherein the target large model is used for outputting a question-answer result according to the input user. According to the method, the dynamic adaptability and the accuracy of the output result are improved, so that the questioning experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large model training and evaluation method. Background Art

[0002] With the development of artificial intelligence technology, large models for intelligent question answering are becoming more and more widely used in the field of natural language processing.

[0003] Currently, in the existing technology, the corpus-based supervised learning method is used to train this type of large model, supplemented by some online learning and incremental learning techniques to continuously improve model performance.

[0004] However, the inventors have found that the prior art has at least the following technical problems: the dynamic adaptability of the trained large model is poor, resulting in poor accuracy of the question and answer results output when the large model interacts with the user, seriously affecting the user's question-asking experience. Summary of the invention

[0005] The embodiment of the present application provides a large model training and evaluation method, so that the final large model can generate more accurate and humanized interaction results based on user feedback and context information, improve the dynamic adaptability and accuracy of the output results, and thus improve the user's questioning experience.

[0006] In a first aspect, the present application provides a large model training and evaluation method, which is applied to a server, comprising:

[0007] Acquire intelligent question and answer data and a training data set, wherein the intelligent question and answer data is generated by the user end in response to the user's interactive operation with the pre-trained large model in a dynamic and changing environment;

[0008] Performing online annotation processing on the intelligent question and answer data to obtain online annotated data;

[0009] Performing data integration processing on the online labeled data and the training data set to obtain an integrated training data set;

[0010] Using the integrated training data set to perform model training on the pre-trained large model to obtain an intermediate large model;

[0011] The intermediate large model is dynamically simulated and evaluated to obtain a first model performance evaluation result, and model parameters are adjusted according to the model performance evaluation result to obtain a target large model; wherein the target large model is used to output question and answer results according to input user questions.

[0012] In a possible implementation, the real-time online annotation processing of the intelligent question and answer data to obtain online annotated data includes: performing real-time language recognition processing on the intelligent question and answer data to obtain interaction data for each type of question; indexing the interaction data for each type of question to obtain reply data corresponding to each type of question from a pre-stored corpus, and performing language recognition processing on the reply data and the interaction data for each type of question to obtain a correlation value; when it is detected that the correlation value is less than a preset threshold, annotating the reply data and the interaction data for each type of question as online annotated data.

[0013] In a possible implementation, the online annotated data and the training data set are subjected to data integration processing to obtain an integrated training data set, including: performing retrieval processing on the training data set to obtain a data subset of the same type as the online annotated data; performing language recognition processing on the data subset and the online annotated data to select interaction data and response data whose fit with each type of question reaches a preset fit threshold from the data subset and the online annotated data to determine a target data subset; and storing the target data subset in the training data set to obtain an integrated training data set.

[0014] In a possible implementation, the dynamic simulation evaluation processing is performed on the intermediate large model to obtain a model performance evaluation result, and the model parameter adjustment processing is performed according to the model performance evaluation result to obtain a target large model, including: setting an upper limit value of iteration rounds and an evaluation problem to be input for the intermediate large model, inputting the evaluation problem to be input into the intermediate large model to dynamically evaluate the intermediate large model, and determining the accuracy value of the result output by the intermediate large model when interacting with the evaluation problem to be input in each iteration as the model performance evaluation result of this iteration, until the upper limit value of the iteration rounds is reached or the accuracy value of the model performance result reaches a preset result accuracy; adjusting the model parameters of the intermediate large model according to the model performance evaluation result obtained in each round of iteration to obtain an adjusted intermediate large model after the end of the round of iteration, and when it is detected that the upper limit value of the iteration rounds is reached or the accuracy value of the model performance result reaches the preset result accuracy, the adjusted intermediate large model is determined as the target large model.

[0015] In a possible implementation, before inputting the integrated training data set into the pre-trained large model for model training, it also includes: sending the intelligent question and answer data to the data annotation end, so that the data annotation end performs offline annotation processing in response to the annotation operation of the staff to obtain offline annotation data; receiving the offline annotation data, the online annotation data and the training data set sent by the data annotation end to perform data integration processing to obtain an integrated training data set.

[0016] In a possible implementation, it also includes: obtaining user satisfaction results and error analysis logs, evaluating the target large model according to the user satisfaction results and error analysis logs, and obtaining a second model performance evaluation result; adjusting model parameters of the target large model according to the model performance evaluation result to obtain an adjusted target large model.

[0017] In a possible implementation, it also includes: performing language recognition processing on the intelligent question and answer data to obtain abnormal interaction data whose answer accuracy is lower than a preset accuracy threshold; sending the abnormal interaction data to a data annotation end, so that the data annotation end performs manual annotation operations on the abnormal interaction data in response to the annotation operations of the staff to obtain abnormal annotated data; inputting the abnormal annotated data into a target large model for question and answer self-training processing to obtain user feedback data; and adjusting model parameters of the target large model according to the user feedback data to obtain an adjusted target large model.

[0018] In a possible implementation, it also includes: when the detection target large model is adjusted, a dynamic loading operation is performed to update the original target large model to the adjusted target large model; in response to the interaction operation between the user and the pre-trained large model, the user question to be input is input into the adjusted target large model to obtain the query result.

[0019] In a second aspect, the present application provides a large model training and evaluation device, applied to a server, comprising:

[0020] An intelligent question-answering module, used to obtain intelligent question-answering data and a training data set, wherein the intelligent question-answering data is generated by the user end in response to the user's interactive operation with the pre-trained large model in a dynamic and changing environment;

[0021] A labeling module, used to perform online labeling processing on the intelligent question and answer data to obtain online labeling data;

[0022] A training data collating module is used to perform data integration processing on the online labeled data and the training data set to obtain an integrated training data set;

[0023] A model training iteration module, used to perform model training on the pre-trained large model using the integrated training data set to obtain an intermediate large model;

[0024] The model evaluation and adjustment module is used to perform dynamic simulation evaluation processing on the intermediate large model to obtain a first model performance evaluation result, and to adjust the model parameters according to the model performance evaluation result to obtain a target large model; wherein the target large model is used to output question and answer results according to the input user questions.

[0025] In a third aspect, the present application provides an electronic device, comprising: at least one processor and a memory;

[0026] The memory stores computer-executable instructions;

[0027] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the large model training and evaluation method described in the first aspect above.

[0028] The present application provides a large model training and evaluation method, which first obtains the training data set and the intelligent question and answer data generated by the user end in response to the interactive operation performed by the user on the pre-trained large model in a dynamic and changeable environment, and then performs online annotation processing on the intelligent question and answer data to obtain online annotation data, and then performs data integration processing on the online annotation data and the training data set to obtain an integrated training data set. The pre-trained large model is then trained with the integrated training data set to obtain an intermediate large model, and then the intermediate large model is dynamically simulated and evaluated to obtain the first model performance evaluation result, and the model parameters are adjusted according to the model performance evaluation result to obtain the target large model. Finally, the user question is input into the target large model and the question and answer result is output. In the process of obtaining the question and answer result, online annotation and data integration are used to make the sample quality used for training the pre-trained model higher, thereby accelerating the model training speed while improving the accuracy of the output result of the final target large model, producing more accurate and humanized interaction results based on the user's real-time feedback and context information, improving the dynamic adaptability and accuracy of the output results, and thus improving the user's question experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0030] Figure 1 A schematic diagram of an application scenario of the large model training and evaluation method provided in an embodiment of the present application;

[0031] Figure 2 A flowchart of a large model training and evaluation method provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the execution flow of the large model training and evaluation method provided in the embodiment of the present application;

[0033] Figure 4 A schematic diagram of the structure of a large model training and evaluation device provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] At present, the traditional large model training method focuses on static data set testing, while ignoring the performance of the model in a dynamic and changing environment. The traditional large model training method is a supervised learning method based on a large-scale corpus, supplemented by some online learning and incremental learning techniques to continuously improve model performance. However, the inventors found that traditional training methods usually require a large amount of labeled data to be prepared in advance, which is not only time-consuming and labor-intensive, but also difficult to cover all possible scenarios. Therefore, the large model obtained by training has low accuracy in outputting results in response to long-tail problems and cold start problems, resulting in a poor service experience for users.

[0037] In order to solve the above technical problems, the embodiments of the present application provide the following technical ideas for solving the problems: by introducing online and offline annotation mechanisms for training data, the quality and quantity of data can be effectively improved, thereby improving the training effect of the model. And by establishing an intelligent interactive model self-evaluation mechanism, the performance of the model is dynamically evaluated by simulating real user scenarios, and the model parameters are automatically adjusted according to the evaluation results, so that the large model obtained after training and evaluation can better adapt to various dynamically changing environments, providing users with a more intelligent and efficient questioning experience.

[0038] Figure 1 A schematic diagram of an application scenario of the large model training and evaluation method provided in the embodiment of the present application, such as Figure 1 As shown, it includes: a user terminal 101, a server 102 and a data annotation terminal 103.

[0039] Among them, the user terminal 101 is used to deploy the big model and respond to the user's interactive operations, and display the results of the big model output. The server 102 is used to train and evaluate the big model, and automatically annotate the relevant data to obtain online annotated data, and execute the relevant steps of the big model training and evaluation method to obtain the target big model. The data annotation terminal 103 is used to analyze the historical interaction data using the conversation analysis method to obtain preliminary training samples when the big model is offline, and respond to manual annotation operations on the preliminary training samples to obtain a higher quality data set for training the big model, and then return the data set to the server 102 for model training to obtain a big model with a higher accuracy of the output results.

[0040] Figure 2 This is a flow chart of the large model training and evaluation method provided in the embodiment of the present application. The execution subject of this embodiment can be Figure 1 The server 102 in the illustrated embodiment may also be other computer-related devices, and there is no particular limitation on this embodiment.

[0041] like Figure 2 As shown, the large model training and evaluation method includes:

[0042] S201: Acquire intelligent question and answer data and a training data set, wherein the intelligent question and answer data is generated by the user end in response to the user's interactive operations with the pre-trained large model in a dynamic and changing environment.

[0043] In this embodiment, obtaining a training data set may be a process of selecting a data sample set that can be used for large model training from a pre-stored large model corpus. Obtaining intelligent question and answer data may be generated when a user dynamically interacts with a pre-trained large model through a user terminal. For example, a user enters "What steps are involved in manufacturing product A?" in the question input field of the user terminal, and the pre-trained large model inputs the result of "Manufacturing product A includes step a1, step a2, step a3, and step a4" according to the question.

[0044] In this embodiment, the pre-trained big model can be a trained artificial intelligence AI big model, a language recognition processing big model or other software, web page or interactive robot that can realize the function of interacting with the user.

[0045] S202: Perform real-time online annotation processing on the intelligent question and answer data to obtain online annotated data.

[0046] In this embodiment, real-time online annotation processing can be a process of annotating the intelligent question and answer data generated by users in real time during the interaction process to obtain data related to the question type and whether the data can be used as training data.

[0047] Specifically, in an optional embodiment of the present application, step S202 includes:

[0048] S202a: Perform real-time language recognition processing on the intelligent question and answer data to obtain interactive data for each type of question.

[0049] In this embodiment, performing real-time language recognition processing on the intelligent question and answer data may be a process of performing language recognition on the intelligent question and answer data obtained in each round of question and answer, and obtaining the question type corresponding to the round of question and answer and the output result corresponding to the question type. For example, the answer results corresponding to the troubleshooting type questions and the troubleshooting type questions in the intelligent question and answer data are extracted as the interactive data of this type of questions.

[0050] In this embodiment, the language recognition processing can be performed by a natural language processing model NLP or other large models that can understand, analyze, process and generate human language.

[0051] S202b: Indexing the interaction data of each type of question to obtain the answer data corresponding to each type of question from the pre-stored corpus, and performing language recognition processing on the answer data and the interaction data of each type of question to obtain a correlation value.

[0052] In this embodiment, the indexing process may be a process of searching the pre-stored corpus based on the keywords in the interactive data of each type of question to obtain the answer result for the type of question that matches the keywords. In this embodiment, the pre-stored corpus may be a database that can be called by the pre-trained large model to answer user questions. When the user inputs a question, the pre-trained large model can perform speech recognition on the question and perform inference analysis, and then select the question with the highest similarity and the answer content corresponding to the question from the pre-stored corpus as the output result based on the inference result, and complete the answer result for the user's question.

[0053] In this embodiment, the interaction data and the answer data corresponding to each type of question in the corpus can calculate the text similarity between the two through a natural language processing model to obtain a correlation value.

[0054] S202c: When it is detected that the correlation value is less than a preset threshold, the reply data and the interaction data of each type of question are marked as online marked data.

[0055] In this embodiment, the preset threshold value may be a preset lower limit of the correlation, for example, the preset threshold value may be 70%. When it is detected that the correlation value is less than the preset threshold value, it can be determined that the reply data and the corresponding interaction data of each type of question will be stored as online annotation data to provide training data for improving the accuracy of the results output by the large model. Otherwise, it means that the pre-trained large model has a relatively accurate response to such questions, and there is no need to perform model training for such questions in the future. Moreover, these reply data and the corresponding interaction data of each type of question can make little contribution to improving the accuracy of the model output results, and are not suitable as training data.

[0056] For example, if a question is an intelligent question-and-answer data of production data. The user asks "Machine No. 1 has a downtime failure, what is the solution?" After the question is input into the pre-trained large model, the pre-trained large model uses the output result "The reasons for the downtime of Machine No. 1 may be reason 1, reason 2 and reason 3, among which the solution for reason 1 is to adjust the input voltage; the solution for reason 2 is to adjust the input current; the solution for problem 3 is to replace accessories (such as conveyor belts, sensors)." At this time, the above intelligent interaction data is labeled online in real time. When the user feedbacks whether the output result of the pre-trained large model is effective for the problem to be solved, the server will calculate the score according to the user's feedback. If there is a description such as "very effective" in the feedback, the score is 15 points, if there is a description such as "qualified answer", the score is 13.5 points, if there is a description such as "not related to the problem", the score is less than 10 points, and the question and the corresponding output result of the question are marked. After multiple rounds of real-time intelligent interaction data labeling, these marked data can be used as backup training data for subsequent model training.

[0057] S203: Perform data integration processing on the online labeled data and the training data set to obtain an integrated training data set.

[0058] In this embodiment, data integration processing refers to the process in which the server automatically integrates the labeled data into the training data set. The integration here refers to a new training data set obtained by merging data of the same type, deleting duplicate content, and selecting better content.

[0059] Based on the above embodiment, in an optional embodiment of the present application, step S203 includes:

[0060] S203a: Perform search processing on the training data set to obtain a data subset of the same type as the online annotated data.

[0061] In this embodiment, the retrieval process may be performed by keyword retrieval or text similarity retrieval to obtain a data subset of the same type as the online annotation data in the training data set. For example, if the online annotation data includes data on the production process flow, then the data subset is the training data related to the production process flow in the training data set. If it is data related to production equipment, then the data subset is the training data related to the production equipment in the training data set.

[0062] S203b: Perform language recognition processing on the data subset and the online annotated data to select the interaction data and the reply data whose fit with each type of question reaches a preset fit threshold from the data subset and the online annotated data to determine the target data subset.

[0063] In this embodiment, the language recognition processing is similar to the language recognition processing principle and process in the above embodiment, so this embodiment will not be repeated here. The fit degree can be the overlap or similarity of two data. The fit threshold can be a pre-set value. If any one of the data subset and the online annotation data reaches the fit threshold, it means that the accuracy of the response data in the data is higher, more in line with user expectations, and more suitable for model training.

[0064] S203c: Storing the target data subset into the training data set to obtain an integrated training data set.

[0065] In this embodiment, storing the target data subset into the training data set refers to a process of inserting the target data set into the storage space of the training data set, for example, adding a document into a folder that integrates the training data set.

[0066] S204: Perform model training on the pre-trained large model using the integrated training data set to obtain an intermediate large model.

[0067] In this embodiment, the model training of the pre-trained large model can be to divide the integrated training data set into a training set, a test set and a validation set. First, the training set is used to input the pre-trained large model, and the error between the output result of the pre-trained large model and the actual target is calculated. Then, the optimization algorithm (such as gradient descent) is used to update the model parameters, iteratively update and reduce the error until the pre-set number of sequences is reached or the error reaches below the pre-set error threshold. Then, the validation set is used to evaluate the performance of the pre-trained large model to prevent overfitting, and the hyperparameters or model structure are adjusted according to the verification results. Then, the test set is used to evaluate the final performance of the model, and various performance indicators (such as accuracy, recall, etc.) are calculated. The final large model is the intermediate large model.

[0068] S205: Perform dynamic simulation evaluation on the intermediate large model to obtain a first model performance evaluation result, and adjust the model parameters according to the model performance evaluation result to obtain a target large model, wherein the target large model is used to output a question-and-answer result according to an input user question.

[0069] In this embodiment, dynamic simulation evaluation processing refers to multiple rounds of intelligent question-and-answer interactions between the user and the intermediate large model, and during each round of interaction, the intermediate large model is evaluated and processed, and the parameters of the model are adjusted to obtain the target large model.

[0070] In an optional embodiment of the present application, in step S205, a dynamic simulation evaluation process is performed on the intermediate large model to obtain a first model performance evaluation result, and a model parameter adjustment process is performed according to the model performance evaluation result to obtain a target large model, including:

[0071] S205a: Set the upper limit of iteration rounds and the evaluation problem to be input for the intermediate large model, input the evaluation problem to be input into the intermediate large model to dynamically evaluate the intermediate large model, and determine the accuracy value of the result output by the intermediate large model when interacting with the input evaluation problem in each iteration as the model performance evaluation result of this iteration, until the upper limit of iteration rounds is reached or the accuracy value of the model performance result reaches the preset result accuracy.

[0072] S205b: Adjust the model parameters of the intermediate large model according to the model performance evaluation results obtained in each round of iteration to obtain the adjusted intermediate large model after the end of the round of iteration, and when it is detected that the upper limit of the iteration round is reached or the accuracy value of the model performance result reaches the preset result accuracy, the adjusted intermediate large model is determined as the target large model.

[0073] In this embodiment, after the target big model is deployed on the user side, the user starts the target big model on the user side to establish a connection with the server, and inputs the question into the target big model, and the target big model infers and outputs the final question and answer result from the pre-stored corpus.

[0074] Based on the above embodiment, in an optional embodiment of the present application, before inputting the integrated training data set into the pre-trained large model for model training in step S203, the following steps are further included:

[0075] Step A: Send the intelligent question and answer data to the data annotation end, so that the data annotation end performs offline annotation processing in response to the annotation operation of the staff to obtain offline annotation data.

[0076] In this embodiment, offline annotated data refers to the intelligent question and answer data saved during the interaction between each user and the pre-trained large model, which is annotated by the server after the user end disconnects from the server. In this embodiment, offline annotation processing is the process in which the staff manually annotates the saved intelligent question and answer data through the data annotation end to obtain offline annotated data.

[0077] Step B: receiving the offline annotation data, online annotation data and training data set sent by the data annotation end, and performing data integration processing to obtain an integrated training data set.

[0078] In this embodiment, the data annotation terminal can send the offline annotation data to the server by wireless or wired data transmission. The data integration process in step B is similar to the data integration process in step S203 of the above embodiment, so it will not be repeated here in this embodiment.

[0079] In an optional embodiment of the present application, offline annotation data can also be used to deeply mine historically saved intelligent question and answer data through big data analysis technology, and generate more and higher-quality training sample data through more detailed annotations. These higher-quality training sample data can be better used for subsequent large model optimization and parameter adjustment, so that the final target large model outputs a higher accuracy result.

[0080] Based on the above embodiment, a large model training and evaluation method provided in an optional embodiment of the present application further includes:

[0081] Step C: Obtain user satisfaction results and error analysis logs, evaluate the target large model based on the user satisfaction results and error analysis logs, and obtain the performance evaluation results of the second model.

[0082] In this embodiment, the satisfaction result can be a satisfaction survey sent by the target large model to the user end after the interaction between the user and the target large model is completed, so that the user can perform scoring and evaluation operations on the user end and return the final satisfaction result to the server. The error analysis log can be data obtained by analyzing the error information automatically generated by the target large model during operation.

[0083] Step D: Adjust the model parameters of the target large model according to the model performance evaluation results to obtain the adjusted target large model.

[0084] In this embodiment, the process of parameter adjustment processing is similar to the principle of the model parameter adjustment processing in step S205, so this embodiment will not be repeated here.

[0085] In this embodiment, error analysis logs and satisfaction results are used to increase the dimension of model evaluation and guide subsequent model improvement and upgrade work, so as to further improve the accuracy of the output results of the target large model.

[0086] The large model training and evaluation method provided in an optional embodiment of the present application further includes:

[0087] Step E: Perform language recognition processing on the intelligent question and answer data to obtain abnormal interaction data whose answer accuracy is lower than a preset accuracy threshold.

[0088] In this embodiment, the abnormal interaction data may be questions that the target large model encounters that cannot be answered accurately and data related to the questions that cannot be answered accurately. Among them, the inability to answer accurately means that the accuracy of the answer is lower than a preset accuracy threshold, and the preset accuracy threshold may be a preset accuracy value, for example, the preset accuracy threshold is 50%.

[0089] Step F: Send the abnormal interaction data to the data annotation end, so that the data annotation end performs manual annotation operations on the abnormal interaction data in response to the annotation operations of the staff to obtain abnormal annotation data.

[0090] In this embodiment, the manual labeling operation of abnormal interaction data refers to the labeling operation performed by the staff at the data labeling end after analyzing the abnormal interaction data to obtain better abnormal labeling data for training the target large model.

[0091] Step G: Input the anomaly labeled data into the target large model for question-answering self-training processing to obtain user feedback data.

[0092] Step H: Adjust the model parameters of the target large model according to the user feedback data to obtain the adjusted target large model.

[0093] In this embodiment, question-answering self-training processing refers to the process of automatically generating training samples through the interaction between the target large model and the user, and then the model parameters can be continuously optimized and adjusted through subsequent user feedback to obtain the adjusted target large model.

[0094] Based on the above embodiment, a large model training and evaluation method provided in an optional embodiment of the present application further includes:

[0095] Step I: When the detection target large model is adjusted, a dynamic loading operation is performed to update the original target large model to the adjusted target large model.

[0096] Step J: In response to the interactive operation between the user and the pre-trained large model, the user question to be input is input into the adjusted target large model to obtain the query result.

[0097] In this embodiment, the dynamic loading operation refers to the process in which the adjusted target large model needs to be reloaded and deployed on the server after the model is adjusted to replace the original target large model. When the subsequent user interacts with the pre-trained large model on the user side, after inputting questions to the updated target large model, the updated target large model will output feedback results. So that the user can use the adjusted target large model at the first time, provide higher quality training samples for subsequent model optimization and parameter adjustment, and obtain a target large model with higher output accuracy.

[0098] Figure 3 A schematic diagram of the execution flow of the large model training and evaluation method provided in an embodiment of the present application.

[0099] like Figure 3 As shown, while the user is conducting intelligent question and answer with the pre-trained large model, online annotation processing is performed to obtain online annotated data. When the pre-trained large model is in an offline use scenario, the previous intelligent question and answer data can also be annotated offline to obtain offline annotated data, and the offline annotated data can be checked and manually annotated to obtain abnormal annotated data. Then the online annotated data, offline annotated data and abnormal annotated data are integrated to obtain an integrated training data set for the model training iteration of the pre-trained large model, and an intermediate large model is obtained. The intermediate large model is dynamically simulated and evaluated to obtain an adjusted target large model. The adjusted target large model is dynamically loaded to the server and deployed to the user end and the data annotation end through hot loading to complete the process of dynamically loading the model. Finally, the user uses the latest deployed adjusted target large model to perform intelligent question and answer interactive operations and obtain the output reply result.

[0100] In summary, the large model training and evaluation method provided in the embodiment of the present application first obtains the training data set and the intelligent question and answer data generated by the user end in response to the interactive operation performed by the user in the dynamic and changing environment with the pre-trained large model, and then performs online annotation processing on the intelligent question and answer data to obtain the online annotation data, and then performs data integration processing on the online annotation data and the training data set to obtain the integrated training data set. Then, the pre-trained large model is trained with the integrated training data set to obtain the intermediate large model, and then the intermediate large model is dynamically simulated and evaluated to obtain the first model performance evaluation result, and the model parameters are adjusted according to the model performance evaluation result to obtain the target large model. The target large model can output the question and answer result according to the input user question. In the process of obtaining the question and answer result, online annotation and data integration are used to make the sample quality used for training the pre-trained model higher, thereby accelerating the model training speed while improving the accuracy of the output result of the final target large model, and producing more accurate and humanized interaction results according to the user's real-time feedback and context information, improving the dynamic adaptability and accuracy of the output results, thereby improving the user's question experience.

[0101] At the same time, offline annotation processing is also used to conduct in-depth mining of intelligent question-answering data to discover hidden patterns and rules when large models answer questions, and more and higher-quality training samples are produced through more detailed offline annotation processing and manual annotation processing, further improving the model training speed and the accuracy of output results.

[0102] At the same time, the subsequent interaction between users and the target large model is used to automatically generate training samples, and the model is continuously optimized and model parameters are adjusted through subsequent user feedback. The use of this self-training mechanism can make the output results of the target large model more accurate.

[0103] At the same time, through dynamic loading processing, users can use the latest deployed adjusted target large model to perform intelligent question-and-answer interactive operations in the first place, making the output response results more accurate and improving the user experience.

[0104] Figure 4 A schematic diagram of the structure of a large model training and evaluation device provided in an embodiment of the present application, the device includes: an intelligent question and answer module 41, a labeling module 42, a training data sorting module 43, a model training iteration module 44 and a model evaluation adjustment module 45.

[0105] Intelligent question and answer module 41, used to obtain intelligent question and answer data and training data sets, wherein the intelligent question and answer data is generated by the user end in response to the user's interactive operation with the pre-trained large model in a dynamic and changing environment;

[0106] The labeling module 42 is used to perform online labeling processing on the intelligent question and answer data to obtain online labeled data.

[0107] The training data sorting module 43 is used to perform data integration processing on the online labeled data and the training data set to obtain an integrated training data set.

[0108] The model training iteration module 44 is used to perform model training on the pre-trained large model using the integrated training data set to obtain an intermediate large model.

[0109] The model evaluation and adjustment module 45 is used to perform dynamic simulation evaluation processing on the intermediate large model to obtain a first model performance evaluation result, and to adjust the model parameters according to the model performance evaluation result to obtain a target large model; wherein the target large model is used to output question and answer results according to the input user questions.

[0110] The intelligent question-answering module 41 is also used to input user questions into the target macro model and output question-answering results.

[0111] In an optional embodiment of the present application, the annotation module 42 is specifically used to: perform real-time language recognition processing on the intelligent question and answer data to obtain interaction data for each type of question; perform index processing on the interaction data for each type of question to obtain reply data corresponding to each type of question from a pre-stored corpus, and perform language recognition processing on the reply data and the interaction data for each type of question to obtain a correlation value; when it is detected that the correlation value is less than a preset threshold, the reply data and the interaction data for each type of question are annotated as online annotation data.

[0112] In an optional embodiment of the present application, the training data sorting module 43 is specifically used to: perform search processing on the training data set to obtain a data subset of the same type as the online annotated data; perform language recognition processing on the data subset and the online annotated data to select interaction data and response data whose fit with each type of question reaches a preset fit threshold from the data subset and the online annotated data to determine a target data subset; store the target data subset into the training data set to obtain an integrated training data set.

[0113] In an optional embodiment of the present application, the model evaluation adjustment module 45 is specifically used to: set an upper limit value of iteration rounds and an evaluation problem to be input for the intermediate large model, input the evaluation problem to be input into the intermediate large model to perform dynamic evaluation processing on the intermediate large model, and obtain the accuracy value of the result output by the intermediate large model when interacting with the input evaluation problem in each iteration as the model performance evaluation result of this iteration, until the upper limit value of the iteration round is reached or the accuracy value of the model performance result reaches the preset result accuracy; adjust the model parameters of the intermediate large model according to the model performance evaluation result obtained in each round of iteration to obtain an adjusted intermediate large model after the end of the round of iteration, and when it is detected that the upper limit value of the iteration round is reached or the accuracy value of the model performance result reaches the preset result accuracy, the adjusted intermediate large model is determined as the target large model.

[0114] In an optional embodiment of the present application, the labeling module 42 is further used to: send the intelligent question and answer data to the data labeling end, so that the data labeling end performs offline labeling processing in response to the labeling operation of the staff to obtain offline labeling data; the training data sorting module 43 is further used to: receive the offline labeling data, online labeling data and training data set sent by the data labeling end for data integration processing to obtain an integrated training data set.

[0115] In an optional embodiment of the present application, the model evaluation and adjustment module 45 is also used to: obtain user satisfaction results and error analysis logs, evaluate the target large model according to the user satisfaction results and error analysis logs, and obtain a second model performance evaluation result; adjust the model parameters of the target large model according to the model performance evaluation result to obtain an adjusted target large model.

[0116] In an optional embodiment of the present application, the annotation module 42 is also used to: perform language recognition processing on the intelligent question and answer data to obtain abnormal interaction data whose answer accuracy is lower than a preset accuracy threshold; send the abnormal interaction data to the data annotation end, so that the data annotation end performs manual annotation operations on the abnormal interaction data in response to the annotation operation of the staff to obtain abnormal annotation data. The model training iteration module 44 is also used to: input the abnormal annotation data into the target large model for question and answer self-training processing to obtain user feedback data; adjust the model parameters of the target large model according to the user feedback data to obtain the adjusted target large model.

[0117] In an optional embodiment of the present application, the model training iteration module 44 is also used to: when the detection target large model is adjusted, perform a dynamic loading operation to update the original target large model to the adjusted target large model. The intelligent question answering module 41 is also used to: in response to the interaction between the user and the pre-trained large model, input the user question to be input into the adjusted target large model to obtain the query result.

[0118] The large model training and evaluation device provided in this embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar, and will not be repeated in this embodiment.

[0119] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the device includes: at least one processor 501 and a memory 502 .

[0120] The memory 502 is used to store computer-executable instructions.

[0121] The processor 501 is used to execute the computer-executable instructions stored in the memory 502 to implement the various steps involved in the above method embodiment. For details, please refer to the relevant description in the above method embodiment.

[0122] Optionally, the memory 502 may be independent or integrated with the processor 501 .

[0123] When the memory 502 is independently provided, the device further includes a bus 503 for connecting the memory 502 and the processor 501 .

[0124] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above large model training and evaluation method is implemented.

[0125] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned large model training and evaluation method when executed by a processor.

[0126] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0127] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0128] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit composed of the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0129] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method of each embodiment of the present application.

[0130] It should be understood that the above processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0131] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0132] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0133] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0134] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0135] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0136] The present invention is intended to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solution from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A large model training and evaluation method, characterized in that: Applicable to servers, including: Acquire intelligent question and answer data and a training data set, wherein the intelligent question and answer data is generated by the user end in response to the user's interactive operation with the pre-trained large model in a dynamic and changing environment; Performing online annotation processing on the intelligent question and answer data to obtain online annotated data; Performing data integration processing on the online labeled data and the training data set to obtain an integrated training data set; Using the integrated training data set to perform model training on the pre-trained large model to obtain an intermediate large model; The intermediate large model is dynamically simulated and evaluated to obtain a first model performance evaluation result, and model parameters are adjusted according to the model performance evaluation result to obtain a target large model; wherein the target large model is used to output question and answer results according to input user questions.

2. The method according to claim 1, characterized in that: The performing real-time online annotation processing on the intelligent question and answer data to obtain online annotated data includes: Performing real-time language recognition processing on the intelligent question and answer data to obtain interactive data for each type of question; Indexing the interactive data of each type of question to obtain the answer data corresponding to each type of question from the pre-stored corpus, and performing language recognition processing on the answer data and the interactive data of each type of question to obtain a correlation value; When it is detected that the correlation value is less than a preset threshold, the reply data and the interaction data of each type of question are marked as online marked data.

3. The method according to claim 2, characterized in that The step of integrating the online labeled data and the training data set to obtain an integrated training data set includes: Performing search processing on the training data set to obtain a data subset of the same type as the online annotated data; Performing language recognition processing on the data subset and the online annotated data to select interaction data and answer data whose fit with each type of question reaches a preset fit threshold from the data subset and the online annotated data to determine a target data subset; The target data subset is stored in a training data set to obtain an integrated training data set.

4. The method according to claim 1, characterized in that The dynamic simulation evaluation process is performed on the intermediate large model to obtain a model performance evaluation result, and the model parameter adjustment process is performed according to the model performance evaluation result to obtain a target large model, including: An upper limit value of iteration rounds and an evaluation question to be input are set for the intermediate large model, and the evaluation question to be input is input into the intermediate large model to perform dynamic evaluation processing on the intermediate large model, and the accuracy value of the result output by the intermediate large model when interacting with the evaluation question to be input in each iteration is determined as the model performance evaluation result of this iteration, until the upper limit value of iteration rounds is reached or the accuracy value of the model performance result reaches a preset result accuracy; The model parameters of the intermediate large model are adjusted according to the model performance evaluation results obtained in each round of iteration to obtain the adjusted intermediate large model after the end of the round of iteration. When it is detected that the upper limit value of the iteration round is reached or the accuracy value of the model performance result reaches the preset result accuracy, the adjusted intermediate large model is determined as the target large model.

5. The method according to claim 1, characterized in that: Before inputting the integrated training data set into the pre-trained large model for model training, the method further includes: Sending the intelligent question and answer data to a data annotation end, so that the data annotation end performs offline annotation processing in response to the annotation operation of the staff to obtain offline annotation data; The offline annotation data sent by the data annotation end is received, and the offline annotation data, the online annotation data and the training data set are integrated to obtain an integrated training data set.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: Obtaining user satisfaction results and error analysis logs, and evaluating the target large model according to the user satisfaction results and error analysis logs to obtain a second model performance evaluation result; The target large model is subjected to model parameter adjustment processing according to the model performance evaluation result to obtain an adjusted target large model.

7. The method according to claim 6, characterized in that Also includes: Performing language recognition processing on the intelligent question and answer data to obtain abnormal interaction data whose answer accuracy is lower than a preset accuracy threshold; Sending the abnormal interaction data to a data annotation end, so that the data annotation end performs a manual annotation operation on the abnormal interaction data in response to the annotation operation of the staff, to obtain abnormal annotation data; Inputting the anomaly labeled data into the target large model for question-answering self-training processing to obtain user feedback data; The target large model is subjected to model parameter adjustment processing according to the user feedback data to obtain an adjusted target large model.

8. The method according to claim 7, characterized in that Also includes: When the detection target large model is adjusted, a dynamic loading operation is performed to update the original target large model to the adjusted target large model; In response to the interactive operation between the user and the pre-trained large model, the user question to be input is input into the adjusted target large model to obtain the query result.

9. A large model training and evaluation device, characterized in that: Applicable to servers, including: An intelligent question-answering module, used to obtain intelligent question-answering data and a training data set, wherein the intelligent question-answering data is generated by the user end in response to the user's interactive operation with the pre-trained large model in a dynamic and changing environment; A labeling module, used to perform online labeling processing on the intelligent question and answer data to obtain online labeling data; A training data collating module is used to perform data integration processing on the online labeled data and the training data set to obtain an integrated training data set; A model training iteration module, used to perform model training on the pre-trained large model using the integrated training data set to obtain an intermediate large model; The model evaluation and adjustment module is used to perform dynamic simulation evaluation processing on the intermediate large model to obtain a first model performance evaluation result, and to adjust the model parameters according to the model performance evaluation result to obtain a target large model; wherein the target large model is used to output question and answer results according to the input user questions.

10. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the large model training and evaluation method as described in any one of claims 1 to 8.

Images